CN107102647A - Unmanned plane target tracking and controlling method based on image - Google Patents

Abstract

The invention discloses an image-based UAV target tracking control method. Establish the target reference; (2) establish a virtual image plane, the camera collects the image information of the target during flight, obtains the collected image, and projects the collected image to the virtual image plane; (3), calculates the distance between the projected image and the target in the vertical direction Relative distance and offset from the target in the horizontal direction; (4), calculate the offset between the projected image and the reference image; (5), calculate the output command of the position controller; (6), generate attitude commands and tension instruction. The control method of the present invention corrects the flight position of the UAV by tracking the image feature points, can realize target tracking, and has a small amount of calculation, does not need sensors such as a magnetic compass, and can avoid the influence of magnetic interference.

Description

Image-based UAV target tracking control method

technical field

The invention relates to the technical field of target tracking, in particular to an image-based control method for target tracking of an unmanned aerial vehicle.

Background technique

At present, the UAV target tracking control method needs to calculate the three-dimensional position of the target relative to the UAV through machine vision. This control method has the following disadvantages: 1. The algorithm is complex and the amount of calculation is large; And attitude information, usually the heading information is obtained through the magnetic sensor, but the magnetic heading is extremely susceptible to interference. When tracking a hostile target or approaching a magnetic field such as a high-voltage wire, the magnetic sensor is prone to failure, which will cause the UAV to lose control, resulting in the inability to track the target.

Contents of the invention

In order to solve the technical problem that the existing unmanned aerial vehicle target tracking control method has a large amount of calculation and the heading information is easily disturbed by magnetic sensors, an image-based unmanned aerial vehicle target tracking control method is proposed. By establishing a virtual image plane , collect the target image, and use the virtual image plane to perform coordinate transformation on the target image feature points, and correct the flight attitude of the UAV by tracking the image feature points, which can also achieve the purpose of target tracking, and this tracking method calculates The volume is small, and it does not rely on the heading information obtained by the magnetic sensor, which can avoid the influence of magnetic interference.

In order to solve the above-mentioned technical problems, the present invention adopts the following technical solutions to achieve:

An image-based UAV target tracking control method, a camera is fixed on the UAV, the lens plane of the camera is parallel to the bottom plane of the UAV and faces downward, and the UAV target tracking control method The method includes the following steps:

(1), the camera collects the image of the target vertically downward at a predetermined height d ^* above the target, obtains a reference image, and extracts a group of feature points from the target, and calculates the area a ^* surrounded by the feature points in the reference image and Reference the center position of the image to establish the target benchmark;

(2), using a mathematical method to establish a virtual image plane, the virtual image plane is parallel to the horizontal plane, the camera periodically collects the image of the target when the drone is flying, obtains the collected image, and projects the collected image to the virtual image plane, Obtain the projected image, and calculate the area a' enclosed by the feature points in the projected image;

(3), according to the area a' enclosed by the feature points in the projected image and the area a ^* enclosed by the feature points in the reference image, calculate the relative distance between the projected image and the target in the vertical direction:

Calculate the horizontal offset of the projected image from the target:

in, is the abscissa value of the center point of the projected image, is the vertical coordinate value of the center point of the projected image;

(4), calculating the offset δ between the projected image and the reference image;

(5) Calculate the output command U of the position controller:

Among them, k _p , k _d , and _ki are the control parameters of the UAV, which are set by the user;

(6) An attitude instruction rd and a pulling force instruction _{f d are} generated from the position controller output instruction U, which are used to control the UAV to adjust the flight height and flight deflection angle.

Further, the calculation method of the area a' enclosed by the feature points in the projected image in step (2) is:

Wherein, N is the quantity of characteristic point, is a positive integer, and the projection Cartesian coordinate system is made on the plane where the projected image is located, and _u'k is in the projected Cartesian coordinate system, the abscissa value of the kth feature point in the projected image, v' _k is the vertical coordinate value of the kth feature point in the projected image in the projected Cartesian coordinate system.

Further, the calculation method of the area a ^* enclosed by the feature points in the reference image in step (1) is:

Wherein, in the machine Cartesian coordinate system on the plane where the reference image is located, u _k is the abscissa value of the kth feature point in the reference image in the machine Cartesian coordinate system, and v _k is in the machine Cartesian coordinate system, The vertical coordinate value of the kth feature point in the reference image.

Further, in step (3), the abscissa value of the center point of the projected image The calculation method is:

The vertical coordinate value of the center point of the projected image The calculation method is:

Further, the calculation method of the offset δ in step (4) is:

δ=[δ _x δ _y δ _z ] ^T

in,

Further, the calculation method of attitude instruction r _d in step (6) is:

The calculation method of the pulling force command f _d is:

f _d =m||U||, where m is the mass of the UAV.

Compared with the prior art, the advantages and positive effects of the present invention are: the image-based UAV target tracking control method of the present invention, by establishing a virtual image plane, collecting target images, selecting feature points and using the virtual image plane to track the target Coordinate transformation of the image feature points, correcting the flight position of the UAV by tracking the image feature points, can also achieve the purpose of target tracking, and this tracking method has a small amount of calculation and does not require sensors such as a magnetic compass, which can avoid Effects of magnetic interference.

Other features and advantages of the present invention will become more apparent after reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic diagram of an embodiment of the image-based UAV target tracking control method proposed by the present invention;

Fig. 2 is a flow chart of an embodiment of the image-based UAV target tracking control method proposed by the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiment 1. This embodiment proposes an image-based UAV target tracking control method. As shown in FIG. The plane is parallel and facing downward. Ideally, the UAV keeps flying in a horizontal direction, but in the actual flight process, the flight attitude is not fixed, and there may be a certain angle with the horizontal plane, but the angle does not affect the image of the target by the camera 11 Acquisition, as long as the target is still within the image shooting range of the camera, the camera can obtain the collected image information of the target. This method analyzes the image information of the collected target, and then corrects the flight position of the drone so that the target is within the range of the camera. The captured image is always in the right position and maintains a suitable size in the image. Since the position and size of the image taken by the target are stable, it is reflected in the actual space, that is, the relative position between the drone and the target is stable, and then realized the tracking of the unmanned aerial vehicle to the target, the method will be described with a specific embodiment, and the unmanned aerial vehicle target tracking control method comprises the following steps in the present embodiment:

S1. The camera collects the image of the target vertically downward at a predetermined height d ^* above the target, obtains a reference image, and extracts a set of feature points from the target, and calculates the area a ^* enclosed by the feature points in the reference image and the reference image The center position of the target is used to establish the target reference; the reference image belongs to an image that the UAV can obtain for the target in an ideal state, that is, the camera is placed directly above the target at a height of d ^* , vertically downward The image of the target is collected to obtain a reference image. The flight height, that is, the distance d ^* from the target to the virtual image plane is the expected tracking height. In this ideal situation, the projection of the target on the virtual image plane is used as a reference image. The images taken when the man-machine is not in an ideal state have corresponding references, and the attitude adjustment information of the UAV can be obtained by comparing the difference between the two images.

S2. Use mathematical methods to establish a virtual image plane. The virtual image plane is parallel to the horizontal plane. When the drone is flying, the camera periodically collects the image of the target to obtain the collected image. Project the collected image to the virtual image plane to obtain the projected image and calculate the features. The area a' enclosed by the points in the projected image; as can be seen from the above, the collected image is the image obtained by the actual shooting of the target by the camera. Since the camera moves with the drone, the shooting angle and height change all the time.

By selecting feature points, there is no need to calculate and track all the points in the target image, effectively reducing the amount of calculation.

S3. According to the area a' enclosed by the feature points in the projected image and the area a ^* enclosed by the feature points in the reference image, calculate the relative distance between the projected image and the target in the vertical direction:

The collected image is projected on the virtual image, the area of the target in the projected image, that is, the area a' enclosed by the feature points in the projected image is positively correlated with the height, and the proportional coefficient is obtained by taking the quadratic root of the area, that is, when When the area a' enclosed by the feature points in the projected image is equal to the area a ^* enclosed by the feature points in the reference image, it means that the relative distance between the current projected image and the target in the vertical direction is d ^* , that is, the acquisition The plane where the image is located is on the same plane as the reference image. When the area a' enclosed by the feature points in the projected image is larger than the area a ^* enclosed by the feature points in the reference image, q _z is greater than d, indicating that the plane where the collected image is located is higher than Refer to the image plane, that is, the flying height of the UAV is higher than the expected height d ^* , otherwise, the flying height of the UAV is lower than the expected height d ^* .

Calculate the horizontal offset of the projected image from the target:

in, is the abscissa value of the center point of the projected image, is the vertical coordinate value of the center point of the projected image;

S4. Calculate the offset δ between the projected image and the reference image; when there is an offset between the projected image and the reference image, it means that the UAV is not directly above the target at this time.

S5, calculate position controller output instruction U:

Among them, k _p , k _d , and _ki are the control parameters of the UAV, which are set by the user;

S6. Generate an attitude instruction _{rd and a pulling force instruction fd from} the position controller output instruction U, which are used to control the UAV to adjust the flight height and flight deflection angle. If the attitude command _{rd and the pulling force command f d are} obtained through calculation, the UAV can be directly controlled.

For example, a quadrotor UAV only needs a small change in motor speed to generate sufficient pitch and roll torque, while generating heading torque requires a large change in motor speed and takes a long time to adjust. Since the image feature point area and relative position error of the UAV are only related to the attitude command _{rd and the pulling force command fd ,} but not to the heading, the error tracking control can be realized only by changing the attitude without changing the heading. Thus the attitude control part decouples the pitch part from the heading part.

The calculation method of the area a' enclosed by the feature points in the projected image in step S2 is:

Wherein, N is the quantity of characteristic point, is a positive integer, and the projection Cartesian coordinate system is made on the plane where the projected image is located, and _u'k is in the projected Cartesian coordinate system, the abscissa value of the kth feature point in the projected image, v' _k is the vertical coordinate value of the kth feature point in the projected image in the projected Cartesian coordinate system;

The calculation method of the area a ^* surrounded by the feature points in the reference image in step S1 is:

Wherein, in the machine Cartesian coordinate system on the plane where the reference image is located, u _k is the abscissa value of the kth feature point in the reference image in the machine Cartesian coordinate system, and v _k is in the machine Cartesian coordinate system, The vertical coordinate value of the kth feature point in the reference image.

In step S3, the abscissa value of the center point of the projected image The calculation method is:

The vertical coordinate value of the center point of the projected image The calculation method is:

The calculation method of offset δ in step S4 is:

δ=[δ _x δ _y δ _z ] ^T

in,

It can be seen from the foregoing description that the reference image belongs to an image that the UAV can acquire for the target in an ideal state, that is, the UAV is located in the horizontal direction and directly above the target in the vertical direction, and the flight height is also The distance d* from the target to the virtual image plane is the expected tracking height. Under this ideal condition, the acquired projection of the target on the virtual image plane is used as a reference image. Therefore, the offset δ between the projected image and the reference image reflects The offset between the acquired image in the horizontal direction and the target, that is,

The calculation method of attitude instruction r _d in step S6 is:

The calculation method of the pulling force command f _d is:

f _d =m||U||, where m is the mass of the UAV.

The above formula is the conventional control method of the UAV, which can calculate the corresponding attitude command and pulling force command through the output command U of the position controller.

Of course, the above descriptions are not intended to limit the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention shall also belong to protection scope of the present invention.

Claims (6)

1. An image-based UAV target tracking control method, characterized in that, a video camera is fixed on the UAV, and the lens plane of the video camera is parallel to the bottom plane of the UAV and faces downwards, the The UAV target tracking control method includes the following steps: (1), the camera collects the image of the target vertically downward at a predetermined height d ^* above the target, obtains a reference image, and extracts a group of feature points from the target, and calculates the area a ^* surrounded by the feature points in the reference image and Reference the center position of the image to establish the target benchmark; (2), using a mathematical method to establish a virtual image plane, the virtual image plane is parallel to the horizontal plane, the camera periodically collects the image of the target when the drone is flying, obtains the collected image, and projects the collected image to the virtual image plane, Obtain the projected image, and calculate the area a' enclosed by the feature points in the projected image; (3), according to the area a' enclosed by the feature points in the projected image and the area a ^* enclosed by the feature points in the reference image, calculate the relative distance between the projected image and the target in the vertical direction: Calculate the horizontal offset of the projected image from the target: <mrow> <msub> <mi>q</mi> <mi>x</mi> </msub> <mo>=</mo> <msub> <mi>q</mi> <mi>z</mi> </msub> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mo>;</mo> </mrow> <mrow> <msub> <mi>q</mi> <mi>y</mi> </msub> <mo>=</mo> <msub> <mi>q</mi> <mi>z</mi> </msub> <mover> <mi>v</mi> <mo>&amp;OverBar;</mo> </mover> <mo>;</mo> </mrow> in, is the abscissa value of the center point of the projected image, is the vertical coordinate value of the center point of the projected image; (4), calculating the offset δ between the projected image and the reference image; (5) Calculate the output command U of the position controller: <mrow> <mi>U</mi> <mo>=</mo> <mo>-</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>p</mi> </msub> <mi>&amp;delta;</mi> <mo>+</mo> <msub> <mi>k</mi> <mi>d</mi> </msub> <mover> <mi>&amp;delta;</mi> <mo>&amp;CenterDot;</mo> </mover> <mo>+</mo> <msub> <mi>k</mi> <mi>i</mi> </msub> <mo>&amp;Integral;</mo> <mi>&amp;delta;</mi> <mo>)</mo> </mrow> <mo>;</mo> </mrow> Among them, k _p , k _d , and _ki are the control parameters of the UAV, which are set by the user; (6) An attitude instruction rd and a pulling force instruction _{f d are} generated from the position controller output instruction U, which are used to control the UAV to adjust the flight height and flight deflection angle. 2. image-based unmanned aerial vehicle target tracking control method according to claim 1 is characterized in that, in the step (2), the calculation method that feature point encloses area a ' in projection image is: <mrow> <msup> <mi>a</mi> <mo>&amp;prime;</mo> </msup> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msup> <mrow> <mo>(</mo> <msubsup> <mi>u</mi> <mi>k</mi> <mo>&amp;prime;</mo> </msubsup> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>+</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msup> <mrow> <mo>(</mo> <msubsup> <mi>v</mi> <mi>k</mi> <mo>&amp;prime;</mo> </msubsup> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> Wherein, N is the quantity of characteristic point, is a positive integer, and the projection Cartesian coordinate system is made on the plane where the projected image is located, and _u'k is in the projected Cartesian coordinate system, the abscissa value of the kth feature point in the projected image, v' _k is the vertical coordinate value of the kth feature point in the projected image in the projected Cartesian coordinate system. 3. image-based UAV target tracking control method according to claim 1, is characterized in that, The calculation method of the area a ^* enclosed by the feature points in the reference image in step (1) is: <mrow> <msup> <mi>a</mi> <mo>*</mo> </msup> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msup> <mrow> <mo>(</mo> <msub> <mi>u</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>+</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msup> <mrow> <mo>(</mo> <msub> <mi>v</mi> <mi>k</mi> </msub> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> Wherein, in the machine Cartesian coordinate system on the plane where the reference image is located, u _k is the abscissa value of the kth feature point in the reference image in the machine Cartesian coordinate system, and v _k is in the machine Cartesian coordinate system, The vertical coordinate value of the kth feature point in the reference image. 4. image-based UAV target tracking control method according to claim 2, is characterized in that, in step (3), the abscissa value of the central point of projection image The calculation method is: <mrow> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mo>=</mo> <mfrac> <mrow> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msubsup> <mi>u</mi> <mi>k</mi> <mo>&amp;prime;</mo> </msubsup> </mrow> <mi>N</mi> </mfrac> </mrow> The vertical coordinate value of the center point of the projected image The calculation method is: <mrow> <mover> <mi>v</mi> <mo>&amp;OverBar;</mo> </mover> <mo>=</mo> <mfrac> <mrow> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msubsup> <mi>v</mi> <mi>k</mi> <mo>&amp;prime;</mo> </msubsup> </mrow> <mi>N</mi> </mfrac> <mo>.</mo> </mrow> 5. image-based UAV target tracking control method according to claim 4, is characterized in that, the computing method of offset δ in step (4) is: δ=[δ _x δ _y δ _z ] ^T in, 6. image-based UAV target tracking control method according to claim 5, is characterized in that, in the step (6), the computing method of _posture instruction r is: <mrow> <msub> <mi>r</mi> <mi>d</mi> </msub> <mo>=</mo> <mfrac> <mi>U</mi> <mrow> <mo>|</mo> <mo>|</mo> <mi>U</mi> <mo>|</mo> <mo>|</mo> </mrow> </mfrac> </mrow> The calculation method of the pulling force command f _d is: f _d =m||U||, where m is the mass of the UAV.