Unmanned aerial vehicle adopting visual image positioning in power inspection and positioning method thereof
Technical Field
The invention relates to the field of unmanned aerial vehicle power grid inspection, in particular to an unmanned aerial vehicle adopting visual image positioning in power inspection and a positioning method thereof.
Background
In recent years, the unmanned aerial vehicle trade is steadily developed, and the unmanned aerial vehicle performance obtains greatly promoting, and four rotor unmanned aerial vehicle especially have received the favor of trades such as fan and agricultural, express delivery owing to its stability is good, and is easy and simple to handle, characteristics such as can hover have received the hobbyist of taking photo by plane and agriculture.
Unmanned aerial vehicle electric wire netting is patrolled and examined has obtained high attention more recently. Because it has safe and reliable, high-efficient nimble, low cost's characteristic, must replace artifical the inspection step by step.
In addition, taking southern power grid as an example, the asset scale of 110kV and above power transmission lines is rapidly increased, and the annual average growth rate reaches 9.6%. The east-west span of the whole network overhead transmission line is 2000 kilometers, the altitude span is 4300 meters, wherein more than 80 percent of lines are located in mountain and green areas far away from towns, traffic main lines and rare people, the specific weight of the extra-dimensional lines needing extra patrol accounts for 20 percent of the total lines, and the transmission line has high operation and inspection difficulty and high quality requirement. And traditional manual patrol inspection work degree of difficulty is big, and the working cycle is long, and factor of safety is low, and the cost of labor is high, and it is serious to patrol inspection personnel's disappearance phenomenon in recent years simultaneously, and the annual average growth rate is less than 3%. The traditional manual mode can not meet the requirements of operation and maintenance of power grids in China more and more, and under the background, the exploration of a new mode of unmanned aerial vehicle power inspection is very necessary.
Secondly, unmanned aerial vehicles used for power inspection currently rely on a GPS global positioning system. Because the coordinates of towers of general power transmission lines are mostly in regions with obvious topographic relief and remote positions, the positioning effect of the unmanned aerial vehicle is not ideal. Furthermore, the positioning effect is also affected by obstacles and electromagnetic interference. Once GPS satellite positioning is disturbed or signals are lost, operation of the drone in the event of a positioning information error must lead to catastrophic results.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to solve the problems that the existing unmanned aerial vehicle power inspection only adopts GPS positioning, because the tower coordinates of a common power transmission line are mostly in a region with obvious topographic relief and remote position, the positioning effect of the GPS on the unmanned aerial vehicle is not ideal, in addition, the positioning effect can be influenced by obstacles and electromagnetic interference, and once the GPS satellite positioning is interfered or signals are lost, the unmanned aerial vehicle can work under the condition of wrong positioning information to certainly cause disastrous results.
The technical scheme is as follows: in order to solve the above problems, the present invention provides the following technical solutions:
the utility model provides an adopt visual image to carry out unmanned aerial vehicle of location in electric power patrols and examines, has the cloud platform including the unmanned aerial vehicle base member that can realize flight function in unmanned aerial vehicle's bottom, is equipped with the camera lens on the cloud platform and can moves towards the monocular camera of bottom surface all the time.
Furthermore, a three-dimensional laser radar scanning device, an infrared thermal imager, an ultraviolet imager and a multispectral imager which can assist in imaging are further arranged in the base body of the unmanned aerial vehicle, and a high-resolution visible light camera/camera which can detect the electric power tower is further arranged on the holder.
Further, still be equipped with the wireless communication module that can carry out the communication with the ground satellite station in unmanned aerial vehicle's the base member.
An unmanned aerial vehicle positioning method for positioning by adopting visual images in power inspection comprises the following steps:
1) the unmanned aerial vehicle communicates with a ground station after taking off, and the ground station receives image information obtained by shooting of the unmanned aerial vehicle through a monocular camera without stopping;
2) the ground station processes the image frame returned by the unmanned aerial vehicle in real time, calculates R and t of the attitude change of the previous frame, and carries out real-time processing on the image frame
Storing the key frame;
3) eliminate the error, improve unmanned aerial vehicle positioning accuracy.
Further, in the step 2), the change of the pose between adjacent frames is described by a rotation matrix R and a translation vector t, and specifically includes the following steps:
a) the ground station extracts ORB feature points from the image frame and performs matching according to the feature points of the last frame stored in the system;
b) obtaining a plurality of pairs of feature point groups (x) after matching
i1,x
i2) Using epipolar constraint equations
Where E is called the essential matrix and E ═ t
ΛR, and at least 8 pairs of the matched characteristic points construct an equation set; the essence matrix E obtained by adopting the eight-point method comprises the pose transformation information of the camera;
then, the intrinsic matrix E is subjected to SVD (singular value decomposition), i.e. E is decomposed into E ═ U ∑ VTIn the form of (a); there are two groups of solutions for each of R and t:
because the positions of the characteristic points are necessarily in front of the monocular camera, three groups of solutions which are not in conformity can be eliminated;
c) continuously inserting key frames, recording feature point information, performing local nonlinear optimization on the monocular camera pose information by adopting a BA algorithm, screening the recorded key frames, and removing redundant key frames.
Further, the step 3) is divided into a closed-loop detection link and a closed-loop correction link,
and a closed loop detection link calculates bag-of-word information of the key frame, and if similar bag-of-word descriptions exist in the system, the monocular camera returns to a scene which is obtained before.
When a closed loop is detected, calculating similarity transformation by a monocular SLAM through a Sim3 algorithm;
in the closed-loop correction link, firstly, repeated point clouds are fused, and a new edge is inserted into the Coovirility Graph to connect a closed loop; the pose information of the current frame and the key frames connected with the current frame are corrected. And then optimizing a pose Graph through the Essential Graph, and dispersing errors into the whole Graph.
Has the advantages that: compared with the prior art, the invention has the advantages that: according to the invention, a positioning monocular camera is arranged on the unmanned aerial vehicle holder to obtain a real-time image of an object below the unmanned aerial vehicle, the image is transmitted back to the ground station, and the position and attitude of the unmanned aerial vehicle visual positioning camera are calculated by a visual positioning algorithm program ORB-SLAM2 of the ground station, so that the position information of the unmanned aerial vehicle is represented, and the visual positioning of the unmanned aerial vehicle is realized. And a 3D digital map is constructed in the ground station system according to the characteristic point information, the movement track of the unmanned aerial vehicle and the distributed characteristic points are recorded, and meanwhile, the geographic environment information around the power grid transmission line tower can be intuitively known.
Drawings
FIG. 1 is a schematic structural diagram of a technical scheme of visual positioning adopted in the unmanned aerial vehicle power inspection of the invention;
FIG. 2 is a schematic diagram of feature points and monocular camera poses for visual positioning using the ORB algorithm according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The utility model provides an adopt visual image to carry out unmanned aerial vehicle of location in electric power patrols and examines, has the cloud platform including the unmanned aerial vehicle base member that can realize flight function in unmanned aerial vehicle's bottom, is equipped with the camera lens on the cloud platform and can moves towards the monocular camera of bottom surface all the time.
And a high-resolution visible light camera/camera capable of assisting in imaging, three-dimensional laser radar scanning equipment, an infrared thermal imager, an ultraviolet imager and a multispectral imager are further arranged in the base body of the unmanned aerial vehicle.
Still be equipped with the wireless communication module that can carry out the communication with the ground satellite station in unmanned aerial vehicle's the base member.
An unmanned aerial vehicle positioning method for positioning by adopting visual images in power inspection comprises the following steps:
1) the unmanned aerial vehicle communicates with a ground station after taking off, and the ground station receives image information obtained by shooting of the unmanned aerial vehicle through a monocular camera without stopping;
2) the ground station processes the image frame returned by the unmanned aerial vehicle in real time, calculates R and t of the attitude change of the previous frame, and carries out real-time processing on the image frame
Storing the key frame;
3) eliminate the error, improve unmanned aerial vehicle positioning accuracy.
The change in pose between adjacent frames is described by the rotation matrix R and the translation vector t.
The ORB-SLAM2 program is divided into three threads, tracking, local mapping, and closed loop detection.
1) The tracking thread is responsible for extracting ORB feature points from the image frame and matching according to the last frame feature points stored in the system. Obtaining a plurality of pairs of feature point groups (x) after matching
i1,x
i2) Using epipolar constraint equations
(where E is called the essential matrix, and E ═ t
ΛR) and at least 8 pairs of the above matched characteristic points construct an equation set. The essence matrix E obtained by adopting the eight-point method comprises the pose transformation information of the camera.
Then, the intrinsic matrix E is subjected to SVD decomposition (eigenvalue decomposition), i.e., E is decomposed into E ═ U Σ VTIn the form of (1). There are two groups of solutions for each of R and t:
since the positions of the feature points are necessarily in front of the monocular camera, three sets of solutions which are not in conformity can be eliminated.
2) Continuously inserting key frames into the local graph building process, recording the characteristic point information, and performing local nonlinear optimization on the monocular camera position and attitude information by adopting a BA (bundle adjustment) algorithm. And then screening the recorded key frames and eliminating redundant key frames.
3) Errors exist in the positions of the monocular cameras calculated by the first two threads and the point cloud positions of the feature points obtained by triangulation, and even if BA is adopted in the local mapping thread for local or global optimization, accumulated errors still exist. The closed-loop detection thread is used for eliminating accumulated errors and mainly comprises closed-loop detection and closed-loop correction.
And a closed loop detection link calculates bag of words (BOW) information of the key frame, and if similar bag of words description exists in the system, the monocular camera returns to a scene which is obtained before. The author of ORB-SLAM2 trained a large number of word bags based on ORB descriptions offline, requiring loading into the program.
When a closed loop is detected, the monocular SLAM computes a similarity transformation using the Sim3 algorithm.
The closed loop correction link first fuses the duplicate point clouds and inserts a new edge in the Covisibility Graph to connect the closed loops. The pose information of the current frame and the key frames connected with the current frame are corrected. And then optimizing a pose Graph through the Essential Graph, and dispersing errors into the whole Graph.
According to the attached drawing 1, the unmanned aerial vehicle adopts the visual positioning to firstly configure a monocular camera on an unmanned aerial vehicle cloud platform in the electric power inspection, the camera is different from a high-resolution camera used in the electric power facility inspection, the camera is not used for acquiring detection data, but is only used for shooting image data under the unmanned aerial vehicle, therefore, the monocular camera is required to be arranged on a movable cloud platform of the unmanned aerial vehicle, the camera is ensured to be always right on the ground, and the unmanned aerial vehicle is not influenced by the motion attitude of the unmanned aerial vehicle.
The real-time image that monocular camera acquireed is transmitted back to the ground station through wireless transmission module by unmanned aerial vehicle.
And after the ground station obtains the image returned by the unmanned aerial vehicle, the image is subjected to information extraction by using an ORB-SLAM2 program in the ground station system.
The monocular ORB-SLAM2 program requires an initialization process before positioning of the drone can be achieved. The purpose of initialization is to obtain the scale factor of the real physical world and the 3D digital world map by utilizing a plurality of frames of images after starting up, set the coordinates of the flying point of the unmanned aerial vehicle and record the coordinates in the 3D digital map data of the system.
The ground station utilizes an ORB-SLAM2 program to position the unmanned aerial vehicle in real time, records the flight track of the unmanned aerial vehicle, and presents the flight track on a ground station screen in a 3D digital map form. The recorded feature points are all points on an object below the unmanned aerial vehicle, so that the geographic environment condition below the unmanned aerial vehicle can be observed more intuitively by the feature point cloud pictures presented by the 3D digital map at the same time.
Unmanned aerial vehicle can acquire the 3D digital map information that the ground satellite station found through wireless transmission module to know the positional information of self, according to the orbit information of record, unmanned aerial vehicle can be full independently return voyage accurately.