CN108919838B - An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision - Google Patents
An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision Download PDFInfo
- Publication number
- CN108919838B CN108919838B CN201810978744.9A CN201810978744A CN108919838B CN 108919838 B CN108919838 B CN 108919838B CN 201810978744 A CN201810978744 A CN 201810978744A CN 108919838 B CN108919838 B CN 108919838B
- Authority
- CN
- China
- Prior art keywords
- power line
- unmanned aerial
- aerial vehicle
- line
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/12—Target-seeking control
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Aviation & Aerospace Engineering (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an unmanned aerial vehicle power transmission line automatic tracking method based on binocular vision. The power line area in the image is extracted by using a human eye vision significance detection technology, and the range is accurately measured according to a binocular vision imaging model, so that the flight direction and the attitude of the unmanned aerial vehicle are controlled. The binocular vision-based unmanned aerial vehicle power transmission line automatic tracking system and method can accurately measure the distance between the unmanned aerial vehicle and the laser mark point on the power line, and meet the requirement of the unmanned aerial vehicle on safe automatic detection and tracking of the power transmission line.
Description
Technical Field
The invention relates to the field of power transmission line inspection, in particular to an unmanned aerial vehicle power transmission line automatic tracking method based on binocular vision.
Background
With the rapid development of economy, the demand of society on electric power is increased rapidly, and power transmission line routing inspection becomes an important means for guaranteeing national power utilization safety. However, the power transmission line has the characteristics of long span, complex terrain of a region where the power transmission line passes through, and the like, and manual routing inspection cannot meet requirements, so that most of the existing power companies adopt a cooperative routing inspection mode of 'machine routing inspection as a main mode and human routing inspection as an auxiliary mode' to carry out routing inspection on the power transmission line. And current unmanned aerial vehicle patrols and examines and is the manual operation unmanned aerial vehicle operation, will guarantee to keep certain safe distance between unmanned aerial vehicle and the transmission line all the time, requires that the unmanned aerial vehicle operator has abundant cumulative experience. However, traditional unmanned aerial vehicle patrols and examines the distance that relies on people's eye to observe unmanned aerial vehicle and power line, requires highly to staff's experience, visual angle, operating condition, is difficult to guarantee the high security that unmanned aerial vehicle patrolled and examined, and shoots the distance and neglects nearly neglected and neglected far away suddenly, easily leads to shooting the quality poor, reduces and patrols and examines efficiency, influences transmission line normal function scheduling problem even. Therefore, the distance between the unmanned aerial vehicle and the high-voltage transmission line in the unmanned aerial vehicle inspection is accurately measured, the high-voltage transmission line is automatically tracked, the safety threat of the unmanned aerial vehicle to the transmission line is avoided, the shooting quality of the unmanned aerial vehicle is guaranteed, and the key factor influencing the unmanned aerial vehicle transmission line inspection system is formed.
In an actual measurement environment, the background of the power line is complex, the power line is accurately extracted from the image, and the characteristic points on the power line are registered, which is an important premise for distance measurement and tracking of the power line.
When utilizing unmanned aerial vehicle to carry out the power line and detecting, because unmanned aerial vehicle flies in power line side below, the power line in the shooting picture is nearly on a parallel with aircraft advancing direction. The image edge can reflect the main characteristics of the target in the image to a certain extent, and Canny as a classic edge detection algorithm can effectively detect the boundaries of all objects, but cannot highlight the edge of a remarkable target. The DoG algorithm can achieve the effects of exciting a local central region and inhibiting surrounding neighborhoods, and accords with the visual characteristics of human eyes, so that the significance of an image can be reflected to a certain degree. Therefore, the method adopts the DoG (difference of Gaussian) algorithm and the Canny edge detection algorithm to extract the significant edge and the power line, can weaken the influence caused by the background, quickly screens out the power line from the complex background, and has wider application range.
The invention realizes the automatic distance measurement from the unmanned aerial vehicle to the power line by using the binocular vision model and combining with the auxiliary light source to mark the characteristic points. Most of existing unmanned aerial vehicles adopt monocular cameras as image acquisition methods, but monocular vision contains a small amount of information, three-dimensional information cannot be embodied, and meanwhile, the visual field range is limited and is easily influenced by external environments. Compared with monocular vision, the binocular vision technology simulates a human binocular vision system and obtains three-dimensional information of an environment and a target by using parallax information in left and right eye images. The most important steps of actual distance measurement of the binocular image are identifying the same-name points and carrying out image matching, and common matching algorithms can be divided into dense matching and sparse matching. The dense matching algorithm can theoretically complete depth calculation of a complete picture, obtain the three-dimensional coordinates of each point in an image and realize three-dimensional reconstruction, but the precision and the running speed of the algorithm are difficult to meet the requirement of real-time patrol. The sparse matching algorithm comprises a SIFT algorithm and a SURF algorithm, and the SURF algorithm is used as an improved algorithm of SIFT, and has the advantages of high speed and good stability, and high feature point identification rate and accuracy can be still maintained under the conditions of view angle, illumination, scale change and the like. However, the traditional sparse matching algorithm is difficult to ensure that the characteristic points on the power line can be identified certainly, so that the sparse matching algorithm cannot be used for distance measurement of the unmanned aerial vehicle and the power line.
Disclosure of Invention
The invention provides an unmanned aerial vehicle power transmission line automatic tracking method based on binocular vision, aiming at the problems that the existing unmanned aerial vehicle power transmission line routing inspection cannot automatically detect and track a power transmission line and the safe distance between an unmanned aerial vehicle and the power transmission line is difficult to guarantee.
The technical scheme adopted by the invention is as follows: an unmanned aerial vehicle power transmission line automatic tracking method based on binocular vision combines a binocular camera and a line laser transmitter which are carried by an unmanned aerial vehicle, and adopts a vision significance detection and binocular ranging technology to realize extraction, ranging and tracking of a power line.
The method comprises the following steps:
1) calibrating a binocular camera: completing calibration of the binocular camera by using a Zhang calibration method, and acquiring internal and external parameters of the binocular camera, wherein the internal and external parameters comprise a focal length, a base line distance, a rotation matrix, a translation matrix and the like;
2) acquiring binocular images to be detected: the line laser emitter emits red line laser, the line laser is dispersed and refracted by using the cylindrical lens to form a fan-shaped light beam, a line irradiated by the line laser emitter is kept vertical, and the line is irradiated to each power line for marking the power lines; after the unmanned aerial vehicle takes off at a designated position, shooting binocular images of a power line tracked and inspected, transmitting the images to an embedded chip, and correcting the images according to internal and external parameters of a binocular camera;
3) detecting a power line region: for the left and right eye images obtained in the step 2), firstly, calculating salient points by using a Gaussian difference method, then, carrying out edge detection by using a Canny algorithm, and screening according to the number of the salient points on the edges to obtain candidate elements of the power line;
4) detecting power lines and labeling: because unmanned aerial vehicle and power line are close parallel direction flight, so the power line is approximate parallel state. Calculating the slope of each power line candidate element, reserving the power line candidate elements with the slope dip angle smaller than a set value, further grouping the broken candidate power line elements according to the distance from the midpoint of each power line candidate element to the straight line of all other power line candidate elements, grouping the power line candidate elements belonging to the same power line into one group, splicing all the power line candidate elements in the same group into a complete power line, and simultaneously labeling each power line in the sequence from top to bottom;
5) extracting laser marking points on all power lines: searching and extracting the positions of the left and right eye image laser marking points of each power line according to the marks for all the power lines extracted in the step 4) according to the pixel colors;
6) unmanned aerial vehicle-power line distance calculation: extracting the parallax and camera parameters of the laser mark points of the left and right eye images according to the previous step, calculating three-dimensional coordinates of two point pairs by using a binocular vision model, and calculating the distance from the unmanned aerial vehicle to each power line;
7) adjusting the flight direction of the unmanned aerial vehicle according to the obtained distance between the unmanned aerial vehicle and the power line, and if the distance between the unmanned aerial vehicle and each power line is larger than the minimum distance threshold ThreshminAnd is less than a maximum distance threshold ThreshmaxThe current direction is proved to be correct, and the flying vehicle continues to fly forwards; if the distance between the unmanned aerial vehicle and each power line is less than the minimum distance threshold ThreshminControlling the unmanned aerial vehicle to obliquely fly towards the opposite direction of the power line; if the distance between the unmanned aerial vehicle and each power line is greater than the set maximum distance threshold ThreshmaxAnd controlling the unmanned aerial vehicle to obliquely fly towards the direction close to the power line, and transmitting the extracted power line position, the corresponding GPS information and the video recorded by the binocular camera back to the ground through the wireless transmission module.
The invention has the beneficial effects that: according to the invention, the power line is detected by using the visual significance of human eyes, the distance measurement between the unmanned aerial vehicle and the power line is realized by using a binocular vision model, the detection result is transmitted to a flight control system as a flight instruction, the flight direction and the attitude of the unmanned aerial vehicle are adjusted, the automatic tracking of the unmanned aerial vehicle on the power transmission line is completed, and the shot picture and the corresponding GPS positioning information are transmitted back to the ground by using a wireless transmission module, so that interfaces are reserved for the functions of power line obstacle detection and the like, and the function expansion is convenient. The system designed by the invention has the advantages of high inspection efficiency, high safety, strong universality and the like.
Drawings
Fig. 1 is an operation schematic diagram of the automatic tracking method for the power transmission line of the unmanned aerial vehicle of the invention;
fig. 2 is a structural diagram of an automatic tracking system of an unmanned aerial vehicle power transmission line;
FIG. 3 is a schematic diagram of a dual camera and infrared laser transmitter;
FIG. 4 is an algorithm flow chart of the unmanned aerial vehicle power transmission line automatic tracking method of the present invention;
FIG. 5(a) is a grayscale image of an original image during saliency edge extraction;
FIG. 5(b) a DoG saliency map in the saliency edge extraction process;
FIG. 5(c) a Canny edge detection effect diagram in the salient edge extraction process;
FIG. 5(d) is a graph of the power line candidate effect in the salient edge extraction process;
FIG. 6 is a schematic diagram of laser mark point distance measurement.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings.
The invention relates to an unmanned aerial vehicle power transmission line automatic tracking system and a method based on binocular vision, the unmanned aerial vehicle power transmission line automatic tracking system is provided with a GPS module, a wireless transmission module and a holder module provided with a binocular camera and a line laser emitter, the power line extraction, the distance measurement and the unmanned aerial vehicle power line tracking are realized by adopting the vision significance and the binocular distance measurement technology, and the operation schematic diagram of the system is shown in figure 1.
Fig. 2 is a structural diagram of an automatic tracking system for an unmanned aerial vehicle power transmission line. The system comprises an embedded platform A, an unmanned aerial vehicle flight control module B, GPS module C, a wireless transmission module D and a holder E. The embedded platform A adopts a Radar Rock microcomputer to control the unmanned aerial vehicle flight control system B and plan a cruising route, analyzes a data packet received by the GPS module C, analyzes an image acquired by the binocular camera and controls the holder E; the unmanned aerial vehicle flight control module B controls the unmanned aerial vehicle to fly according to the cruise route and receives an instruction transmitted by the embedded platform A to adjust the flight attitude of the unmanned aerial vehicle; the GPS module C receives a data packet of related geographic information and transmits the data packet to the embedded platform A for analysis; the wireless transmission module D adopts a 4G communication module to send the processed information and the video back to the ground together; the pan-tilt E adopts a three-axis pan-tilt, the left camera, the right camera and the line laser emitter are kept horizontal, and the plane of the line laser is ensured to be vertical, as shown in figure 3.
Firstly, a worker sets a routing inspection route for an embedded platform A and a safe distance range of the unmanned aerial vehicle and a power transmission line as Threshmin，Threshmax]The power line location area is detected by using a visual saliency detection technology, the distance between the unmanned aerial vehicle and the power line is accurately measured by using an auxiliary light combined binocular vision model, the unmanned aerial vehicle and the power line are guaranteed to be kept within a safe distance range, safe automatic routing inspection of the power transmission line is realized, and finally, the recorded video is combined with GPS information and transmitted back to the ground by using a wireless transmission module D.
FIG. 4 is a flow chart of the automatic tracking method for the power transmission line of the unmanned aerial vehicle according to the present invention, which comprises calibrating a binocular camera by the Zhang calibration method, using the calibrated binocular camera to shoot left and right eye images of a target to be detected, calculating a saliency edge as a candidate power line primitive by combining a Gaussian difference algorithm (DoG) and a Canny algorithm, grouping and splicing the broken candidate power line primitives into real power lines according to the statistical characteristics of the saliency edge, sorting and labeling all the power lines from top to bottom, adjusting the holder angle to move each power line to the horizontal plane of the image center, re-shooting the binocular image, solving the distance between the unmanned aerial vehicle and each power line according to the positions of the laser labeling points on the corresponding power lines in the left and right eyes, adjusting the flight direction of the unmanned aerial vehicle according to the distance between the unmanned aerial vehicle and each power line, and further completing the automatic tracking of the, if the power line target is lost in tracking, the position of the power line target is adjusted according to a preset routing inspection power line area, then the power line is shot again through the angle adjustment of the holder, and the power line is cut apart and tracked again.
The following details the steps of unmanned aerial vehicle-power line ranging using visual saliency and binocular ranging techniques:
1) and calibrating the binocular camera by using a Zhang calibration method. Shooting a plurality of groups of binocular images by using a 7 multiplied by 7 chessboard model, completing camera calibration, and acquiring internal and external parameters of the binocular camera, including focal length, base line distance, rotation matrix, translation matrix and the like.
2) Acquiring binocular images of the power line: after the unmanned aerial vehicle takes off at the designated position, the unmanned aerial vehicle firstly shoots binocular images of the power line tracked and patrolled, transmits the images to the embedded chip, and corrects the binocular images according to the internal and external parameters of the binocular camera.
3) Detecting the power line region by using the visual significance detection and power line characteristic statistical analysis results:
since the edges extracted by the Canny algorithm are all continuous edges in the image, all the edges are not only edges of the power lines, and a certain number of significant points on the edges are ensured according to the relevant principle of significance, which is the first screening in the process of constructing the power lines.
(31) As shown in fig. 5(b), Gaussian Difference saliency map points of the left and right eye images are respectively extracted, Difference of Gaussian (DoG) means that feature detection on a certain scale can obtain a response value image of DoG by subtracting two adjacent Gaussian scale space images, so as to achieve the effects of exciting a local central region and suppressing surrounding neighborhoods, and conform to human visual characteristics, thereby reflecting the saliency of the images to a certain extent, and the initial Gaussian Difference saliency map points of the left and right eye images are calculated according to the following formula:
wherein σ1And σ2Representing excitation and suppression bandwidth, respectively, by a value σ1＝0.7，σ20.9, I is a grayscale image, symbol
Representing that sliding frequency filtering is carried out on the image, wherein DoG (x, y) is an obtained significance metric value, and x and y are horizontal and vertical coordinates of pixel points in an image coordinate system;
measure of significance DoG (x, y) is set to 0 and the mean of the significance measure is set to the threshold T:
wherein, count (DoG > 0) represents the number of significant points with the significance metric value larger than 0 in DoG, sum (DoG > 0) represents the sum of the significance metric values larger than 0;
finally, a saliency map D (x, y) is calculated:
(32) as shown in fig. 5(c), all the continuous edges of the image are detected by using the Canny edge detection algorithm, where the edges are not only the edges of the power lines, but also various kinds of edge information of the background. Because the background of the unmanned aerial vehicle for shooting the image of the power line is relatively complex, under the condition, the Canny detection result has a plurality of short edges caused by factors such as shadow, reflection and the like, the significance of the power line cannot be highlighted, the extraction of the target edge is not facilitated, in order to ensure the accuracy of the extraction of the power line in the image, the significant points of the DoG are utilized to restrain the Canny edge, the significant edge of the target is extracted to serve as a candidate element of the power line, and the effect is as shown in fig. 5 (d);
(33) processing a result obtained by a Canny edge detection algorithm, wherein the specific process comprises the steps of firstly, in order to remove false detection boundaries, extracting edge information, screening the length of edges, and deleting edges with the length smaller than the set pixel length in an image, such as 10 pixels; then sorting the edge significance metric values in a descending order according to the number of the Gaussian difference significant points on each effective boundary, and reserving the edges arranged in the front and provided with proportion values as candidate elements of the power line;
4) calculating the slopes of all the candidate elements for all the extracted power line candidate elements, wherein the power lines in the binocular picture should be nearly horizontal as the unmanned aerial vehicle flies in a direction nearly parallel to the power lines, so that the power line candidate elements with the inclination angles smaller than the set degrees, such as the power line candidate elements with the inclination angles smaller than 15 degrees, are reserved.
Judging whether two candidate elements of the power line belong to the same power line in the original image by calculating the lateral distance between every two candidate elements of the power line, wherein the calculation method of the lateral distance comprises the following steps: suppose the left and right endpoints of the candidate primitive for powerline L1 are A, B and the equation is A1x+B1y+C10, the endpoint of the candidate power line primitive L2 is C, D, and equation is a2x+B2y+C20 and A, B, C, D with the coordinate of four points as A (a)1,b1)、B(a2,b2)、C(a3,b3)、D(a4,b4)，A1，B1，C1，A2，B2，C2Is a constant number of times, and is,
according to the two-point equation of the straight line, the straight line segments L1 and L2 corresponding to the candidate primitive of the power line can be expressed as:
two-point conversion to the general formula:
according to the distance formula from the point to the straight line, the distance from the midpoint of the line segment to the line segment 2 is D1：
Calculating the distance from the midpoint of each power line candidate element to the straight line of all other power line candidate elements in the main direction, namely the lateral distance, by using the formula, and then sequencing all the obtained lateral distances; when the straight line distance between a certain candidate element of the power line and another candidate element of the power line is less than a certain threshold (the threshold is 2mm), the two candidate elements of the power line are considered to belong to the same power line and can be grouped; counting the candidate power line primitives with the lateral distances larger than 50 pixels in the image, and if the distances between the straight line where one candidate power line primitive is located and the straight line formed by other candidate power line primitives are larger than 50 pixels, considering the candidate power line primitives as non-power line primitives with the same main direction, and removing the non-power line primitives. In each group of power line elements, sequentially connecting the horizontal coordinates of the left end point and the right end point of each power line element from left to right, and acquiring all coordinates covered by one complete power line, namely the position of each power line; finally, labeling each power line of the image from top to bottom;
5) because most of the power lines are single in color, the laser mark points are obvious red points on the power lines, the positions of the power lines extracted according to the step 4) are found, and the positions are not described in the step 4), continuous red pixel points in the power line area are extracted, and the middle points of the pixel points are calculated, wherein the middle points are mark points of the laser of one line on the power line. All power lines are processed in sequence, and the distance between the unmanned aerial vehicle and each power line can be obtained. The position of the power line is all coordinates passed by the power line which can be solved after the head and the tail of the power line are reconnected, and the position is the accurate position of the power line.
Calculating three-dimensional coordinates of two point pairs according to binocular parallax of two infrared laser marking points in a binocular image and camera parameters, obtaining the distance from the unmanned aerial vehicle to each power line, selecting the base line distance between known binocular vision cameras as b, the focal length of the cameras as f, expressing the binocular parallax by d, assuming that left and right eye images are registered, determining the parallax by the points marked by laser, and expressing the position difference of the same laser marking point as d ═ x (x is the position difference of the same laser marking point)l-xr) Wherein x isl、xrRespectively, calculating the distance between the unmanned aerial vehicle and a marked power line by the abscissa of the laser marking point in the left and right eye images, namely the depth of an infrared laser marking point P in a left camera coordinate system:
the marker distance measurement is shown in fig. 6.
6) Finally, the flight direction and the flight attitude of the unmanned aerial vehicle are adjusted according to the distance between the unmanned aerial vehicle and the power lines, and if the distance between the current unmanned aerial vehicle and each power line is greater than the minimum distance threshold ThreshminAnd the distance is less than the maximum distance threshold ThreshmaxThe current direction is proved to be correct, and the flying vehicle continues to fly forwards; if the distance is less than the minimum distance threshold ThreshminControlling the unmanned aerial vehicle to fly obliquely in the opposite direction of the power line; if the unmanned plane reaches the set maximum distance threshold ThreshmaxAnd controlling the unmanned aerial vehicle to fly obliquely towards the direction close to the power line. The environments such as wind power and the like in different places are different, and the flying stability of different unmanned aerial vehicles in the air is not determined, so that the minimum distance threshold Thresh of the unmanned aerial vehicle relative to the power transmission lineminAnd a maximum distance threshold ThreshmaxAll set by the staff by oneself to satisfy different service environment, guarantee the quality of shooting again on the basis of guaranteeing safety.
The invention completes the automatic tracking of the unmanned aerial vehicle on the power transmission line, accurately calculates the distance between the unmanned aerial vehicle and the laser mark point on the power line, ensures the safe tracking of the unmanned aerial vehicle on the power transmission line, further ensures the stability of the shooting picture of the unmanned aerial vehicle, and has the characteristics of high polling efficiency, high universality, high safety and the like.
Claims (7)
1. An unmanned aerial vehicle power transmission line automatic tracking method based on binocular vision is characterized by comprising the following steps:
1) calibrating a binocular camera: completing calibration of a binocular camera by using a Zhang calibration method, and acquiring internal and external parameters of the binocular camera;
2) acquiring binocular images to be detected: the line laser emitter emits red line laser, the line laser is dispersed and refracted by using the cylindrical lens to form a fan-shaped light beam, and a line irradiated by the line laser emitter is kept vertical and is irradiated to each power line; after the unmanned aerial vehicle takes off at a designated position, shooting binocular images of a power line tracked and inspected, transmitting the images to an embedded chip, and correcting the images according to internal and external parameters of a binocular camera;
3) detecting a power line region: for the left and right eye images obtained in the step 2), firstly, calculating salient points by using a Gaussian difference method, then, carrying out edge detection by using a Canny algorithm, and screening according to the number of the salient points on the edges to obtain candidate elements of the power line;
4) detecting power lines and labeling: calculating the slope of each power line candidate element, reserving the power line candidate elements with the slope dip angle smaller than a set value, further grouping a plurality of broken candidate power line elements according to the distance from the midpoint of each power line candidate element to the straight line of all other power line candidate elements, grouping the power line candidate elements belonging to the same power line into one group, splicing all the power line candidate elements in the same group into a complete power line, and simultaneously labeling each power line in the sequence from top to bottom;
5) extracting laser marking points on all power lines: searching and extracting the positions of the left and right eye image laser marking points of each power line according to the marks for all the power lines extracted in the step 4) according to the pixel colors;
6) unmanned aerial vehicle-power line distance calculation: calculating three-dimensional coordinates of two point pairs according to binocular parallax of two infrared laser marking points in a binocular image and camera parameters, obtaining the distance from the unmanned aerial vehicle to each power line, selecting the base line distance between known binocular vision cameras as b, the focal length of the cameras as f, expressing the binocular parallax by d, assuming that left and right eye images are registered, determining the parallax by the points marked by laser, and expressing the position difference of the same laser marking point as d ═ x (x is the position difference of the same laser marking point)l-xr) Wherein x isl、xrRespectively, calculating the distance between the unmanned aerial vehicle and a marked power line by the abscissa of the laser marking point in the left and right eye images, namely the depth of an infrared laser marking point P in a left camera coordinate system:
7) adjusting the distance between the unmanned aerial vehicle and the power line according to the obtained distanceThe flight direction of the unmanned aerial vehicle is determined if the distance between the unmanned aerial vehicle and each power line is greater than the minimum distance threshold ThreshminAnd is less than a maximum distance threshold ThreshmaxContinuing to fly forwards; if the distance between the unmanned aerial vehicle and each power line is less than the minimum distance threshold ThreshminControlling the unmanned aerial vehicle to obliquely fly towards the opposite direction of the power line; if the distance between the unmanned aerial vehicle and each power line is greater than the set maximum distance threshold ThreshmaxAnd controlling the unmanned aerial vehicle to obliquely fly towards the direction close to the power line, and transmitting the extracted power line position, the corresponding GPS information and the video recorded by the binocular camera back to the ground through the wireless transmission module.
2. The binocular vision based unmanned aerial vehicle power transmission line automatic tracking method of claim 1,
in the step 3), calculating an initial gaussian difference salient point of the left-eye image and the right-eye image according to the following formula:
wherein σ1And σ2Respectively representing excitation and suppression bandwidths, I being a grey-scale image, symbol
Representing that sliding frequency filtering is carried out on the image, wherein DoG (x, y) is an obtained significance metric value, x and y are horizontal and vertical coordinates of a pixel point under an image coordinate system, and e is a mathematical constant;
the negative value generated by the significance metric DoG (x, y) is set to 0, and the significance metric mean is set to the threshold T:
wherein, count (DoG > 0) represents the number of significant points with the significance metric value larger than 0 in DoG, sum (DoG > 0) represents the sum of the significance metric values larger than 0;
finally, a saliency map D (x, y) is calculated:
3. the binocular vision-based unmanned aerial vehicle power transmission line automatic tracking method of claim 1, wherein: in the step 3), a specific method for screening according to the number of salient points on the edge is as follows:
after edge detection is carried out by using a Canny algorithm, the length of the edge is screened, and the edge with the length smaller than the set pixel length in the image is deleted; and then sorting the edge significance metric values in a descending order according to the number of the Gaussian difference significant points on each effective boundary, and reserving the edges arranged in the front and provided with proportion values as candidate elements of the power line.
4. The binocular vision-based unmanned aerial vehicle power transmission line automatic tracking method of claim 1, wherein: in the step 4), the distance D from the midpoint of each power line candidate primitive to the straight line of all other power line candidate primitives is calculated by the following formula1I.e., the lateral distance,
wherein, let the left and right endpoints of the candidate primitive L1 for the power line be A, B, and the equation be A1x+B1y+C10, the endpoint of the candidate power line primitive L2 is C, D, and equation is a2x+B2y+C20 and A, B, C, D with the coordinate of four points as A (a)1,b1)、B(a2,b2)、C(a3,b3)、D(a4,b4)，A1，B1，C1，A2，B2，C2Are all constants.
5. The binocular vision based unmanned aerial vehicle power transmission line automatic tracking method according to claim 4, wherein in the step 4), all obtained lateral distances are sorted, and when a straight line distance between a certain power line candidate primitive and another power line candidate primitive is smaller than a certain threshold value, the two power line candidate primitives are considered to belong to the same power line and are grouped together.
6. The binocular vision based unmanned aerial vehicle power transmission line automatic tracking method of claim 5, wherein the threshold is 2 millimeters.
7. The binocular vision based unmanned aerial vehicle power transmission line automatic tracking method of claim 5, wherein in the step 4), in each group of power line elements, the abscissa of the left end point and the abscissa of the right end point of each power line element are sequentially connected end to end from left to right, and all coordinates covered by one complete power line, namely the position of each power line, are obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810978744.9A CN108919838B (en) | 2018-08-27 | 2018-08-27 | An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810978744.9A CN108919838B (en) | 2018-08-27 | 2018-08-27 | An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108919838A CN108919838A (en) | 2018-11-30 |
| CN108919838B true CN108919838B (en) | 2021-03-26 |
Family
ID=64406725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810978744.9A Active CN108919838B (en) | 2018-08-27 | 2018-08-27 | An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108919838B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109683629B (en) * | 2019-01-09 | 2020-08-21 | 燕山大学 | Unmanned aerial vehicle electric power overhead line system based on combination navigation and computer vision |
| CN110120074B (en) * | 2019-05-10 | 2020-08-25 | 清研同创机器人（天津）有限公司 | Cable positioning method for live working robot in complex environment |
| CN110132226B (en) * | 2019-05-14 | 2021-05-28 | 广东电网有限责任公司 | System and method for measuring distance and azimuth angle of unmanned aerial vehicle line patrol |
| CN110245701A (en) * | 2019-06-11 | 2019-09-17 | 云南电网有限责任公司曲靖供电局 | A kind of electric power line detecting method based on unmanned plane image |
| CN110119161B (en) * | 2019-06-14 | 2022-03-01 | 河海大学常州校区 | Unmanned aerial vehicle power transmission line inspection visual obstacle avoidance method and system |
| CN110599513B (en) * | 2019-09-04 | 2022-02-11 | 南京邮电大学 | Binocular vision image edge detection and target tracking method |
| CN111626095B (en) * | 2020-04-06 | 2023-07-28 | 连云港市港圣开关制造有限公司 | Power distribution inspection system based on Ethernet |
| CN111814547A (en) * | 2020-06-03 | 2020-10-23 | 国网浙江嘉善县供电有限公司 | A system and method for tracking and detecting overhead high-voltage conductors based on visual recognition |
| CN112493228B (en) * | 2020-10-28 | 2021-12-14 | 河海大学 | Laser bird repelling method and system based on three-dimensional information estimation |
| CN112649003B (en) * | 2020-12-11 | 2024-03-12 | 国网辽宁省电力有限公司沈阳供电公司 | A UAV altitude-changing circle method based on combined navigation and image information |
| CN112815911B (en) * | 2020-12-30 | 2022-01-04 | 河海大学 | Measurement method of transmission line crossing distance based on trinocular vision |
| CN113219447B (en) * | 2021-04-09 | 2024-07-16 | 国电南瑞科技股份有限公司 | Stable distance measurement method for power transmission line based on millimeter wave array |
| CN113358033B (en) * | 2021-05-25 | 2022-09-16 | 晋能控股煤业集团有限公司 | Method for judging safe distance of trees under power transmission line based on visual analysis soft measurement |
| CN114778560B (en) | 2022-06-20 | 2022-09-27 | 国网江苏省电力有限公司常州供电分公司 | Overhead line detection method and system based on cable inspection robot |
| CN115290038B (en) * | 2022-09-29 | 2022-12-30 | 中科南京人工智能创新研究院 | Binocular distance measurement method for linear barrier |
| CN115629619A (en) * | 2022-11-04 | 2023-01-20 | 广东电网有限责任公司 | Unmanned aerial vehicle power transmission line inspection system and method |
| CN116012728A (en) * | 2023-02-07 | 2023-04-25 | 北京交通大学 | Track line identification tracking method and system applied to unmanned aerial vehicle inspection |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393961A (en) * | 2011-06-29 | 2012-03-28 | 山东电力研究院 | Computer vision-based real-time detecting and tracking method for electric power transmission circuit of aircraft |
| CN103090807A (en) * | 2013-01-11 | 2013-05-08 | 北京国网富达科技发展有限责任公司 | Method and system for detecting safe operations of electric transmission line |
| CN104091168A (en) * | 2014-07-18 | 2014-10-08 | 深圳供电局有限公司 | Power line automatic extraction and positioning method based on unmanned aerial vehicle image |
| CN204679793U (en) * | 2015-05-26 | 2015-09-30 | 河海大学 | A kind of full-automatic remote distance power circuit unmanned plane inspection tour system |
| EP3062123A1 (en) * | 2015-02-27 | 2016-08-31 | GE Aviation Systems LLC | System and methods of detecting an intruding object in a relative navigation system |
| CN106340009A (en) * | 2016-08-18 | 2017-01-18 | 河海大学常州校区 | Parallel-binocular-based power line detection method and system |
| CN106356757A (en) * | 2016-08-11 | 2017-01-25 | 河海大学常州校区 | Method for inspecting electric power lines by aid of unmanned aerial vehicle on basis of human vision characteristics |
| CN106595499A (en) * | 2016-11-21 | 2017-04-26 | 云南电网有限责任公司电力科学研究院 | Wire icing thickness measuring image processing system |
| CN106682579A (en) * | 2016-11-21 | 2017-05-17 | 云南电网有限责任公司电力科学研究院 | Unmanned aerial vehicle binocular vision image processing system for detecting power transmission line icing |
| CN107462217A (en) * | 2017-07-07 | 2017-12-12 | 北京航空航天大学 | Unmanned aerial vehicle binocular vision barrier sensing method for power inspection task |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
| WO2018056498A1 (en) * | 2016-09-21 | 2018-03-29 | 한국전력공사 | Transmission line electromagnetic field and instantaneous inspection image acquisition device and method |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6940994B2 (en) * | 2001-03-09 | 2005-09-06 | The Boeing Company | Passive power line detection system for aircraft |
| US7543780B1 (en) * | 2004-10-04 | 2009-06-09 | The United States Of America As Represented By The Secretary Of The Air Force | Unmanned air vehicle transmission line docking surveillance |
| CN103353297A (en) * | 2013-06-03 | 2013-10-16 | 长春理工大学 | Airborne photoelectric measurement apparatus of dimensions and spacing of electric transmission line and target, and method thereof |
| US20150130840A1 (en) * | 2013-11-08 | 2015-05-14 | Sharper Shape Ltd. | System and method for reporting events |
| CN103812052B (en) * | 2014-03-07 | 2016-06-01 | 国家电网公司 | A kind of for without the centralized monitoring system of man-machine polling transmission line and monitoring method |
| EP3251193A4 (en) * | 2015-01-28 | 2018-08-08 | Lockheed Martin Corporation | In-situ power charging |
| US20160229533A1 (en) * | 2015-02-06 | 2016-08-11 | Izak Jan van Cruyningen | Efficient Flight Paths for Aerial Corridor Inspection |
| WO2016140985A1 (en) * | 2015-03-02 | 2016-09-09 | Izak Van Cruyningen | Flight planning for unmanned aerial tower inspection |
| US11029352B2 (en) * | 2016-05-18 | 2021-06-08 | Skydio, Inc. | Unmanned aerial vehicle electromagnetic avoidance and utilization system |
-
2018
- 2018-08-27 CN CN201810978744.9A patent/CN108919838B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393961A (en) * | 2011-06-29 | 2012-03-28 | 山东电力研究院 | Computer vision-based real-time detecting and tracking method for electric power transmission circuit of aircraft |
| CN103090807A (en) * | 2013-01-11 | 2013-05-08 | 北京国网富达科技发展有限责任公司 | Method and system for detecting safe operations of electric transmission line |
| CN104091168A (en) * | 2014-07-18 | 2014-10-08 | 深圳供电局有限公司 | Power line automatic extraction and positioning method based on unmanned aerial vehicle image |
| EP3062123A1 (en) * | 2015-02-27 | 2016-08-31 | GE Aviation Systems LLC | System and methods of detecting an intruding object in a relative navigation system |
| CN204679793U (en) * | 2015-05-26 | 2015-09-30 | 河海大学 | A kind of full-automatic remote distance power circuit unmanned plane inspection tour system |
| CN106356757A (en) * | 2016-08-11 | 2017-01-25 | 河海大学常州校区 | Method for inspecting electric power lines by aid of unmanned aerial vehicle on basis of human vision characteristics |
| CN106340009A (en) * | 2016-08-18 | 2017-01-18 | 河海大学常州校区 | Parallel-binocular-based power line detection method and system |
| WO2018056498A1 (en) * | 2016-09-21 | 2018-03-29 | 한국전력공사 | Transmission line electromagnetic field and instantaneous inspection image acquisition device and method |
| CN106595499A (en) * | 2016-11-21 | 2017-04-26 | 云南电网有限责任公司电力科学研究院 | Wire icing thickness measuring image processing system |
| CN106682579A (en) * | 2016-11-21 | 2017-05-17 | 云南电网有限责任公司电力科学研究院 | Unmanned aerial vehicle binocular vision image processing system for detecting power transmission line icing |
| CN107462217A (en) * | 2017-07-07 | 2017-12-12 | 北京航空航天大学 | Unmanned aerial vehicle binocular vision barrier sensing method for power inspection task |
| CN107729808A (en) * | 2017-09-08 | 2018-02-23 | 国网山东省电力公司电力科学研究院 | A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection |
Non-Patent Citations (5)
| Title |
|---|
| Power Lines Extraction and Distance Measurement from Binocular Aerial Images for Power Lines Inspection using UAV;Chen Shuai 等;《2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics》;20171231;69-74 * |
| Study on Algorithms of Power Line Extraction Based on Power Line Image Feature;Wen-Yu WANG 等;《3rd Annual International Conference on Electronics, Electrical Engineering and Information Science (EEEIS 2017)》;20171231;240-244 * |
| 基于无人机航拍输电线路杆塔的三维重建;郑恩;《中国优秀硕士学位论文全文数据库信息科技辑》;20180115(第01期);I138-1369 * |
| 无人机电力巡检视觉避障技术研究;周士超;《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》;20160815(第08期);C042-640 * |
| 无人机航拍图像的输电线识别方法;孙实超;《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》;20170215(第02期);C042-2269 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108919838A (en) | 2018-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108919838B (en) | An automatic tracking method for unmanned aerial vehicle transmission line based on binocular vision | |
| CN110147706B (en) | Obstacle recognition method and device, storage medium, and electronic device | |
| CN104197928B (en) | Multi-camera collaboration-based method for detecting, positioning and tracking unmanned aerial vehicle | |
| CN111436216B (en) | Method and system for color point cloud generation | |
| CN109765930B (en) | Unmanned aerial vehicle vision navigation | |
| CN109060281B (en) | Integrated bridge detection system based on unmanned aerial vehicle | |
| CN104215239B (en) | Guidance method using vision-based autonomous unmanned plane landing guidance device | |
| CN112017251B (en) | Calibration method and device, road side equipment and computer readable storage medium | |
| CN113298035B (en) | UAV power tower detection and autonomous cruising method based on image recognition | |
| CN110988912A (en) | Road target and distance detection method, system and device for automatic driving vehicle | |
| CN110850723A (en) | Fault diagnosis and positioning method based on transformer substation inspection robot system | |
| CN109934230A (en) | A Radar Point Cloud Segmentation Method Based on Visual Aid | |
| WO2020090428A1 (en) | Geographic object detection device, geographic object detection method, and geographic object detection program | |
| CN106019264A (en) | Binocular vision based UAV (Unmanned Aerial Vehicle) danger vehicle distance identifying system and method | |
| CN109063532A (en) | A kind of field lost contact personnel's method for searching based on unmanned plane | |
| CN106289186A (en) | The airborne visual detection of rotor wing unmanned aerial vehicle and multi-target positioning system and implementation method | |
| CN114972177B (en) | Road hazard identification and management method, device and intelligent terminal | |
| CN114034296A (en) | Navigation signal interference source detection and identification method and system | |
| CN108564787A (en) | Traffic observation procedure, system and equipment based on Floating Car method | |
| CN118015377B (en) | A road vehicle trajectory extraction method based on multi-UAV images | |
| CN109816697A (en) | A system and method for building a map for an unmanned model vehicle | |
| CN115131360A (en) | Road pit detection method and device based on 3D point cloud semantic segmentation | |
| CN115493594A (en) | Patrol control system and method based on BIM | |
| CN112116637A (en) | Automatic detection method and system of power tower based on UAV 3D laser scanning technology | |
| CN110058263A (en) | A kind of object positioning method in vehicle travel process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |