Disclosure of Invention
The purpose of the present application is: by adopting the neural network technology, according to the characteristic that the tangent rule of each point changes due to the inherent characteristics of line stress, the three-section line image comprising the tower foundation is used for reasoning the shape of the whole transmission line, the whole transmission line is fitted, the problem that the whole transmission line is difficult to identify is solved, and the image extraction of the transmission line without a spacer (a span) is realized.
The technical scheme of the first aspect of the application is that: the method for fitting the transmission line between the towers comprises the following steps: step 1, extracting convolution center points of a multi-layer convolution neural network layer by layer according to a first set of fitting data in historical data, calculating scale invariant features of a final layer of convolution layers in the multi-layer convolution neural network according to the features of the extracted convolution center points, and generating a tower line fitting neural network; step 2, training a tower line fitting neural network by using a classification function according to a second group of fitting data in the historical data; and step 3, acquiring image data to be fitted of the tower foundation transmission line, and fitting the image data to be fitted by utilizing the trained tower foundation fitting neural network and the input tower foundation data to generate a fitted transmission line.
In any of the above solutions, further, each layer of convolutional neural network includes a classifier and a plurality of layers of convolutional layers, and in
step 1, the method specifically includes: step 11, inputting a first group of fitting data into a multi-layer convolutional neural network, and determining convolutional center point sampling points in the convolutional layer by layer in a random sampling mode; step 12, calculating the isomorphism characteristic of the convolution center point sampling point layer by layer in an iterative manner according to the convolution center point sampling point of the convolution layer, the convolution center point coordinates and the corresponding local point characteristics, and marking the isomorphism characteristic as the characteristic of the convolution center point; step 13, calculating the scale invariant feature of the final layer of convolution layer according to the feature of the final layer of convolution layer, wherein the scale invariant feature
The calculation formula of (2) is as follows:
wherein, pooling (·) is an average pooling operation,
for the index set of convolution center point sampling points in the final layer of convolution layers, phi (·) is an inner product operation,>
for the combination of the non-normalized eigenvectors corresponding to the c-th convolution center sampling point, s
F Is the largest dimension of the local feature;
step 14, the scale-invariant features are obtained
And (3) inputting the data into a classifier, and generating a tower line fitting neural network by combining a vertical line fitting calculation equation.
In any of the above technical solutions, further, in step 2, the calculation formula of the classification function is:
wherein D is j (x i ) For the ith image data x i Classified output value of jth tower base data, D yi (x i ) For the ith image data x i Belonging to line data y i B is the total number of tower-based data.
In any of the above technical solutions, further, after acquiring the image data to be fitted of the tower foundation transmission line, step 3 specifically includes: step 31, carrying out gray processing on the acquired image data to be fitted, and generating a plurality of small view windows according to a preset window width; step 32, determining a binarization threshold value corresponding to the view window according to the pixel points contained in the view window and the pixel points of all rows in the image data to be fitted corresponding to the view window; and step 33, carrying out binarization processing on pixel points in the image data to be fitted according to the binarization threshold value, and carrying out power transmission line fitting according to the binarized image data to be fitted.
The technical scheme of the second aspect of the application is that: the utility model provides a transmission line detection device between tower base, this device includes: the system comprises an image acquisition unit, an image processing unit and a fault judging unit; the image acquisition unit is used for acquiring image data and sending the acquired image data to the image processing unit; the image processing unit generates a fitting transmission line according to the image data by using the method for fitting transmission lines between foundation according to any one of the technical schemes of the first aspect; and the fault judging unit is used for generating power transmission line alarm information when judging that the fitted power transmission line exceeds a line threshold curve.
In any one of the above solutions, further, the apparatus further includes: a data transmission unit; the data transmission unit is used for uploading the image data, the fitting transmission line and the transmission line alarm information to a data storage device.
The beneficial effects of this application are:
according to the method, the tower line fitting neural network is generated and trained through historical data, the trained tower line fitting neural network is used for carrying out fitting processing on acquired image data to be fitted of the tower foundation power transmission line, line data corresponding to the power transmission line due to stress are determined, the line data comprise vertical line vertex coordinates and load ratios, further, a fitting power transmission line is generated according to a tower foundation data and vertical line fitting calculation equation, power transmission line image extraction without a spacer (span) is achieved, power transmission line fitting efficiency and reliability are improved, and further, fault of the power transmission line is checked fast.
According to the method and the device, the acquired image data to be fitted are subjected to binarization processing, the influence of illumination on the image data is reduced, the value of the binarization threshold is determined between the first gray threshold and the third gray threshold by calculating the gray value variance of the pixel points in the view window, the accuracy of the value of the binarization threshold is improved, and meanwhile the problem of binarization probability errors of the same-color area caused by the fact that a plurality of continuous windows are located in the same-color area can be solved.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced otherwise than as described herein, and thus the scope of the present application is not limited to the specific embodiments disclosed below.
Embodiment one:
an embodiment one will be described below with reference to fig. 1 and 2.
As shown in fig. 1, this embodiment provides a method for fitting a transmission line between bases, including:
step 1, extracting convolution center points of a multi-layer convolution neural network layer by layer according to a first set of fitting data in historical data, calculating scale invariant features of a final layer of convolution layers in the multi-layer convolution neural network according to the features of the extracted convolution center points, and generating a tower line fitting neural network;
specifically, historical data of the power transmission line between the towers is randomly divided into two sets of fitting data, wherein the first set of fitting data is used for generating a tower line fitting neural network, and the second set of fitting data is used for training the tower line fitting neural network so as to improve the accuracy and reliability of power transmission line fitting of the tower line fitting neural network, and the historical data comprises tower base data (such as tower base coordinates and tower base heights), image data of the power transmission line between the towers and corresponding line data (such as vertical line vertex coordinates and historical load-weight ratios).
Further, in this embodiment, each layer of convolutional neural network includes a classifier and multiple layers of convolutional layers, and step 1 specifically includes:
in step 11,inputting the first group of fitting data into a multi-layer convolutional neural network, and determining convolutional center point sampling points in the convolutional layer by adopting a random sampling mode
Specifically, let the index set of the convolution center point sampling points of the first layer be:
wherein n (l) is the number of convolution center point sampling points in the first convolution layer, which can be set manually according to the requirement,
the c-th convolution center point sampling point in the first convolution layer.
In this embodiment, the initial value of the convolution center point sampling point is set as follows:
in the formula, n represents the number of pixel points in the input first set of fitting data, namely, represents the whole pixel points.
Step 12, sampling points according to the convolution center points of the convolution layers
Convolved center point coordinates +.>
And corresponding local point features, iterative calculation of convolution center point sampling points layer by layer>
Is->
Features noted as convolution center points;
specifically, for each convolution layer, a kNN proximity algorithm is adopted, and a convolution center point sampling point is used as a local point, wherein each local point has local point characteristics, namely a characteristic vector
Further, according to the kNN proximity algorithm, local features within a certain range of the local point can be determined (here, the prior art is not described in detail). An iterative algorithm is adopted among the multiple layers of convolution layers of the convolution neural network, namely, the output of the first layer-1 convolution layer is the input of the first layer of convolution layer, and the input of the first layer of convolution layer is set as a set of characteristics and corresponding points ∈ ->
In the method, in the process of the invention,
for the coordinate corresponding to the c-th convolution center, < >>
Is the feature vector corresponding to the c-th convolution center.
For the first layer convolution layer, the constant characteristics
For empty, these edge features->
When the vector splicing operation in the following calculation is participated, ambiguity can be generated, therefore, any vector is specified to splice an empty vector, and the spliced result is kept unchanged.
Further, an alike feature
The calculation formula of (2) is as follows:
where c is the index of the sampling point,
l is the number of layers of the convolution layer, l=1, 2, …, L is the number of the last convolution layer, s
F For the largest scale of the local feature, Φ (·) is the inner product operation of the input and convolution kernel,
Representing the corresponding non-return of the c-th convolution center as an intermediate calculation variableNormalized eigenvector combinations (matrix form), s
P For maximum scale of local points, MLP
δ (. Cndot.) is a multi-layer perceptron, (. Cndot.)>
Relative coordinate matrix of local points, < >>
The vector stitching operation is characterized in that,
for a feature vector matrix corresponding to a local point, k is the number of local points (i.e. the number of k nearest samples in kNN's neighbor algorithm), i is used as an index of local points, i=1, …, k,
For the combination of the relative coordinates of all points in the kNN part corresponding to the c-th convolution center,/->
The coordinates corresponding to the c-th convolution center.
The maximum dimension s of the local feature F And the maximum dimension s of the local point P Defined as the maximum of the two norms, which characterizes the physical properties of the first set of fitting data, should not participate in the process of iterative optimization of the convolutional neural network.
For vector concatenation operations
If vector a= (a)
1 ,…,a
θ ),b=(b
1 ,…,b
β ) Then there is
The input set of the last convolution layer can be calculated by the calculation formula
In this embodiment, the index set of the convolution center point sampling points of the final layer of convolution layer is set as
The operation of the last layer of convolution layer is very similar to the previous layers of convolution layer operation, except that the largest dimension s of the local feature is not made
F Is multiplied by (a) back to (b). The purpose of this is to obtain a feature that is "invariant" to the input scale, rather than a feature that is "alike". This is in view of the output constant characteristics of the last layer of convolution layer +.>
After direct pooling, the output result of the convolutional neural network is obtained by a classifier, thus, the characteristic of the convolutional neural network is changed uniformly>
The relative scale information between them is not important, but its invariance is significant for the correct output of the classifier.
Step 13, according to the characteristics of the last convolution layer
Calculating scale invariant features of the last convolutional layer
Wherein the scale invariant feature->
The calculation formula of (2) is as follows:
where L is the number of the last convolutional layer and pooling (·) is the average pooling operation.
Step 14, the scale-invariant features are obtained
And (3) inputting the data into a classifier, and generating a tower line fitting neural network by combining a vertical line fitting calculation equation.
Specifically, the scale-invariant feature having invariance is obtained in step 13 above
After the convolution characteristic is transmitted into a classifier of the convolution neural network, a neural network which can determine line data according to the input tower foundation data and image data can be obtained, and then a vertical line fitting calculation equation is combined to generate a tower line fitting neural network, so that fitting of the transmission line in the input image data is realized, wherein the vertical line fitting calculation equation is as follows:
f=kx+b-ach(x/a+arch w-l/a)+ach(arch w-l/a)
wherein kx+b is a linear equation between two tower bases, which is determined by tower base data, x is a horizontal distance of fitting points, l is an abscissa of a vertex fitting a vertical line, and a is a weight ratio.
Step 2, training a tower line fitting neural network by using a classification function according to a second group of fitting data in the historical data;
further, in step 2, the calculation formula of the classification function is:
wherein D is j (x i ) For the ith image data x i A classification output value belonging to the jth tower base data,D yi (x i ) For the ith image data x i Belonging to line data y i And B is the total number of tower-based data, and the values of i, j and B are determined by the second group of fitting data.
Specifically, the tower line fitting neural network is denoted as D, and the input image data is denoted as x 1 …x N The corresponding line data is denoted y 1 …y N . Setting the sample number of the second group of fitting data as N and the total number of the corresponding tower base data as B, inputting the second group of fitting data into a tower line fitting neural network D, and defining a classification function of the ith image data through the concept of cross entropy:
where i=1, 2, …, N is determined by the second set of fitting data.
The loss function is then optimized using conventional algorithms, such as a random gradient descent algorithm
I.e. the network parameters (such as convolution kernel parameters, full connection parameters) are adjusted according to the gradient direction update, the value of the loss function is reduced. And iterating the process until the classification accuracy rate converges, so that a tower line fitting neural network with trained parameters is obtained, and corresponding line data is determined according to the input image data and tower base data, so that fitting of the power transmission line in the image data is realized.
And step 3, acquiring image data to be fitted of the tower foundation transmission line, and fitting the image data to be fitted by using the trained tower foundation fitting neural network and the input tower foundation data to generate a fitted transmission line.
Specifically, for any two adjacent tower bases, image data to be fitted of at least three power transmission lines are obtained, wherein the image data to be fitted of the joints of the two ends of the power transmission lines and the tower bases and the image data to be fitted of the power transmission lines between the at least two tower bases are included. The tower base data is known data available to power transmission line maintenance personnel, and the tower base data can be directly input into the trained tower line fitting neural network.
Setting the central position of a first tower foundation as an origin of coordinates, establishing a rectangular coordinate system, utilizing a trained tower line fitting neural network to process acquired image data to be fitted, determining line data corresponding to a power transmission line in the image, such as a load ratio a and vertical vertex coordinates, and fitting the image data to be fitted according to the input tower foundation data.
It should be noted that, because the method in this embodiment is a tower line fitting neural network generated and trained through historical data, accuracy of line data determined by the tower line fitting neural network according to collected image data to be fitted can be effectively ensured, and reliability of fitting a power transmission line is further improved. Meanwhile, the tower line fitting neural network can output multiple groups of line data with high accuracy according to actual demands, and a corresponding fitting transmission line is generated by fitting each group of line data, so that the reliability of transmission line fitting is further improved.
Further, after the image data to be fitted of the tower foundation transmission line is collected, the step 3 specifically includes:
step 31, carrying out gray processing on the acquired image data to be fitted, and generating a plurality of small view windows according to a preset window width;
specifically, the image data to be fitted is subjected to graying treatment, for example, a maximum value method is adopted to adjust the RGB value corresponding to each pixel point in the image data to be fitted, and the corresponding calculation formula is as follows:
F’(B,G,R)=B*0.114+G*0.587+R*0.229,
in the formula, F' (B, G, R) is a gradation value of the pixel after the gradation processing.
Setting the preset window width as 16 and the window height as 1, namely selecting pixel points in row pixels row by using the view windows, and in order to equally divide the image data to be fitted after the graying treatment according to the column number n=16, inserting redundant pixel points into the image data to be fitted, wherein the pixel value can be set by itself, so that the image data to be fitted can be divided into an integer number of view windows by taking the row pixel points as a unit.
Step 32, determining a binarization threshold value corresponding to the view window according to the pixel points contained in the view window and the pixel points of all rows in the image data to be fitted corresponding to the view window;
in this embodiment, a method for determining a binarization threshold is shown, which includes:
step 321, calculating a first gray average value of pixel points contained in the view window, and calculating a first gray threshold value of the view window according to a preset correction coefficient and the first gray average value;
specifically, according to the line pixel points, temporarily storing the pixel points in the image data to be fitted, sequentially selecting the generated view windows, counting the sum of the gray values of the pixel points in each view window, calculating the corresponding first gray average value according to the number of the pixel points in the view window, and recording as gMean win Integrating the gray value with a preset correction coefficient theta and recording the gray value as a first gray threshold TH corresponding to a current view window 1 The corresponding calculation formula is:
in the formula g i For the gray value of the pixel point of the ith pixel point in the current view window, the preset correction coefficient θ may be set to 0.87, where N is the total number of pixel points in the view window, and n=16.
Step 322, calculating a second gray level average value of all the row pixel points in the image data to be fitted corresponding to the view window, and calculating a second gray level threshold value of the row pixel points according to the preset correction coefficient and the second gray level average value;
specifically, according to the pixel points of the rows, temporarily storing the pixel points in the image data to be fitted, counting the sum of the pixel gray values of all the pixel points in the current row of the pixel points row by row, calculating a second gray average value according to the number of the pixel points of the rows, and recording as gMean line Will beIt is integrated with the preset correction coefficient theta and is recorded as the second gray level mean value TH corresponding to the current row pixel point 2 The corresponding calculation formula is:
in the formula g j For the pixel gray value of the j-th pixel point in the current row of pixel points, the value of the preset correction coefficient theta can be set to be 0.87, and M is the total number of pixel points in the row of pixel points.
Step 323, calculating a third gray threshold according to the first gray threshold and the second gray threshold by using a weighting algorithm;
specifically, a first weight w is set 1 The value is 9, and the second weight value w 2 According to a weighted calculation formula, a first weight, a second weight, a first gray threshold and a second gray threshold, a third gray threshold is calculated, and the weighted calculation formula is as follows:
in TH crct Is the third gray threshold, w 1 As the first weight, TH 1 Is a first gray threshold, w 2 As the second weight, TH 2 Is the second gray level threshold.
Step 324, according to the gray value variance of the pixel points in the view window, determining a binarization threshold, wherein the binarization threshold is one of a first gray threshold and a third gray threshold;
in particular, in the conventional binarization process, a same-color region with extremely small pixel gray level difference is usually encountered, wherein the definition of the same-color region is that the pixel gray level in the region is distributed near the region gray level mean and obeys sigma 2 Extremely small normal distribution N (mu, sigma) 2 ) Is a region of (a) in the above-mentioned region(s). The same color region is most likely to be an ideal region with equal gray values, but there is a possibility that the gray values after photographing and sampling are different, and black-white phase is generated after binarization, so in the present inventionThe application introduces a preset variance threshold, the size of which can be adjusted according to an empirical value, calculates the gray value variances of all pixel points in a view window, and determines a binarization threshold according to the size relation between the two.
Taking the view window as a unit, calculating gray value variance S of all pixel points in the view window
2 Setting a preset variance threshold
According to the gray value variance S
2 And a preset variance threshold->
The magnitude relation between the two is used for determining a binarization threshold value TH, and the corresponding calculation formula is as follows:
and step 33, carrying out binarization processing on pixel points in the image data to be fitted according to the binarization threshold value, and generating a fitting power transmission line according to the binarized image data to be fitted and the trained tower line fitting neural network.
Specifically, when binarization is performed, pixel gray values g of pixel points in image data to be fitted are compared point by point x The magnitude of the binarization threshold TH corresponding to the view window, when the pixel gray value of the pixel point is determined to be smaller than the binarization threshold, the pixel gray value of the pixel point is set to 0, and when the pixel gray value of the pixel point is determined to be larger than or equal to the binarization threshold, the pixel gray value of the pixel point is set to 255, namely, the corresponding binarization formula is:
in the formula, g' x The pixel gray value of the pixel point after the binarization processing is used.
By the power transmission line fitting method in the embodiment, the generated fitting power transmission line is shown in fig. 2, and the fitting process of the power transmission line can be completed on the premise of no span. In order to improve accuracy of the final fitting line, the number of the fitting transmission lines is set to be three in the embodiment.
Embodiment two:
the embodiment provides a transmission line detection device between tower footing, and the device includes: the system comprises an image acquisition unit, an image processing unit and a fault judging unit; the image acquisition unit is used for acquiring image data and transmitting the acquired image data to the image processing unit; the image processing unit generates a fitting transmission line according to the image data by using the method for fitting the transmission line between the towers according to any of the above embodiments; the fault judging unit is used for generating power transmission line alarm information when judging that the fitted power transmission line exceeds a line threshold curve.
Specifically, when at least three images of the power transmission line to be fitted are acquired by the image acquisition unit (such as a camera), the image processing unit processes the acquired images, and the specific processing process is as described in the first embodiment, which is not repeated here, after the fitted power transmission line is generated, the fault judgment unit judges the generated profile of the fitted power transmission line according to the preset line threshold curve, when the profile of the fitted power transmission line exceeds the line threshold curve, the radian indicating the sagging of the power transmission line exceeds the warning value, so that the problem of ageing, loosening and even breaking of the power transmission line can be judged, and at the moment, the fault judgment unit generates power transmission line alarm information.
Further, in order to realize the big data management of the state information of the power transmission line, a data storage device is arranged in each work area, and the data storage devices of each work area are commonly connected to a cloud server, so that the data sharing is carried out among each work area, the fault corresponding rate of the power transmission line is improved, and the device further comprises: a data transmission unit; the data transmission unit is used for uploading image data, fitting the power transmission line and alarming information of the power transmission line to the data storage device.
The technical scheme of the application is explained in detail above with reference to the accompanying drawings, and the application provides a method for fitting and detecting a transmission line between towers, wherein the method comprises the following steps: step 1, extracting convolution center points of a multi-layer convolution neural network layer by layer according to a first set of fitting data in historical data, calculating scale invariant features of a final layer of convolution layers in the multi-layer convolution neural network according to the extracted convolution center points, and generating a tower line fitting neural network; step 2, training a tower line fitting neural network by using a classification function according to a second group of fitting data in the historical data; and step 3, acquiring image data to be fitted of the tower foundation transmission line, and fitting the image data to be fitted by using the trained tower foundation fitting neural network and the input tower foundation data to generate a fitted transmission line. According to the technical scheme, the fitting curve of the transmission line is generated by utilizing the neural network fitting, so that the problem that the outline of the whole transmission line is difficult to extract is solved, and the efficiency of transmission line fault detection is improved.
The steps in the present application may be sequentially adjusted, combined, and pruned according to actual requirements.
The units in the device can be combined, divided and pruned according to actual requirements.
Although the present application is disclosed in detail with reference to the accompanying drawings, it is to be understood that such description is merely exemplary and is not intended to limit the application of the present application. The scope of the present application is defined by the appended claims and may include various modifications, alterations, and equivalents to the invention without departing from the scope and spirit of the application.