CN113284144A - Tunnel detection method and device based on unmanned aerial vehicle - Google Patents

Abstract

The invention discloses a tunnel detection method and a tunnel detection device based on an unmanned aerial vehicle, wherein the method comprises the following steps: receiving a video image of a tunnel acquired by an unmanned aerial vehicle; preprocessing and standardizing the video image; and inputting the video image after preprocessing and standardization processing into a trained three-dimensional convolutional neural network (3D-CNN) model, and outputting a tunnel disease detection result by the three-dimensional convolutional neural network model according to the video image. According to the invention, the video image of the tunnel is acquired by the unmanned aerial vehicle, and the video image is input into the trained three-dimensional convolutional neural network model, so that the characteristics of space and time in the video can be better captured, the tunnel disease detection result is rapidly output, the tunnel disease identification rate, the detection frequency and the processing speed are improved, and the artificial dependence degree is reduced.

Description

A method and device for tunnel detection based on unmanned aerial vehicle

technical field

The invention relates to the technical field of tunnel detection, and in particular, to a method and device for tunnel detection based on unmanned aerial vehicles.

Background technique

With the acceleration of urbanization, urban subways have developed rapidly, and more and more underground infrastructures have been built. In recent years, tunnel safety problems have occurred frequently, such as tunnel cracks, water seepage, settlement, lining peeling, and falling blocks, which have caused serious casualties and huge economic losses. Therefore, tunnel safety is a key issue during tunnel operation.

The traditional manual inspection and closed-circuit television system find problems in time, which will lead to serious consequences. Tunnel disease inspection should be carried out by a combination of manual and information-based methods. With the continuous development of unmanned aerial vehicle technology, the application field of unmanned aerial vehicle has become more and more extensive, and it has been used in aerial photography, agriculture, plant protection, express transportation, disaster rescue, surveying and mapping, power inspection and other fields. With the continuous innovation of UAV positioning, perception, control and data transmission technology, future UAVs will replace humans to go deep into underground dangerous areas, detect tunnel diseases, and ensure the safety of infrastructure.

If the tunnel disease is not discovered in time, it will affect the normal use of the tunnel, and even affect the structural safety. The traditional detection method that relies on manual labor and equipment not only consumes a lot of manpower and material resources, but also has problems such as untimely detection and low detection efficiency, which completely rely on the subjective judgment of testing technicians.

Therefore, the existing technology still needs to be improved and developed.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a method and device for tunnel detection based on unmanned aerial vehicle, which aims to solve the problem that traditional disease inspection mainly relies on visual inspection to preliminarily determine the disease type, and the recognition rate depends on the experience of the inspecting personnel. Certainty issues.

In order to achieve the above object, the present invention provides a method for detecting a tunnel based on an unmanned aerial vehicle, and the method for detecting a tunnel based on an unmanned aerial vehicle comprises the following steps:

Receive video images of the tunnel obtained by the drone;

preprocessing and standardizing the video image;

The preprocessed and standardized video images are input into the trained three-dimensional convolutional neural network model, and the three-dimensional convolutional neural network model outputs tunnel disease detection results according to the video images.

Optionally, in the method for detecting a tunnel based on a drone, wherein the receiving video information of the tunnel obtained by the drone further includes:

The UAV collects image information of the surrounding environment in real time by carrying a lidar or a thermal imaging high-definition camera;

Obtain the position and attitude information of the UAV through an inertial navigation device;

transmitting the image information and the pose information to the supercomputing platform in real time;

The supercomputing platform builds a three-dimensional high-precision map according to the image information and the pose information, obtains a planned route, and sends it to the UAV;

The UAV performs autonomous navigation and obstacle avoidance in tunnel flight according to the planned route.

Optionally, in the described UAV-based tunnel detection method, the training process of the trained three-dimensional convolutional neural network model is:

Train the 3D convolutional neural network model by applying the tunnel detection professional database;

Acquiring the first percentage of video data in the application tunnel detection professional database for training of the three-dimensional convolutional neural network model;

Acquiring the remaining second percentage of video data in the application tunnel detection professional database for verifying the trained three-dimensional convolutional neural network model;

When the recognition rate of the three-dimensional convolutional neural network model reaches the preset requirement, the current three-dimensional convolutional neural network model is saved as the trained three-dimensional convolutional neural network model.

Optionally, in the UAV-based tunnel detection method, the first percentage is greater than the second percentage.

Optionally, in the UAV-based tunnel detection method, wherein the preprocessing is: performing brightness adjustment, contrast adjustment and smoothing filtering on the video image, and the smoothing filtering is used to filter out sharp discontinuities noise.

Optionally, in the UAV-based tunnel detection method, the video information input into the trained three-dimensional convolutional neural network model is a video image of multiple consecutive frames.

Optionally, in the UAV-based tunnel detection method, the standardization process is to divide the continuous video into several short videos of 6 frames.

Optionally, the UAV-based tunnel detection method, wherein the three-dimensional convolutional neural network model consists of an input layer, three convolutional layers, two sampling layers, a fully connected layer and an output layer. composition.

Optionally, in the UAV-based tunnel detection method, the tunnel disease detection results include: cracks, water leakage, settlement, exposed steel bars, misalignment, lining peeling, and falling blocks.

In addition, in order to achieve the above object, the present invention also provides a tunnel detection device based on drones, wherein the tunnel detection device based on drones includes:

UAV, supercomputing platform, application tunnel detection professional database and 3D convolutional neural network model;

The drone is used to obtain video images of the tunnel;

The supercomputing platform is used to establish a three-dimensional high-precision map according to the image information collected by the UAV and the position and attitude information obtained by the inertial navigation device, and send the planned route to the UAV after obtaining the planned route;

The application tunnel detection professional database is used for training the three-dimensional convolutional neural network model;

The three-dimensional convolutional neural network model is used to output the tunnel disease detection result according to the video image after the training is completed.

The present invention obtains the video image of the tunnel by receiving the drone; preprocesses and standardizes the video image; inputs the preprocessed and normalized video image into the trained three-dimensional convolutional neural network model , the three-dimensional convolutional neural network model outputs a tunnel disease detection result according to the video image. The present invention obtains the video image of the tunnel through the drone, and inputs the video image into the trained three-dimensional convolutional neural network model, so that the spatial and temporal features in the video can be better captured, so as to quickly output the tunnel disease detection result , which improves the identification rate, detection frequency and processing speed of tunnel diseases, and reduces the degree of manual dependence.

Description of drawings

Fig. 1 is the flow chart of the preferred embodiment of the tunnel detection method based on unmanned aerial vehicle of the present invention;

2 is a schematic diagram of the realization process of autonomous navigation and obstacle avoidance of unmanned aerial vehicles in the preferred embodiment of the tunnel detection method based on unmanned aerial vehicles of the present invention;

3 is a schematic diagram of a sliding window operation performed by the convolution kernel of the 3D convolution in the three-dimensional space of the input image in the preferred embodiment of the UAV-based tunnel detection method of the present invention;

4 is a schematic diagram of the composition structure of a three-dimensional convolutional neural network model in a preferred embodiment of the UAV-based tunnel detection method of the present invention;

FIG. 5 is a schematic diagram of the principle of a preferred embodiment of the UAV-based tunnel detection device of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer and clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Convolutional neural network is a kind of neural network method based on convolution operation, which belongs to one of the representative networks of deep neural network. Convolutional neural network has the characteristics of parameter sharing, local perception and multi-core, etc. The product kernel performs operations, which can effectively extract different features. 3D-CNN usually consists of three-dimensional convolution kernels, two-dimensional convolution (CNN, Convolutional Neural Networks, convolutional neural network) is a convolution in the spatial dimension, and a three-dimensional convolution is both a convolution in space and a convolution in space. Convolution over time. CNN is widely used in image recognition, and does not take into account the data information contained in consecutive frames, while 3D-CNN can perform different convolution operations in the same area, extract different features for different convolution operations, and then Analyze video data, generate multi-channel information from continuous multi-frame video, and perform convolution kernel sampling on each channel, and obtain the final feature representation through all channel information; compared with other neural networks, convolutional neural networks are It has excellent performance in image understanding tasks, especially in the field of image classification, and can automatically extract effective high-dimensional features.

The traditional detection method that relies on manual labor and equipment not only consumes a lot of manpower and material resources, but also has problems such as untimely detection and low detection efficiency, which completely rely on the subjective judgment of testing technicians. The future direction of tunnel disease detection should be automatic detection, highlighting the characteristics of modern technology automation and real-time. If the tunnel disease is not discovered in time, it will affect the normal use of the tunnel, and even affect the structural safety. With the development of technology, it is very important to provide a method for intelligent detection of tunnels and an efficient detection system. A large number of video images can be obtained through the drone, and the drone is combined with the big data cloud platform and 3D-CNN to establish a set of equipment for data collection, data storage and data processing. This intelligent system equipment can carry out daily tunnel diseases. Check, adapt to the future development trend of tunnel disease detection.

The UAV-based tunnel detection method according to the preferred embodiment of the present invention, as shown in FIG. 1 , the UAV-based tunnel detection method includes the following steps:

Step S10, receiving a video image of the tunnel obtained by the drone.

Specifically, the present invention collects real-time video images of tunnels (the tunnels can also be large-scale underground infrastructure) through drones. The drones have functions of autonomous navigation and obstacle avoidance, and solve the problem of large-scale underground infrastructure such as tunnels. The problem of autonomous navigation and obstacle avoidance of UAVs under the condition of no GPS signal and insufficient light in the facility, the realization process is shown in Figure 2:

Step A1: The drone is equipped with an OS-1 lidar (a radar system that emits a laser beam to detect the position, speed and other characteristic quantities of the target. The OS-1 lidar is launched by the lidar manufacturer Ouster, and the OS-1 can output fixed output in real time. High-resolution depth images, signal images and environmental images, which can be used by deep learning algorithms) or thermal imaging high-definition cameras (sensors that can detect extremely small temperature differences, convert the temperature differences into real-time video images and display them) Real-time capture of images of the surrounding environment information;

Step A2: Obtain the position and attitude information of the UAV through IMU (Inertial Measurement Unit, inertial navigation device, a sensor mainly used to detect and measure acceleration and rotational motion);

Step A3: transmitting the environmental image information and pose information around the UAV to the supercomputing platform in real time;

Step A4: The supercomputing platform uses SLAM (Simultaneous Localization and Mapping, real-time positioning and map construction, or simultaneous positioning and mapping) to build a three-dimensional high-precision map with the surrounding environment image and the UAV pose information, and carry out route planning , sending the planned route to the UAV;

Step A5: The UAV receives the path planning information to realize the autonomous navigation and obstacle avoidance of the UAV in the tunnel (using the three-dimensional route planning flight control algorithm to calculate the flight path, and realize the autonomous navigation and obstacle avoidance of the UAV in the tunnel). ).

Further, after the drone acquires the video image of the tunnel, upload it to a data storage platform such as HBase (Hadoop Database) to store massive data and form a big data platform data warehouse. The real-time data transmission adopts WiFi transmission. When the tunnel has no WiFi When the signal is used, COFDM (Coded Orthogonal Frequency Division Multiplexing) technology can be used, and COFDM wireless real-time video transmission can be realized by installing transmitters and standard high-definition COFDM receiver equipment; Managed NoSQL service, compatible with HBase, Phoenix, OpenTSDB and other open source standard interfaces, suitable for GB-PB level data storage, query and analysis, etc., supports SQL analysis, secondary index, time series query and other functions; HBase is built on the Hadoop file system The distributed column-oriented database above is a horizontal development type database that provides fast random access to massive structured data; high-quality analysis results can be obtained. By establishing a database, it provides big data support for tunnel disease detection and increases the objectivity of detection. .

Step S20, performing preprocessing and normalization processing on the video image.

Specifically, the preprocessing is to perform brightness adjustment, contrast adjustment and smooth filtering on the image, so that the image features are obvious and easy to identify. Among them, the purpose of smoothing filtering is to remove sharp discontinuous noise. The normalization process is to divide the continuous video into several short videos of 6 frames. This is because with the increase of the input window (the time dimension of the convolution), the number of trainable parameters also increases, and the input limit of the 3D-CNN network is limited. for a very small number of consecutive video frames.

Step S30: Input the preprocessed and standardized video image into a trained 3D convolutional neural network model, and the 3D convolutional neural network model outputs a tunnel disease detection result according to the video image.

Specifically, the video information input into the trained 3D convolutional neural network model is a video image of multiple consecutive frames. For example, the normalization process is to divide the continuous video into several short videos of 6 frames, and then Input into the trained 3D convolutional neural network model.

Among them, the training process of the trained 3D convolutional neural network model (that is, the 3D-CNN model) is as follows: training the 3D convolutional neural network model by applying the tunnel detection professional database; A percentage of the video data is used for training the 3D convolutional neural network model; the remaining second percentage of the video data in the application tunnel detection professional database is obtained to verify the trained 3D convolutional neural network model ; When the recognition rate of the 3D convolutional neural network model reaches the preset requirement, save the current 3D convolutional neural network model as the trained 3D convolutional neural network model (the training process is the model self-learning process). The first percentage is greater than the second percentage, for example, the first percentage is 70%, then the second percentage is 30%.

For example, use the tunnel detection professional database to train the 3D-CNN model, 70% of the data is used for training, and 30% of the data is used to verify the model. When the model reaches a high recognition rate, save the model as the trained 3D-CNN model. , after saving the model, it has the ability to identify various diseases.

Wherein, the three-dimensional convolutional neural network model outputs tunnel disease detection results according to the video image, which can achieve higher accuracy and more stable judgment; the tunnel disease detection results include: cracks, water leakage, settlement, Exposed steel bars, misalignment, peeling of linings, and falling blocks, of course, can also be the result of other diseases.

As shown in Figure 3, Figure 3 shows a schematic diagram of the sliding window operation of the convolution kernel of 3D convolution in the three-dimensional space of the input image. Figure 3 wants to illustrate the difference between 3D-CNN and 2D-CNN. CNN is based on 2D-CNN and has a sliding window in time series. The convolution kernel moves in three directions. The main difference between 3D-CNN and 2D-CNN is that it can not only capture spatial information, but also Time information can also be captured. Among them, Sliding Windows is referred to as "sliding window", which is the work performed on the input layer; by designing a sliding window to traverse the input video images, and detect the local images corresponding to each window, it can effectively overcome the scale, The problem of input heterogeneity caused by position and deformation, etc., improves the detection effect.

The previous technology cannot extract the temporal features in the video, but the 3D-CNN model significantly improves the extraction of sample features, which can better capture the spatial and temporal features in the video, and improve the identification rate of tunnel diseases.

As shown in FIG. 4 , the three-dimensional convolutional neural network model (3D-CNN model) in the present invention consists of an input layer, three convolutional layers, two sampling layers (ie, pooling layers), and a fully connected layer and an output layer.

Among them, the input layer is used for 6 consecutive video frame images as input, and the sliding window operation is performed to improve the detection effect.

Among them, the convolutional layer is a 3D convolutional layer. First, a series of small 3D feature extractors (kernels) need to be defined to extract stacked high-level representations. In order to generate a new feature space, different 3D kernels are used to extract the input space. Different features, then add a bias term, use a linear activation function, the formula is as follows:

；

;

表示第1层的第i个3D特征；

表示前一层的第k个3D特征空间；

、

、

分别代表值

在坐标轴空间

中的值，

代表三维特征抽取器空间

中的坐标；

表示激活函数ReLU（Rectified Linear Unit）。in,

represents the i-th 3D feature of layer 1;

represents the kth 3D feature space of the previous layer;

,

,

respectively represent the value

in axis space

value in ,

represents the 3D feature extractor space

the coordinates in ;

Represents the activation function ReLU (Rectified Linear Unit).

Among them, the pooling layer is a 3D maximum pooling layer. Unlike the convolutional layer that calculates the cross-correlation between the input and the kernel, the pooling layer directly calculates the maximum value of the elements in the pooling window. This operation is called maximum pooling.

Among them, in the fully connected layer, each neuron is connected to all neurons in the adjacent layer. Before the fully connected layer, the feature space needs to be flattened to a neuron vector, and then vector-matrix multiplication is performed, and then Add a bias term and apply a linear activation function.

；

;

为输入特征向量，从

层的3D特征空间（flatten）取得的；

是第

层的输出特征向量（全连接层）；

表示权重矩阵；

表示偏置项；

表示激活函数ReLU。in,

is the input feature vector, from

The 3D feature space (flatten) of the layer is obtained;

is the first

The output feature vector of the layer (fully connected layer);

represents the weight matrix;

represents the bias term;

represents the activation function ReLU.

Among them, the output layer adopts the Softmax function, and its outputs are all positive values between (0, 1), and the sum is 1; through the Softmax function, the output value of the multi-classification can be converted into a range of [0, 1] and 1 The probability distribution of ; the calculation formula of the Softmax function is as follows:

；

;

为第i个节点的输出值；k为输出节点的个数，即分类的类别个数。in,

is the output value of the ith node; k is the number of output nodes, that is, the number of categories of classification.

The Softmax function is used in the multi-classification process. It maps the possibility of cracks, leaking water, settlement, exposed steel bars, misalignment, lining peeling, and falling blocks to the [0, 1] interval, which can be regarded as a probability Understand that when the output node is finally selected, the node with the highest probability is finally selected as the output result.

The invention can obtain a large number of video images through the unmanned aerial vehicle, and combine the unmanned aerial vehicle with the big data cloud platform and 3D-CNN to establish a set of equipment for data collection, data storage and data processing, and the intelligent system equipment can carry out daily tunneling. It can adapt to the development trend of tunnel disease detection in the future.

Further, as shown in FIG. 5 , based on the above-mentioned UAV-based tunnel detection method, the present invention also provides a UAV-based tunnel detection device, wherein the UAV-based tunnel detection device includes: :

The drone 100, the supercomputing platform 200, the application tunnel detection professional database 300 and the three-dimensional convolutional neural network model 400; the drone 100 is used for acquiring video images of the tunnel; the supercomputing platform 200 is used for The image information collected by the UAV 100 and the pose information obtained by the inertial navigation device establish a three-dimensional high-precision map, and the planned route is sent to the UAV 100; the application tunnel detection professional database 300 is used for the three-dimensional volume. The convolutional neural network model 400 is used for training; the three-dimensional convolutional neural network model 400 is used to output the tunnel disease detection result according to the video image after the training is completed.

The intelligent system equipment has the characteristics of automation, real-time and integration, etc. It is an efficient comprehensive tunnel disease detection system equipment, which improves the detection frequency, processing speed and labor dependence of tunnel diseases; it can realize one type of equipment to detect multiple diseases at one time. , belonging to a comprehensive disease detection equipment; reduce labor costs.

To sum up, the present invention provides a method and device for tunnel detection based on unmanned aerial vehicles. The method includes: receiving a video image of a tunnel obtained by an unmanned aerial vehicle; preprocessing and standardizing the video image; The preprocessed and standardized video images are input into the trained three-dimensional convolutional neural network model, and the three-dimensional convolutional neural network model outputs tunnel disease detection results according to the video images. The present invention obtains the video image of the tunnel through the drone, and inputs the video image into the trained three-dimensional convolutional neural network model, so that the spatial and temporal features in the video can be better captured, so as to quickly output the tunnel disease detection result , which improves the identification rate, detection frequency and processing speed of tunnel diseases, and reduces the degree of manual dependence.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above descriptions, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.

Claims (10)

1. A tunnel detection method based on an unmanned aerial vehicle is characterized by comprising the following steps:

receiving a video image of a tunnel acquired by an unmanned aerial vehicle;

preprocessing and standardizing the video image;

and inputting the video image after preprocessing and standardization processing into a trained three-dimensional convolution neural network model, and outputting a tunnel disease detection result by the three-dimensional convolution neural network model according to the video image.

2. The drone-based tunnel detection method of claim 1, wherein the receiving drone obtains video information of the tunnel, and wherein the method further comprises:

the unmanned aerial vehicle acquires image information of the surrounding environment in real time by carrying a laser radar or a thermal imaging high-definition camera;

acquiring pose information of the unmanned aerial vehicle through an inertial navigation device;

transmitting the image information and the pose information to a supercomputing platform in real time;

the super computing platform establishes a three-dimensional high-precision map according to the image information and the pose information, obtains a planned route and sends the planned route to the unmanned aerial vehicle;

and the unmanned aerial vehicle performs autonomous navigation and obstacle avoidance in tunnel flight according to the planned route.

3. The unmanned aerial vehicle-based tunnel detection method of claim 1, wherein the training process of the trained three-dimensional convolutional neural network model is as follows:

training the three-dimensional convolutional neural network model by applying a tunnel detection professional database;

acquiring a first percentage of video data in the application tunnel detection professional database for training a three-dimensional convolutional neural network model;

acquiring the video data of the second percentage left in the application tunnel detection professional database for verifying the trained three-dimensional convolution neural network model;

and when the recognition rate of the three-dimensional convolutional neural network model reaches a preset requirement, saving the current three-dimensional convolutional neural network model as a trained three-dimensional convolutional neural network model.

4. The drone-based tunnel detection method of claim 3, wherein the first percentage is greater than the second percentage.

5. The unmanned aerial vehicle-based tunnel detection method of claim 1, wherein the preprocessing is: and carrying out brightness adjustment, contrast adjustment and smooth filtering on the video image, wherein the smooth filtering is used for filtering sharp and discontinuous noise.

6. The unmanned-aerial-vehicle-based tunnel detection method of claim 1, wherein the video information input into the trained three-dimensional convolutional neural network model is a continuous multi-frame video image.

7. The drone-based tunnel detection method of claim 6, wherein the normalization process is to segment consecutive video into several 6-frame short videos.

8. The unmanned-aerial-vehicle-based tunnel detection method of claim 1, wherein the three-dimensional convolutional neural network model is composed of one input layer, three convolutional layers, two sampling layers, one fully-connected layer, and one output layer.

9. The unmanned aerial vehicle-based tunnel detection method of claim 1, wherein the tunnel defect detection result comprises: cracks, water leakage, settlement, steel bar exposure, slab staggering, lining stripping and chipping.

10. The utility model provides a tunnel detection device based on unmanned aerial vehicle, its characterized in that, tunnel detection device based on unmanned aerial vehicle includes:

the system comprises an unmanned aerial vehicle, an ultra-computation platform, an application tunnel detection professional database and a three-dimensional convolution neural network model;

the unmanned aerial vehicle is used for acquiring a video image of the tunnel;

the super-computation platform is used for establishing a three-dimensional high-precision map according to the image information acquired by the unmanned aerial vehicle and the pose information acquired by the inertial navigation device, and sending the map to the unmanned aerial vehicle after a planned route is obtained;

the application tunnel detection professional database is used for training the three-dimensional convolution neural network model;

and the three-dimensional convolutional neural network model is used for outputting a tunnel disease detection result according to the video image after training is finished.

Images