CN112462346B - Ground penetrating radar subgrade disease target detection method based on convolutional neural network - Google Patents

Abstract

The invention discloses a ground penetrating radar subgrade disease target detection method based on a convolutional neural network, which comprises the steps of obtaining original image data which are simulated and actually collected by a ground penetrating radar; respectively carrying out corresponding pretreatment on the images to obtain ground penetrating radar pretreatment images; marking and storing target information in the original and preprocessed images; constructing a ground penetrating radar image and a marking information file into a PASCAL VOC data set, and dividing the PASCAL data set into a training set, a verification set and a test set; dynamically setting an initial value of anchor frame parameters by combining the aspect ratio of the marked target frame; and then, training and verifying the Cascade R-CNN network which is finely built by the training set to obtain a convolutional network model, evaluating the performance of the network model by using the testing set, and finally, accurately and quickly detecting the subgrade disease target of the ground penetrating radar. The method does not depend on artificial identification, has stronger generalization capability, and can realize rapid and accurate detection of roadbed disease targets.

Description

A method for detecting roadbed defects using ground penetrating radar based on convolutional neural network

Technical Field

The present invention relates to the field of ground penetrating radar signal processing, and in particular to a ground penetrating radar roadbed defect target detection method based on convolutional neural network.

Background Art

Roadbed is crucial for highways and railways. Due to construction conditions, geographical environment, climate, vehicle driving and other reasons, there are many road diseases. Road and railway surface and shallow diseases are easy to observe and detect, but roadbed diseases are not easy to find. If they are not handled in time and effectively, they will affect the use of roads and railways and seriously threaten the lives of drivers. Ground penetrating radar is a non-destructive, highly accurate, efficient and adaptable detection technology that replaces the original destructive and non-destructive detection methods and is widely used in roadbed disease detection projects.

The transmitting antenna in the ground penetrating radar system emits short pulse electromagnetic waves. The electromagnetic waves pass through the surface and underground media, and are reflected when encountering interfaces and targets with different electrical properties. The receiving antenna receives the reflected echo and forms an A-Scan signal. The transmitting/receiving antenna moves at fixed intervals along the highway and railway survey line, and the A-Scan signals received at different positions constitute the B-Scan image data.

Early ground penetrating radar target detection methods were based on A-Scan signals. According to the distribution characteristics of different disease targets and stratum structures in the time profile and frequency domain, they mainly used information such as energy distribution, waveform characteristics, amplitude phase and cross-correlation between target signals, and used calculation methods such as Fourier transform and wavelet transform to artificially extract the characteristics of different targets for analysis, identification and positioning. Although the above methods can detect targets, they rely on manual analysis and identification, require technicians with rich experience and prior knowledge, and understand and master a large number of roadbed disease target structure characteristics, and contain many subjective factors; they take a lot of energy and time, and the detection efficiency is low; and due to manual operation, the obtained feature parameters and feature representations are few, lacking generalization ability, resulting in low detection accuracy, affecting the judgment of roadbed diseases.

With the development of machine learning in recent years, combined with the representation of different targets in the ground penetrating radar B-Scan image data, such as circular targets of different media presenting a hyperbolic structure with different polarity, square targets presenting a structure with hyperbolas on both sides and parallel lines in the middle, etc., according to their shape, polarity and other performance characteristics, machine learning methods are used to realize automatic detection of roadbed disease targets. Although there is no need for manual extraction of target recognition and detection, it is still impossible to accurately detect complex roadbed environments due to reasons such as algorithm design. Therefore, efficient and accurate identification and positioning of roadbed diseases in complex environments is of great significance for highway and railway maintenance.

Summary of the invention

The purpose of the present invention is to overcome the shortcomings of the above-mentioned technology and provide a ground penetrating radar roadbed disease target detection method based on convolutional neural network, which does not rely on human identification, can achieve fast and accurate detection of different types of roadbed disease targets, is adaptable to different complex roadbed environments, and has generalization ability.

In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions to achieve it.

A method for detecting roadbed defects using ground penetrating radar based on convolutional neural network is implemented in the following steps:

Step 1: Obtaining raw ground penetrating radar image data

The ground penetrating radar system is used to detect the actual roadbed and collect the actual ground penetrating radar B-Scan image data. The FDTD-based gprMax software is used to perform forward simulation on three common types of roadbed diseases to generate ground penetrating radar B-Scan simulation images.

Step 2: GPR data preprocessing

The collected ground penetrating radar image data is normalized, de-biased, and subjected to mean filtering to remove direct waves and automatic gain processing. The simulated ground penetrating radar image data is subjected to mean filtering to remove direct waves and automatic gain amplification signal processing to obtain corresponding preprocessed two-dimensional image data, and then the preprocessed images and the original image data in step 1 are scaled to a uniform pixel size.

Step 3: Marking targets in the GPR image

Use labelImg software to label the targets in the simulated and collected GPR images, and store the target category, coordinates and other information in .xml files;

Step 4: Build the PASCAL VOC dataset

The ground penetrating radar image data in .jpg format and the tag information in .xml format are organized to construct the PASCAL VOC dataset, and divided into training set, validation set and test set according to a certain ratio;

Step 5: Dynamically set anchor box parameters

Calculate the number of target bounding boxes marked in the training set at different aspect ratios, and select the aspect ratio greater than the set threshold and its reciprocal as the initial value of the preset anchor box aspect ratio parameter in network training;

Step 6: Get the convolutional neural network model

Use the built Cascade R-CNN model to train the training set data to obtain a network model that fits the data, and use the validation set data generated in step 4 to fine-tune the network hyperparameters to obtain the final convolutional neural network model;

Step 7: Evaluate Convolutional Neural Network Model Performance

Use the test set generated in step 4 to evaluate the model performance, with recall and average precision as evaluation indicators;

Step 8: Detecting roadbed damage targets using ground penetrating radar

The ground penetrating radar data B-Scan data is input into the trained Cascade R-CNN model in .jpg format for detection, and the category, confidence and detection box coordinates of the target are output.

The present invention is further characterized in that:

In step 1, the original ground penetrating radar image data is obtained. The specific process is as follows:

(1) Obtaining ground penetrating radar image data

The ground penetrating radar system is used to detect the actual roadbed in different places, collect ground penetrating radar images, and form images in the form of B-Scan.

(2) Obtaining ground penetrating radar simulation image data

The FDTD-based gprMax software was used to perform forward simulation on the three main types of roadbed defects to generate ground penetrating radar B-Scan simulation images. The gprMax software constructed the roadbed model and the three defect models respectively. The roadbed consists of a three-layer structure of surface layer, base layer and cushion layer. The three defects include voids, voids and faults.

Adjust the size, shape and burial depth of different types of disease targets, as well as the center frequency of the transmitting antenna, move the transmitting/receiving antenna along the lateral line with a fixed step length, simulate the ground penetrating radar data image of the roadbed disease, and display it in the form of B-Scan.

The present invention is further characterized in that:

In step 2, ground penetrating radar data is preprocessed, and the specific processing flow is as follows:

The collected ground penetrating radar image data are normalized, de-biased, and processed by the mean filter method to remove direct waves and automatic gain. The simulated ground penetrating radar image data are processed by the mean filter method to remove direct waves and automatic gain.

(1) Normalization of ground penetrating radar data

Normalize the two-dimensional B-Scan image so that the value range of all sampling points in the two-dimensional B-Scan becomes [-1,1]. The calculation formula is as follows:

The two-dimensional B-Scan data B (M × N) is composed of N channels of A-Scan signal data, M represents the number of sampling points, N represents the total number of scanning channels, B _min and B _max represent the minimum and maximum values of the image matrix B respectively, and _Bi ′ _j is the normalized sampling point value.

(2) Debiasing of Ground Penetrating Radar Data

The two-dimensional B-Scan image is debiased, and the calculation formula is as follows:

Wherein, x _ij is the i-th sampling point of the j-th A-Scan data X _j =[x _j1 ,x _j2 ,...,x _jM ] ^T , and x _i ′ _j is the data sampling point value after zero bias removal, so as to obtain the ground penetrating radar data after zero bias removal.

(3) Mean filtering of ground penetrating radar data

The two-dimensional B-Scan image is subjected to mean filtering to remove direct waves. The specific process is as follows:

Subtract the mean value of all A-Scan data corresponding to each sampling point from each A-Scan signal of the B-Scan data at each sampling point. The calculation formula is as follows:

Wherein, x _ij is the i-th sampling point of the j-th A-Scan data X _j =[x _j1 ,x _j2 ,...,x _jM ] ^T , and x _i ′ _j is the data sampling point value after removing the direct wave.

(4) Automatic gain processing of ground penetrating radar data

Automatic gain is performed on the two-dimensional B-Scan image to achieve signal amplification. The specific process is as follows:

Each A-Scan signal is divided into T time windows, with 50% overlap between adjacent time windows. The gain value corresponding to the starting point of the time window is calculated by the average amplitude of the sampling points in each time window. The gain values of adjacent time windows are calculated by linear interpolation. The calculation formula is as follows:

表示下取整，A_tj表示第j道A-Scan数据中第t个时窗的平均幅值，x_ij为第j道A-Scan数据X_j＝[x_j1,x_j2,...,x_jM]^T的第i个采样点。The size of each time window is

represents rounding down, _Atj represents the average amplitude of the tth time window in the jth A-Scan data, _{and xij} is the i-th sampling point of the jth A-Scan data _Xj = [ _xj1 , _xj2 , ..., _xjM ] ^T .

The gain value _Gtj of each time window is calculated as follows:

The calculation formula for the gain value of each sampling point in the time window is as follows:

Where _Gsj represents the gain value corresponding to each sampling point in the [t,t+W] time window in the j-th A-Scan data, _Gtj represents the gain value of the t-th time window, _Gt+W,j represents the gain value of the t+1-th time window, and s represents the sampling point index in the time window.

After processing, the preprocessed data of the simulated and collected ground penetrating radar data are obtained respectively, and then they are scaled together with the original image data in step 1 to a uniform size of 375×500 pixels.

The present invention is further characterized in that:

In step 4, the PASCAL VOC dataset is constructed. The specific process is as follows:

The original ground penetrating radar data image data generated in step 1, the .jpg format image data preprocessed in step 2, and the file data marked with targets and stored in .xml format in step 3 are constructed according to the standard format of the PASCAL VOC dataset and divided into training set, validation set and test set in a ratio of 8:1:1.

The present invention is further characterized in that:

In step 5, the anchor frame parameters are dynamically set. The specific process is as follows:

The aspect ratios of the manually marked target bounding boxes in the training set are counted, the number of target bounding boxes in the training set corresponding to different statistical aspect ratios is calculated, and the bounding box aspect ratio greater than the threshold 0.65 and its reciprocal are selected as the initial value of the anchor box aspect ratio parameter in network training.

The present invention is further characterized in that:

In step six, the convolutional neural network model structure is obtained. The specific operations are as follows:

The FPN multi-scale feature mapping module in the Cascade R-CNN model is improved. On the basis of the original _P3 - _P6 fusion feature mapping layers, the _P2 fusion layer is added to detect small targets and the _P7 fusion layer is added to detect larger targets, where _P2 - _P7 represent the output layers of the 2nd to 7th stage fusion feature maps, respectively. After the 5 stage output feature maps of the FPN module are unified to 256 channels by 1×1 convolution, 1×1 convolution kernels and ReLU activation functions are added to enhance the nonlinear expression ability of the network. The IOU thresholds of the three-stage cascade are set to 0.5, 0.6 and 0.7, respectively.

When the stochastic gradient descent algorithm is used to train the Cascade R-CNN model, the total loss function is the weighted sum of the classification loss and the regression loss, and the calculation formula is as follows:

L(x,g)＝L _cls (h(x),y)+λL _reg (f(x,b),g)

Where L _cls (·) represents the classification loss function, using the cross entropy loss function, L _loc (·) represents the regression loss function, using the smooth L1 loss function, h(x) represents the classifier function, f(x,b) represents the regressor function, x represents the partitioned image block input during training, y represents the true category label, λ represents the weighting coefficient, b represents the predicted bounding box, and g represents the true bounding box.

The initial learning rate of the training network model is 0.0025, and the Step learning rate change strategy is adopted. The maximum iteration cycle epoch is set to 50. When the network is trained to the 38th and 48th epochs, the learning rate decreases by 0.1 respectively. After the training, the network model generated uses the validation set generated in step 4 to further fine-tune the hyperparameters to generate a network model that better fits the data.

The present invention is further characterized in that:

In step 7, the performance of the convolutional neural network model is evaluated. The specific operations are as follows:

The test set generated in step 4 is used to evaluate the model performance, with recall and average precision as evaluation indicators.

The recall calculation formula is as follows:

TP represents true positives, that is, the number of samples predicted by the model to be positive and actually are positive; FN represents false negatives, that is, the number of samples predicted by the model to be negative and actually are positive.

The average precision calculation formula is as follows:

FP represents the number of samples predicted as positive but actually negative, and M represents the number of positive samples in a category.

Compared with the prior art, the present invention has the following beneficial technical effects:

1. In the present invention, gprMax software is used to perform forward simulation of roadbed diseases, and the ground penetrating radar system is used to detect the actual roadbed conditions to obtain ground penetrating radar B-Scan data images, which meet the requirements of a large number of data sets in convolutional neural network training and have rich target feature information;

2. The present invention adopts the mean filtering method to quickly and effectively remove the direct wave, has strong real-time performance, and the automatic gain processing effectively realizes the target signal amplification. In addition, a variety of preprocessing methods are used to expand the data set and enrich the target features;

3. The present invention uses convolutional neural network to realize automatic detection of roadbed disease targets by ground penetrating radar, which does not need to rely on manual identification and processing, reduces labor costs and data processing volume, and saves manpower and resources;

4. The present invention uses a deep convolutional neural network to detect ground penetrating radar roadbed disease targets, which has high efficiency, accurate detection accuracy, strong generalization ability, and is adaptable to different complex roadbed environments;

5. Based on the FPN module structure of the original Cascade R-CNN model, the present invention adds a 1×1 convolution kernel and activation function to increase the nonlinear expression ability of the network, reduce the amount of calculation and complexity, and add a _P2 convolution layer to detect small targets, thereby achieving accurate detection of multi-scale target sizes;

6. The present invention adopts dynamic setting of anchor frame parameters, which does not require manual setting and is adaptable to various target boundary box marking situations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for detecting roadbed defects using ground penetrating radar based on a convolutional neural network;

Figure 2 is the B-Scan original image of some ground penetrating radar roadbed disease targets;

Figure 3 shows the Cascade R-CNN network model structure;

Figure 4 shows the FPN module structure of the Cascade R-CNN model in step 6;

Figure 5 is a diagram showing some of the ground penetrating radar roadbed disease target detection results;

DETAILED DESCRIPTION

The present invention is further described in detail below with reference to the accompanying drawings.

As shown in FIG1 , the method for detecting roadbed defects using a ground penetrating radar based on a convolutional neural network according to the present invention is processed according to the following steps:

Step 1: Obtaining raw ground penetrating radar image data

The ground penetrating radar system is used to detect the actual roadbed and collect the actual ground penetrating radar B-Scan image data. The FDTD-based gprMax software is used to perform forward simulation on three common types of roadbed diseases to generate ground penetrating radar B-Scan simulation images.

Step 2: GPR data preprocessing

The collected ground penetrating radar image data is normalized, de-biased, and subjected to mean filtering to remove direct waves and automatic gain processing. The simulated ground penetrating radar image data is subjected to mean filtering to remove direct waves and automatic gain amplification signal processing to obtain corresponding preprocessed two-dimensional image data, and then the preprocessed images and the original image data in step 1 are scaled to a uniform pixel size.

Step 3: Marking targets in the GPR image

Use labelImg software to label the targets in the simulated and collected GPR images, and store the target category, coordinates and other information in .xml files;

Step 4: Build the PASCAL VOC dataset

The ground penetrating radar image data in .jpg format and the tag information in .xml format are organized to construct the PASCAL VOC dataset, and divided into training set, validation set and test set according to a certain ratio;

Step 5: Dynamically set anchor box parameters

Calculate the number of target bounding boxes marked in the training set at different aspect ratios, and select the aspect ratio greater than the set threshold and its reciprocal as the initial value of the preset anchor box aspect ratio parameter in network training;

Step 6: Get the convolutional neural network model

Use the built Cascade R-CNN model to train the training set data to obtain a network model that fits the data, and use the validation set data generated in step 4 to fine-tune the network hyperparameters to obtain the final convolutional neural network model;

Step 7: Evaluate Convolutional Neural Network Model Performance

Use the test set generated in step 4 to evaluate the model performance, with recall and average precision as evaluation indicators;

Step 8: Detecting roadbed damage targets using ground penetrating radar

The ground penetrating radar data B-Scan data is input into the trained Cascade R-CNN model in .jpg format for detection, and the category, confidence and detection box coordinates of the target are output.

The first step is to obtain the original ground penetrating radar image data, which is further divided into obtaining actual acquisition data and simulation data.

For the actual data collection of GPR roadbed disease targets, a GPR acquisition system is used to conduct field detection of roadbeds in different places, collect GPR images, and form images in the form of B-Scan.

For the simulation data of GPR roadbed disease targets, the FDTD-based gprMax software was used to perform forward simulation on the three main types of diseases in the roadbed to generate GPR B-Scan images. The gprMax software constructed the road model and the three disease target models respectively. The simulation model has a width of 10m and a height of 3m. Since the highway and railway are composed of three layers of surface layer, base layer and subbase layer, the surface layer is mainly composed of asphalt and concrete, the base layer is mainly composed of mixed soil, and the subbase layer is mainly composed of sand and gravel, so the thickness is set to 20cm, 30cm and 2.5m respectively, the relative dielectric constants are 4, 9 and 12 respectively, and the conductivity is 0.05, 0.05 and 0.1 respectively. The three diseases include voids, voids and faults, which are placed at different positions in the subbase layer.

The size, shape and burial depth of different types of disease targets are adjusted. The target data are grouped into groups of 10 data. The targets in each group are at different locations and have different sizes. The center frequencies of the transmitting antennas are set to 300 MHz, 900 MHz and 2 GHz, respectively. The ground penetrating radar roadbed disease target data images are simulated and displayed in the form of B-Scan.

In step 2, the ground penetrating radar data is preprocessed by normalizing, removing zero bias, and using the mean filtering method to remove direct waves and automatic gain processing for the collected ground penetrating radar image data, and by using the mean filtering method to remove direct waves and automatic gain processing for the simulated ground penetrating radar image data.

(1) Normalization of ground penetrating radar data

Normalize the two-dimensional B-Scan image so that the value range of all sampling points in the two-dimensional B-Scan becomes [-1,1], which is convenient for subsequent preprocessing operations. The calculation formula is as follows:

The two-dimensional B-Scan data B (M × N) is composed of N channels of A-Scan signal data, M represents the number of sampling points, N represents the total number of scanning channels, B _min and B _max represent the minimum and maximum values of the image matrix B respectively, and _Bi ′ _j is the normalized sampling point value.

(2) Debiasing of Ground Penetrating Radar Data

In order to make the mean value of each A-Scan data of the ground penetrating radar 0 and ensure that the A-Scan signal waveform is non-offset, the two-dimensional B-Scan image is de-biased. Calculate the mean value of each A-Scan data, and then subtract the mean value from each sampling point. The calculation formula is as follows:

Wherein, x _ij is the i-th sampling point of the j-th A-Scan data X _j =[x _j1 ,x _j2 ,...,x _jM ] ^T , and x _i ′ _j is the data sampling point value after zero bias removal, so as to obtain the ground penetrating radar data after zero bias removal.

(3) Mean filtering of ground penetrating radar data

Since the direct wave signal has strong energy and tends to exist stably, it will hide the real target signal, and it appears as a horizontal straight line in the B-Scan image. Therefore, the mean filter method is used to remove the direct wave from the two-dimensional B-Scan image. The specific process is as follows:

Subtract the mean value of all A-Scan data corresponding to each sampling point from each A-Scan signal of the B-Scan data at each sampling point. The calculation formula is as follows:

Wherein, x _ij is the i-th sampling point of the j-th A-Scan data X _j =[x _j1 ,x _j2 ,...,x _jM ] ^T , and x _i ′ _j is the data sampling point value after removing the direct wave.

(4) Automatic gain processing of ground penetrating radar data

Different disease targets are located at different depths. When detecting targets at deeper locations, the signal is weakened due to the long round-trip time, and the target cannot be directly observed with the naked eye in the B-Scan image. Therefore, the two-dimensional B-Scan image is automatically amplified to achieve signal amplification and balance. The specific process is as follows:

Each A-Scan signal is divided into T time windows, with 50% overlap between adjacent time windows. The gain value corresponding to the starting point of the time window is calculated by the average amplitude of the sampling points in each time window. To ensure that the image is not distorted, the gain value of each sampling point in the adjacent time windows is calculated by linear interpolation. The specific calculation formula is as follows:

表示下取整，A_tj表示第j道A-Scan数据中第t个时窗的平均幅值，x_ij为第j道A-Scan数据X_j＝[x_j1,x_j2,...,x_jM]^T的第i个采样点值。The size of each time window is

represents rounding down, _Atj represents the average amplitude of the tth time window in the jth A-Scan data, and _xij is the i-th sampling point value of the jth A-Scan data _Xj = [ _xj1 , _xj2 , ..., _xjM ] ^T .

The gain value _Gtj at the starting point of each time window is calculated as follows:

The calculation formula for the gain value of each sampling point in the time window is as follows:

Where G _sj represents the gain value corresponding to each sampling point in the [t, t+W] time window in the j-th A-Scan data, G _tj represents the gain value of the t-th time window, G _t+W,j represents the gain value of the t+1-th time window, and s represents the index of the sampling point in the time window. For shallow target signals, the corresponding gain value is small, and for deep target signals, the corresponding gain value is large.

The above four processing methods are used to perform corresponding preprocessing on the original ground penetrating radar data simulated and collected in step 1, and the respective preprocessed image data are obtained, and then the preprocessed image data and the original image data are scaled to a uniform size of 375×500 pixels.

In step three, mark the targets in the ground penetrating radar image. Use labelImg software to mark the targets in the ground penetrating radar image obtained in step two, and mark different roadbed disease targets as one type of target. After marking, the software automatically stores the image file name, location, marked target category, coordinates and other information, and generates the corresponding .xml file.

In step 4, the PASCAL VOC dataset is constructed. The original ground penetrating radar data image data generated in step 1, the .jpg format image data preprocessed in step 2, and the target marked and stored in .xml format file data in step 3 are used to construct a standard dataset according to the standard format of the PASCAL VOC dataset, and divided into training set, validation set and test set in a ratio of 8:1:1. The PASCAL VOC dataset includes three folders: Annotations, ImageSets and JPEGImages. Annotations stores .xml target marking files, ImageSets stores .txt files consisting of the data paths and names of the training set, validation set and test set generation, and JPEGImages stores .jpg format ground penetrating radar data images.

In step 5, the anchor box parameters are dynamically set, the aspect ratio of the length to the width of the marked target bounding boxes in the training set is counted, and the number of target bounding boxes marked by image data in the training set at different aspect ratios is calculated respectively. The aspect ratio greater than the threshold 0.65 and its reciprocal are selected as the initial value of the preset anchor box aspect ratio parameter in the network training. The initial value of the anchor box scale adopts the default value of the network.

In step 6, the convolutional neural network model is obtained, and the Cascade R-CNN model structure is rebuilt. Figure 3 is a schematic diagram of the Cascade R-CNN network model structure, where "Input" represents image input, "Conv" represents the convolution layer in the backbone network Backbone, "Pool" represents regional feature extraction, "Head" represents the Head part of the network, which is used to predict image features, generate prediction boxes and classifications, "B" represents bounding box regression operation, "C" represents classification operation, "B0" represents the initially generated bounding box, and "1", "2" and "3" represent the three stages of the network model respectively. It can be seen from the figure that the Cascade R-CNN network model mainly includes four stages. The first stage uses the RPN module to generate a preliminary bounding box. The other three stages use cascade IOU thresholds set to 0.5, 0.6 and 0.7 respectively to obtain more accurate bounding boxes. By resampling and adjusting the bounding box in the previous stage, positive samples with higher IOU values are found for training in the next stage.

In order to detect disease targets of different sizes, the FPN module in the Cascade R-CNN model is improved to obtain multi-scale feature maps. On the basis of the originally generated _P3 - _P6 fusion feature mapping layer, the _P2 fusion layer is added to detect small targets and the _P7 fusion layer is added to detect larger targets. After the 5-stage output feature maps of the FPN module are subjected to 1×1 convolution to unify the number of channels to 256, 1×1 convolution kernels and ReLU activation functions are added to enhance the nonlinear expression ability of the network and reduce the amount of calculation and complexity. The structure of the improved FPN module is shown in Figure 4, where Conv1-5 represents the output feature maps of the five stages of the backbone network in the model, Conv_R1-5 represents the output convolutional layer after the channels of Conv1-5 layers are unified to 256, P2-7 is the output fusion feature layer of the FPN module, where P6 is generated by P5 through a 3×3 convolution kernel with a step size of 2, P7 is generated by P6 through a 3×3 convolution kernel with a step size of 2 and a ReLU activation function, and P2 is generated by Conv_R2 through a 3×3×256 convolution operation.

The purpose of network training is to find the network parameter weights and biases corresponding to the minimum total loss function. Therefore, the stochastic gradient descent algorithm is used to train the Cascade R-CNN model. Its total loss function is the weighted sum of classification loss and regression loss. The calculation formula is as follows:

L(x,g)＝L _cls (h(x),y)+λL _reg (f(x,b),g)

Where L _cls (·) represents the classification loss function, using the cross entropy loss function, L _loc (·) represents the regression loss function, using the smooth L1 loss function, h(x) represents the classifier function, f(x,b) represents the regressor function, x represents the partitioned image block input during training, y represents the true category label, λ represents the weighting coefficient, b represents the predicted bounding box, and g represents the true bounding box.

The initial learning rate for training the Cascade R-CNN network model is 0.0025, and the Step learning rate change strategy is adopted. The maximum iteration cycle epoch is set to 50, and the learning rate is reduced by 0.1 when the network is trained to the 38th and 48th epochs respectively. After the training, the network model generated uses the validation set generated in step 4 to further fine-tune the hyperparameters to generate a network model that better fits the data.

In step 7, the performance of the convolutional neural network model is evaluated. The test set generated in step 4 is used to evaluate the model performance, and the recall rate and average precision are used as evaluation indicators.

The recall calculation formula is as follows:

TP represents true positives, that is, the number of samples predicted by the model to be positive and actually are positive; FN represents false negatives, that is, the number of samples predicted by the model to be negative and actually are positive.

The average precision calculation formula is as follows:

FP represents the number of samples predicted as positive but actually negative, and M represents the number of positive samples in a category.

In step eight, the ground penetrating radar roadbed disease targets are detected. The ground penetrating radar data B-Scan data is input into the trained Cascade R-CNN model in .jpg format for detection, and the category, confidence and detection box coordinates of the existing target are output.

The following is an explanation of the experimental effect of the ground penetrating radar roadbed disease target detection method of this scheme with specific examples:

The ground penetrating radar system and gprMax software are used to generate the ground penetrating radar roadbed disease target image. Figure 2 is the original ground penetrating radar disease target image. Figure 2(a) is the B-Scan data image obtained by using a 2GHz ground penetrating radar antenna to detect the actual roadbed road conditions, mainly including faults, voids and holes at different depths. Figure 2(b) is the simulated B-Scan data image of the ground penetrating radar roadbed disease target. The target echoes formed by the transmitting antennas with different center frequencies at different positions for different types of targets constitute the B-Scan simulation image. The data disease targets mainly include holes, voids and faults. In the simulation, the holes are set to be circular and square, the voids are set to be square and inverted triangle, and the faults have two different inclination angles. In this example, labelImg software is used to annotate the target bounding box and category in the generated ground penetrating radar image data, select the minimum bounding box that can frame the target, and uniformly annotate it as a type of disease target, and generate the corresponding .xml file. Then, the generated original images of ground penetrating radar disease targets, preprocessed image data and marked .xml file data are used to generate training sets, validation sets and test sets in a ratio of 8:1:1, and then they are organized to construct the PASCAL VOC dataset.

This example is implemented on the detection toolbox mmdetection based on PyTorch. The Cascade R-CNN network model in step 6 is used for training detection. Its backbone network Backbone is ResNet101. The network is trained for 50 epochs on 1 GPU (each GPU trains 2 images). The initial learning rate is set to 0.0025. The learning rate uses the Step mechanism and decreases by 0.1 when training to 38 and 48 epochs. The momentum is set to 0.9 and the weight decay is 0.0005. The image input to the network is unified to 375×500 pixels, and flipping in the framework is used as the only online data enhancement technology.

The network model obtained after fine-tuning the network hyperparameters using the validation set is used to evaluate the network performance using the test set, with the recall rate Recall and the average precision AP as evaluation indicators. Whether there is a target mark in the test set is divided into positive samples and negative samples. The network is detected according to the IOU threshold of the NMS post-processing set to obtain the result. The network detection performance is evaluated by comparing the two results using the recall rate Recall and the average precision AP to determine whether the network performance is good and whether the detection result is accurate. The IOU threshold of the NMS used in the network post-processing is set to 0.6, that is, the correct result is output only when the confidence of the detected bounding box is greater than 0.6. The recall rate obtained in this example reaches 94.5%, the average precision reaches 90.1%, and the time taken by the network to detect the target in an image is in the order of milliseconds, which shows that the network model has good detection performance and can be used to accurately and efficiently detect roadbed disease targets. Figure 5 shows the detection results of some ground penetrating radar roadbed disease targets, which can accurately detect roadbed disease targets in the image.

Claims (7)

1. The ground penetrating radar subgrade disease target detection method based on the convolutional neural network is characterized by comprising the following steps of:

step one: acquiring original image data of ground penetrating radar

Detecting and collecting actual image data of a ground penetrating radar B-Scan by using a ground penetrating radar system, and forward modeling common 3 disease types in the roadbed by using FDTD-based gprMax software to generate a ground penetrating radar B-Scan simulation image;

step two: ground penetrating radar data preprocessing

The method comprises the steps of carrying out normalization, zero offset removal and mean filtering on collected ground penetrating radar image data to remove a direct reaching wave and automatic gain processing, carrying out mean filtering on simulated ground penetrating radar image data to remove the direct reaching wave and automatic gain amplification signal processing to obtain two-dimensional image data after corresponding preprocessing respectively, and scaling the preprocessed image and original image data in the first step to uniform pixel size;

step three: marking targets in ground penetrating radar images

Marking a target in the simulated and collected ground penetrating radar image by using labelImg software, and storing the type and coordinate information of the target in an xml file;

step four: construction of PASCALVOC dataset

The method comprises the steps of sorting ground penetrating radar image data in a jpg format and marking information in an xml format to construct a PASCALVOC data set, and dividing the PASCALVOC data set into a training set, a verification set and a test set according to a certain proportion;

step five: dynamically setting anchor frame parameters

Calculating the corresponding quantity of the target boundary boxes marked in the training set under different aspect ratios, and selecting the aspect ratio and the reciprocal thereof larger than a set threshold value as an initial value of an aspect ratio parameter of a preset anchor frame in network training;

step six: acquiring convolutional neural network model

Training the training set data by using the established CascadeR-CNN model to obtain a network model of fitting data, and fine-tuning the network super-parameters by using the verification set data generated in the fourth step to obtain a final convolutional neural network model;

step seven: evaluation of convolutional neural network model Performance

The performance of the model is evaluated by adopting the test set generated in the step four, and the recall rate and the average precision are used as evaluation indexes;

step eight: detecting ground penetrating radar subgrade disease target

And (3) inputting the ground penetrating radar data B-Scan data into a trained CascadeR-CNN model in a jpg format for detection, and outputting the category, confidence coefficient and detection frame coordinates of the existing target.

2. The method for detecting the ground penetrating radar subgrade disease target based on the convolutional neural network according to claim 1, wherein the specific process of acquiring the ground penetrating radar original image data in the first step is as follows:

(1) Acquisition of ground penetrating radar acquired image data

Detecting actual roadbed at different places by adopting a ground penetrating radar system, collecting ground penetrating radar images, and imaging in a B-Scan mode;

(2) Obtaining ground penetrating radar simulation image data

Forward modeling is carried out on 3 common disease types in roadbed by using FDTD-based gprMax software to generate a ground penetrating radar B-Scan simulation image, the gprMax software respectively constructs a road model and 3 disease target models, the width of the simulation model main body is 10m, the height is 3m, the model is composed of a surface layer, a base layer and a base layer, the thickness is respectively set to be 20cm, 30cm and 2.5m, the relative dielectric constants are respectively 4, 9 and 12, the conductivities are respectively 0.05, 0.05 and 0.1,3 diseases including hollowness, void and faults, and the model is placed at different positions in the base layer;

the size, shape and burial depth of different types of disease targets are adjusted, every 10 data of target data are in a group, the positions of the targets in each group are different, the sizes of the targets among the groups are different, the center frequencies of the transmitting antennas are respectively 300MHz, 900MHz and 2GHz, a ground penetrating radar roadbed disease target data image is simulated, and the ground penetrating radar roadbed disease target data image is imaged and displayed in a B-Scan mode.

3. The method for detecting the ground penetrating radar subgrade disease target based on the convolutional neural network according to claim 1, wherein the ground penetrating radar data preprocessing in the second step comprises the following specific procedures:

carrying out normalization, zero offset removal, mean filtering, direct arrival removal and automatic gain treatment on the collected ground penetrating radar image data; adopting an average filtering method to remove the direct arrival wave and automatic gain processing on the simulated ground penetrating radar image data;

(1) Normalization processing of collected data of ground penetrating radar

Normalizing the two-dimensional B-Scan image to enable the value range of all sampling point values in the two-dimensional B-Scan to be [ -1,1], wherein the calculation formula is as follows:

wherein the two-dimensional B-Scan data B (MXN) is composed of N-channel A-Scan data, M represents the number of sampling points, N represents the number of scanning channels, B _min 、B _max Respectively representing the minimum and maximum values of the image matrix B _ij The ith sample point value, B ', representing the jth track A-Scan before normalization' _ij The ith sampling point value of the j-th track A-Scan after normalization is represented;

(2) Zero offset removal processing for collected data of ground penetrating radar

Zero offset is removed on the two-dimensional B-Scan image, and the calculation formula is as follows:

obtaining ground penetrating radar data after zero offset removal, wherein x is _ij For the jth track A-Scan data X _j ＝[x _j1 ,x _j2 ,...,x _jM ] ^T Is the ith sample point, x _mj For the jth track A-Scan data X _j ＝[x _j1 ,x _j2 ,...,x _jM ] ^T M is the whole sampling point number M, x' _ij The data sampling point value is the data sampling point value after zero offset removal;

(3) Ground penetrating radar data average value filtering processing

The method for removing direct waves by the mean value filtering method is carried out on the two-dimensional B-Scan image, and comprises the following specific processes:

subtracting the average value of all the channel A-Scan data corresponding to the sampling point from each channel A-Scan signal of the B-Scan data, wherein the calculation formula is as follows:

wherein x is _ij For the jth track A-Scan data X _j ＝[x _j1 ,x _j2 ,...,x _jM ] ^T Is the ith sample point, x _ik The value range of k is the whole scanning track number N, x', which is the ith sampling point of the kth track A-Scan data _ij Removing the direct post-data sampling point value;

(4) Automatic gain processing of ground penetrating radar data

The two-dimensional B-Scan image is automatically gain to realize signal amplification, and the specific process is as follows:

dividing each channel of A-Scan signal into T time windows, wherein adjacent time windows have 50% overlap, calculating a time window starting point corresponding gain value by the average amplitude of sampling points in each time window, and calculating the gain value of the adjacent time windows by adopting linear interpolation, wherein the calculation formula is as follows:

wherein each time window has a size of

Lower and rounded representation, A _tj Represents the average amplitude, x of the t-th time window in the j-th track A-Scan data _ij For the jth track A-Scan data X _j ＝[x _j1 ,x _j2 ,...,x _jM ] ^T Is the i-th sampling point of (a);

gain value G for each time window _tj The calculation is as follows:

the calculation formula of the gain value of each sampling point in the time window is as follows:

wherein G is _sj Represents [ t, t+W ] in the jth lane A-Scan data]Gain value, G, corresponding to each sampling point in the time window _tj Represents the gain value of the t-th time window, G _t+W,j The gain value of the (t+1) th time window is represented, and s represents the index of sampling points in the time window;

preprocessing data of simulation and acquisition of ground penetrating radar data are obtained through processing, and then the preprocessing data and the original image data in the step one are scaled to be of a uniform 375 multiplied by 500 pixel size.

4. The method for detecting the ground penetrating radar subgrade disease target based on the convolutional neural network according to claim 1, wherein a PASCALVOC data set is constructed in the fourth step, and the specific flow is as follows:

and E, constructing the original ground penetrating radar data image data generated in the first step, the image data in the jpg format preprocessed in the second step and the file data marked with the target and stored as the xml format in the third step according to the standard format of the PASCALVOC data set, and dividing the data into a training set, a verification set and a test set according to the ratio of 8:1:1.

5. The method for detecting the ground penetrating radar subgrade disease target based on the convolutional neural network according to claim 1, wherein in the fifth step, anchor frame parameters are dynamically set, and the specific flow is as follows:

and counting the aspect ratio of the artificially marked target boundary boxes in the training set, calculating the corresponding quantity of the target boundary boxes in the training set under different statistical aspect ratios, and selecting the boundary box aspect ratio and the reciprocal thereof which are larger than a threshold value of 0.65 as initial values of anchor box aspect ratio parameters in the network training.

6. The method for detecting the ground penetrating radar subgrade fault target based on the convolutional neural network according to claim 1, wherein the convolutional neural network model structure is obtained in the sixth step, and the specific operation is as follows:

FPN acquisition multi-scale feature mapping module in improved Cascade R-CNN model, and the FPN acquisition multi-scale feature mapping module is in original P ₃ -P ₆ Based on the fusion feature mapping layer, P is added ₂ Fusion of feature map layers to detect small objects and P ₇ Fusing feature map layers to detect larger targets, where P ₂ -P ₇ Respectively representing the 2 nd to 7 th fusion feature mapping layers; after the number of the unified channels of the 5 stage output feature graphs of the FPN module is 256 through 1×1 convolution, 1×1 convolution kernels and ReLU activation functions are respectively added to enhance the nonlinear expression capacity of the network; the three-stage cascade IOU thresholds are set to 0.5, 0.6 and 0.7, respectively;

when a random gradient descent algorithm is adopted to train a Cascade R-CNN model, the total loss function is a weighted sum of classification loss and regression loss, and the calculation formula is as follows:

L(x,g)＝L _cls (h(x),y)+λL _reg (f(x,b),g)

wherein L is _cls (. Cndot.) represents the class loss function, employing the cross entropy loss function, L _reg Representing a regression loss function, adopting a smoothL1 loss function, h (x) representing a classifier function, f (x, b) representing a regressor function, x representing divided image blocks input in the training process, y representing a real class label, lambda representing a weighting coefficient, b representing a prediction boundary box, and g representing a real boundary box;

the initial learning rate of the training network model is 0.0025, a Step learning rate change strategy is adopted, the maximum iteration period epoch is set to be 50, the learning rate is respectively reduced by 0.1 when the network is trained to the 38 th and 48 th epochs, the network model is generated after the training is finished, and the verification set generated in the fourth Step is adopted to further fine tune the super parameters to generate the network model of more fitting data.

7. The method for detecting the ground penetrating radar subgrade fault target based on the convolutional neural network according to claim 1, wherein in the seventh step, the performance of the convolutional neural network model is evaluated, and the specific operation is as follows:

the performance of the model is evaluated by adopting the test set generated in the step four, and the recall rate and the average precision are used as evaluation indexes;

the recall ratio calculation formula is as follows:

wherein TP represents true positive, i.e. the number of samples of which the model predicts as positive and the model actually is positive, and FN represents false negative, i.e. the number of samples of which the model predicts as negative and the model actually is positive;

the average accuracy calculation formula is as follows:

wherein the FP representation predicts the number of samples as positive examples and actually negative examples, and S represents the number of positive examples existing in one class sample.

Images