CN107133960A - Image crack dividing method based on depth convolutional neural networks - Google Patents

Abstract

The invention discloses an image crack segmentation method based on a deep convolutional neural network, comprising: inputting an original image into a deep convolutional neural network, learning features through convolution, pooling and activation layers, and obtaining a feature map; The feature map with the same size as the original image is obtained by upsampling; the softmax prediction is performed on the feature map with the same size as the original image, and the category of the corresponding position is obtained, so as to realize the segmentation of the crack area. The invention can learn multi-level features from low to high, can quickly realize high-precision crack region segmentation, and is especially suitable for bridge structure crack detection.

Description

Image crack segmentation method based on deep convolutional neural network

technical field

The invention relates to the technical field of crack detection, in particular to an image crack segmentation method based on a deep convolutional neural network.

Background technique

With the rapid development of national projects such as my country's economic development, urbanization, and high-speed rail, the number of bridges built to cross obstacles in the construction of roads, railways, or urban and rural water conservancy has increased rapidly. Bridges play an important role in the development of the national economy. It plays a pivotal role and is also a manifestation of our country's comprehensive strength. Due to the ubiquity of bridges, the safety and durability of bridge structures cannot be ignored. Cracks, as a major bridge structure disease feature, are the most harmful to the durability and safety of the bridge structure. Therefore, cracks are one of the main evaluation indicators of its health status.

The current detection method is still mainly manual detection, which has many shortcomings:

(1) Low detection efficiency: time-consuming, need to install or disassemble handstand and other equipment;

(2) Low detection accuracy: it is mainly observed and detected by human eyes, which is easily affected by human subjective factors;

(3) High labor intensity: there are many bridges, and the inspection workload is heavy, and it is only done manually, which is very intensive;

(4) Low safety: inspectors need to go down to the bottom of the bridge for inspection, and safety is not guaranteed;

(5) High cost: use a lot of manpower and material resources for testing, and the cost is high;

(6) Low degree of informatization: It is impossible to accurately establish historical data of bridge cracks, it is not convenient for the management and maintenance of dangerous bridges, and it is also unable to provide decision-making support information for government management departments.

The above deficiencies lead to the fact that the current detection status cannot adapt to the current bridge construction and development at all.

In recent years, algorithms for detecting and extracting road cracks based on image vision methods have been proposed one after another, which has made great progress in the automatic and intelligent detection of road cracks. Bridge structure crack detection is similar to road pavement crack detection, but the former is more complicated, mainly in two aspects: first, the complexity of the bridge structure makes it extremely difficult to obtain data based on visual image methods, and the upper surface of the bridge structure It is basically consistent with the road, and the data is relatively easy to obtain. At present, there are practical systems put into production. However, there has been no effective data acquisition and processing system for its lower surface so far. Second, the pavement texture features of the road are relatively simple and single, and the crack features are generally consistent, while the surface texture of the bridge bottom is relatively complex, and there are a lot of "noise" such as spots, stains, water stains, and detection marking lines. Extraction is more difficult. These two points greatly limit the automatic detection of bridge structural cracks and intelligent structural safety monitoring.

Different understandings of crack characteristics lead to a variety of crack detection methods proposed by people. Most of the algorithms use the same basic features, and the algorithm flow is roughly the same: preprocessing, crack region detection and segmentation, postprocessing and feature descriptions. As a seemingly simple object, cracks have variability and complexity due to their background and structural characteristics. The existing road crack detection algorithms still have many defects, which are far from meeting their needs.

In short, there are a variety of features used to detect cracks, but simple and efficient detection is still a difficulty. How to better separate the diverse cracks from background features, how to quickly extract crack features and quickly reconstruct crack structure features These are very challenging questions.

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention provides an image crack segmentation method based on a deep convolutional neural network. The method is based on a deep learning method and utilizes the low-to-high multi-layer features for efficient and high-precision fracture region segmentation.

The idea of the present invention is:

Based on Fully Convolutional Networks (FCN) and Deeply-Supervised Nets (DSN), the present invention provides a deep convolutional neural network for learning hierarchical features, which can automatically learn multiple scales, Multiple levels of multi-level fracture features to achieve end-to-end, pixel-level prediction. Since the deep convolutional neural network holistically and directly supervises the learning strategy, it can be applied to process crack images of various scenes and scales.

In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

Image crack segmentation method based on deep convolutional neural network, including:

The deep convolutional neural network used includes 5 convolutional stages, among which the first convolutional stage includes convolutional layer Conv1_1 and convolutional layer Conv1_2, and convolutional layer Conv1_1 and convolutional layer Conv1_2 use 64 convolution kernels respectively ; The second convolution stage includes convolutional layer Conv2_1 and convolutional layer Conv2_2, convolutional layer Conv2_1 and convolutional layer Conv2_2 use 128 convolution kernels respectively; the third convolutional stage includes convolutional layer Conv3_1, convolutional layer Conv3_2 and convolutional layer Conv3_3, convolutional layer Conv3_1, convolutional layer Conv3_2 and convolutional layer Conv3_3 use 256 convolution kernels respectively; the fourth convolution stage includes convolutional layer conv4_1, convolutional layer conv4_2 and convolutional layer conv4_3 , convolutional layer conv4_1, convolutional layer Conv4_2, and convolutional layer Conv4_3 use 512 convolution kernels respectively; the fifth convolution stage includes convolutional layer Conv5_1, convolutional layer Conv5_2, and convolutional layer Conv5_3, convolutional layer conv5_1, Convolution layer conv5_2 and convolution layer conv5_3 use 512 convolution kernels respectively; convolution layer conv1_2, convolution layer conv2_2, convolution layer conv3_3 and convolution layer conv4_3 are followed by batch normalization layer, nonlinear activation layer, Pooling layer; other convolutional layers are followed by batch normalization layer and nonlinear activation layer in turn;

The batch normalization layer and nonlinear activation layer following the convolutional layer conv1_1 are recorded as the first batch of normalization layers and the first nonlinear activation layer; the batch normalization layer, nonlinear activation layer, and The pooling layer is recorded as the second batch of normalization layer, the second nonlinear activation layer, and the first pooling layer;

The batch normalization layer and nonlinear activation layer following the convolutional layer conv2_1 are recorded as the third batch normalization layer and the third nonlinear activation layer; the batch normalization layer, nonlinear activation layer, and The pooling layer is recorded as the fourth batch of normalization layer, the fourth nonlinear activation layer, and the second pooling layer;

The batch normalization layer and nonlinear activation layer following the convolutional layer conv3_1 are denoted as the fifth batch normalization layer and the fifth nonlinear activation layer; the batch normalization layer and nonlinear activation layer immediately following the convolutional layer conv3_2 are denoted as It is the sixth batch of normalization layer and the sixth non-linear activation layer; the batch normalization layer, non-linear activation layer and pooling layer following the convolutional layer conv3_3 are recorded as the seventh batch of normalization layer, the seventh non-linear activation layer, The third pooling layer;

The batch normalization layer and nonlinear activation layer following the convolutional layer conv4_1 are denoted as the eighth batch normalization layer and the eighth nonlinear activation layer; the batch normalization layer and nonlinear activation layer immediately following the convolutional layer conv4_2 are denoted as It is the ninth batch of normalization layer and the ninth non-linear activation layer; the batch normalization layer, non-linear activation layer and pooling layer following the convolutional layer conv4_3 are recorded as the tenth batch of normalization layer, the tenth non-linear activation layer, The fourth pooling layer;

The batch normalization layer and nonlinear activation layer immediately following the convolutional layer conv5_1 are recorded as the eleventh batch normalization layer and the eleventh nonlinear activation layer; the batch normalization layer and nonlinear activation layer immediately following the convolutional layer conv5_2 The layer is marked as the twelfth batch normalization layer and the twelfth nonlinear activation layer; the batch normalization layer, nonlinear activation layer, and pooling layer immediately following the convolutional layer conv5_3 are recorded as the thirteenth batch normalization layer and the tenth batch normalization layer. Three non-linear activation layers;

The original image is input from the convolutional layer conv1_1; in the first convolution stage, the original image is sequentially passed through the convolutional layer conv1_1, the first batch of normalization layers, the first nonlinear activation layer, the convolutional layer conv1_2, and the second batch of normalization layers , The second nonlinear activation layer outputs the first feature map; the first feature map is input to the second convolution stage after the first pooling layer, and then passes through the convolution layer conv2_1, the third batch of normalization layers, and the third nonlinear The activation layer, convolutional layer conv2_2, the fourth batch of normalization layers, and the fourth nonlinear activation layer output the second feature map; the second feature map is input to the third convolution stage after the second pooling layer, and is convolved in sequence Layer conv3_1, fifth batch normalization layer, fifth nonlinear activation layer, convolutional layer conv3_2, sixth batch normalization layer, sixth nonlinear activation layer, convolutional layer conv3_3, seventh batch normalization layer, seventh nonlinear activation Layer outputs the third feature map; the third feature map is input to the fourth convolution stage after passing through the third pooling layer, and then passes through the convolution layer conv4_1, the eighth batch normalization layer, the eighth nonlinear activation layer, and the convolution layer conv4_2, the ninth batch of normalization layer, the ninth non-linear activation layer, the convolutional layer conv4_3, the tenth batch of normalization layer, the tenth non-linear activation layer output the fourth feature map; the fourth feature map is input after the fourth pooling layer The fourth convolution stage, sequentially through the convolutional layer conv5_1, the eleventh batch of normalization layer, the eleventh nonlinear activation layer, the convolutional layer conv5_2, the twelfth batch of normalization layer, the twelfth nonlinear activation layer, The convolutional layer conv5_3, the thirteenth batch normalization layer, and the thirteenth non-linear activation layer output the fifth feature map;

The first feature map, the second feature map, the third feature map, the fourth feature map, and the fifth feature map respectively go through the convolutional layer Conv1, the convolutional layer Conv2, the convolutional layer Conv3, the convolutional layer Conv4, and the convolutional layer The output feature maps of Conv5, Conv1, convolutional layer Conv2, convolutional layer Conv3, convolutional layer Conv4, and convolutional layer Conv5 are respectively deconvoluted and upsampled by deconvolution layers Deconv2, Deconv3, Deconv4, and Deconv5 to obtain The feature map with the same size as the original image; five feature maps with the same size as the original image are fused through the connection layer, and the output of the connection layer is sequentially reduced through the convolutional layer Conv-1-2, and the corresponding pixel value is obtained through the softmax function. The prediction probability of cracks and non-cracks is recorded as the fusion prediction probability, so as to realize the image crack segmentation of the original image;

Record the output of the second nonlinear activation layer, deconvolution layer Deconv2, Deconv3, Deconv4, Deconv5 as side output;

The described depth convolutional neural network adopts the following method to carry out sample training:

Collect training sample set I _i represents the i-th training sample, G _i represents the manual label corresponding to I _i , count the number of non-crack pixels and crack pixels in each training sample, and record them as C ₀ and C ₁ respectively;

With the loss cost function L(I,G,Q)=L _side (I,G,Q,q)+L _fuse (I,G,Q) minimum, optimize the parameters of the deep convolutional neural network; where:

L _side (I,G,W,w) represents the side loss cost,

ω ₀ and ω ₁ are class balance weights, ω ₀ =1, ω ₁ =C ₀ /C ₁ ; I(k) and G(k) refer to the values corresponding to the kth pixel of I and G respectively, logPr (*) refers to the softmax function;

L _fuse (I,G,Q) represents the fusion loss cost,

Further, all convolutional layers use a convolutional kernel window with a size of 3×3.

Further, all pooling layers use a convolution kernel window of size 2×2 for maximum pooling.

Further, all batch normalization layers use the formula Perform a batch normalization operation on the feature map processed by the convolutional layer, where BN _{γ, β} (x) represents the result of the batch normalization operation; β=E[x], ε is a constant approaching to 0; Var[·] means variance, E[·] means mean value.

Further, the nonlinear activation layer adopts a nonlinear function f(x)=max(0,x).

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The invention can learn multi-level features from low to high, can quickly realize high-precision crack region segmentation, and is especially suitable for bridge structure crack detection.

Description of drawings

Fig. 1 is a schematic diagram of a deep convolutional neural network of the present invention;

Fig. 2 is the output schematic diagram of deep convolutional neural network of the present invention;

Figure 3 is a schematic diagram of the convolution kernel window, where Figure (a) shows the case where the periphery of the image is not filled with 0, and Figure (b) shows the situation where the periphery of the image is filled with 0.

detailed description

The technical solutions of the present invention will be described in detail below in conjunction with the accompanying drawings.

The deep convolutional neural network of the present invention is abbreviated as DeepCrack, which is a deep convolutional neural network for learning hierarchical features, based on fully convolutional neural networks (Fully Convolutional Networks, FCN) and deep supervision networks (Deeply-Supervised Nets, DSN) implementation, the loss cost of crack and non-crack prediction results can be balanced in the training phase.

The invention includes: inputting the original image into a deep convolutional neural network, learning features through convolution, pooling and activation layers, and obtaining a feature map; performing up-sampling on the feature map to obtain a feature map with the same size as the original image; The feature maps of the same size are predicted by softmax, and the category of the corresponding position is obtained, so as to realize the segmentation of the crack area. The feature learning and fracture region segmentation of the present invention are both independently realized by a deep convolutional neural network, as shown in FIG. 1 .

Fully convolutional neural network refers to the use of deep convolutional network to learn effective image semantic features, and finally generate a prediction map with the same size as the original image. The deep supervision network refers to the features learned by supervising each convolution stage and the fused features of each convolution stage, see the side output shown in Figure 2, and the loss cost of semantic prediction for statistically learned features, thus forming an overall supervision. In the present invention, the original image directly obtains the prediction result through the deep convolutional neural network, and the prediction result corresponds to each pixel on the original image. The prediction result here is to obtain the probability value of each pixel belonging to cracks and non-cracks at the same time.

The deep convolutional neural network for learning hierarchical features of the present invention has a multi-layer convolutional layer (Convolutional layer), and each layer of convolutional layer is followed by a batch normalization layer (Batch Normalization, BN) and a nonlinear activation layer (Rectified Linear Units). , ReLu). Convolution is divided into 5 convolution stages, the first convolution stage includes conv1_1 convolution layer and conv1_2 convolution layer; the second convolution stage includes conv2_1 convolution layer and conv2_2 convolution layer; the third convolution stage includes conv3_1 convolution layer, conv3_2 convolution layer and conv3_3 convolution layer; the fourth convolution stage includes conv4_1 convolution layer, conv4_2 convolution layer and conv4_3 convolution layer, and the fifth convolution stage includes conv5_1 convolution layer, conv5_2 convolution layer and conv5_3 convolutional layer. Each convolution stage uses a 3×3 convolution kernel window, and the movement step of the convolution kernel window is 1. At the end of the first 4 convolutional stages, after the conv1_2 convolutional layer, conv2_2 convolutional layer, conv3_3 convolutional layer, and conv4_3 convolutional layer, in addition to the BN batch normalization layer and the ReLu nonlinear activation layer, there is also pooling Layer (Pooling). The pooling layer uses a 2×2 convolution kernel window to downsample the feature map of the convolution stage. For the features learned in the last layer of each convolution stage, namely conv1_2 convolutional layer, conv2_2 convolutional layer, conv3_3 convolutional layer, conv4_3 convolutional layer and conv5_3 convolutional layer, upsample 0, 2, 4, 8 respectively , 16 times to get a feature map with the same size as the original image. Then reduce the dimension of the feature map through a convolutional layer, fuse the obtained five feature maps through the connection layer Concat layer, and then reduce the dimensionality through a layer of convolutional layer, and get the crack corresponding to each pixel after passing through Softmax and the predicted probability of non-crack.

In the convolution stage, the pooling layer is used to make the receptive field size of the convolution window corresponding to the original image in different convolution stages increase step by step, conv1_2 convolution layer, conv2_2 convolution layer, conv3_3 convolution layer, The conv4_3 convolutional layer and conv5_3 correspond to receptive fields on the original image of 5, 14, 40, 92, and 196, respectively. This feature makes the image features learned in different convolution stages show the characteristics of from shallow to deep, from low-level to abstract, which can automatically learn multi-scale and multi-level crack features.

The following will describe the data processing process of the deep convolutional neural network in conjunction with Table 1. See Table 1, Conv<a_b>-<c>-<d>, a represents the convolution stage, b represents the convolution layer number of the convolution stage, c represents the size of the convolution kernel, and d represents the number of convolution kernels. For example, Conv1_1-3-64 represents the convolutional layer conv1_1, which uses 64 convolution kernels, and the size of the convolution kernel used is 3×3 (that is, the convolution window size is 3×3). BN represents the batch normalization layer, ReLU represents the nonlinear activation layer, and Pool1, Pool2, Pool3, and Pool4 represent the pooling layer. Conv<a>-<c>-<d> where a, c, and d have the same meaning as the previous characters, for example, Conv1-3-1 means a convolution layer using a convolution kernel window with a size of 3×3 . Deconv<a>-<e> represents the deconvolution layer, e represents the multiple of upsampling, and Crop represents the cropping of the upsampling result so that the output feature map size remains consistent. Conv1 represents the output of the convolutional layer Conv1-3-1, and Deconv2, Deconv3, Deconv4, and Deconv5 represent the output of the deconvolutional layers Deconv2-2, Deconv3-4, Deconv4-8, and Deconv5-16, respectively.

In the present invention, a convolution kernel with a size of 3×3 is used in the convolution stage, and a circle of 0 is added to the periphery of the image, as shown in FIG. 3 . Among them, the gray color represents the convolution kernel window. In the case where the periphery of the image is not filled with 0, see Figure 3(a). The center of the convolution kernel window cannot be located at the edge pixel position of the image. Taking the edge pixel in the upper left corner as an example, the initial convolution The center of the kernel window can only reach (1, 1). If the height and width of the input image are M and N respectively, the height and width of the output image data can only be M-2 and N-2. In the case of adding 0 to the periphery of the image, see Figure 3(b), the center of the convolution kernel window can be located at the edge pixel position of the image. Still taking the upper left corner as an example, the center of the initial convolution kernel window can reach (0, 0). If the height and width of the input image are M and N respectively, the height and width of the output image are also M and N respectively.

Table 1 Schematic diagram of the data processing process of the deep convolutional neural network

The present invention includes a training phase and a testing phase. In the training phase, loss statistics are performed on the output of each softmax, wherein the loss cost of the Conv1, Deconv2, ... DeConv5 part statistics is the loss cost of the side output, and Conv-1-2 corresponds to the softmax is the loss cost after fusion. In the test phase, the softmax on the side input side is removed, and the rest is reserved.

In the present invention, for a convolution kernel window whose size is N×N, its weight matrix is denoted as Let the corresponding image pixel feature set when the convolution kernel window slides to a certain position be x, then the convolution operation result y is:

In formula (1):

x _{i, j} represent the eigenvalues of the i-th row and j-th column pixel in the convolution kernel window;

w _{i, j} represents the element in the i-th row and j-column in the weight matrix w, that is, the weight corresponding to x _{i, j} ;

b is the offset.

In the present invention, w and b are parameters to be optimized.

In the present invention, the batch normalization layer is used for batch normalization operation, which is a normalization method, and the K images input to the deep convolutional neural network are recorded as {p _k |k=1,2,. ..K}, perform batch normalization on the eigenvalues x _i,j,k output by the convolutional layer at the pixel position (i,j) of the k-th image:

In formula (2):

BN _γ,β ( _xi,j,k ) represents the result of the batch normalization operation;

The initial values of γ and β are respectively: β ₀ ＝E[ _xi,j ], during the training process, the gradient descent method is used to optimize γ and β;

ε is a constant close to 0, generally in the range of (1e-10,1e-3);

E[·] represents mean value,

Var[ ] means to find the variance,

In this embodiment, the nonlinear activation layer ReLu uses a nonlinear function to perform nonlinear activation processing, and the adopted nonlinear function f(x)=max(0,x).

In this embodiment, the pooling layer adopts the most value pooling, and for the pixel feature values in the convolution kernel window with a size of N×N, only the maximum value in the image pixel feature set x is retained, namely:

f(x)=max(x) (3)

In formula (3), f(x) represents the pooling operation result.

In the present invention, the connection layer is used to combine data, and is used to convert the feature matrix For fusion, the operation formula of the connection layer is as follows:

In formula (4), W, H, and c refer to the width, height, and number of channels of the feature matrix, respectively.

In the present invention, the softmax method is used to calculate the probability value of the category to which each pixel belongs. For D-dimensional feature vector x and weight vector Q, the predicted probability that feature vector x belongs to category a is:

There are only two classes in the present invention, non-crack class and crack class, ie a=0 or 1.

According to the features learned in each convolution stage of supervision, the loss cost of the prediction results of each convolution stage and the loss cost of the fusion prediction results are counted. In the present invention, the loss cost of crack and non-crack prediction results is balanced. First, the training sample set is collected Among them, R represents the number of training samples, I _r refers to the r-th original image input for training, that is, the r-th training sample; G _r is the manual label corresponding to I _r . Record all the parameters of the deep convolutional neural network as W, W includes the weight matrix and bias of all convolutional layers in the deep convolutional neural network; the parameters of the deep supervision part are w={(w ⁽¹⁾ ,w ⁽²⁾ ,...,w ^(M) )}, M is the total number of indexes, m represents the index number, w ^(m) respectively correspond to the network parameters of the side output part, therefore, the loss cost function of the side depth supervision part for:

In formula (6):

l _side (I,G,W,w ^(m) ) represents the image-level loss function of the deep supervision part;

Indicates the prediction result DSN-side output by the deep supervision part, that is, the probability map of cracks and non-cracks;

a _m is a constant coefficient, the default value is 1.0;

Δ( ) represents the cross-entropy loss function. Considering that more than 90% of an image is non-crack pixels, Δ( ) is defined as:

In formula (7):

ω ₀ and ω ₁ are the class balance weights;

|I| represents the number of pixels of the training sample image I;

I(k) and G(k) refer to the value corresponding to the kth pixel in I and G respectively;

logPr(*) refers to the logarithm of the result of the softmax function Pr(*).

For a training sample, count the number of non-crack pixels and crack pixels therein, which are recorded as C ₀ and C ₁ respectively. In the present invention, ω ₀ =1, ω ₁ =C ₀ /C ₁ .

In addition to the supervision of the side output part, the final fusion prediction part will also count the loss during the training process. The loss function L _fuse (I, G, W) of this part is recorded as:

Therefore, the overall loss cost function L(I,G,W) is written as:

L(I,G,W)=L _side (I,G,W,w)+L _fuse (I,G,W) (9)

The overall optimization goal of the present invention is to obtain the parameters that minimize the overall loss cost value L (I, G, W):

Claims (6)

1. the image crack dividing method based on depth convolutional neural networks, it is characterized in that：

The depth convolutional neural networks used include 5 convolution stages, wherein, first convolution stage includes convolutional layer Conv1_1 and convolutional layer Conv1_2, convolutional layer Conv1_1 and convolutional layer Conv1_2 are respectively using 64 convolution kernels；Second The convolution stage includes convolutional layer Conv2_1 and convolutional layer Conv2_2, and convolutional layer Conv2_1 and convolutional layer Conv2_2 are used respectively 128 convolution kernels；3rd convolution stage includes convolutional layer Conv3_1, convolutional layer Conv3_2 and convolutional layer Conv3_3, convolution Layer Conv3_1, convolutional layer Conv3_2 and convolutional layer Conv3_3 are respectively using 256 convolution kernels；4th convolution stage includes Convolutional layer conv4_1, convolutional layer conv4_2 and convolutional layer conv4_3, convolutional layer conv4_1, convolutional layer Conv4_2 and convolution Layer Conv4_3 is respectively using 512 convolution kernels；5th convolution stage include convolutional layer Conv5_1, convolutional layer Conv5_2 and Convolutional layer Conv5_3, convolutional layer conv5_1, convolutional layer conv5_2 and convolutional layer conv5_3 are respectively using 512 convolution kernels； Successively immediately following batch standardization after convolutional layer conv1_2, convolutional layer conv2_2, convolutional layer conv3_3 and convolutional layer conv4_3 Layer, nonlinear activation layer, pond layer；Successively immediately following batch normalization layer, nonlinear activation layer after other convolutional layers；

Batch normalization layer closelyed follow after convolutional layer conv1_1, nonlinear activation layer are designated as first normalization layer, the first non-thread Property active coating；Batch normalization layer closelyed follow after convolutional layer conv1_2, nonlinear activation layer, pond layer are designated as second batch standard Change layer, the second nonlinear activation layer, the first pond layer；

Batch normalization layer closelyed follow after convolutional layer conv2_1, nonlinear activation layer are designated as the 3rd batch of normalization layer, the 3rd non-thread Property active coating；Batch normalization layer closelyed follow after convolutional layer conv2_2, nonlinear activation layer, pond layer are designated as the 4th batch of standard Change layer, the 4th nonlinear activation layer, the second pond layer；

Batch normalization layer closelyed follow after convolutional layer conv3_1, nonlinear activation layer are designated as the 5th batch of normalization layer, the 5th non-thread Property active coating；Batch normalization layer closelyed follow after convolutional layer conv3_2, nonlinear activation layer are designated as the 6th batch of normalization layer, the Six nonlinear activations layer；Batch normalization layer closelyed follow after convolutional layer conv3_3, nonlinear activation layer, pond layer are designated as the 7th Criticize normalization layer, the 7th nonlinear activation layer, the 3rd pond layer；

Batch normalization layer closelyed follow after convolutional layer conv4_1, nonlinear activation layer are designated as the 8th batch of normalization layer, the 8th non-thread Property active coating；Batch normalization layer closelyed follow after convolutional layer conv4_2, nonlinear activation layer are designated as the 9th batch of normalization layer, the Nine nonlinear activations layer；Batch normalization layer closelyed follow after convolutional layer conv4_3, nonlinear activation layer, pond layer are designated as the tenth Criticize normalization layer, the tenth nonlinear activation layer, the 4th pond layer；

Batch normalization layer closelyed follow after convolutional layer conv5_1, nonlinear activation layer are designated as the tenth a collection of normalization layer, the 11st Nonlinear activation layer；Batch normalization layer closelyed follow after convolutional layer conv5_2, nonlinear activation layer are designated as the 12nd batch of standardization Layer, the 12nd nonlinear activation layer；By batch normalization layer closelyed follow after convolutional layer conv5_3, nonlinear activation layer, pond layer note For the 13rd batch of normalization layer, the 13rd nonlinear activation layer；

Original image is inputted from convolutional layer conv1_1；In first convolution stage, original image sequentially through convolutional layer conv1_1, First normalization layer, the first nonlinear activation layer, convolutional layer conv1_2, second batch normalization layer, the second nonlinear activation layer Export fisrt feature figure；Fisrt feature figure inputs second convolution stage after the first pond layer, sequentially through convolutional layer conv2_ 1st, the 3rd batch of normalization layer, the 3rd nonlinear activation layer, convolutional layer conv2_2, the 4th batch of normalization layer, the 4th nonlinear activation Layer output second feature figure；Second feature figure inputs the 3rd convolution stage after the second pond layer, sequentially through convolutional layer Conv3_1, the 5th batch of normalization layer, the 5th nonlinear activation layer, convolutional layer conv3_2, the 6th batch of normalization layer, the 6th non-thread Property active coating, convolutional layer conv3_3, the 7th batch of normalization layer, the 7th nonlinear activation layer output third feature figure；Third feature Figure inputs the 4th convolution stage after the 3rd pond layer, sequentially through convolutional layer conv4_1, the 8th batch of normalization layer, the 8th non- Linear active coating, convolutional layer conv4_2, the 9th batch of normalization layer, the 9th nonlinear activation layer, convolutional layer conv4_3, the tenth batch Normalization layer, the tenth nonlinear activation layer output fourth feature figure；Fourth feature figure inputs the 4th volume after the 4th pond layer Product the stage, sequentially through convolutional layer conv5_1, the tenth a collection of normalization layer, the 11st nonlinear activation layer, convolutional layer conv5_2, 12nd batch of normalization layer, the 12nd nonlinear activation layer, convolutional layer conv5_3, the 13rd batch of normalization layer, the 13rd non-thread Property active coating output fifth feature figure；

To fisrt feature figure, second feature figure, third feature figure, fourth feature figure, fifth feature figure respectively through convolutional layer Conv1, convolutional layer Conv2, convolutional layer Conv3, convolutional layer Conv4, convolutional layer Conv5, Conv1, convolutional layer Conv2, convolution Layer Conv3, convolutional layer Conv4, convolutional layer Conv5 output characteristic figure respectively through layer Deconv2 that deconvolute, Deconv3, Deconv4, Deconv5 are deconvoluted, up-sampled successively, are obtained and original image size identical characteristic pattern；By five with Original image size identical characteristic pattern is merged by articulamentum, and the output of articulamentum passes sequentially through convolutional layer Conv-1-2 drops Dimension, the prediction probability by softmax functions each corresponding crack of pixel of acquisition and non-crack, are designated as fusion forecasting probability, from And realize the image crack segmentation of original image；

Second nonlinear activation layer, deconvolute layer Deconv2, Deconv3, Deconv4, Deconv5 output are designated as side Output；

Described depth convolutional neural networks are adopted to be trained with the following method：

Gather training sample setTraining sample set is inputted into depth convolutional neural networks, wherein, I_rRepresent the R training samples, G_rFor I_rCorresponding artificial mark, counts the quantity of non-crack pixel and crack pixel in each training sample, point C is not designated as it₀And C₁；

To lose the minimum parameter as objective optimization depth convolutional neural networks of cost value, described loss cost function is：

L (I, G, W)=L_side(I,G,W,w)+L_fuse(I,G,W)

Wherein：

L_side(I, G, W, w) represent side loss cost,

<mrow> <mi>&amp;Delta;</mi> <mo>=</mo> <mo>-</mo> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> </mrow> </mfrac> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> </mrow> </munderover> <mo>{</mo> <msub> <mi>&amp;omega;</mi> <mn>0</mn> </msub> <mi>log</mi> <mi> </mi> <mi>Pr</mi> <mrow> <mo>(</mo> <mi>G</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>=</mo> <mn>0</mn> <mo>|</mo> <mi>I</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>;</mo> <mi>W</mi> <mo>,</mo> <msup> <mi>w</mi> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </msup> <mo>)</mo> </mrow> <mo>}</mo> <mo>+</mo> <msub> <mi>&amp;omega;</mi> <mn>1</mn> </msub> <mi>log</mi> <mi> </mi> <mi>Pr</mi> <mrow> <mo>(</mo> <mi>G</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>=</mo> <mn>1</mn> <mo>|</mo> <mi>I</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>;</mo> <mi>W</mi> <mo>,</mo> <msup> <mi>w</mi> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </msup> <mo>)</mo> </mrow> <mo>}</mo> </mrow>

ω₀And ω₁It is class balance weight, ω₀=1, ω₁=C₀/C₁；

I (k) and G (k) are the corresponding values of k-th of pixel for referring to I and G respectively, and logPr (*) refers to softmax functions；

L_fuse(I, G, Q) represents fusion loss cost,

<mrow> <msub> <mi>L</mi> <mrow> <mi>f</mi> <mi>u</mi> <mi>s</mi> <mi>e</mi> </mrow> </msub> <mrow> <mo>(</mo> <mi>I</mi> <mo>,</mo> <mi>G</mi> <mo>,</mo> <mi>W</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> </mrow> </mfrac> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mo>|</mo> <mi>I</mi> <mo>|</mo> </mrow> </munderover> <mo>{</mo> <msub> <mi>&amp;omega;</mi> <mn>0</mn> </msub> <mi>log</mi> <mi> </mi> <mi>Pr</mi> <mrow> <mo>(</mo> <mi>G</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>=</mo> <mn>0</mn> <mo>|</mo> <mi>I</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>;</mo> <mi>W</mi> <mo>)</mo> </mrow> <mo>}</mo> <mo>+</mo> <msub> <mi>&amp;omega;</mi> <mn>1</mn> </msub> <mi>log</mi> <mi> </mi> <mi>Pr</mi> <mrow> <mo>(</mo> <mi>G</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>=</mo> <mn>1</mn> <mo>|</mo> <mi>I</mi> <mo>(</mo> <mi>k</mi> <mo>)</mo> <mo>;</mo> <mi>W</mi> <mo>)</mo> </mrow> <mo>}</mo> <mo>.</mo> </mrow>

2. the image crack dividing method as claimed in claim 1 based on depth convolutional neural networks, it is characterized in that：

Convolutional layer conv1_1 and convolutional layer conv1_2 are respectively using 64 convolution kernels, convolutional layer conv2_1 and convolutional layer Conv2_2 is respectively using 128 convolution kernels, and convolutional layer conv3_1, convolutional layer conv3_2 and convolutional layer conv3_3 are used respectively 256 convolution kernels, convolutional layer conv4_1, convolutional layer conv4_2 and convolutional layer conv4_3 are respectively using 512 convolution kernels, volume Lamination conv5_1, convolutional layer conv5_2 and convolutional layer conv5_3 are respectively using 512 convolution kernels.

3. the image crack dividing method as claimed in claim 1 based on depth convolutional neural networks, it is characterized in that：

All convolutional layers are using the convolution kernel window that size is 3 × 3.

4. the image crack dividing method as claimed in claim 1 based on depth convolutional neural networks, it is characterized in that：

All pond layers use size to carry out maximum pond for 2 × 2 convolution kernel window.

5. the image crack dividing method as claimed in claim 1 based on depth convolutional neural networks, it is characterized in that：

All batch normalization layers use formulaCharacteristic pattern after convolutional layer processing is criticized Normalizing operation, wherein, BN_γ,β(x_i,j,k) represent to criticize the result of normalizing operation；ε is levels off to 0 Constant,β=E [x],Note is each The image of convolutional layer has K, and convolutional layer refers to the convolutional layer before batch normalization layer, x here_i,j,kRepresent the kth of convolutional layer output The characteristic value of image pixel positions (i, j).

6. the image crack dividing method as claimed in claim 1 based on depth convolutional neural networks, it is characterized in that：

Described nonlinear activation layer using nonlinear function f (x)=max (0, x), wherein, x represents the defeated of batch normalization layer Go out.