CN111598854A - Complex texture small defect segmentation method based on rich robust convolution characteristic model - Google Patents

Abstract

The invention discloses a method for segmenting small defects of complex textures based on a rich and robust convolution feature model. The image of the segmented object undergoes feature reorganization to obtain the feature map of each side output layer; the feature map of each side output layer is sequentially connected to a deconvolution layer and a fine loss function of the side output layer, and the output layer of each stage is obtained. Predict the feature map; at the same time, a fusion layer is added to the model to fuse the feature maps after deconvolution of the feature maps of all side output layers, and then connect a fusion layer fine loss function to obtain the final prediction map to achieve defect segmentation. This method solves the problem of inaccurate prediction results caused by the unbalanced ratio between target pixels and background pixels, and can predict fine targets.

Description

A Segmentation Method for Small Defects in Complex Textures Based on Rich Robust Convolutional Feature Models

technical field

The invention relates to the technical field of lithium battery surface defect detection, in particular to a segmentation method for small defects of complex textures based on a rich robust convolution feature model.

Background technique

Lithium battery surface defect detection has become an important technical means of lithium battery surface quality control. The surface quality of lithium battery can not only improve the service life of battery components, but also improve the power generation efficiency of lithium battery.

For the crack segmentation method based on convolutional neural network (CNN), there are usually two problems, one is that the crack is missed or falsely detected, and the other is that the predicted crack segmentation result is coarse and requires complex post-processing to obtain Fine cracks. The complex non-uniform texture background on the surface of the lithium battery is one of the main reasons for the above problems. Another reason is that the proportion of crack pixels and background pixels in the defect image is extremely unbalanced. For example, the size of the defect image is 1 million pixels, and the defect image Pixels only occupy dozens or even a dozen pixels.

The information obtained through different convolutional layers becomes rougher and more "global" as the number of layers becomes deeper. The lower convolutional layer contains complex random texture background and target detail information, and the distinction between target information and background It is not obvious that what the network learns is only some indistinguishable information such as shape and corner features. Important target information is preserved in the higher convolutional layers, while the intermediate convolutional layers contain essential information. target details. However, the general convolutional neural network model only uses the output features of the last convolutional layer or the convolutional layers before each stage pooling layer, ignoring the target details contained in the intermediate convolutional layers; for crack segmentation, The key problem it faces is that the similarity between the background and target information is very high, and excessive fusion will inevitably lead to serious false detections.

Although the segmentation method based on convolutional neural network is good at predicting features such as contours and edges rich in semantic information, it can be seen from the analysis that the prediction result of crack segmentation based on convolutional neural network directly is much thicker than the labeled cracks of real labels. As a result, crack pixels cannot be accurately located. The problem of predicting cracks, edges, contours, or lines that are too thick is rarely discussed in the existing literature. One possible reason is that these methods usually apply post-processing methods that thin the cracks, edges, contours, or lines after generating the initial prediction results. In order to obtain prediction results close to the real labels, the width of the processed prediction results seems to have no effect on the results, which will actually reduce the prediction accuracy. Therefore, in some detection tasks that require high pixel-level accurate positioning, cannot meet demand.

The loss function is used to evaluate the degree of inconsistency between the predicted value of the model and the real value. The smaller the loss function, the better the robustness of the model, and the loss function can guide the learning of the model. However, the crack defect image of lithium battery is extremely unbalanced in the proportion of crack pixels and background pixels, resulting in negative samples (background pixels) occupying a large proportion of the model loss, which will make the learning process fall into the local minimum value of the loss function, resulting in more accurate predictions. It is biased towards background pixels, and the trained model cannot detect cracks, which is an "uneasy event". Therefore, the unbalanced proportion of crack pixels and background pixels in the crack defect image of lithium batteries has an impact on the loss function, which is the cause of cracks. The root cause of coarse segmentation results.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the technical problem to be solved by the present invention is to provide a segmentation method for small defects of complex textures based on a rich and robust convolution feature model.

The technical scheme adopted by the present invention to solve the technical problem is:

A method for segmenting small defects of complex textures based on a rich robust convolution feature model, characterized in that the method includes acquiring an image containing an object to be segmented, and using the rich robust convolution feature model to segment the image containing the object to be segmented Perform feature reorganization to obtain the feature map of each side output layer; the feature map of each side output layer is sequentially connected to a deconvolution layer and a fine loss function of the side output layer to obtain the predicted feature map of the side output layer of each stage;

At the same time, a fusion layer is added to the model to fuse the feature maps after deconvolution of the feature maps of all side output layers, and then connect a fusion layer fine loss function to obtain the final prediction map to achieve defect segmentation;

Among them, the fine loss function of the side output layer satisfies formula (1):

P _side =σ(A _side ), A _side ={a _j , j=1,...|Y|} (2)

In the formula, L ^(k) (P _side , G) represents the distance loss function of the k-th stage; L (W, w ^(k) ) represents the weighted cross-entropy loss function of the k-th stage; P _side represents the k-th stage side output layer prediction feature map; σ is the sigmoid activation function; A _side represents the set of activation values at all pixels of the prediction feature map of the output layer of the k stage side; a _j represents any one of the prediction feature maps of the output layer of the kth stage side The activation value at pixel j; Y represents the sum of defective and non-defective pixels in the figure;

The fine loss function of the fusion layer is obtained by the following formula:

L _fuse (W, w)=L _c (P _fuse , G) (3)

In the formula, L _c represents the standard cross-entropy loss function; P _fuse represents the fusion of the prediction feature maps of the output layers of the k stages, that is, the weight of the fusion layer; K represents the total number of stages;

Use the argmin function to summarize the fine loss function of the fusion layer and the fine loss function of the side output layer of all stages to obtain the objective function L, which is expressed by formula (5);

Finally, the objective function is optimized to obtain the fine loss function of the side output layer and the weight of the fine loss function of the fusion layer.

The specific process of feature reorganization using a rich and robust convolutional feature model is as follows:

On the basis of the original ResNet40 network, the fully connected layer and the fifth-stage pooling layer are removed. The first-stage identification block layer and the second-stage identification block layer of the original ResNet40 network are respectively connected to a convolutional layer laterally, and the first stage is obtained. , the feature map of the output layer of the two-stage side;

Connect a convolutional layer laterally after each block layer in the third, fourth and fifth stages of the original ResNet40 network to obtain the feature map after convolution of the respective block layer, and then convolve all the block layers in the same stage. The feature maps of , respectively, are added element by element to obtain the feature map of the output layer on the corresponding stage side.

The convolution kernel size of the convolutional layers connected laterally in the first and second stages of the identification block layer is 1 × 1, the stride and the number of channels are both 1; in the third, fourth and fifth stages, the lateral The connected convolutional layers have a kernel size of 1 × 1, a stride of 1, and a number of channels of 21.

The original ResNet40 network includes 40 convolutional layers and a fully connected layer at the last layer of the network, which is divided into 5 stages, each stage includes a convolutional block layer and one or more identification block layers, of which the first , the second stage respectively contains a convolution block layer and a marker block layer, the third, fourth and fifth stages respectively include a convolution block layer and two marker block layers, each convolution block layer and marker block layer contains Multiple convolutional layers; each stage adds a pooling layer with a pooling window size of 2×2 and stride 1 after all the marker block layers.

The specific structure of the original ResNet40 network is: first, the input target image is successively passed through the convolution kernel size of 5 × 5, the step size is 1, the number of channels is 32, and the convolution kernel size is 2 × 2, the step size is 2 The input features of the first stage are obtained after the maximum pooling layer of The output features of the first-stage convolution block layer are obtained after the three convolutions and a convolution kernel size of 1 × 1, stride of 1, and channel number of 32 residuals are connected to obtain the output features of the first-stage convolutional block layer; The output features are successively passed through three convolutions with the convolution kernel size of 1×1, 3×3, and 1×1, the step size is 1, and the number of channels is 32 to obtain the output features of the first-stage identification block layer; The output features of the first-stage identification block layer are passed through a pooling layer with a convolution kernel size of 2 × 2 and a stride of 2 to obtain the output features of the first stage;

The output features of the first stage go through three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, the stride size is 1, and the number of channels is 64, and a convolution kernel size is 1. The output features of the second-stage convolution block layer are obtained after residual connections with a step size of 1, a stride of 1, and a channel number of 64; After three convolutions of 3 × 3, 1 × 1, the step size is 1, and the number of channels is 64, the output features of the second-stage identification block layer are obtained; the output features of the second-stage identification block layer are passed through a convolution kernel. The output features of the second stage are obtained after a pooling layer with a size of 2×2 and a stride of 2;

The output features of the second stage go through three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, the stride size is 1, the number of channels is 256, and a convolution kernel size is 1. The output features of the third-stage convolution block layer are obtained after residual connections with a step size of 1, a stride of 1, and a channel number of 256; After three convolutions of 3 × 3, 1 × 1, the step size is 1, and the number of channels is 256, the output features of the first identification block layer in the third stage are obtained; the output features of the first identification block layer in the third stage are sequentially After three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, stride of 1, and number of channels of 256, the output features of the second identification block layer in the third stage are obtained; The output features of the second identification block layer in the stage are passed through a pooling layer with a convolution kernel size of 2 × 2 and a stride of 2 to obtain the output features of the third stage;

The operation process of the fourth stage is the same as that of the third stage. The output features of the third stage are repeated to obtain the output features of the fourth stage after repeating the operation of the third stage;

The operation process of the fifth stage is the same as the operation process of the convolution block layer and the two identification block layers in the fourth stage. The output features of the fourth stage repeat the operations of the convolution block layer and the two identification block layers in the fourth stage. Get the output features of the fifth stage.

A segmentation method of complex texture small defects based on a rich robust convolutional feature model, the specific steps of the method are:

S1 image preprocessing

Collect images containing defects to be segmented, and normalize the collected images to 1024×1024 pixels; add pixel-level labels to the normalized images, and these labeled images are the target images; The proportion is divided into different sample sets;

S2 builds a rich and robust convolutional feature model

Based on the original ResNet40 network, a convolutional layer is laterally connected to the first-stage identification block layer and the second-stage identification block layer of the original ResNet40 network to obtain the feature maps of the first and second-stage output layers;

Connect a convolutional layer laterally after each block layer in the third, fourth and fifth stages of the original ResNet40 network to obtain the feature map after convolution of the respective block layer, and then convolve all the block layers in the same stage. The feature maps are added element by element to obtain the feature map of the output layer of the corresponding stage;

The feature maps of the output layers of the above five stages are respectively connected to a deconvolution layer (deconv) for upsampling to obtain the feature maps after deconvolution of each stage, and the feature maps after deconvolution of each stage are respectively connected. A fine loss function of the side output layer performs pixel-by-pixel classification to obtain the predicted feature map of the side output layer at each stage;

Connect the feature maps after deconvolution of the above stages together, and then fuse all the feature maps after deconvolution through a convolution layer with a convolution kernel size of 1 × 1 and a stride of 1 to obtain a fusion layer feature map; The fusion layer feature map is finally connected to a fusion layer fine loss function to obtain the final predicted feature map;

S3 model training and testing

Initialize the model parameters, input the target image for training and its corresponding pixel-level label; in the model training process, the loss is passed to the weight of each convolutional layer through the stochastic gradient descent method, and its weight value is updated, the stochastic gradient descent method The momentum is 0.9, and the weight decay is 0.0005; 1 image is randomly sampled during each training process, and the training is stopped when the number of iteration cycles reaches 100 cycles, and the training of the model is completed;

The target image for testing is scaled to 1024x1024 pixels, and the scaled target image is input into the model after training; the test time for a single image is 0.1s, and the operation of the model is repeated to complete the model test.

The objects to be segmented are cracks, edges or linear structures.

Compared with the prior art, the beneficial effects of the present invention are:

From the perspective of rational utilization of convolution features and design of loss functions, the present invention aims to make the model learn as rich and complete defect features as possible, and predict robust fine defects without using post-processing methods. , so that the defect features learned by the model can generate a predicted feature map that is as similar as possible to the real label; therefore, the present invention builds a rich and robust convolutional feature model based on the original ResNet40 network, and uses it in the Keras1.13 deep learning framework. The end-to-end deep learning is performed under the model. The model adopts a network structure of multi-scale and multi-level features, and more high-level features (third, fourth, and fifth stages) are fused, and lower layers (first, second, and second stages) are fused less. At the same time, the convolution kernel size of 1x1 is used for superposition fusion at each stage, and all convolution features are encapsulated into a richer and more robust expression, which improves the expression ability of features; and uses The output feature map of the intermediate layer in each stage overcomes the fact that the existing conventional convolutional neural network model only uses the output features of the last convolutional layer or the convolutional layer before the pooling layer of each stage and ignores the target details contained in the intermediate layer. Information flaws.

Aiming at the problem of inaccurate prediction results caused by the unbalanced ratio between cracked pixels and non-cracked pixels, the present invention introduces the fine loss function of the side output layer in the side output layer of each stage in order to predict the fine crack. The fusion layer introduces the fusion layer fine loss function, and the side output layer fine loss function combines the weighted cross entropy loss function and the distance loss function. Layer fine loss function optimizes the weight of each convolutional layer in the side output layer; the fusion layer fine loss function fuses the side output layer fine loss function to refine and finally predict the defect features in the feature map, and at the same time, in the model training process, the fusion layer The fine loss function optimizes the weights of each convolutional layer in the fusion layer, and realizes the prediction of cracks from global to local.

From the experimental results, compared with the traditional filter segmentation method and the conventional convolutional neural network, the rich and robust convolutional feature model can predict finer cracks, and the crack segmentation recognition accuracy can reach 79.64%.

This method can provide ideas for the segmentation of targets with extreme aspect ratios similar to crack structures and high requirements for fineness.

Description of drawings

Fig. 1 is the network structure diagram of the rich and robust convolution feature model of the present invention;

Fig. 2 is the crack segmentation result diagram of different segmentation methods of the present invention;

FIG. 3 is a comparison diagram of evaluation results of different segmentation methods of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Hereinafter, the method of the present application will be described in detail by taking the application in the lithium battery for the surface crack defect of the lithium battery as an example.

The present invention provides a segmentation method (method for short) of small defects in complex textures based on a rich and robust convolution feature model, comprising the following steps:

S1 image preprocessing

S1-1 get image

The lithium battery image is collected by a 1.4 million near-infrared camera. The actual size of the lithium battery image collected by the present invention is 165mm×165mm, and the collected image is normalized to 1024×1024 pixels. The image is subjected to a complex preprocessing process to ensure that the size can be used for model input after normalization; the setting of the image size is almost the same as the image size collected by the original camera, which can better preserve the original image information, and there is no complicated The processing process improves the processing speed of the algorithm and meets the real-time requirements of production line detection; the original image includes the image containing the object to be segmented and the image that does not contain the object to be segmented;

S1-2 Make Image Labels

Use Labelimg software to manually label all the original images containing the objects to be segmented in step S1-1, and add pixel-level labels. The pixel-level labels contain the area size and spatial position information of the defects. These labeled images are used. target images for model training, testing and validation;

S1-3 Making a sample set

The target images in step S1-2 are grouped, and 20% (default value) of the target images are randomly selected as the test sample set, and the remaining target images are randomly divided into the training sample set and the verification sample set according to the ratio of 4:1;

S2 builds a rich and robust convolutional feature model (Rich and Robust Convolutional Features, RRCF)

S2-1 original ResNet40 network

The present invention is based on the improvement of the original ResNet40 network. The original ResNet40 network includes 40 convolutional layers (Conv) and a fully connected layer (Fully Connected Layer) located at the last layer of the network. It is mainly divided into 5 stages (Stage), each Each stage includes a convolution block layer (Conv Block) and one or more identity block layers (IdentityBlock), which are represented by stagek_blockm, k represents the number of stages, m represents the number of block layers in the corresponding stage, and the first and second stages are respectively It consists of a convolution block layer and a marker block layer. The third, fourth, and fifth stages respectively include a convolution block layer and two marker block layers. Each convolution block layer and marker block layer contains multiple convolution blocks. layer; in each stage, a pooling layer with a pooling window size of 2×2 and a stride of 1 is added after all the identification block layers; the specific parameters of each convolutional layer of the original ResNet40 network are shown in Table 1;

First, the input target image goes through a convolution with a kernel size of 5×5, a stride of 1, and a number of channels of 32, and a max pooling layer with a kernel size of 2×2 and a stride of 2 (Maxpool). Then, the input features of the first stage are obtained; the input features of the first stage are successively subjected to three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, the stride size is 1, and the number of channels is 32. And a residual connection (Shortcut) with a convolution kernel size of 1 × 1, a stride of 1, and a channel number of 32 to obtain the output features of the first-stage convolutional block layer; the output features of the first-stage convolutional block layer After successively going through three convolutions with the convolution kernel size of 1×1, 3×3, and 1×1, the step size is 1, and the number of channels is 32, the output features of the first-stage identification block layer are obtained; The output features of the identification block layer go through a pooling layer with a convolution kernel size of 2 × 2 and a stride of 2 to obtain the output features of the first stage;

The output features of the first stage go through three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, the stride size is 1, and the number of channels is 64, and a convolution kernel size is 1. The output features of the second-stage convolution block layer are obtained after residual connections with a step size of 1, a stride of 1, and a channel number of 64; After three convolutions of 3 × 3, 1 × 1, the step size is 1, and the number of channels is 64, the output features of the second-stage identification block layer are obtained; the output features of the second-stage identification block layer are passed through a convolution kernel. The output features of the second stage are obtained after a pooling layer with a size of 2×2 and a stride of 2;

The output features of the second stage go through three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, the stride size is 1, the number of channels is 256, and a convolution kernel size is 1. The output features of the third-stage convolution block layer are obtained after residual connections with a step size of 1, a stride of 1, and a channel number of 256; After three convolutions of 3 × 3, 1 × 1, the step size is 1, and the number of channels is 256, the output features of the first identification block layer in the third stage are obtained; the output features of the first identification block layer in the third stage are sequentially After three convolutions with convolution kernel sizes of 1×1, 3×3, and 1×1, stride of 1, and number of channels of 256, the output features of the second identification block layer in the third stage are obtained; The output features of the second identification block layer in the stage are passed through a pooling layer with a convolution kernel size of 2 × 2 and a stride of 2 to obtain the output features of the third stage;

The operation process of the fourth stage is the same as that of the third stage. The output features of the third stage are repeated to obtain the output features of the fourth stage after repeating the operation of the third stage;

The operation process of the fifth stage is the same as the operation process of the convolution block layer and the two identification block layers in the fourth stage. The output features of the fourth stage repeat the operations of the convolution block layer and the two identification block layers in the fourth stage. Get the output features of the fifth stage;

The input target image is calculated layer by layer to extract features. For example, the size of the target image is 1024 × 1024 × 32, of which the length and width are 1024, and the number of channels is 32; the size of the convolution kernel is 5 × 5, and the step size is 1. , the output size after convolution with a channel number of 32 is 1024×1024×32; then the output size after max pooling with a convolution kernel size of 2×2 and a stride of 2 is 512×512×32, That is, the input feature size of the first stage is 512×512×32; after the above operations, the output feature size of the first stage is 256×256×32, the output feature size of the second stage is 128×128×64, and the third stage’s output feature size is 128×128×64. The output feature size of the stage is 64×64×256, the output feature size of the fourth stage is 32×32×256, and the output feature size of the fifth stage is 16×16×256;

Table 1 Specific parameters of the original ResNet40 network

In the table, Identity Block×2 means that the Identity Block operation has been performed twice;

Feature Map Reorganization for S2-2 Rich Robust Convolutional Feature Model

On the basis of the original ResNet40 network constructed in step S2-1, the fully connected layer and the fifth-stage pooling layer are removed; on the one hand, the fully connected layer is removed to provide a fully convolutional network whose output is predicted from image to image, At the same time, the computational complexity of the model can be reduced; on the other hand, the pooling layer in the fifth stage will double the step size and affect the positioning of defects; although the pooling layer will affect the positioning, the pooling layer is mainly used in the first four stages. To speed up training;

In the original ResNet40 network, the first-stage identification block layer (stage1_block2) and the second-stage identification block layer (stage2_block2) are respectively connected laterally to a convolutional layer with a convolution kernel size of 1×1 and a stride and channel number of 1. Reduce the dimensionality of the number of channels, and obtain the feature maps of the output layers of the first and second stages respectively, so as to realize the integration of feature information;

After each block layer in the third, fourth and fifth stages of the original ResNet40 network, namely stage3_block1, stage3_block2, stage3_block3, stage4_block1, stage4_block2, stage4_block3, stage5_block1, stage5_block2, stage5_block3, a convolution kernel size of 1× 1. A convolutional layer with a stride of 1 and a channel number of 21, obtains the feature maps after the convolution of the respective block layers, and then adds the feature maps after the convolution of all the block layers in the same stage element by element to obtain the corresponding stage. The feature map of the side output layer;

S2-3 Build the predicted feature map of a rich and robust convolutional feature model

The feature maps of the output layers of the above five stages are respectively connected to a deconvolution layer (deconv) for upsampling, so as to predict the feature maps of the same scale as the target image, and obtain the feature maps after deconvolution of the respective stages. The feature map after deconvolution retains the spatial location information of defects in the target image;

The feature map after deconvolution at each stage is connected to a fine loss function of the side output layer for pixel-by-pixel classification, and the predicted feature map of the side output layer of each stage is obtained, that is, for each feature map on the deconvolved feature map. All pixels are classified, and the defect features in the prediction feature map of the fine side output layer are refined, and the weight of each convolutional layer in the side output layer is optimized through the fine loss function of the side output layer during the model training process;

In order to directly utilize the predicted feature maps of the output layer at each stage, a fusion layer is added to the model, and the fusion layer weights are learned during the training process. Then, all the deconvolutional feature maps are fused through a convolutional layer with a convolution kernel size of 1×1 and a stride of 1 to obtain a fusion layer feature map; the fusion layer feature map is finally connected to a fusion layer fine loss function to obtain the final Predict the feature map, refine the defect features in the final prediction feature map, and optimize the weights of each convolutional layer in the fusion layer through the fusion layer fine loss function during the model training process; the final predicted feature map is the rich and robust convolutional feature model. The predicted feature map of ;

Design of S3 Fine Loss Function

S3-1 weighted cross entropy loss function

Due to the complex texture background in the defect image, the distribution of small defects on the pixels is very uneven, and most of the pixels are randomly distributed non-defect pixels, that is, the background, such as crack pixels and non-crack pixels, so directly use the cross entropy loss function to Defective pixels cannot be accurately segmented from non-defective pixels; the weighted cross-entropy loss function (weighted cross-entropy loss) introduces a class-balanced weight coefficient β to offset the imbalance between defective and non-defective pixels, each The loss of pixels satisfies equation (1):

β=|Y _- |/|Y|, 1-β=|Y ₊ |/|Y| (2)

In the formula, X represents the target image; W represents the set of all network layer parameters; w ^(k) represents the weight of the predicted feature map of the output layer of the kth stage; Y ₊ and Y _- represent defective pixels and non-defective pixels, respectively; β represents the class balance weight coefficient; Y=Y ₊ and Y _{_} sum; y _j represents any pixel in the target image; Pr(y _j =1|X; W, w ^(k) ) represents the use of sigmoid at pixel y _j The category score calculated by the activation function, and Pr∈[0,1];

S3-2 distance loss function (Dice loss, written as Dice loss function)

Given a target image X and the corresponding real label G, the predicted image of the target image X is P, the distance loss function (Dice loss function) can compare the similarity between the predicted image P and the real label G, and can minimize the two. The distance between the two, the Dice loss function (Dist(P,G)) formula is as follows:

Among them, p _j ∈ P, is any pixel in the predicted image P; g _j ∈ G, is any pixel in the real label G; N represents the total number of pixels in the target image;

Design of S3-3 Fine Loss Function

In order to obtain better defect prediction performance, a Precise Loss Function that combines the weighted cross-entropy loss function and the Dice loss function is proposed; the Dice loss function is considered as an image-level loss, focusing on two groups of The similarity between image pixels, the Dice loss function can reduce redundant information, which is the key to generating fine cracks in this application, and the Dice loss function is prone to incomplete prediction and loss of targets, such as a predicted crack missing part; The weighted cross-entropy loss function focuses on pixel-level dissimilarity, because it is the sum of the distances between each corresponding pixel between the predicted image and the true label, and the prediction is comprehensive and will not cause target loss, but the weighted cross-entropy loss function is easy to Introducing more background information, resulting in inaccurate prediction results; therefore, the combination of the two can minimize the distance between the image level and the pixel level, and achieve global to local prediction;

In order to obtain a more refined prediction feature map of the side output layer at each stage, a fine loss function of the side output layer is proposed, which satisfies the following formula:

P _side =σ(A _side ), A _side ={a _j , j=1,...|Y|} (5)

Among them, L ^(k) (P _side , G) represents the distance loss function of the k-th stage; L (W, w ^(k) ) represents the weighted cross-entropy loss function of the k-th stage; P _side represents the k-th stage side output layer σ is the sigmoid activation function; A _side represents the set of activation values at all pixels of the prediction feature map of the output layer of the k stage side; a _j represents any pixel in the prediction feature map of the output layer of the kth stage side activation value at j;

The fine loss function of the fusion layer is obtained by the following formula:

L _fuse (W, w)=L _c (P _fuse , G) (6)

Among them, L _c represents the standard cross-entropy loss function; P _fuse represents the fusion of the prediction feature maps of the output layers of the k stages, that is, the weight of the fusion layer; K represents the total number of stages;

Use the argmin function (minimum function, indicating the variable value when the objective function takes the minimum value) to sum up the fine loss function of the fusion layer and the fine loss function of the side output layer of all stages to obtain the objective function, as shown in formula (8) ; Optimize the objective function through the standard stochastic gradient descent method, and then optimize the weight of the fine loss function of each side output layer and the weight of the fine loss function of the fusion layer;

S4 model training and testing

S4-1 Model parameter initialization: initialize all weight values, bias values, and batch normalized scale factor values, input the initialized parameter data into the rich robust convolution feature model established in step S2, and set the initial learning rate λ of the model = 0.001; the weight standard deviation of the convolutional layers in the first to fifth stages is initialized to 0.01, and the weight deviation is initialized to 0; the weight standard deviation of all convolutional layers of the fusion layer is initialized to 0.2, and the weight deviation is initialized to 0;

S4-2 Model training: Input the target images in the training sample set and their corresponding pixel-level labels into the rich robust convolution feature model after the parameters are initialized in step S4-1; in the model training process, the stochastic gradient descent method (Stochastic gradient descent, SGD) passes the loss to the weight of each convolutional layer, and updates its weight value. The momentum of the stochastic gradient descent method is 0.9, and the weight decay is 0.0005; 1 image is randomly sampled in each training process, and the number of iteration cycles Stop training when it reaches 100 cycles, and complete the training of the rich and robust convolutional feature model; the above operations are all completed under the window10 system, the computer used for training is the Core i7 series, the memory is 32GB, and the graphics card is NIVIDIAGeforce GTX2080ti; based on Keras1. 13 The deep learning framework realizes the training of the model;

S4-3 model test: scale the target image in the test sample set to 1024x1024 pixels, and input the scaled target image into the rich robust convolution feature model trained in step S4-2; the test time for a single image is 0.1 s, to meet the requirements of production efficiency, repeat the operation of step S4-2 to complete the model test.

In order to verify the effectiveness of this method, experiments were carried out on lithium battery images containing crack defects using this method. For comparison, the comparison results are shown in Figure 2; among them, (a1) represents the original image containing crack defects, (a5) is the corresponding real label; (a2) is the result of feature extraction using the Gabor filter method; (a3) ) is the result of feature extraction using the UNet model (U-shaped network) method; (a4) is the result of feature extraction using the rich robust convolutional feature model (RRCF) proposed by this method;

As can be seen from Figure 2, the RRCF model proposed by this method can learn more crack information due to the superposition and fusion of convolution features at each stage, which overcomes the Gabor filter method which is easy to integrate grid lines similar to crack structures. Structural misdetection, and the UNet model is easy to misdetect the part with grain occlusion as a crack defect; the crack line predicted by the RRCF model proposed by this method is thinner and closer to the real label. The results show that the two methods in this method The fine loss function helps to predict fine cracks, improves the prediction results of the Gabor filter and the UNet model that are not fine enough, and has higher prediction accuracy.

In order to quantitatively evaluate the performance of each method, three indicators of cpt (completeness), crt (accuracy) and F-measure (F-measure) are used for quantitative analysis. F-measure is the calculation result based on cpt and crt. The higher the F-measure value, the more effective the method used; the expressions are shown in equations (9)-(11);

Among them, L _g represents the number of cracked pixels in the real label marked manually; L _t is the number of pixels extracted by the detection method; L is the number of pixels that match the real label in the extraction result of the detection method;

The index values of the three methods are shown in Figure 3. Both the UNet model and the rich and robust convolutional feature model show high integrity cpt, reflecting the advantages of convolutional neural networks in solving the crack detection problem under complex background interference; The F-measure of the rich robust convolutional feature model is 85.81%, which is better than the other two methods; the completeness and accuracy of the rich robust convolutional feature model are 93.02% and 79.64%, respectively. The multi-level fusion is beneficial to improve the integrity of crack segmentation. On the other hand, according to the extreme aspect ratio of cracks, two fine loss functions are designed to reduce background information interference and improve accuracy, and the accuracy of recognition is significantly improved. In this process It is not easy to lose the crack feature and will not miss the crack detection; the accuracy of the UNet model is the lowest (69.5%), because it is greatly affected by the background interference, it will introduce too much background information, and the fine segmentation of the crack cannot be achieved. Therefore, it shows the lowest accuracy. In summary, this method has the highest completeness and accuracy of crack segmentation, and has the best segmentation effect, which can achieve fine segmentation of cracks.

What is not described in the present invention applies to the prior art.

Claims (7)

1. A segmentation method of complex texture small defects based on a rich robust convolution feature model is characterized in that the method comprises the steps of obtaining an image containing an object to be segmented, and performing feature recombination on the image containing the object to be segmented by using the rich robust convolution feature model to obtain a feature map of each side output layer; the feature diagram of each side output layer is sequentially connected with a deconvolution layer and a side output layer fine loss function to obtain a prediction feature diagram of each stage side output layer;

meanwhile, a fusion layer is added in the model, the characteristic graphs after deconvolution of the characteristic graphs of all side output layers are fused together, and then a fusion layer fine loss function is connected to obtain a final prediction graph, so that defect segmentation is realized;

wherein, the side output layer fine loss function satisfies formula (1):

P_side＝σ(A_side)，A_side＝{a_j，j＝1，……|Y|} (2)

in the formula, L^(k)(P_sideG) represents the distance loss function of the k stage; l (W, W)^(k)) Representing a weighted cross-entropy loss function for the kth stage; p_sideA prediction feature map representing a k-th stage side output layer; sigma is a sigmoid activation function; a. the_sideA set of activation values at all pixels of a predicted feature map representing k stage-side output layers; a is_jRepresenting an activation value at any pixel j in the prediction feature map of the k-th stage side output layer; y represents the sum of the defective pixel and the non-defective pixel in the diagram;

the fusion layer fine loss function is given by:

L_fuse(W，w)＝L_c(P_fuse，G) (3)

in the formula, L_cA cross entropy loss function representing a criterion; p_fuseRepresenting the fusion of the prediction characteristic graphs of the k stage side output layers, namely the fusion layer weight; k represents the total number of stages;

summarizing the fusion layer fine loss function and the side output layer fine loss functions of all stages by using an argmin function to obtain a target function L, and expressing the target function L by using a formula (5);

and finally, optimizing the objective function to obtain the weights of the side output layer fine loss function and the fusion layer fine loss function.

2. The segmentation method according to claim 1, wherein the specific process of using the rich robust convolution feature model to perform the feature reconstruction is as follows:

removing the full-connection layer and the pooling layer in the fifth stage on the basis of the original ResNet40 network, and respectively laterally connecting the identification block layer in the first stage and the identification block layer in the second stage of the original ResNet40 network with one convolutional layer to obtain characteristic diagrams of side output layers in the first and second stages;

and respectively connecting a convolution layer laterally behind each block layer in the third, fourth and fifth stages of the original ResNet40 network to obtain feature maps after convolution of the respective block layers, and then respectively adding element by element the feature maps after convolution of all the block layers in the same stage to obtain the feature maps of the output layers at the corresponding stages.

3. The segmentation method according to claim 2, wherein the convolution kernel size of the convolutional layer laterally connected to the first and second stages of the identifier block layer is 1 × 1, and the step size and the number of channels are all 1; the convolution kernel size of the convolution layer connected laterally behind each block layer in the third, fourth and fifth stages is 1 × 1, the step size is 1, and the number of channels is 21.

4. The segmentation method according to claim 2, wherein the original ResNet40 network comprises 40 convolutional layers and a fully-connected layer located at the last layer of the network, and is divided into 5 stages, each stage comprises a convolutional block layer and one or more flag block layers, wherein the first and second stages respectively comprise a convolutional block layer and a flag block layer, the third, fourth and fifth stages respectively comprise a convolutional block layer and two flag block layers, and each convolutional block layer and flag block layer comprises a plurality of convolutional layers; each stage adds a pooling layer with a pooling window size of 2 x2 and a step size of 2 after all the flag layers.

5. The segmentation method according to claim 4, wherein the original ResNet40 network has a specific structure:

firstly, sequentially carrying out convolution with convolution kernel size of 5 multiplied by 5, step length of 1, channel number of 32 and maximum pooling layer with convolution kernel size of 2 multiplied by 2 and step length of 2 on an input target image to obtain input characteristics of a first stage; the input features of the first stage are connected with a residual error with convolution kernel size of 1 × 1, step size of 1 × 3 and channel number of 32 through three convolution kernels with convolution kernel size of 1 × 1, convolution kernel size of 3 × 3 and convolution kernel size of 1 × 1 and channel number of 1 × 1 in sequence to obtain output features of the convolution block layer of the first stage; the output characteristics of the first stage convolution block layer are subjected to three convolutions with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 32 in sequence to obtain the output characteristics of the first stage identification block layer; the output characteristic of the first stage identification block layer is subjected to a pooling layer with a convolution kernel size of 2 multiplied by 2 and a step length of 2 to obtain the output characteristic of the first stage;

the output characteristics of the first stage are connected by three convolution volumes with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 64 and a residual error with convolution kernel sizes of 1 × 1, step lengths of 1 and channel numbers of 64 in sequence to obtain the output characteristics of the convolution block layer of the second stage; the output characteristics of the second stage convolution block layer are subjected to three convolutions with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 64 in sequence to obtain the output characteristics of the second stage identification block layer; the output characteristics of the second stage identification block layer pass through a pooling layer with convolution kernel size of 2 multiplied by 2 and step length of 2 to obtain the output characteristics of the second stage;

the output characteristics of the second stage are connected by three convolution volumes with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 256 and a residual error with convolution kernel sizes of 1 × 1, step lengths of 1 and channel numbers of 256 in sequence to obtain the output characteristics of the convolution block layer of the third stage; the output characteristics of the convolution block layer in the third stage are subjected to three convolutions with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 256 in sequence to obtain the output characteristics of the first identification block layer in the third stage; the output characteristics of the first identification block layer in the third stage are subjected to three convolutions with convolution kernel sizes of 1 × 1, 3 × 3 and 1 × 1, step lengths of 1 and channel numbers of 256 in sequence to obtain the output characteristics of the second identification block layer in the third stage; the output characteristic of the second identification block layer in the third stage passes through a pooling layer with the convolution kernel size of 2 multiplied by 2 and the step length of 2 to obtain the output characteristic of the third stage;

the operation process of the fourth stage is the same as that of the third stage, and the output characteristic of the fourth stage is obtained after the output characteristic of the third stage is repeated in the operation of the third stage;

the operation process of the fifth stage is the same as the operation process of the convolution block layer and the two identification block layers of the fourth stage, and the output characteristic of the fourth stage is obtained after the operation of the convolution block layer and the two identification block layers of the fourth stage is repeated.

6. The segmentation method according to any one of claims 1 to 5, characterized in that the method comprises the specific steps of:

s1 image preprocessing

Collecting an image containing a defect to be segmented, and normalizing the collected image into 1024 multiplied by 1024 pixels; adding pixel level labels to the normalized images, wherein the images added with the labels are target images; dividing the target image into different sample sets according to the proportion;

s2 construction of rich robust convolution characteristic model

Based on an original ResNet40 network, respectively and laterally connecting a convolutional layer on an identification block layer at a first stage and an identification block layer at a second stage of the original ResNet40 network to obtain characteristic diagrams of output layers at the first and second stages;

respectively connecting a convolution layer laterally behind each block layer in the third, fourth and fifth stages of the original ResNet40 network to obtain feature maps after convolution of the respective block layers, and then respectively adding element by element the feature maps after convolution of all the block layers in the same stage to obtain the feature maps of the output layers at the corresponding stages;

respectively connecting the feature maps of the five stage side output layers with a deconvolution layer (deconv) for up-sampling to obtain feature maps after respective stage deconvolution, and respectively connecting the feature maps after each stage deconvolution with a side output layer fine loss function for pixel-by-pixel classification to obtain a prediction feature map of each stage side output layer;

connecting the characteristic diagrams after deconvolution of each stage, and then fusing all the characteristic diagrams after deconvolution through a convolution layer with convolution kernel size of 1 multiplied by 1 and step length of 1 to obtain a fused layer characteristic diagram; finally, connecting the fusion layer feature graph with a fusion layer fine loss function to obtain a final prediction feature graph;

s3 model training and testing

Initializing model parameters, and inputting a target image for training and a pixel level label corresponding to the target image; in the model training process, the weight value of each convolution layer is transferred to the loss through a random gradient descent method, the weight value is updated, the momentum of the random gradient descent method is 0.9, and the weight attenuation is 0.0005; randomly sampling 1 image in each training process, stopping training when the number of iterative cycles reaches 100 cycles, and finishing the training of the model;

scaling the target image for testing to 1024x1024 pixels, and inputting the scaled target image into the trained model; the testing time of a single image is 0.1s, and the operation of the model is repeated to complete the model test.

7. The segmentation method according to claim 1, wherein the object to be segmented is a crack, an edge or a line structure.

Images