CN111738991A - A method of creating a digital radiographic inspection model for weld defects - Google Patents

Abstract

The invention provides a method for creating a digital ray detection model of weld defects, which extracts a weld in an image by using a digital image processing technology; expanding the number of samples of the welding seam image by applying an image enhancement technology; the model is trained by utilizing the built deep learning model and the designed loss function, so that the defect rule can be learned from the existing defect image, all images to be detected can be more flexibly batch-processed by using the deep neural network, the model is more flexible and more stable in performance, and higher recall rate and certain accuracy rate of the model are ensured; and the model is convenient to call and use by packaging and storing the model. Reduce artificial work load, improve work efficiency.

Description

A method of creating a digital radiographic inspection model for weld defects

technical field

The invention relates to the technical field of image detection model creation methods, in particular to a creation method of a digital ray detection model for weld defects.

Background technique

In recent years, artificial intelligence has brought new developments to the field of computer vision, especially in image classification and object detection. Commonly used image classification models such as Alexnet, VGG, Resnet, etc. can handle extremely complex image data, and are often used to classify public datasets. The performance and accuracy of these models have been continuously improved. However, the data in actual production is not as easy to distinguish as the common public data sets. For example, in the field of weld inspection, there is a problem of difficulty in evaluating films. There are certain combing defects in welded joints, such as cracks, pores, slag inclusions, incomplete penetration, lack of fusion, etc. The existence of these defects will reduce the welding strength, cause stress concentration, and cause damage to the welded structure. All of the above will affect the integrity and safety of the welded structure, and the film reviewers need to judge the photographed weld images, so as to screen out the images with defects. However, this manual detection method is greatly affected by the professional knowledge and physical fatigue of the inspector, and a more objective and intelligent defect image detection method needs to be adopted. At present, there are also some methods for automatic identification and classification of ray weld images, such as methods based on image difference, methods based on edge detection, and similar defect detection algorithms based on template matching, etc. These methods based on traditional image processing are to a large extent It still relies on human knowledge to judge, and it is extremely unstable and requires high image resolution and contrast. The existing neural network models applied to public datasets have poor performance in recognizing such images with inconspicuous features and cannot be used. Meet the requirements of defect detection for high recall rate.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a method for creating a digital ray detection model for weld defects, so as to solve the problem that the neural network model used for public data sets in the prior art has a poor effect on such images with insignificant features, and The problem that cannot meet the requirements of defect detection for high recall rate. The specific technical solutions are as follows:

An embodiment of the present invention provides a method for creating a digital radiographic detection model for weld defects, including:

S110. Obtain the original picture;

S120. Using a predetermined method, extract the weld seam in the original image to obtain a weld seam image;

S130. Use a data enhancement algorithm to perform sample expansion on the weld image to obtain multiple sample maps based on the weld image;

S140. Use the sample graph to train a convolutional neural network to construct and obtain a deep learning model suitable for weld defect detection;

S150. Use a loss function to calibrate the deep learning model, and calculate the accuracy and recall of the deep learning model;

S160. Repeat steps S110-S150, and use at least two of the original pictures to train the deep learning model to obtain a basic model, so that the basic model meets a predetermined standard; wherein, the predetermined standard includes: a predetermined accuracy rate, predetermined recall rate;

S170. Save the basic model, and perform interface encapsulation on the basic model to obtain a digital ray detection model for weld defects.

Optionally, the predetermined method is:

Obtain the region of interest of the original image, and outline the region of interest;

Weld seam images are extracted from the region of interest using edge detection methods.

Optionally, the edge detection method includes:

Slicing the region of interest along the length direction to obtain a plurality of segment images;

performing an open operation on the segment images to obtain consecutive image segments;

performing local histogram equalization with limited contrast on the continuous image segments, and then performing median filtering to obtain smooth image segments;

Using a discrete first-order difference operator to calculate a first-order gradient approximation of the Y-axis of the image brightness function of the smoothed image segment;

converting the first-order gradient approximation into a binarized image;

Perform grayscale inversion on the binarized image, and use an edge detection operator to perform edge detection to obtain a weld outline image;

The Y-axis coordinate of the weld outline image is recorded, the region of interest is cropped according to the Y-axis coordinate of the weld outline image, and the weld image is extracted from the region of interest.

Optionally, the loss function is:

Loss=-[y _t logy _p +w*(1-y _t )log(1-y _p )]

In the formula, w is paranoia, (x, y) is the sample point, assuming that the true label of a sample point is y _t , the probability that the sample point takes y _t =1 is y _p , Loss is the loss function, and the value of y _p The value is between (0, 1).

Optionally, in step S160, the deep learning model is trained by using at least two of the original pictures to obtain a basic model, and making the basic model meet a predetermined standard includes:

Acquiring a plurality of defect-free original images, and acquiring defective original images equal to the number of the defect-free original images;

Using the defect-free original image and the defective original image as a model training set, and randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training;

until the base model reaches a predetermined standard.

Optionally, in step S160, the deep learning model is trained by using at least two of the original pictures to obtain a basic model, and making the basic model meet a predetermined standard includes:

Randomly selects the basic model from the database and randomly selects half the number of samples without defects from the database, and randomly selects the original images with defects equal to the number of the original images without defects from the database;

Using the defect-free original image and the defective original image as a model training set, and randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training;

until the base model reaches a predetermined standard.

Optionally, the basic model includes: the structure of the model, and the weights after training the deep learning model.

The embodiment of the present invention provides a method for creating a digital ray detection model for weld defects, which uses digital image processing technology to extract welds in an image; expands the number of samples of weld images by using image enhancement technology; The deep learning model and the designed loss function are used to train the model, which can learn the defect rules from the existing defect images. Using the deep neural network can more flexibly batch batch all the images to be tested, which is more flexible and has more stable performance. The model has a high recall rate and a certain accuracy rate; it is convenient to call and use the model by encapsulating and saving the model. Reduce manual workload and improve work efficiency.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required in the description of the embodiments or the prior art.

1 is a flowchart of a method for creating a digital radiographic detection model for weld defects provided by an embodiment of the present invention;

FIG. 2 is a flowchart of weld extraction provided by an embodiment of the present invention;

3 is a structural diagram of a deep neural network model provided by an embodiment of the present invention;

4 is a flowchart of two training strategies provided by an embodiment of the present invention;

5 is an original image with outline lines provided by an embodiment of the present invention;

6 is a schematic diagram of a welding seam extraction process provided by an embodiment of the present invention;

FIG. 7 is a final extracted weld diagram provided by an embodiment of the present invention;

8 is a schematic diagram of convolutional layer splicing provided by an embodiment of the present invention;

FIG. 9 is an existing cross-entropy function error sensitivity table provided by an embodiment of the present invention;

10 is a three-dimensional diagram of error sensitivity of an existing cross-entropy function provided by an embodiment of the present invention;

11 is a three-dimensional diagram of error sensitivity of cross-entropy functions under different paranoia provided by an embodiment of the present invention;

12 is an image model training fitting diagram of Strategy 1 provided by an embodiment of the present invention;

13 is an image model training fitting diagram of Strategy 2 provided by an embodiment of the present invention;

14 is a training fitting diagram of an image model of recall provided by an embodiment of the present invention;

FIG. 15 is a training result icon when the basic model provided by an embodiment of the present invention terminates training.

Detailed ways

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

The purpose of the embodiments of the present invention is to provide a digital ray detection method, system and server for welding seam defects, so as to solve the problem that the neural network model used for public data sets in the prior art has a poor effect on such images with insignificant features, Moreover, it cannot meet the problem that defect detection requires a high recall rate.

Example 1

Referring to FIG. 1, an embodiment of the present invention provides a method for creating a digital ray inspection model for weld defects, including:

S110. Obtain the original picture; the above original picture can be obtained directly from the database. Specifically, in this application, the above original picture is the original picture with the welding seam. Specifically, whether the above welding seam is defective can be used as the above original picture of this solution. .

S120. Using a predetermined method, extract the weld seam in the original image to obtain a weld seam image.

S130. Use a data enhancement algorithm to perform sample expansion on the weld image to obtain multiple sample maps based on the weld image; use a data enhancement algorithm to expand the existing positive and negative samples, thereby increasing the capacity of the samples . According to the characteristics of the weld image, the original weld image is expanded by using the image inversion symmetry technique.

S140. Use the sample graph to train a convolutional neural network to construct and obtain a deep learning model suitable for weld defect detection; construct a deep learning model suitable for weld defect detection according to data characteristics.

Specifically, in view of the problems of low contrast of the weld image, small defects, and inconspicuous features, the present invention adopts the method of splicing to perform feature fusion. In the design module, the features are spliced together in the channel dimension to form thicker features. The block is shown in Figure 8. After the data from the previous layer enters the module, the features are extracted through two independent convolution kernels with different specifications, and the obtained features are directly spliced together in the dimension scale to form new features, and the features are put into the dropout layer. The existence of this layer avoids the overfitting of the model to a certain extent, and then enters the BN layer for normalization, and finally passes through the pooling layer to form the feature data after this module. Use this module to build a deep neural network, see Figure 3.

S150. Use a loss function to calibrate the deep learning model, and calculate the accuracy and recall of the deep learning model; in order to ensure a higher recall rate for the model, a designed loss function that guarantees a high recall rate is used for model training .

S160. Repeat steps S110-S150, and use at least two of the original pictures to train the deep learning model to obtain a basic model, so that the basic model meets a predetermined standard; wherein, the predetermined standard includes: a predetermined accuracy rate, Predetermined recall; different training methods are used to enable stable training of the model even when the amount of data is too large to fit in memory.

S170. Save the basic model, and perform interface encapsulation on the basic model to obtain a digital ray detection model for weld defects. Encapsulate and save the trained model to facilitate calls from remote servers.

Specifically, an embodiment of the present invention provides a method for creating a digital ray detection model for weld defects, which uses digital image processing technology to extract welds in an image; and uses image enhancement technology to expand the number of samples in the weld image. ;Using the built deep learning model and the designed loss function to train the model, can learn the defect rules from the existing defect images, and use the deep neural network to more flexibly batch all the images to be tested, which is more flexible and has better performance. Stable, to ensure a high recall rate and a certain accuracy of the model; it is convenient to call and use the model by encapsulating and saving the model. Reduce manual workload and improve work efficiency.

Compared with the existing weld defect detection algorithms, this algorithm has the following advantages:

1. Compared with traditional image processing algorithms, it has higher efficiency and more flexibility

Weld defect detection through traditional image processing techniques such as image difference, template matching, etc. relies on human expertise to perform image comparison to a certain extent, and cannot truly achieve end-to-end detection. The core detection algorithm can learn the defect rules from the existing defect images. Using the deep neural network can more flexibly batch batch all the images to be tested, which is more flexible and has more stable performance.

2. Compared with existing deep learning models, it has higher recall rate and accuracy rate

The existing deep learning model is not aimed at welding seam images, so the algorithm used in the present invention has stronger pertinence, so it has higher precision, especially the loss function designed to ensure a high recall rate in this algorithm, so it can guarantee a higher recall rate. The high recall rate can also ensure a certain accuracy rate, which can better meet the requirements of production practice.

Example 2

On the basis of the foregoing Embodiment 1, this embodiment further describes the solution in detail with reference to FIGS. 2 to 15 . details as follows:

Further, the predetermined method is:

Obtain the region of interest of the original image, and outline the region of interest;

Weld seam images are extracted from the region of interest using edge detection methods.

Specifically, the first is to extract the region of interest from the image, that is, the weld.

The most important thing in image processing is the ROI (region of interest), the region of interest. In machine vision and image processing, the region to be processed is outlined from the processed image in the form of boxes, circles, ellipses, irregular polygons, etc., which is called region of interest, ROI. Various operators (Operator operation symbols) and functions are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the ROI of the region of interest, and perform the next processing of the image. In the image of the weld seam below, the section is framed with a rectangular frame, see Figure 5.

Further, the edge detection method includes:

Slicing the region of interest along the length direction to obtain a plurality of segment images;

performing an open operation on the segment images to obtain consecutive image segments;

performing local histogram equalization with limited contrast on the continuous image segments, and then performing median filtering to obtain smooth image segments;

Using a discrete first-order difference operator to calculate a first-order gradient approximation of the Y-axis of the image brightness function of the smoothed image segment;

converting the first-order gradient approximation into a binarized image;

Perform grayscale inversion on the binarized image, and use an edge detection operator to perform edge detection to obtain a weld outline image;

Record the Y-axis coordinate of the weld outline image, and crop the region of interest according to the Y-axis coordinate of the weld outline image, and extract the weld image from the region of interest, see FIG. 7 , Figure 7 is the final extracted weld image.

Specifically, in order to enable the model to be trained more accurately, that is to say, the neural network only processes the part framed by the rectangular frame, and treats other parts of the image as interference processing. The first thing to do is to extract the weld seam, which is mainly used The technique is edge detection.

Edge detection is mainly based on the gradient change of the image to obtain the shape of the image to be detected. The analysis based on edge detection is not easily affected by changes in the overall light intensity. Many image understanding methods are based on edges. Edge detection emphasizes the contrast of the image. Detecting the contrast, that is, the difference in brightness, can enhance the boundary features in the image, and the appearance of these boundaries is the difference in the brightness of the image. The target boundary is actually the step change in brightness level, and the edge is the location of the step change. The first-order differential can be used to enhance the edge change to detect the edge position. The commonly used operators are Sobel, Canny, etc.

Since the contrast of the image itself is not particularly obvious, it is necessary to perform some operations to enhance the contrast of the image. Please refer to Figure 2 for the specific process.

Please refer to FIG. 6 . In FIG. 6 , from left to right, the slices are binarized after the opening operation is performed, and the edge detection is performed on the binarized image. The position of the weld in the image is generally across the image, but there will be other disturbances in the image besides the weld, so the image needs to be sliced to make the disturbance as small as possible. Since the sliced image has low contrast and there are burrs, the image opening operation is performed, that is, the image is eroded first and then the image is expanded, so that the obtained image will be more coherent and have less burr interference. Perform local histogram equalization to limit the contrast of the image after the opening operation, and then perform median filtering to make the image smoother. Use the Sobel operator to calculate the gradient of the y-axis of the image, and convert the gradient value into a binarized image, so that The edges of the welds are sharper. Finally, the grayscale of the image is flipped, and the Canny operator is used to perform edge detection, filter out the contours that meet the conditions, and record the position of the weld on the y-axis, so as to cut the original image and realize the weld extraction.

Further, the loss function is:

Loss=-[y _t logy _p +w*(1-y _t )log(1-y _p )]

In the formula, w is paranoia, (x, y) is the sample point, assuming that the true label of a sample point is y _t , the probability that the sample point takes y _t =1 is y _p , Loss is the loss function, and the value of y _p The value is between (0, 1).

Specifically, please refer to FIG. 9. In order to ensure that the model has a higher recall rate and improve the accuracy of the model, the present invention modifies the traditional cross-entropy function and adds a bias term. The existing cross-entropy function is:

Loss=-[y _t logy _p +(1-y _t )log(1-y _p )]

In the formula, (x, y) is the sample point, assuming that the real label of a sample point is y _t , the probability that the sample point takes y _t =1 is y _p , Loss is the loss function, and the value of y _p is in ( 0, 1) between. The error sensitivity of the traditional cross-entropy function to the false positive class and the false negative class is the same, please refer to Fig. 10. Fig. 10 also shows that the loss function is symmetric between the two, and the embodiment of the present invention provides The algorithm modifies the cross entropy function as:

Loss=-[y _t logy _p +w*(1-y _t )log(1-y _p )]

Multiplying the cross-entropy function by a paranoid w, according to the superficial understanding, is to give different penalties to the two types of errors, as shown in Figure 11, it can be seen from Figure 11 that when w < 1, 1 is judged as 0. The penalty for this type of error is more important than the other, and the smaller w is, the more obvious this bias is; and when w = 1, the penalty for the two errors is the same; when w > 1, give 0 An error of 1 imposes a huge penalty, and the magnitude is more severe than when it is less than 1.

Therefore, in order to reduce the occurrence of defective images judged as non-defective, the above mechanism is used to add a bias to the loss function, so that the model finally obtains a higher recall rate.

Further, referring to FIG. 4, in step S160, the deep learning model is trained by using at least two of the original pictures to obtain a basic model, and making the basic model meet a predetermined standard includes:

Acquiring a plurality of defect-free original images, and acquiring defective original images equal to the number of the defect-free original images;

Using the defect-free original image and the defective original image as a model training set, and randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training;

until the base model reaches a predetermined standard.

As the strategy 1 for training the basic model in this embodiment, images with the same number of defective samples are randomly selected from the non-defective images and put into the model training before the program training starts, which ensures that the positive and negative samples are 1:1, which is beneficial to the model learning features. Please refer to Figure 12. From Figure 12, it can be seen that the model has a slight overfitting at around 50 epochs, but the overall performance is still good, but considering that the model trained in this case only uses and the number of defective images For the same normal image, a large amount of data will be wasted. Although the randomness in the sampling process may alleviate this waste, for a model, the probability of each data entering the model training should be the same.

Further, referring to FIG. 4, in step S160, the deep learning model is trained by using at least two of the original pictures to obtain a basic model, and making the basic model meet a predetermined standard includes:

Randomly selects the basic model from the database and randomly selects half the number of samples without defects from the database, and randomly selects the original images with defects equal to the number of the original images without defects from the database;

Taking the defect-free original image and the defective original image as a model training set, randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training;

until the base model reaches a predetermined standard.

Specifically, the embodiment of the present invention also provides another strategy for training the basic model, strategy 2: randomly select half batch_size images from all normal images for training in each batch_size. The balanced ratio of positive and negative samples also allows each image to have the opportunity to enter the model training. Please refer to Figure 13. It can be seen from Figure 13 that the learning behavior occurs at 80 epochs. It happens that all the data may have entered the model for learning, and the overfitting of the model also occurs relatively late, about 110 epochs.

The final recall image after training is shown in Figure 14, where the top line represents the recall rate, the middle line represents f1_score, and the bottom line represents the accuracy rate. It can be seen that since our loss function increases the Defective images are judged as no-defective images. At the beginning, the model tends to judge all images as defective images, so that the model obtains a higher recall rate of 1.0. As the training process proceeds, the model seeks more The small loss has to find the perfect image as much as possible on the basis of ensuring that the defective image is not judged as the perfect image, so the accuracy rate is improved. Since the accuracy rate and the recall rate are contradictory, so the Lose some recall rate, and stop training when the final model meets the requirements, that is, when the recall rate and accuracy rate are fitted, stop training, and get the training result at this time, as shown in Figure 15.

Further, the basic model includes: the structure of the model and the weights after training the deep learning model.

Specifically, the model after training needs to be saved. The saved part includes the structure of the model and the weight after training. When there is a need for testing, the saved model can be called. For this purpose, the interface is encapsulated, and the model building and weight encapsulation are completed. Easy to call directly.

The embodiment of the present invention provides a method for creating a digital ray detection model for weld defects, which uses digital image processing technology to extract welds in an image; expands the number of samples of weld images by using image enhancement technology; The deep learning model and the designed loss function are used to train the model, which can learn the defect rules from the existing defect images. Using the deep neural network can more flexibly batch batch all the images to be tested, which is more flexible and has more stable performance. The model has a high recall rate and a certain accuracy rate; it is convenient to call and use the model by encapsulating and saving the model. Reduce manual workload and improve work efficiency.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Each embodiment in this specification is described in a related manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the partial descriptions of the method embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (7)

1. a method for creating a digital radiographic detection model for weld defects, comprising: S110. Obtain the original picture; S120. Using a predetermined method, extract the weld seam in the original image to obtain a weld seam image; S130. Use a data enhancement algorithm to perform sample expansion on the weld image to obtain multiple sample maps based on the weld image; S140. Use the sample graph to train a convolutional neural network to construct and obtain a deep learning model suitable for weld defect detection; S150. Use a loss function to calibrate the deep learning model, and calculate the accuracy and recall of the deep learning model; S160. Repeat steps S110-S150, and use at least two of the original pictures to train the deep learning model to obtain a basic model, so that the basic model meets a predetermined standard; wherein, the predetermined standard includes: a predetermined accuracy rate, predetermined recall rate; S170. Save the basic model, and perform interface encapsulation on the basic model to obtain a digital ray detection model for weld defects. 2. The method for creating a digital radiographic inspection model for weld defects according to claim 1, wherein the predetermined method is: Obtain the region of interest of the original image, and outline the region of interest; Weld seam images are extracted from the region of interest using edge detection methods. 3. The method for creating a digital radiographic detection model for weld defects according to claim 2, wherein the edge detection method comprises: Slicing the region of interest along the length direction to obtain a plurality of segment images; performing an open operation on the segment images to obtain consecutive image segments; performing local histogram equalization with limited contrast on the continuous image segments, and then performing median filtering to obtain smooth image segments; Using a discrete first-order difference operator to calculate a first-order gradient approximation of the Y-axis of the image brightness function of the smoothed image segment; converting the first-order gradient approximation into a binarized image; Perform grayscale inversion on the binarized image, and use an edge detection operator to perform edge detection to obtain a weld outline image; The Y-axis coordinate of the weld outline image is recorded, the region of interest is cropped according to the Y-axis coordinate of the weld outline image, and the weld image is extracted from the region of interest. 4. The method for creating a digital radiographic detection model for weld defects according to claim 1, wherein the loss function is: Loss=-[y _t logy _p +w*(1-y _t )log(1-y _p )] In the formula, w is paranoia, (x, y) is the sample point, assuming that the true label of a sample point is y _t , the probability that the sample point takes y _t =1 is y _p , Loss is the loss function, and the value of y _p The value is between (0, 1). 5. The method for creating a digital ray detection model for weld defects according to claim 1, wherein in step S160, at least two of the original pictures are used to train the deep learning model to obtain a basic model, Bringing the base model to predetermined criteria includes: Acquiring a plurality of defect-free original images, and acquiring defective original images equal to the number of the defect-free original images; Using the defect-free original image and the defective original image as a model training set, and randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training; until the base model reaches a predetermined standard. 6. The method for creating a digital radiographic detection model for weld defects according to claim 1, wherein in step S160, at least two of the original pictures are used to train the deep learning model to obtain a basic model, Bringing the base model to predetermined criteria includes: Randomly selects the basic model from the database and randomly selects half the number of samples without defects from the database, and randomly selects the original images with defects equal to the number of the original images without defects from the database; Using the defect-free original image and the defective original image as a model training set, and randomly acquiring the original image from the model training set, and placing the original image in the deep learning model for training; until the base model reaches a predetermined standard. 7 . The method for creating a digital ray detection model for weld defects according to claim 1 , wherein the basic model comprises: the structure of the model and the weights after training the deep learning model. 8 .

Images