CN111127454A

CN111127454A - Method and system for generating industrial defect sample based on deep learning

Info

Publication number: CN111127454A
Application number: CN201911376015.7A
Authority: CN
Inventors: 乐心怡; 李钧正; 尤志远; 陈李洋; 习俊通
Original assignee: Shanghai Jiao Tong University
Current assignee: Shanghai Jiao Tong University
Priority date: 2019-12-27
Filing date: 2019-12-27
Publication date: 2020-05-08

Abstract

The invention provides a method and a system for generating an industrial defect sample based on deep learning, which comprises the following steps: step 1: acquiring an industrial defect product picture, and marking defect information on the industrial defect product picture; step 2: constructing an confrontation generation depth model; and step 3: according to the labeled defect information, performing countermeasure training in an antibiotic formation depth model; and 4, step 4: training to obtain a defect sample; and 5: and screening the defective samples, and removing the defective samples which do not accord with the preset conditions to obtain industrial defective samples. The industrial defect picture generated by the invention has weak cross correlation, prominent defect characteristics and higher fine structure quality; the quality of the industrial defect picture generated by the method is high, and the performance index of the deep defect detection network trained according to the method can be obviously improved.

Description

Method and system for generating industrial defect sample based on deep learning

Technical Field

The invention relates to the technical field of computer vision and deep learning, in particular to a method and a system for generating an industrial defect sample based on deep learning.

Background

In industrial production, almost all products need to be inspected, and most of the inspection processes are performed by visual inspection (hereinafter referred to as visual inspection) by quality inspectors, especially some surface defects such as decorative plates, metal surfaces, keyboard surfaces, etc., which is very common in actual industry. Because the variety of product, the variety of defect, for example, there are mar, stain, plaque, wearing and tearing, piece etc. promptly to the defect of dalle, greatly increased quality control person's work load and work degree of difficulty, lead to artifical visual inspection efficiency to descend and easily because the fatigue of quality control person and error lead to the condition such as lou examining, wrong detection, improve the time cost of production line and probably influence the quality of the product on market. With the development of deep learning technology, industrial defect detection technology based on deep learning is widely applied to the field of defect detection.

The essence of deep learning is that the characteristics are learned by constructing a machine learning model with multiple hidden layers and massive training data, so that the accuracy and universality of classification or prediction are finally improved. However, training an effective deep learning model requires a large amount of labeled data, a large amount of graphics card resources, and a long training time, and in many industrial scenarios, the acquisition cost of defect images is very high, so that the number of samples is very limited, and it is difficult to directly train the deep learning model. Therefore, the false detection rate and the missing detection rate of the trained defect detection model are high, and the requirements of enterprises are difficult to meet.

With the development of data enhancement technology, some scientific researchers have developed some technical schemes capable of improving data enhancement. Patent document CN108932735A discloses a method of generating deep learning samples, which proposes to generate some deep learning samples using poisson distribution. However, because only the traditional noise-like method is used, the quality of the generated industrial defect samples is not high, that is, the cross correlation between the industrial defect samples is stronger, and the degree of improving the detection capability of the deep learning defect detection model is smaller. Other anti-generation techniques are also often plagued by problems of pattern collapse, overfitting, etc., and it is difficult to generate high quality industrial defect samples. The existing patent technology is a method for generating high-quality industrial defect samples, and the requirement of an actual enterprise on training of an automatic defect detection system based on a deep learning network is difficult to solve.

Patent document CN109559298A (application number: 201811357765.5) discloses a method for detecting a defect of a lotion pump based on deep learning, which is to respectively construct classification models of various angles based on the principles of transfer learning and convolutional neural network in deep learning to detect a defect sample. First, the network model was pre-trained using the Mini-ImageNet dataset. And then, adjusting the model structure and loading parameters of a pre-training network, inputting the training set and the verification set of each angle of the emulsion pump into a convolutional neural network for training after an image preprocessing algorithm, automatically performing the processes of feature extraction and classification in the network, and adjusting the network hyper-parameters according to the change of the accuracy rate of the verification set in the training process to obtain a final network model. And finally, inputting the preprocessed emulsion pump test sample into the trained model, and detecting the defect identification effect of the final model.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a method and a system for generating an industrial defect sample based on deep learning.

The method for generating the industrial defect sample based on the deep learning provided by the invention comprises the following steps:

step 1: acquiring an industrial defect product picture, and marking defect information on the industrial defect product picture;

step 2: constructing an confrontation generation depth model;

and step 3: according to the labeled defect information, performing countermeasure training in an antibiotic formation depth model;

and 4, step 4: training to obtain a defect sample;

and 5: and screening the defective samples, and removing the defective samples which do not accord with the preset conditions to obtain industrial defective samples.

Preferably, an error function is adopted in the confrontation training of step 3, and the formula is as follows:

wherein Gn is a generator network, Dn is a discriminator network, Ladv is a countermeasure loss function, Lrec is a reconstruction loss function, and a is a fixed coefficient and is a reconstruction error weight.

Preferably, the defect information includes marking data of a defect location.

Preferably, the construction of the contrast generating depth model is performed according to the image samples;

and image sampling is carried out according to the labeled data, so that the sampling probability is improved.

Preferably, the countermeasure generation depth model comprises a generator network and a discriminator network;

the generator network comprises one or more fully connected layers, and one or more volume blocks; the full-connection layer maps the uniform distribution or Gaussian distribution into a certain specific distribution and generates an industrial defect picture with the resolution ratio conforming to a first set range; each convolution block comprises one or more convolution layers and one-time up-sampling operation, an input industrial defect picture with the resolution conforming to a first set range is added with noise to generate an industrial defect picture with the resolution conforming to a second set range, semantic content on the industrial defect picture with the resolution conforming to the second set range is enriched, and a high-definition picture which is mapped into industrial defects by a generator network is obtained;

the discriminator network comprises one or more convolutional blocks, each convolutional block comprising one or more convolutional layers; and distinguishing the industrial defective product picture from the defect picture generated by the generator network.

The system for generating the industrial defect sample based on the deep learning provided by the invention comprises the following steps:

module M1: acquiring an industrial defect product picture, and marking defect information on the industrial defect product picture;

module M2: constructing an confrontation generation depth model;

module M3: according to the labeled defect information, performing countermeasure training in an antibiotic formation depth model;

module M4: training to obtain a defect sample;

module M5: and screening the defective samples, and removing the defective samples which do not accord with the preset conditions to obtain industrial defective samples.

Preferably, the module M3 uses an error function in the confrontation training, and the formula is:

Preferably, the defect information includes marking data of a defect location.

Compared with the prior art, the invention has the following beneficial effects:

1. according to the method, a neural network is generated by training countermeasures based on a small number of image samples with industrial defects, and a large number of image samples with industrial defects can be generated in a short time by means of manual discrimination;

2. the industrial defect picture generated by the invention has weak cross correlation, prominent defect characteristics and higher fine structure quality;

3. the quality of the industrial defect picture generated by the method is high, and the performance index of the deep defect detection network trained according to the method can be obviously improved.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a flow chart of a high quality industrial defect sample of the present invention;

FIG. 2 is a network architecture diagram of a generator network and an arbiter network employed in the present invention;

FIG. 3 is a diagram of a high quality industrial sample of a steel sheet blister defect of the present invention;

FIG. 4 is a diagram of a high quality industrial sample of an aluminum sheet blistering defect of the present invention.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.

As shown in fig. 1, which is a flow chart of an embodiment of the present invention, the method includes the following steps:

step A: collecting a small amount of real industrial defect samples, and marking the positions of the defects on the real industrial defect samples

And B: according to fig. 2, a countermeasure generation depth model is constructed.

And C: for the depth model, the labeled data is utilized to carry out countermeasure training

Step D: executing generator network in trained model to generate defect sample

Step E: manually discriminating the defect sample generated by the generator network, and eliminating the over-distorted picture generated by the generator to obtain the high-quality industrial defect sample

As shown in fig. 2, the countermeasure generation depth model is composed of a generator network and a discriminator network.

Wherein the generator network comprises one or more fully connected layers, and one or more volume blocks. The full-connection layer maps the uniform probability distribution (or Gaussian distribution) into a certain specific distribution and generates an industrial defect picture with lower resolution. Each convolution block comprises one or more convolution layers, and an up-sampling operation can add low-resolution pictures input into the convolution block and noise to generate a picture with higher resolution and enrich semantic content on the picture. Finally, the uniform probability distribution (or gaussian distribution, etc.) is mapped by the generator network into a high definition picture of the industrial defect.

Wherein the network of discriminators comprises one or more volume blocks. Each convolution block includes one or more convolution layers. Their role is to gradually reduce whether the input picture is a real defect picture or a defect picture generated by the generator network.

The training method adopts countertraining, and the error function adopted by the countertraining is as follows:

wherein Gn is a generator network, Dn is a discriminator network, Ladv is a countermeasure loss function, Lrec is a reconstruction loss function, and a is a fixed coefficient called reconstruction error weight.

The opposition loss function is a measure of the ability of the discriminator network to distinguish between real industrial defect samples and industrial defect samples generated by the generator network. When the discriminator network is optimized, the error function is improved as much as possible, so that the discriminator network can more accurately distinguish samples generated by the generator network. The error function is reduced as much as possible while optimizing the generator network, thereby enabling the generator network to better spoof the arbiter network. The antagonistic training can improve the ability of the generator network to generate as realistic industrial defects as possible.

The reconstruction loss function measures the ability of the generator network to map a particular input to a real industrial defect. Defined as the root mean square value of the pixel-by-pixel deviation of the defect image and the real defect image output by the generator network, with all noise inputs of the generator network set to zero. The reconstruction loss function can improve the fine structure quality of the generated picture.

When the size of the processed image is trained to be a rolling block which is closer to the size of a real industrial defect image, in order to avoid video memory overflow, a small-picture sampling training method is adopted to calculate and resist generation errors. The method can obviously reduce the video memory occupation of the training process under the condition of not reducing the training quality too much. However, the method of small-frame sampling training cannot be adopted when the reconstruction error is calculated, otherwise style imbalance is caused.

When sampling a small image, the probability of sampling each position of the original image by using the interested region can be controlled. That is, the defect location identified in step a is selected as the region of interest, and the sampling probability in this region is increased. The method can obviously improve the generation quality of the defect part, enrich the semantic information of the defect part and improve the training speed.

When the generator network is actually used for generating the high-quality industrial defect picture, the whole generator network can be used, or only the pure convolution part of the generator network can be used without using the full-connection layer part of the generator network according to the generation effect. When the full connection layer is not used, the industrial defect picture with low resolution needs to be input to the position of the pure convolution layer input of the generator network, so that the industrial defect picture with high quality is generated.

Bubble defects often occur in the field of steel plate defect detection, high-quality sample generation of the bubble defects of the steel plates is achieved by the aid of the technology, convex skin defects often occur in the field of aluminum plate defect detection, and high-quality sample generation of the convex skin defects of the aluminum plates is achieved by the aid of the technology, and the technology is respectively shown in fig. 3 and fig. 4.

Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims

1. a method for generating industrial defect samples based on deep learning, is characterized in that, comprising:

Step 1: Collect pictures of industrial defective products, and mark defect information on the pictures of industrial defective products;

Step 2: Build an adversarial generative deep model;

Step 3: Conduct adversarial training in the anti-generation deep model according to the labeled defect information;

Step 4: Train to obtain defect samples;

Step 5: Screen the defective samples, remove the defective samples that do not meet the preset, and obtain industrial defective samples.

2. The method for generating industrial defect samples based on deep learning according to claim 1, wherein an error function is used in the confrontation training in the step 3, and the formula is:

Among them, Gn is the generator network, Dn is the discriminator network, Ladv is the adversarial loss function, Lrec is the reconstruction loss function, and a is a fixed coefficient, which is the reconstruction error weight.

3 . The method for generating industrial defect samples based on deep learning according to claim 1 , wherein the defect information comprises labeling data of defect positions. 4 .

4. The method for generating industrial defect samples based on deep learning according to claim 3, wherein the construction of the confrontation generation depth model is carried out according to image sampling;

Image sampling is performed according to the labeled data to increase the sampling probability.

5. The method for generating industrial defect samples based on deep learning according to claim 1, wherein the confrontational generation deep model comprises a generator network and a discriminator network;

The generator network includes one or more fully connected layers and one or more convolution blocks; the fully connected layer maps the uniform distribution or Gaussian distribution to a specific distribution, and generates a resolution that conforms to the first A set range of industrial defect pictures; each convolution block includes one or more convolution layers, and an upsampling operation, adding the input resolution of the industrial defect pictures and noise in the first set range to generate a resolution. The rate conforms to the second setting range of the industrial defect pictures, and enriches the semantic content of the industrial defect pictures whose resolution conforms to the second setting range, and obtains the high-definition pictures that are mapped into the industrial defects by the generator network;

The discriminator network includes one or more convolution blocks, and each convolution block includes one or more layers of convolution layers; distinguishes pictures of industrial defective products from pictures of defects generated by the generator network.

6. A system for generating industrial defect samples based on deep learning, comprising:

Module M1: Collect pictures of industrial defective products, and mark defect information on the pictures of industrial defective products;

Module M2: Build an adversarial generative deep model;

Module M3: Conduct adversarial training in the anti-generation deep model according to the labeled defect information;

Module M4: training to obtain defect samples;

Module M5: Screen the defective samples, remove the defective samples that do not meet the preset, and obtain industrial defective samples.

7. The system for generating industrial defect samples based on deep learning according to claim 1, wherein an error function is used in the adversarial training of the module M3, and the formula is:

8 . The system for generating industrial defect samples based on deep learning according to claim 1 , wherein the defect information includes labeling data of defect locations. 9 .

9. The system for generating industrial defect samples based on deep learning according to claim 8, wherein the construction of the confrontation generation depth model is performed according to image sampling;

10. The system for generating industrial defect samples based on deep learning according to claim 1, wherein the adversarial generation deep model comprises a generator network and a discriminator network;