CN116433651A - Small sample panel defect detection method, system, equipment and storage medium - Google Patents
Small sample panel defect detection method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN116433651A CN116433651A CN202310518810.5A CN202310518810A CN116433651A CN 116433651 A CN116433651 A CN 116433651A CN 202310518810 A CN202310518810 A CN 202310518810A CN 116433651 A CN116433651 A CN 116433651A
- Authority
- CN
- China
- Prior art keywords
- defect detection
- sample image
- defect
- image
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, a system, equipment and a storage medium for detecting defects of a panel with a small sample, which relate to the technical field of defect detection, and the method comprises the following steps: pre-training a deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier; performing sample fusion processing and/or sample balancing processing on the original sample image and the new sample image to obtain a training data set; defect detection model M based on training data set 1 Classifier parametersAdjusting to obtain a defect detection model M 2 The method comprises the steps of carrying out a first treatment on the surface of the Based on defect detection model M 2 And performing defect detection on the image to be detected to output defect identification and classification results. According to the invention, a training mode of fine adjustment of the model is adopted, so that relevant characteristic information can be better learned for defects of a small number of samples, and further, the accuracy of defect detection is improved.
Description
Technical Field
The invention relates to the technical field of defect detection, in particular to a small sample panel defect detection method, a system, equipment and a storage medium.
Background
The panel processing factory can generate a plurality of defects in the panel production process, however, the whole panel product has complex production process flow and long production period, and long time is often required from the base plate to the production and processing, so that the defects generated in each process section need to be monitored at all times, and the defect is prevented from flowing into the next production Cheng Zaocheng to reduce the yield.
The traditional defect detection method is to collect images of panel products through an AOI (Automatic Optical Inspection) instrument, then manually judge the defects of the images, and once the defects occur, the defects are modified so as not to flow into the next process. The traditional defect detection relies on an artificial naked eye graph judgment, is easily influenced by personnel experience differences and mental states, and has high misjudgment rate and relatively high labor cost; therefore, many panel processing factories begin to introduce automatic defect detection and classification systems for replacing manual defect detection, however, the systems generally adopt a target detection algorithm based on deep learning as a core detection algorithm, the accuracy of graph judgment applied to defect detection and classification is limited by the defect distribution condition of training set pictures, when the number of samples is unbalanced, image characteristic information cannot be learned when the number of samples is less, so that the problem of missed detection easily occurs due to lower accuracy of small sample defect prediction.
Disclosure of Invention
The invention provides a small sample panel defect detection method, a system, equipment and a storage medium, which solve the problems of low accuracy of small sample panel defect prediction and easy occurrence of false detection and missing detection.
In a first aspect, an embodiment of the present invention provides a method for detecting a defect in a panel leakage process, the method including the following steps:
pre-training a deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier;
performing sample fusion processing and/or sample balancing processing on the original sample image and the new sample image to obtain a training data set;
defect detection model M based on training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 ；
Based on defect detection model M 2 Treatment ofThe detected image is subjected to defect detection to output defect identification and classification results.
In the above embodiment, the present invention first performs preliminary training on the deep learning model through a large number of original sample images to obtain the defect detection model M 1 At this time, defect detection model M 1 The feature extractor can extract the defect features, and the classifier can accurately identify and classify the base class defects; then carrying out sample fusion processing and/or sample equalization processing on a large number of original samples and a small number of new samples, wherein the number of images contained in the obtained training data set is definitely larger than that of the new samples because of various image fusion modes, and the related operation of marking the positions of the new defects can be omitted because the relative positions of the image fusion are known; finally, the defect detection model M is subjected to training data set 1 The classifier is trained again, so that fine adjustment of the model is realized, and the model after fine adjustment can accurately identify and classify new defects.
As some optional embodiments of the application, the defect detection model M 1 The classifier based on cosine similarity is used for calculating similarity scores among different defect categories in a cosine similarity calculation mode, so that the intra-class variance is reduced, and the detection accuracy of new types of defects is improved under the condition of less training quantity.
As some optional embodiments of the present application, the defect detection model M is based on a training data set 1 When the classifier of (1) performs parameter adjustment, relevant parameters of the feature extractor are frozen, namely, the relevant parameters of the feature extractor are not changed.
As some optional embodiments of the present application, a deep learning model is pre-trained based on the original sample image to obtain a defect detection model M 1 The flow of (2) is as follows:
marking the basic type defects on the original sample image, namely marking the positions and types of the basic type defects;
and inputting the marked original sample image into a deep learning model for feature extraction and defect classification.
As some optional embodiments of the present application, the flow of performing the sample fusion processing on the original sample image and the new sample image is as follows:
performing defect extraction processing on the new type defect image on the new sample image to obtain the new type defect image;
and carrying out image fusion processing on the new type defect image and the original sample image in a pixel superposition mode, so that different position relations exist between the new type defect and the basic type defect to obtain a fused sample image, and adding the fused sample image into a training data set.
The sample equalization processing flow for the original sample image and the new sample image is as follows:
screening the original sample image, and marking new defects of the new sample image, namely marking the positions and types of the new defects;
and adding the screened original sample image and the new sample image after labeling into the training data set in the same proportion.
In the above embodiment, by screening the original sample image, a small amount of images most representative of the basic defects can be obtained, and the images are added to the training data set in the same proportion as the new sample image, so that the problem of low detection precision of the small sample defects caused by sample imbalance can be solved.
As some alternative embodiments of the application, the positional relationship of the new class defect and the base class defect includes overlapping, intersecting and separating.
As some optional embodiments of the present application, the defect detection model M is based on a training data set 1 The flow of parameter adjustment by the classifier is as follows:
inputting the fused sample image, the original sample image and the new sample image in the training dataset into the defect detection model M 1 ；
And gradually reducing the loss function value by adopting a back propagation mode so as to realize parameter adjustment of the classifier.
In the above embodiment, the present invention first passes through the defect detection model M 1 Is subjected to new class defects by a feature extractor of (1)Line feature map extraction and then by applying to the defect detection model M 1 Parameter adjustment is carried out by the classifier of (1) through a defect detection model M 1 And the classifier is used for classifying the defects of the feature extraction graph, and based on a back propagation mode, the parameters are continuously updated by calculating the loss function value, so that the loss function value is smaller.
In a second aspect, the present invention provides a small sample panel defect detection system, the system comprising:
a model pre-training unit for pre-training the deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier;
the image processing unit is used for carrying out sample fusion processing and/or sample balance processing on the original sample image and the new sample image so as to obtain a training data set;
model parameter adjustment unit for adjusting the defect detection model M based on the training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 ；
A defect detection unit based on a defect detection model M 2 And performing defect detection on the image to be detected to output defect identification and classification results.
In a third aspect, the present invention provides a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the small sample panel defect detection method when executing the computer program.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the small sample panel defect detection method.
The beneficial effects of the invention are as follows: according to the invention, the deep learning model is trained in a mode of fine tuning, so that a small number of samples can learn related characteristic information better, and further the accuracy of panel defect detection is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a small sample panel defect detection method according to an embodiment of the present invention;
FIG. 2 is a flow chart of model pre-training according to an embodiment of the present invention;
FIG. 3 is a flow chart of model fine-tuning according to an embodiment of the present invention;
fig. 4 is a block diagram of a small sample panel defect detection system according to an embodiment of the present invention.
Detailed Description
In order to better understand the above technical solutions, the following detailed description of the technical solutions of the present invention is made by using the accompanying drawings and specific embodiments, and it should be understood that the specific features of the embodiments and the embodiments of the present invention are detailed descriptions of the technical solutions of the present invention, and not limiting the technical solutions of the present invention, and the technical features of the embodiments and the embodiments of the present invention may be combined with each other without conflict.
It should also be appreciated that in the foregoing description of at least one embodiment of the invention, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of at least one embodiment of the invention. This method of disclosure, however, is not intended to imply that more features than are required by the subject invention. Indeed, less than all of the features of a single embodiment disclosed above.
Example 1
Referring to fig. 1, an embodiment of the present invention provides a method for detecting a defect of a panel with a small sample, the method includes the following steps:
(1) Pre-training a deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprising a feature extractor and a classifier, the deep learning model can be Faster RCNN, etc., and the defect detection model M 1 The feature extractor is used for extracting the feature map, and the classifier is used for classifying the defect categories of the feature map.
Specifically, referring to fig. 2, the feature extractor includes a backbone network, a region candidate network, a region of interest pooling network, and a region of interest feature extraction network.
Specifically, referring to fig. 2, a depth network model is pre-trained based on an original sample image to obtain a defect detection model M 1 The flow of (2) is as follows:
the method comprises the steps of (1.1) marking basic defects on an original sample image, namely marking positions and types of the basic defects, wherein the basic defects are common panel defects, so that the number of the original sample images is large, and image collection is easy to carry out;
and (1.2) inputting the marked sample image into a deep learning model for feature extraction and defect classification, and inputting the original sample image into the deep learning model for model training, so that the model has the feature extraction capability on panel defects and the classification processing capability on basic defects.
(2) Performing sample fusion processing and/or sample equalization processing on the original sample image and the new sample image to obtain a training data set;
specifically, the flow of sample fusion processing for the original sample image and the new sample image is as follows:
(2.1) performing defect extraction processing on the new type defect image on the new sample image to obtain the new type defect image; the new defects are rare panel defects, so that the number of new sample images is small, and image collection is difficult;
(2.2) performing image fusion processing on the new type defect image and the original sample image in a pixel superposition mode, so that different position relations exist between the new type defect and the basic type defect to obtain a fused sample image, and adding the fused sample image into a training data set; the image only containing the new type of defects (excluding the background image) is subjected to image fusion with the original sample image (including the background image) so that the fused sample image contains the new type of defects and the basic type of defects, and therefore the position of the new type of defects and the type of the new type of defects can be marked according to the basic type of images, and the auxiliary feature extractor can extract feature images of the new type of images; the position relation of the new type defect and the basic type defect comprises overlapping, intersecting and separating.
Specifically, the sample equalization processing procedure for the original sample image and the new sample image is as follows:
(2.3) screening the original sample image, and marking new defects of the new sample image, namely marking the positions and types of the new defects;
(2.4) adding the screened original sample image and the noted new sample image to the training dataset.
In the embodiment of the invention, a small number of images which can most represent basic defects can be obtained by screening the original sample images, and the images are added into the training data set in the same proportion as the new sample images, so that the problem of low detection precision of the small sample defects caused by sample imbalance can be solved.
(3) Defect detection model M based on training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 The defect detection model M 1 The classifier based on cosine similarity; the defect detection model M 1 The classifier based on cosine similarity is used for calculating similarity scores among different defect categories in a cosine similarity calculation mode, so that the intra-class variance is reduced, and the detection accuracy of new types of defects is improved under the condition of less training quantity.
Specifically, a defect detection model M is paired based on a fused sample image 1 Freezing the relevant parameters of the feature extractor when the classifier of (a) performs parameter adjustmentThis is because the feature extractor already has feature map extraction capabilities, so no corresponding parameter adjustments are necessary;
specifically, the defect detection model M is based on a training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 The flow of (1) is as follows, please refer to fig. 3:
(3.1) inputting the fused sample image, the original sample image and the new sample image in the training dataset into the defect detection model M 1 ；
(3.2) gradually reducing the loss function value by adopting a counter-propagation mode so as to realize parameter adjustment of the classifier;
specifically, the calculation formula of the loss function value is:
wherein L is i (S i,j ,y i ) The classification category calculated on behalf of the classifier belongs to the true category y i N represents the number of training dataset sample images;
specifically, the calculation formula of cosine similarity is:
wherein α represents a scaling factor, F (X) i Weights representing input feature patterns, W j Representing weights of other different classes, gamma m Representing parameter adjustment data;
in the embodiment of the invention, when the defect detection model M is input 1 When the sample of (a) is an original sample image or a new sample image after labeling: gamma ray m Takes a value of 1, and properly adjusts F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; when the defect detection model M is input 1 Is fusion of samples (1)Sample image: when the fused sample image is that the new type defect is overlapped with the basic type defect, the gamma is correspondingly reduced m Properly adjust F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; (II) when the fused sample image is the intersection of the new type defect and the basic type defect, properly increasing gamma based on the first type defect m Properly adjust F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; thirdly, when the fused sample image is that the new type defect is separated from the basic type defect, gamma m Takes a value of 1, and properly adjusts F (X) i And weight W j So that the cosine similarity S i,j And the value of the loss function value is continuously calculated in a back propagation manner, so that the loss function value is smaller.
(4) Based on defect detection model M 2 Performing defect detection on the image to be detected to output defect identification and classification results, namely inputting the image to be detected which can contain new types of defects into a defect detection model M 2 Through a defect detection model M 2 Judging whether a new type of defect exists.
In the embodiment of the invention, after the model fine adjustment is performed based on the new type of defects, relevant parameters of the model fine adjustment can be stored in an independent space, and when the defects need to be detected, only the relevant parameters need to be called, so that the model can accurately detect various defects in a parameter calling mode.
In the above embodiment, the present invention first performs preliminary training on the deep learning model through a large number of original sample images to obtain the defect detection model M 1 At this time, defect detection model M 1 The feature extractor can extract the defect features, and the classifier can accurately identify and classify the base class defects; then, image fusion processing and/or sample equalization processing are/is carried out on a large number of original samples and a small number of new samples, andthe number of images contained in the obtained training data set is definitely larger than that of the new sample images because of various image fusion modes, and related operation of labeling new defect positions can be omitted because the relative positions of the image fusion are known; finally, the defect detection model M is subjected to training data set 1 The classifier is trained again, so that fine adjustment of the model is realized, and the model after fine adjustment can accurately identify and classify new defects.
Example 2
The present invention provides a small sample panel defect detection system, which corresponds to the method of embodiment 1 one by one, referring to fig. 4, and the system comprises:
(1) A model pre-training unit for pre-training the deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier;
(2) The image processing unit is used for carrying out sample fusion processing and/or sample equalization processing on the original sample image and the new sample image so as to obtain a training data set;
(3) Model parameter adjustment unit for adjusting the defect detection model M based on the training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 ；
(4) A defect detection unit based on a defect detection model M 2 And performing defect detection on the image to be detected to output defect identification and classification results.
Specifically, the model pre-training unit pre-trains the depth network model based on the original sample image to obtain a defect detection model M 1 The flow of (2) is as follows:
the method comprises the steps of (1.1) marking basic defects on an original sample image, namely marking positions and types of the basic defects, wherein the basic defects are common panel defects, so that the number of the original sample images is large, and image collection is easy to carry out;
and (1.2) inputting the marked sample image into a deep learning model for feature extraction and defect classification, and inputting the original sample image into the deep learning model for model training, so that the model has the feature extraction capability on panel defects and the classification processing capability on basic defects.
Specifically, the flow of the sample fusion processing of the original sample image and the new sample image by the image processing unit is as follows:
(2.1) performing defect extraction processing on the new type defect image on the new sample image to obtain the new type defect image; the new defects are rare panel defects, so that the number of new sample images is small, and image collection is difficult;
(2.2) performing image fusion processing on the new type defect image and the original sample image in a pixel superposition mode, so that different position relations exist between the new type defect and the basic type defect to obtain a fused sample image, and adding the fused sample image into a training data set; the image only containing the new type of defects (excluding the background image) is subjected to image fusion with the original sample image (including the background image) so that the fused sample image contains the new type of defects and the basic type of defects, and therefore the position of the new type of defects and the type of the new type of defects can be marked according to the basic type of images, and the auxiliary feature extractor can extract feature images of the new type of images; the position relation of the new type defect and the basic type defect comprises overlapping, intersecting and separating.
Specifically, the flow of sample equalization processing of the original sample image and the new sample image by the image processing unit is as follows:
(2.3) screening the original sample image, and marking new defects of the new sample image, namely marking the positions and types of the new defects;
(2.4) adding the screened original sample image and the noted new sample image to the training dataset.
Specifically, the model parameter adjustment unit adjusts the defect detection model M based on the training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 The flow of (2) is as follows:
(3.1) Inputting the fused sample image, the original sample image and the new sample image in the training dataset into the defect detection model M 1 ；
(3.2) gradually reducing the loss function value by adopting a counter-propagation mode so as to realize parameter adjustment of the classifier;
specifically, the calculation formula of the loss function value is:
wherein L is i (S i,j ,y i ) The classification category calculated on behalf of the classifier belongs to the true category y i N represents the number of training dataset sample images;
specifically, the calculation formula of cosine similarity is:
wherein α represents a scaling factor, F (X) i Weights representing input feature patterns, W j Representing weights of other different classes, gamma m Representing parameter adjustment data;
in the embodiment of the invention, when the defect detection model M is input 1 When the sample of (a) is an original sample image or a new sample image after labeling: gamma ray m Takes a value of 1, and properly adjusts F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; when the defect detection model M is input 1 When the sample of (a) is a fused sample image: when the fused sample image is that the new type defect is overlapped with the basic type defect, the gamma is correspondingly reduced m Properly adjust F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; (II) when the fused sample image is a new type of defectWhen the notch intersects with the basic defect, the gamma is properly increased on the basis of (one) m Properly adjust F (X) i And weight W j So that the cosine similarity S i,j The values of (2) are kept balanced, and a back propagation mode is adopted to continuously calculate the loss function value, so that the loss function value is smaller; thirdly, when the fused sample image is that the new type defect is separated from the basic type defect, gamma m Takes a value of 1, and properly adjusts F (X) i And weight W j So that the cosine similarity S i,j And the value of the loss function value is continuously calculated in a back propagation manner, so that the loss function value is smaller.
In the embodiment of the invention, the system adopts a training mode of model fine adjustment, so that relevant characteristic information can be better learned for defects of a small number of samples, and further the accuracy of defect detection is improved.
Example 3
The invention provides a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing a small sample panel defect detection method as described in embodiment 1 when executing the computer program.
The computer device provided in this embodiment may implement the method described in embodiment 1, and in order to avoid repetition, a description thereof will be omitted.
Example 4
The present invention provides a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, implements a small sample panel defect detection method as described in embodiment 1.
The computer readable storage medium provided in this embodiment may implement the method described in embodiment 1, and will not be described herein in detail to avoid repetition.
The processor may be a central processing unit (CPU, central Processing Unit), other general purpose processors, digital signal processors (digital signal processor), application specific integrated circuits (Application Specific Integrated Circuit), off-the-shelf programmable gate arrays (Fieldprogrammable gate array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (11)
1. A method for detecting defects of a small sample panel, the method comprising the steps of:
pre-training a deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier;
performing sample fusion processing and/or sample equalization processing on the original sample image and the new sample image to obtain a training data set;
defect detection model M based on training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 ；
Based on defect detection model M 2 And performing defect detection on the image to be detected to output defect identification and classification results.
2. A method for detecting defects of a small sample panel according to claim 1, wherein the defect detection model M 1 Is a cosine similarity based classifier.
3. The method of claim 1, wherein the defect detection is modeled based on a training data setM is a kind of 1 And (3) freezing the relevant parameters of the feature extractor when the classifier is subjected to parameter adjustment.
4. The method for detecting defects of a small sample panel according to claim 1, wherein a deep learning model is pre-trained based on an original sample image to obtain a defect detection model M 1 The flow of (2) is as follows:
marking the basic type defects on the original sample image, namely marking the positions and types of the basic type defects;
and inputting the marked original sample image into a deep learning model for feature extraction and defect classification.
5. The method for detecting defects of a small sample panel according to claim 1, wherein the process of performing sample fusion processing on an original sample image and a new sample image is as follows:
performing defect extraction processing on the new type defect image on the new sample image to obtain the new type defect image;
and carrying out image fusion processing on the new type defect image and the original sample image in a pixel superposition mode, so that different position relations exist between the new type defect and the basic type defect to obtain a fused sample image, and adding the fused sample image into a training data set.
6. The method for detecting defects of a small sample panel according to claim 1, wherein the sample equalization process for the original sample image and the new sample image is as follows:
screening the original sample image, and marking new defects of the new sample image, namely marking the positions and types of the new defects;
and adding the screened original sample image and the new sample image after labeling into the training data set in the same proportion.
7. The method of claim 5, wherein the positional relationship between the new type of defect and the basic type of defect includes overlapping, intersecting, and separating.
8. The method for detecting defects of a small sample panel according to claim 5, wherein the defect detection model M is based on a training data set 1 The flow of parameter adjustment by the classifier is as follows:
inputting the fused sample image, the original sample image and the new sample image in the training dataset into the defect detection model M 1 ；
And gradually reducing the loss function value by adopting a back propagation mode so as to realize parameter adjustment of the classifier.
9. A small sample panel defect detection system, the system comprising:
a model pre-training unit for pre-training the deep learning model based on the original sample image to obtain a defect detection model M 1 Wherein the defect detection model M 1 Comprises a feature extractor and a classifier;
the image processing unit is used for carrying out sample fusion processing and/or sample balance processing on the original sample image and the new sample image so as to obtain a training data set;
model parameter adjustment unit for adjusting the defect detection model M based on the training data set 1 Parameter adjustment is performed on the classifier of (2) to obtain a defect detection model M 2 ；
A defect detection unit based on a defect detection model M 2 And performing defect detection on the image to be detected to output defect identification and classification results.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized by: the processor, when executing a computer program, implements a small sample panel defect detection method as claimed in any one of claims 1-8.
11. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements a small sample panel defect detection method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310518810.5A CN116433651A (en) | 2023-05-10 | 2023-05-10 | Small sample panel defect detection method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310518810.5A CN116433651A (en) | 2023-05-10 | 2023-05-10 | Small sample panel defect detection method, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116433651A true CN116433651A (en) | 2023-07-14 |
Family
ID=87079814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310518810.5A Pending CN116433651A (en) | 2023-05-10 | 2023-05-10 | Small sample panel defect detection method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116433651A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119205743A (en) * | 2024-11-25 | 2024-12-27 | 深圳新视智科技术有限公司 | Defect recognition method and device integrating discriminant model and generative model |
-
2023
- 2023-05-10 CN CN202310518810.5A patent/CN116433651A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119205743A (en) * | 2024-11-25 | 2024-12-27 | 深圳新视智科技术有限公司 | Defect recognition method and device integrating discriminant model and generative model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109977191B (en) | Problem map detection method, device, electronic equipment and medium | |
| CN110992329A (en) | Product surface defect detection method, electronic device and readable storage medium | |
| CN113222913B (en) | Circuit board defect detection positioning method, device and storage medium | |
| CN111242899B (en) | Image-based flaw detection method and computer-readable storage medium | |
| CN109344851B (en) | Image classification display method and device, analysis instrument and storage medium | |
| CN113222938A (en) | Chip defect detection method and system and computer readable storage medium | |
| KR100868884B1 (en) | Glass Substrate Glass Defect Information System and Classification Method | |
| CN112819821A (en) | Cell nucleus image detection method | |
| CN109146873A (en) | A kind of display screen defect intelligent detecting method and device based on study | |
| CN115861170A (en) | Surface defect detection method based on improved YOLO V4 algorithm | |
| US12094098B2 (en) | Defect management apparatus, method and non-transitory computer readable medium | |
| CN114926407A (en) | Steel surface defect detection system based on deep learning | |
| CN114462469B (en) | Training method of target detection model, target detection method and related device | |
| CN113962980A (en) | Glass container defect detection method and system based on improved YOLOV5X | |
| CN115409797A (en) | A PCB defect image detection method based on improved deep learning algorithm | |
| CN113313678A (en) | Automatic sperm morphology analysis method based on multi-scale feature fusion | |
| CN116188432A (en) | Training method and device of defect detection model and electronic equipment | |
| TWI747686B (en) | A defect detection method and a defect detection device | |
| CN113936294A (en) | Construction site personnel identification method, readable storage medium and electronic device | |
| CN116433651A (en) | Small sample panel defect detection method, system, equipment and storage medium | |
| Choi et al. | Deep learning based defect inspection using the intersection over minimum between search and abnormal regions | |
| CN206897873U (en) | A kind of image procossing and detecting system based on detection product performance | |
| CN109741351A (en) | A class-sensitive edge detection method based on deep learning | |
| CN106447045B (en) | The assessment method of ADAS system based on machine learning | |
| CN113642473A (en) | Mining coal machine state identification method based on computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |