Disclosure of Invention
The invention aims to provide an image-based method for detecting abnormal space defects, which realizes measurement and detection through images, integrates a dimension measurement function, replaces a measurement mode of manual spot check and improves efficiency.
In order to achieve the above object, the present invention provides an image-based pitch anomaly type defect detection method,
S1, template matching;
S2, morphological detection and measurement detection;
s3, performing defect fusion and outputting a detection result;
Step S3 also includes step S31, arranging the output detection results according to particle level;
S32, fusing defects under the same ROI to be detected, judging whether the defect areas detected under the current ROI to be detected have intersection, judging whether two or more defect connected domains are overlapped, if so, merging, and fusing a plurality of overlapped defects into one defect to be used as an output detection result.
Preferably, the step S1 further comprises a step S11 of correcting the image to be detected through a rotation characteristic area, wherein the rotation characteristic area is formed by framing two independent communication areas with the same size and shape on the left side of the image to be detected, the two areas are in a vertical and upper-lower position relation on the original material, and the matching template is searched on the image to be detected and an affine matrix is obtained.
Preferably, the step S2 further comprises a step S21 of mapping MASK onto the image to be detected in an affine manner, wherein the area covered by MASK on the image to be detected is the area to be detected.
Preferably, the step S21 further includes a step S210 of processing each MASK using gray values and morphology on the template map, when the MASK is required to be reduced, performing etching operation with the structural element S for the number of repeated iterations with a positive integer greater than 0 as the number of iterations, and when the MASK is required to be enlarged, performing expansion operation with the same structural element, and obtaining the enlarged MASK.
Preferably, S211, when the extraction result is not empty, the defect size screening is performed.
Preferably, the step S2 further comprises a step S221 of mapping the ROI to be detected on the template diagram of the MASK onto the image to be detected in an affine manner by using an affine matrix;
s222, denoising the image to be detected;
s223 of enhancing the boundary of the image to be detected,
imgR=η*round(imgO-imgM)+imgO (5),
Wherein imgR represents an enhanced image, imgO represents a smoothed and denoised image, imgM represents an image subjected to mean filtering, η represents a contrast enhancement weight, round represents a result obtained by rounding and rounding the result of subtraction of the two images;
s224, extracting the boundary of the structure in the ROI to be detected, and extracting a sub-pixel boundary line on the obtained enhanced image;
the image g (x, y) subjected to gaussian filtering is:
Wherein, Performing convolution operation on the two-dimensional Gaussian kernel and the original image to obtain a smoothed image g (x, y), and calculating gradient amplitude and direction of the smoothed image according to a Sobel filter, wherein the kernel used by the Sobel filter is as follows:
Gradient values g x(m,n),gy (m, n) in different directions are obtained through the two operators, and the actual gradient values and the gradient directions are calculated by using the gradient values in the two directions:
Performing non-maximum suppression to determine a final unique boundary line, and then generating a boundary line of the sub-pixels to obtain a boundary line which can be used for calculating the distance;
s225, judging the calculated distance direction;
The structure in the image to be detected has various directions, the direction of the distance to be calculated is firstly judged, the directions of two boundary lines are calculated by extracting the boundary on the template diagram of the MASK, a rectangular coordinate system is established in the width-height direction of the image to be detected, the angle of the boundary line is calculated from the positive X-half axis in the clockwise direction,
S226, calculating the distance between the extraction boundaries;
before calculating the distance, judging whether the number of the extracted boundary lines exceeds 2, and if so, merging the boundary lines once;
Traversing two boundary lines, picking out a pair of points on the boundary lines to form a point pair during each calculation, calculating the distance D of a certain point pair on the two boundary lines through a formula (9),
D=L*sin(|α-β|) (9);
Wherein L represents Euclidean distance between two points, and sin (|alpha-beta|) is a sine value of an included angle between a straight line where the point pair is located and the spacing direction;
Traversing all boundary lines needing to calculate the distance in the image to be detected, comparing the calculation result with the set distance value, and marking the ROI to be detected where the current boundary is located as a defect if the numerical value of the calculation result is smaller than the set value.
Compared with the prior art, the method has the beneficial effects that 1) the dimension measuring function is integrated on the basis of the traditional appearance detection, the measurement detection is realized through the image, the measurement mode of manual spot check is replaced, the manual detection cost is reduced, and the production efficiency is improved;
2) The defect merging is performed efficiently, the defect information is output accurately without redundancy, so that an equipment operator can quickly and accurately perform repeated judgment and acceptance, and the detection rate of defects with different severity degrees is effectively improved;
3) The problem that the area edge detection is increased due to the fact that the traditional method excessively depends on affine areas to be detected after template matching is solved, and accuracy is effectively improved and detection omission ratio is reduced through detection of the structural edge spacing with higher accuracy.
Detailed Description
In order to make the objects, technical solutions and advantageous technical effects of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and detailed description. It should be understood that the detailed description is intended to illustrate the invention, and not to limit the invention.
Referring to fig. 1, the present invention provides a method for detecting abnormal pitch defects based on images,
S1, template matching;
referring to fig. 2 to 6, step S1 further includes step S11 of correcting the image to be detected by rotating a rotation feature region, wherein the rotation feature region is formed by framing two independent communication regions with the same size and shape on the left side of the image to be detected, and the two regions are in a vertical and upper-lower position relationship on the raw material;
searching a matching template on the image to be detected and obtaining an affine matrix, wherein the obtained affine matrix comprises parameters of image rotation, size scaling and translation in a 2D plane.
S2, morphological detection and measurement detection;
Step S2 also includes step S21, morphological detection, mapping MASK (template MASK) onto the image to be detected in an affine mode, wherein the area covered by MASK on the image to be detected is the area to be detected;
The MASK refers to a region for defect detection extracted through morphological processing in the to-be-detected ROI (region of interest) on the template map, and is smaller than the detection region in consideration of affine accuracy and compatible material tolerance, so as to avoid a large number of false detections, so that the MASK is used for performing conventional pitch anomaly defect detection:
S210, each MASK is obtained by using gray values and morphological processing on a template diagram, the gray values are intuitively seen from the gray diagram of the template diagram, and the gray range for binarization extraction corresponds to a normal and defect-free template diagram;
The structural elements used for the corrosion and expansion treatment are:
In the formula (1), S represents a structural element, and the center point of the structural element is set to be a reference point. When the MASK needs to be reduced, the positive integer larger than 0 is used as the iteration number, and the structural element S is used for carrying out the corrosion operation of the repeated iteration number, so that the reduced MASK is finally obtained. When expansion of MASK is required, expansion operation is performed by using the same structural element to obtain expanded MASK.
It should be noted that, the range of the defect is outside the gray value, the defect gray range is used for performing binarization extraction on the area covered by the map MASK to be detected, and when the extraction result is not empty, the defect point is indicated to exist;
S211, when the extraction result is not empty, performing defect size screening. Because of the factors of materials and imaging, part of defect points with small size are actually noise points or non-defect structures, the non-defect points are thrown away through a set size threshold value in a template, and finally the defects of conventional detection are obtained.
S22, measuring and detecting, namely mapping the ROI to be detected on the template diagram of the MASK to an image to be detected in an affine mode;
The ROI is a rough region for extracting MASK when a template is created, and comprises a region to be detected and other regions, wherein critical defects cannot be well dealt with by morphological processing, and detection of gap defects which are easy to leak and difficult to detect is needed to be carried out again based on the ROI to be detected;
S221, mapping the ROI to be detected onto the image to be detected in an affine mode by using the affine matrix obtained in the S1;
S222, denoising the image to be detected firstly due to noise of the light source and the imaging system;
Taking a bilateral filtering as an example of this,
The bilateral filtering is nonlinear filtering, and the filter kernel is generated by two functions, wherein one function determines the template coefficient of the filter by the Euclidean distance of the pixel, and the other function determines the coefficient of the filter by the gray level difference value of the pixel;
in the above formula (2), g (i, j) represents a pixel to be finally output, f (k, l) represents a pixel value at a template window coordinate of the bilateral filter, and ω (i, j, k, l) is a template of the bilateral filter:
the template of the bilateral filter is obtained by multiplying a distance template d (i, j, k, l) and a value domain template r (i, j, k, l), and meanwhile, spatial domain information and gray level similarity are considered, so that the boundary can be well reserved while smooth denoising is achieved.
In the above formula (4), the distance template is a two-dimensional gaussian function, the value domain template is a one-dimensional gaussian function, wherein exp represents an exponential function, sigma represents a standard deviation of the gaussian function, i, j represents coordinates of an output pixel point, and k, l represents coordinates of a template window.
S223 of enhancing the boundary of the image to be detected,
imgR=η*round(imgO-imgM)+imgO (5),
Wherein imgR represents an enhanced image, imgO represents a smoothed and denoised image, imgM represents an image subjected to mean filtering, η represents a contrast enhancement weight, round represents a result obtained by rounding and rounding the result of subtraction of the two images;
Further, η is a contrast enhancement weight, which acts on the result of subtraction of two images, and represents the degree of scaling of the enhancement result, and a larger value is required for a larger contrast, and vice versa.
S224, extracting the boundary of the structure in the ROI to be detected, and extracting a sub-pixel boundary line on the obtained enhanced image;
Further, taking Canny operator as an example, gaussian filtering is first performed, and since gaussian filtering can blur an image, an image to be detected is enhanced first, so that precision loss is reduced, and an image g (x, y) subjected to gaussian filtering is:
Wherein, Performing convolution operation on the two-dimensional Gaussian kernel and the original image to obtain a smoothed image g (x, y), and calculating gradient amplitude and direction of the smoothed image according to a Sobel filter, wherein the kernel used by the Sobel filter is as follows:
Gradient values g x(m,n),gy (m, n) in different directions are obtained through the two operators, and the actual gradient values and the gradient directions are calculated by using the gradient values in the two directions:
and then performing non-maximum suppression to determine the last unique boundary line, and then generating the boundary line of the sub-pixels to finally obtain the boundary line which can be used for calculating the distance.
S225, judging the calculated distance direction;
The structure in the image to be detected has various directions, the direction of the distance to be calculated is firstly judged, the directions of two boundary lines are calculated by extracting the boundary on the template diagram of the MASK, a rectangular coordinate system is established in the width-height direction of the image to be detected, the angle (0-180 DEG) of the boundary line is calculated from the X-positive half axis (the width-side right direction is the X-positive half axis) in the clockwise direction,
S226, calculating the distance between the extraction boundaries;
Before calculating the distance, judging whether the number of the extracted boundary lines exceeds 2, if so, merging the boundary lines once, and restoring the boundary which is supposed to be a disconnected boundary;
in the calculation, two boundary lines are expressed by a set of discrete points, so that the distance between the two boundary lines is converted into the distance between the points, the two boundary lines are traversed, a point pair on a pair of boundary lines is selected for each calculation, the distance D of a certain point pair on the two boundary lines is calculated through a formula (9),
D=L*sin(|α-β|) (9);
Wherein L represents Euclidean distance of two points, sin (|alpha-beta|) is a sine value of an included angle between a straight line where a point pair is located and the distance direction, so that the value after projection is calculated on the Euclidean distance L conveniently;
Traversing all boundary lines needing to calculate the distance in the image to be detected, comparing the calculation result with the set distance value, and marking the ROI to be detected where the current boundary is positioned as a defect if the numerical value of the calculation result is smaller than the set value;
s3, performing defect fusion and outputting a detection result;
S31, sorting the output detection results according to the particle level (namely, merging and judging the defects detected in one particle);
S32, fusing defects under the same ROI to be detected (a plurality of ROIs to be detected exist on one particle), judging whether the defect areas detected under the current ROI to be detected have intersection, judging whether two or more defect connected domains are overlapped, if so, merging, and fusing a plurality of overlapped defects into one defect to serve as an output detection result, and if not, indicating that the defects are independent defects and are not required to be fused.
It should be noted that there is also a merging mode, namely distance merging. The method is an optional merging mode, for some defects which are easy to occur in a discrete mode, a merging interval value is manually set, when the distance between two defects is smaller than the merging interval value, the defects which meet the conditions are merged, and the Euclidean distance of a pixel level is adopted in a distance calculation mode.
The present invention is not limited to the details and embodiments described herein, and thus additional advantages and modifications may readily be made by those skilled in the art, without departing from the spirit and scope of the general concepts defined in the claims and the equivalents thereof, and the invention is not limited to the specific details, representative apparatus and examples shown and described herein.