CN119515825A

CN119515825A - A method for detecting industrial glass defects based on Visionmaster

Info

Publication number: CN119515825A
Application number: CN202411578936.2A
Authority: CN
Inventors: 曾伟伟; 闫浩霖; 安伯阳; 刘成午; 龚骏宇; 王正松; 杨乐
Original assignee: Northeastern University China
Current assignee: Northeastern University China
Priority date: 2024-11-07
Filing date: 2024-11-07
Publication date: 2025-02-25

Abstract

The innovation provides a Visionmaster-based industrial glass defect detection method which is provided for pain points of current glass defect detection and is combined with actual demands of enterprises. The traditional industrial glass defect detection method has a plurality of problems and defects, such as poor environmental adaptability, lack of robustness, low precision, dependence on manual adjustment and the like. In order to solve the problems, the innovation adopts the steps of image preprocessing, detecting whether glass is transversely or longitudinally divided, detecting glass edge defects and detecting glass surface defects, and judging glass qualification by summarizing the results. Firstly, noise interference is removed through an image processing technology, the position and the edge of a glass area are obtained, and the contrast is improved by utilizing an image enhancement means, so that the image is clearer. Based on Visionmaster, it is determined whether the glass is intersected or divided longitudinally, whether an edge defect exists, and whether a surface defect exists by using conventional vision techniques and setting an appropriate threshold range. Finally, by summarizing the detection results of all links, whether the industrial glass is qualified or not is determined, and if the industrial glass is unqualified, what defects exist in the glass is determined. The method has the advantages of reducing the coding difficulty, simplifying the parameter adjusting process in the detection process, improving the accuracy, the universality and the anti-interference capability of the industrial glass defect detection, reducing the labor cost, providing higher guarantee for the product quality and accelerating the detection speed and the detection efficiency. In addition, the method has strong interpretability and adaptability, and is suitable for complex and changeable industrial field environments. The method effectively solves the challenges of the traditional method while improving the production efficiency, and provides a more effective and reliable detection method for the industrial glass production process.

Description

Visionmaster-based industrial glass defect detection method

Technical Field

The invention relates to the technical field of industrial glass defect detection, in particular to a Visionmaster-based industrial glass defect detection method.

Background

Industrial glass plays an important role in numerous industrial applications, involving multiple fields of high temperature, high pressure, and special optical requirements. With the increasing complexity of industrial production, the standard of quality control of glass is continuously improved, and in particular, in the aspect of detecting defects on the surface of glass, such defects may include scratches, bubbles, cracks and the like, and any minor flaws may affect the quality and safety of the product. Therefore, it is important to develop an efficient and accurate glass defect detection system.

The traditional glass defect detection mainly depends on a manual mode, so that the efficiency is low, the influence of visual fatigue and subjective judgment is easy to influence, and particularly when a large number of products are faced, the rapid and accurate detection requirement is difficult to meet. The introduction of machine vision techniques provides an effective solution for this, utilizing image processing and computer vision techniques to identify and locate various defects.

However, the existing machine vision technology still faces some challenges in glass defect detection, namely firstly, the reflection and refraction characteristics of the glass surface lead to the formation of a plurality of outlines of light, difficulty in defect detection is brought to result inaccuracy or false alarm, secondly, the detection of micro-bubble defects is troublesome, the detection algorithm is difficult to accurately identify due to the fact that the sizes and the shapes of the micro-bubble defects are variable, furthermore, the stability and the accuracy can be affected when the traditional rule-based image processing algorithm processes complex defects, and finally, although the detection method based on a neural network can improve the detection accuracy under certain conditions, a large amount of labeling data is usually required for training, and the training process and the data quality are extremely sensitive.

Thus, industrial glass defect detection is a challenging task, requiring us to continually study and explore new technologies and methods that are more efficient and superior. In addition to the above solutions, there are some emerging technical approaches that deserve research, such as three-dimensional vision systems. The three-dimensional vision system captures three-dimensional structural information of an object by utilizing a depth sensor and a stereo camera, and is widely applied to the fields of industrial detection, robot navigation, automatic manufacturing and the like. The method can more accurately identify the shape, the size and the position by calculating the depth information of the object, thereby improving the detection precision. For example, in defect detection, three-dimensional vision can effectively identify microscopic defects on complex surfaces, overcoming the limitations of conventional two-dimensional images. In addition, the technology can be combined with a machine learning algorithm, so that the recognition and classification capabilities are improved, and stronger support is provided for an automatic process.

In summary, industrial glass defect detection is a significant problem related to the actual production and wide application scenarios of enterprises. In view of the above challenges and problems and in combination with the actual needs of enterprises, we have studied an industrial glass defect detection method based on Visionmaster. By using a proper algorithm and a proper model, the method can rapidly and accurately identify the glass defects, and provides a guarantee for the product quality of glass manufacturing enterprises.

Disclosure of Invention

Aiming at the defects of the prior art and combining with the actual demands of enterprises, the invention provides a Visionmaster-based industrial glass defect detection technology.

The method comprises the following steps:

And step 1, preprocessing an image. First, gray level matching is performed on an image captured by an industrial line camera to obtain the position of a glass region in the image. The gray-scale-based matching algorithm is also called a template-based matching algorithm, the gray values of the corresponding positions of the subgraph and the template graph are calculated to obtain a value, the value can be the sum of the absolute values of the subtraction of corresponding pixels (namely SAD algorithm), the calculated sum can be divided by the number of the pixels of the template (namely MAD algorithm), and other gray-scale-based image matching algorithms are based on the idea, but the calculated values are different. The invention adopts SAD algorithm because of simpler detection object and higher detection speed. The SAD algorithm is that the absolute values of the subtraction of the corresponding pixels of the template and the reference picture are summed, and then the template is sequentially slid to traverse each position to obtain a matrix of (P-M) rows and (Q-N) columns, wherein the position with the minimum value is that the similarity between the template and the reference picture is the highest. The SAD similarity formula is as follows (since the algorithm is directly calculated by a matrix, and is different in terms of formulation but substantially the same):

Wherein i is more than or equal to 1 and less than or equal to M-M+1, j is more than or equal to 1 and less than or equal to N-N+1.

The image is then gaussian filtered to remove noise. The main causes of noise generation involved in the process of capturing images are uneven brightness of the bar-shaped light source, fluctuation of ambient light, and interference in image transmission engineering. Aiming at the noise interference, the invention adopts a Gaussian filtering method to carry out filtering denoising on the image. Gaussian filtering is essentially a process of weighted averaging over the entire image. The value of each pixel is obtained by weighted average of the pixel and other pixel values in the neighborhood. The specific operation of the gaussian filtering is that a gaussian convolution kernel is first determined, wherein when an integer gaussian convolution kernel is determined, each pixel value in the fractional gaussian convolution kernel is divided by the pixel value in the upper left corner of the fractional gaussian convolution kernel, and rounded to obtain the integer gaussian convolution kernel. And using the Gaussian convolution kernel to scan each pixel in the image, using the weighted average value of the pixels in the neighborhood determined by the convolution kernel to replace the value of the central pixel point of the convolution area, multiplying the corresponding pixel value of the input image and the corresponding pixel value of the convolution kernel respectively, adding, if the added pixel value is larger than a pixel threshold 255, cutting off, dividing the added pixel value by 255, and taking the remainder as the central pixel value of the convolution area. Therefore, the noise reduction and the smoothing treatment of the image are realized, the image quality is improved, and the subsequent defect detection is convenient.

The edges of the glass in the image are then acquired using a Canny edge detection algorithm. In order to effectively identify the edge of the glass in the image, the influence of noise and false detection of the edge are reduced as much as possible, and meanwhile, in order to make subsequent processing and analysis easier and more efficient, a Canny edge detection method is adopted to acquire the contour of the glass in the image, and the method mainly comprises four parts of image noise reduction, gradient calculation, non-maximum suppression and double-threshold boundary tracking. The Canny edge detection gradient calculation is shown below:

Where f (x, y) is the gray value of the image at (x, y), Is the partial derivative in the x-direction,Is the partial derivative in the y-direction. The gradient amplitude and direction of this point can thus be obtained as:

And then, the image after edge detection is subjected to expansion treatment, so that the glass edge is clearer, and the contrast ratio is enhanced. The dilation process is mainly based on mathematical morphology and updates the pixel values of local areas of the image by means of structural elements, thus changing the characteristics of the image. Are commonly used in denoising, edge detection and morphological transformations of images. It enhances the features of the image by enlarging the target area in the image. The key component in the dilation process is a structural element, which is a variable shape and size template used to traverse the image and compare it to pixels. The structural element can be any shape such as rectangle, circle, cross, etc. The invention combines the characteristics of glass to finally select rectangular structural elements with the size of 5 multiplied by 5. The calculation formula is as follows:

And finally, correcting the image. In order to facilitate subsequent defect detection, the processed image is subjected to image correction, so that the influence caused by different positions and angles of glass is weakened or eliminated. The image correction mainly includes the steps of 1) detecting the reference point. First, the system needs to determine one or more fiducial points, which are typically known or easily identified feature points on the target object. The points may be corner points, edges, circle centers, etc. Here we choose the center position of the partial glass as the reference point. 2) And calculating the position deviation. The positional deviation is calculated by comparing the actually detected reference point position with a theoretical (or preset) position. This deviation may include various types of transformations, such as translation, rotation, scaling, etc. In the device, the matching points after gray level matching are selected as preset positions (under the same illumination environment, the matching points after gray level matching of different glasses are not much different, so that the positions of the corrected glass images are similar). 3) A transformation matrix is generated. A transformation matrix is generated based on the calculated positional deviation. This matrix describes how the image or a certain region of the image is transformed from the current position to the translation, rotation, scaling, etc. required for the correct position. 4) A transformation matrix is applied. And applying the generated transformation matrix to the input image to obtain the corrected glass image.

And 2, detecting whether the glass is transversely or longitudinally divided. According to the demands of enterprises, defects caused by errors can occur in the cutting process of the produced glass, namely, the glass is transversely or longitudinally divided. By observation, it is readily apparent that when the glass is traversed or translated, the interior region of the glass in the image must exhibit a crack through the glass. Thus, one can process the glass image and detect the transverse or longitudinal defects of the glass based on this feature.

Based on the analysis, median filtering is performed on the processed image, and two structural elements with different sizes are needed because of different transverse and longitudinal directions, so that the input image only keeps transverse lines when transverse is detected, and the input image only keeps longitudinal lines when longitudinal is detected. The median filtering (MEDIAN FILTER) is a commonly used nonlinear filtering technique, which is mainly used for removing noise in images or signals, and particularly has excellent performance when salt-and-pepper noise (salt-and-pepper noise) is removed. The basic principle is to replace the value of a point in a digital image or signal with the median of the values of each point in a neighborhood of the point. The median filtering selects pixel values of the pixel points in the digital image or the digital sequence and the adjacent pixel points (a common odd number of pixel points) around the pixel points, sequences the pixel values, and then takes the pixel value at the middle position as the pixel value of the current pixel point to enable the surrounding pixel values to be close to the true value, thereby eliminating isolated noise points. The invention adopts two different structural elements of 25 multiplied by 80 and 100 multiplied by 27 to respectively carry out filtering operation on the processed glass image.

And then using a straight line to search the inner region of the glass in the image after the medium value filtering to check whether a transverse or longitudinal penetrating crack exists in the glass, and if so, transversely or longitudinally dividing the glass, wherein the quality is unqualified.

And 3, detecting glass edge defects. If the glass is not transversely or longitudinally divided, edge defect detection is performed, and the edge defect is detected by adopting an edge-to-model defect detection method in consideration of different shapes and various types of edge defects. The edge pair model defect detection method is to compare the actually extracted edge pair with a pre-established edge pair model, analyze information such as existence, position, width and the like of the edge pair, and judge whether edge pair defects such as edge breakage and the like exist.

And 4, detecting defects on the surface of the glass. According to the enterprise demand, the types of surface defects mainly comprise round surface defects such as bubbles, pits and the like, and line-adjusting surface defects such as scratches, cracks and the like. In view of the defects, the method adopts the Blob analysis method to detect the surface defects. The specific flow of Blob analysis to detect surface defects mainly consists of several processes of image binarization, blob extraction, blob screening, blob sorting and Blob output.

First, the image is binarized. Image binarization refers to converting pixel points in an image to pure black (gray value 255) or pure white (gray value 0). After conversion, the image is divided into two parts of black and white. In the conversion process, the pixel points with gray values converted into 0 and 255 are specifically determined by a preset pixel threshold (i.e., a specific gray value). The present invention 1) the threshold value used by the team is 90, that is, the gray value is greater than or equal to 90 and is set as the foreground (gray value is set as 255), and the gray value is lower than 90 and is set as the background (gray value is set as 0), the specific calculation formula is as follows:

and then performing Blob extraction on the binarized image. The algorithm extracts blobs from the connected domain characteristics, including 4-connected or 8-connected, through the connectivity configuration mentioned in the parameter configuration. The invention adopts 8 communication.

Next, blob screening is performed. The method finally selects the length of the area, the circularity, the major axis and the minor axis as screening conditions by combining the geometric features of the surface defects. The method comprises the steps of using area screening conditions for all surface defects, screening out the surface defects (unit is a pixel) with the area size of 70-10000, using circularity as a screening index for bubbles, pits and other round surface defects, wherein the circularity is in the range of 0.9-1.0 and is considered as bubbles and other round surface defects, using long-axis length and short-axis length as screening indexes for surface scratches, cracks and other strip defects, and using Blob with the long-axis length of more than 30 and the short-axis length of 1-30 as scratches or cracks (unit is a pixel).

The selected blobs are then ranked. The method sorts the blobs screened in the previous process according to specific characteristics, and sorts the blobs according to the area size of the blobs.

And finally outputting a Blob analysis result. The algorithm returns the binarized image, the Blob image, the number of blobs meeting the preset screening conditions, and the basic feature information and feature weighting score of each Blob meeting the preset screening conditions. If the output Blob number is 0, the glass has no surface defect, otherwise, the glass has a defect on the surface.

And 5, judging whether the glass is qualified according to the detection results of the step 2, the step 3 and the step 4, if one step of detection is carried out in the step 2, the step 3 and the step 4, the glass is considered to be a non-qualified product, and if all the detection is qualified in the step 2, the step 3 and the step 4, the glass is considered to be a qualified product.

The benefits generated by adopting the scheme are mainly as follows:

The invention provides a method for detecting defects of industrial glass, which can effectively reduce labor cost, improve production efficiency and product quality and has strong interpretation. The method is based on Visionmaster, reduces the coding difficulty, simplifies the parameter adjustment process of the industrial glass defect detection process, enables the industrial glass defect detection process to be simpler and more convenient, selects a simple and efficient detection algorithm, ensures the accuracy of defect detection, further improves the speed and efficiency of defect detection, and further improves the rigid interference capability of the defect detection method and enhances the universality by selecting a proper threshold. The invention reduces the outflow quantity of defective products, greatly improves the product quality and ensures the reputation of enterprises.

Drawings

FIG. 1 is a flow chart of a defect detection method based on Visionmaster in an example of the invention.

FIG. 2 is a block diagram of a Visionmaster-based industrial glass defect detection system in accordance with an example of the present invention.

Detailed Description

The following describes in further detail the embodiments of the present invention with reference to the drawings and examples. The following examples are illustrative of the invention and are not intended to limit the scope of the invention.

Current conventional industrial glass defect detection methods face many challenges in complex field environments, such as illumination changes, optical interference, and equipment vibration, which all significantly affect the detection results. To solve these problems, researchers have begun to explore the use of deep learning techniques for defect detection. However, deep learning requires a large amount of training data, and defect data on an industrial site is often scarce. In addition, the deep learning model is complex, and the common workers are difficult to manually adjust parameters, so that the deep learning model becomes a big pain point for detecting defects of the current industrial glass. Based on the above problems, as shown in fig. 1, the present invention provides a Visionmaster-based industrial glass defect detection method, which includes the following steps:

Claims

1. A Visionmaster-based industrial glass defect detection method is characterized by comprising the following steps:

The method comprises the steps of 1, preprocessing an image, carrying out gray level matching on the image shot by an industrial linear array camera to obtain the position of a glass region in the image, wherein the gray level matching algorithm is also called a template-based matching algorithm, gray level values on the positions corresponding to the sub-image and the template image are calculated to obtain a value which can be the sum of absolute values of subtraction of corresponding pixels (namely SAD algorithm), the calculated sum can be divided by the number of template pixels (namely MAD algorithm), other gray level-based image matching algorithms are based on the idea, the calculated values are different, and because the detection object is simpler and the detection speed is higher, the method adopts SAD algorithm, the SAD algorithm is the sum of absolute values of subtraction of the corresponding pixels of the template and the reference image, and then the matrix of (P-M) rows and (Q-N) columns is obtained by sequentially sliding the template, wherein the position with the minimum value is the position which indicates that the similarity between the template and the reference image is highest;

Then, gaussian filtering is carried out on the image to filter noise; the method comprises the specific operations of firstly determining a Gaussian convolution kernel, dividing each pixel value in the decimal Gaussian convolution kernel by the pixel value of the upper left corner of the integral Gaussian convolution kernel when determining the integral Gaussian kernel, rounding to obtain the integral Gaussian convolution kernel, using the Gaussian convolution kernel to scan each pixel in the image, using the weighted average value of the pixels in the neighborhood determined by the convolution kernel to replace the value of the central pixel point of the convolution region, multiplying the corresponding pixel value of the input image with the corresponding pixel value of the convolution kernel respectively, adding the pixel value after the summation is larger than the pixel threshold 255, and dividing the pixel value after the summation by 255 to obtain the central pixel value of the convolution region;

In order to effectively identify the edge of the glass in the image, the influence of noise and false detection of the edge are reduced as much as possible, and meanwhile, in order to make the subsequent processing and analysis easier and more efficient, the contour of the glass in the image is obtained by adopting a Canny edge detection method, which mainly comprises four parts of image noise reduction, gradient calculation, non-maximum suppression and double-threshold boundary tracking;

The invention mainly uses the structure element to update the pixel value of local area of the image to change the image characteristic, which is used in image denoising, edge detection and morphological transformation, to enlarge the target area of the image to strengthen the image characteristic, wherein the key component in the expansion process is the structure element which is a template with variable shape and size to traverse the image and compare with the pixels, and the structure element can be rectangle, round, cross and other arbitrary shapes, and the invention combines the glass characteristic to select the rectangle structure element with 5 x 5 size;

The method mainly comprises the steps of 1) detecting datum points, firstly, determining one or more datum points which are usually known or easily identified characteristic points on a target object by a system, wherein the datum points can be corner points, edges, circle centers and the like, selecting the center position of local glass as the datum point, 2) calculating position deviation, calculating the position deviation by comparing the actually detected datum point position with a theoretical (or preset) position, wherein the deviation can comprise translation, rotation, scaling and other various types of transformation, selecting the matched point after gray level matching as the preset position (under the same illumination environment, the matched point after gray level matching of different glasses is not greatly different, thus the position of the corrected glass image is similar), 3) generating a transformation matrix according to the calculated position deviation, and generating a transformation matrix which describes how to transform an image or a certain area in the image from the current position to the required position, and applying the transformation to the correct transformation, 4) generating the transformation matrix by the transformation to the image;

step 2, detecting whether the glass is transversely or longitudinally divided, wherein defects possibly caused by errors occur in the cutting process of the produced glass, namely the glass is transversely or longitudinally divided, and observing that when the glass is transversely or longitudinally divided, the inner area of the glass in the image is always cracked through the glass;

Step 3, detecting the edge defect of the glass, if the glass is not transversely or longitudinally divided, detecting the edge defect, and considering that the edge defects are different in shape and various in variety, the invention adopts an edge pair model defect detection method to detect the edge defect, wherein the edge pair model defect detection method is to compare an actually extracted edge pair with a pre-established edge pair model, analyze information such as existence, position, width and the like of the edge pair, and judge whether edge pair defects such as edge breakage and the like exist;

The method comprises the steps of (4) detecting glass surface defects, wherein the types of the surface defects mainly comprise round surface defects such as bubbles and pits, and line-adjusting surface defects such as scratches and cracks according to enterprise requirements;

The method comprises the steps of firstly, binarizing an image, namely converting pixel points in the image into pure black (gray value 255) or pure white (gray value 0), dividing the image into two parts of black and white after conversion, wherein in the conversion process, the pixel points with gray values converted into 0 and 255 are specifically determined by a preset pixel threshold value (namely a specific gray value), and 1) setting the threshold value used by the team to be 90, namely setting the gray value to be greater than or equal to 90 as a foreground (setting the gray value to be 255), and setting the gray value lower than 90 to be a background (setting the gray value to be 0);

then, performing Blob extraction on the binarized image, wherein the algorithm extracts the Blob according to the connected domain characteristics, wherein the connected characteristics comprise 4 connected or 8 connected and can be through the connected configuration mentioned in the parameter configuration;

the method comprises the steps of selecting a Blob according to preset conditions, selecting the Blob according to an algorithm, finally selecting the lengths of an area, a circularity, a long axis and a short axis as screening conditions by combining geometric features of the surface defects, selecting the Blob with the area size of 70-10000 (the unit is a pixel) for all the surface defects, selecting the round surface defects such as bubbles and pits as screening indexes by additionally using the circularity which is in the range of 0.9-1.0 and is identified as the bubble, and selecting the strip-shaped defects such as surface scratches and cracks by additionally using the length of the long axis and the length of the short axis as screening indexes, wherein the Blob with the length of the long axis which is more than 30 and the length of the short axis which is in the range of 1-30 is identified as scratches or cracks (the unit is a pixel);

Sorting the blobs screened in the previous process according to specific characteristics by an algorithm, and sorting the blobs according to the area size of the blobs;

Finally, outputting a Blob analysis result, returning binary images, blob images, the number of blobs meeting preset screening conditions, basic feature information and feature weighting scores of each Blob meeting the preset screening conditions by an algorithm, and if the result of outputting the number of blobs is 0, ensuring that the glass has no surface defect;

2. The method for detecting industrial glass defects based on Visionmaster as defined in claim 1, wherein the algorithm in step 1 performs glass region searching, and the core idea is SAD similarity calculation, based on which glass region information in an image can be effectively extracted, and SAD similarity calculation formula is as follows (because the algorithm directly uses matrix calculation, the algorithm is different from, but substantially the same as, the formula expression):

3. The method for detecting industrial glass defects based on Visionmaster as defined in claim 1, wherein the glass edge area is reinforced in step 1, and the core idea is gradient calculation and non-maximum suppression of Canny edge detection and edge expansion treatment, and the image processing technology can be used for effectively extracting glass edge information and guaranteeing edge definition;

Edge gradient calculations are shown below:

Where f (x, y) is the gray value of the image at (x, y), Is the partial derivative in the x-direction,Is the partial derivative in the y direction, from which the gradient magnitude and direction of the point can be obtained as:

the expansion process calculation formula is as follows:

4. a Visionmaster-based industrial glass defect detection method according to claim 1 is characterized in that the algorithm for detecting glass is intersected or longitudinally divided in the step 2, median filtering is carried out on processed images, two structural elements with different sizes are needed due to different intersecting and longitudinally dividing directions, so that an input image only keeps transverse lines when the intersecting is detected, an input image only keeps longitudinal lines when the longitudinally dividing is detected, filtering operation is carried out on the processed glass image respectively by adopting two different structural elements of 25 multiplied by 80 and 100 multiplied by 27, then straight line searching is used for the glass inner area in the image with the median filtering, whether transverse or longitudinally penetrated cracks exist in the glass or not is checked, and if yes, the glass is intersected or longitudinally divided, and the quality is disqualified.