CN114689604A

CN114689604A - Image processing method for optical detection of object to be detected with smooth surface and detection system thereof

Info

Publication number: CN114689604A
Application number: CN202011591862.8A
Authority: CN
Inventors: 李明翰; 林祐绅; 莫又宁; 胡皓强
Original assignee: Chroma ATE Suzhou Co Ltd
Current assignee: Chroma ATE Suzhou Co Ltd
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2022-07-01
Also published as: KR20220095100A; KR102602369B1

Abstract

The invention discloses an image processing method and a detection system for optical detection of an object to be detected with a smooth surface, wherein the method is used for capturing and synthesizing images of the object to be detected in different illumination directions based on a three-dimensional optical method, and is characterized in that an image data preprocessing step is carried out after the image is captured and before the images are synthesized, the gray value of each pixel is subjected to nonlinear adjustment, so that the gray change ratios of a plurality of pixels of which the gray value is smaller than a gray threshold before the adjustment can be set relative to the gray change ratios of other pixels of which the gray value is not smaller than the gray threshold before the adjustment. Therefore, the detection system based on the stereoscopic optical method disclosed by the invention can be suitable for detecting the object to be detected with a smooth surface after corresponding image processing, and can not be limited by uniform parallel illumination light.

Description

Image processing method for optical detection of object to be detected with smooth surface and detection system thereof

Technical Field

The present invention relates to an image processing method and an image processing system for optical inspection, and more particularly, to an image processing method and an image processing system for optical inspection of an object to be inspected having a smooth surface.

Background

The stereoscopic optical Method (PSM) is an algorithm for reconstructing information on the surface of an object extended by an optical projection molding Method, and a single camera obtains a plurality of images of an object to be measured under the same shooting angle, the object to be measured is irradiated by irradiation light in different irradiation directions one by one, and then the images are superimposed by the algorithm to generate a synthesized image.

Conventionally, the stereo optical method uses a perfect diffusion (perfect diffusion) model in optics to solve gradient vectors (gradients) on the surface of an object to be measured, and then obtains a three-dimensional model after integration of a vector field, so as to obtain the light intensity distribution condition of the surface of the object to be measured.

The stereo optical method is widely used because it is not necessary to calculate all the parameter values (such as reflection coefficients) that affect the light intensity during the solution process, and good results can be obtained.

However, the stereo optical method requires a perfect diffusion model in optics to solve the problem, which is limited in application to the object to be measured having a rough surface and needs to be matched with uniform parallel light sources, and thus, the object to be measured having a smooth surface or having an easy-reflection characteristic and a non-collimated light source cannot be detected.

Disclosure of Invention

One of the objectives of the present invention is to make the stereoscopic optical method suitable for detecting an object to be detected with a smooth surface, and not limited by the condition of uniform parallel illumination light.

Another object of the present invention is to enable minute defects of protrusions or depressions on a metal material to be detected easily.

The invention further discloses a method for detecting flaws of an object to be detected, which is characterized by using a smaller detection system configuration space and improving the flaw detection accuracy of the object to be detected with a smooth surface.

In order to achieve the above and other objects, the present invention provides an image processing method for an optical inspection of an object to be inspected having a smooth surface, comprising: based on different illumination directions of the object to be detected, sequentially obtaining original images corresponding to the illumination directions, wherein the number of the original images is at least 3; an image data preprocessing step, respectively carrying out nonlinear adjustment on a pre-adjustment gray value of each pixel in each original image to generate a corresponding post-adjustment gray value, wherein when a gray threshold is smaller than a reversal threshold, the gray change ratios of a plurality of pixels of which the pre-adjustment gray values are smaller than the gray threshold are all larger than the gray change ratios of the rest pixels of which the pre-adjustment gray values are not smaller than the gray threshold, and when the gray threshold is larger than the reversal threshold, the gray change ratios of a plurality of pixels of which the pre-adjustment gray values are smaller than the gray threshold are all smaller than the gray change ratios of the rest pixels of which the pre-adjustment gray values are not smaller than the gray threshold; and a synthesizing step of synthesizing each of the original images processed by the image data preprocessing step into a synthesized image for subsequent defect detection based on a stereoscopic optical method.

In an embodiment of the invention, the synthesizing step may further include an image data post-processing step of adjusting all gradient values of the gradient distribution data having gradient values smaller than a first gradient threshold or larger than a second gradient threshold to 0 based on the gray scale distribution data of the synthesized image, and distributing gray scale values of 0 to 255 between the first gradient threshold and the second gradient threshold to generate an image to be analyzed having new gray scale distribution for subsequent defect detection.

In an embodiment of the present invention, the absolute values of the first gradient threshold and the second gradient threshold may be the same.

In one embodiment of the present invention, the first gradient threshold may be-0.5 and the second gradient threshold may be 0.5.

In an embodiment of the invention, the gray value of the new gray distribution corresponding to the gradient value 0 may be 128.

In an embodiment of the present invention, the Ra value of the surface roughness of the measured region of the specimen is less than or equal to 1.6(μm).

In order to achieve the above and other objects, the present invention further provides a detection system using the image processing method, comprising: the system comprises a carrying platform for placing an object to be detected, an image capturing device arranged above the carrying platform, a light source module arranged around the periphery of the image capturing device, and a control host coupled with the image capturing device and the light source module. The light source module comprises a plurality of light source devices, and the control host executes the image processing method.

In an embodiment of the invention, the control host may be configured to execute a defect determining procedure, where a pixel with a gray value larger than a convex defect threshold on the image to be analyzed is defined as a convex defect, and a pixel with a gray value smaller than a concave defect threshold on the image to be analyzed is defined as a concave defect.

In an embodiment of the invention, an included angle between the light irradiation direction of each light source device and the optical axis of the image capturing device may be 0 to 30 degrees.

Thus, in the embodiment disclosed in the present invention, the pre-processing step of non-linear conversion is performed on the input image data, so that the operation model based on the stereo optical method can correctly process the image data from the object to be measured with a smooth surface, thereby improving the problem of uneven brightness in the image, and making the object to be measured which is easy to generate reflection also be suitable. In addition, the post-processing step of limiting normalization is further performed on the synthesized image, so that the problem of inconsistent image data intervals can be improved, and tiny defects can be accurately detected.

The detection system based on the stereoscopic optical method disclosed by the invention can be suitable for detecting the object to be detected with a smooth surface after corresponding image processing, and is not limited by using uniform parallel illumination light.

Drawings

FIG. 1 is a flowchart illustrating an image processing method according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating an image processing method according to another embodiment of the present invention;

FIG. 3 is a schematic diagram of a detection system according to an embodiment of the present invention; and

FIG. 4 is a schematic top view of the inspection system of FIG. 3.

Detailed Description

For a fuller understanding of the objects, features and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which:

the terms "a" or "an" are used herein to describe a procedure, component, structure, device, module, system, part, or region, etc. This is done for convenience of illustration only and to provide a general sense of the scope of the invention. Accordingly, unless clearly indicated to the contrary, such description should be read to include one or at least one and the singular also includes the plural.

The terms "comprises, comprising, having," or any other similar language, when used herein, are not intended to be limited to the elements listed herein but may include other elements not expressly listed or inherent to such step, component, structure, device, module, system, part, or region.

As used herein, the terms "first" or "second," and the like, are used to distinguish or refer to the same or similar steps, processes, or operations, and do not necessarily imply a temporal order of the steps, processes, or operations. It should be understood that in some cases or configurations, ordinal terms may be used interchangeably without affecting the practice of the invention.

In the embodiment disclosed by the invention, the object to be measured is subjected to image capturing more than twice, and the illumination conditions are different during image capturing each time, so that a plurality of original images which can be calculated by a three-dimensional optical method are obtained. However, since the stereo-optical method is suitable for the object having an approximate lambert surface, when the surface profile of the object is smooth and easily reflected, the original image obtained is difficult to be directly applied to the operation model of the stereo-optical method. Therefore, in order to use the advantages of the stereoscopic optical method, in the embodiment disclosed in the present invention, the acquired original image is preprocessed, so that the gray level of the image on the original image can be applied to the operation model of the stereoscopic optical method, and then the image synthesis is performed by using the stereoscopic optical method, and then the normalization of the image data can be further performed based on the defect detection requirement of the object to be detected with a smoother surface, so as to generate the image to be analyzed, thereby enhancing the consistency of the image data interval, further enabling the defect region to be easily identified on the image to be analyzed, and improving the accuracy of the rear-end defect detection.

The degree of profile variation of the surface of the test object can be represented by the surface roughness (measured according to ISO 13565 specifications), for example, by the arithmetic average roughness Ra value, and when the Ra value is less than or equal to 1.6(μm), the degree of diffusion is low, and thus it is difficult to directly apply the method to the operation model of the stereo optical method. The pretreatment disclosed in the embodiments of the present application can be used to solve this problem.

Fig. 1 is a flowchart illustrating an image processing method according to an embodiment of the invention. Firstly, step S100 is carried out to obtain each original image under different illumination directions; subsequently, step S200, pre-processing (non-linear adjustment) of image data is performed; then, step S300 is performed to synthesize images based on the stereoscopic optical method, so that each original image is synthesized into a synthesized image for subsequent defect detection.

In step S100, images of the detected surface of the object under different illumination directions are captured, for example: the irradiation light in three different irradiation directions for the surface to be detected is adopted for sequential irradiation, so as to obtain each original image corresponding to each irradiation direction.

In step S200, the non-linear adjustment is to process the gray scale value of the image. That is, a pre-adjusted gray value G of each pixel in each original image is obtained₁Respectively performing nonlinear adjustment to correspondingly generate an adjusted gray value G₂. Wherein, the ratio G of the gray values before and after adjustment₂/G₁Is the gray scale variation ratio.

The non-linear adjustment in step S200 is to divide the anisotropic adjustment into two regions by a gray threshold. A gray value G before adjustment₁The value corresponds to the gray expression of a pixel, and the gray value of the pixel before and after adjustment has a ratio G₂/G₁. The gray threshold is the gray value G before adjustment₁As a differentiated numerical position and by adjusting the pre-gray value G₁The condition setting for dividing the gradation change ratio into two regions is selected. For smooth surface of the object to be measured, for example, the gray threshold (gray value G before adjustment)₁) Can be selected from 50-60, and can be set at 170-180 in other cases. The gray threshold may be defined by the average gray level of the defect and its surrounding based on the unacceptable defect level, for example, if the set unacceptable defect level and its surrounding average gray level is 40, the gray threshold may be set at a position greater than 40 (e.g., 50); also, for example, if the average gray level around and for the unacceptable defect level is set to 180, the gray level threshold may be set to a position less than 180 (e.g., 170).

In embodiments involving the anisotropic matching of two region values, the pre-adjustment gray level value G is less than the gray level threshold₁The numerical region of (2) is called a first region, and the pre-adjustment gray value G larger than the gray threshold value₁The numerical region of (2) is referred to as a second region. When the gray threshold is less than an inversion threshold, the non-linear adjustment is to make the ratio G of the gray variation of the first region₂/G₁Are all larger than the gray scale change ratio G of the second region₂/G₁The adjustment rule of (2); conversely, when the gray threshold is larger than the inversion threshold, the non-linear adjustment is to make the gray change ratio G of the first region₂/G₁Are all smaller than the gray scale change ratio G of the second region₂/G₁The adjustment rule of (3). That is, based on the adjustment rule, the object to be measured having a smooth surface can be applied to the operation model of the stereo optical method. The original image adjusted in step S200 can be synthesized by the subsequent synthesis steps. The inversion threshold defines the twoThe boundary position of the opposite non-linear adjustment mode is used to determine which adjustment rule of the non-linear adjustment mode corresponding to a gray threshold value should be. Wherein the inversion threshold can be set at any value between 70 and 80.

In step S300, a composite image is generated based on the operation model of the stereo-optical method. In this step, the calculation of the integral surface normal vector is performed on each input original image, an orthogonal vector perpendicular to the surface normal vector is further calculated based on the calculation, and a depth image is obtained through the integration of the vector field, wherein the depth image is a synthesized image which can be used for subsequent flaw detection after synthesis. The calculation model of the stereo optical method is a well-known calculation processing technology, and is based on the light intensity value as a calculation basis of the whole model, and details of the calculation are not described herein.

Referring to fig. 2, a flowchart of an image processing method according to another embodiment of the invention is shown. Compared to the embodiment of fig. 1, the method further includes an image data post-processing step S400. In step S400, the image data is normalized to further generate an image to be analyzed, so as to enhance the consistency of the image data interval and improve the accuracy of subsequent defect judgment.

Step S400 may include: the step of converting the distribution of gray values into the distribution of gradient values S410, the step of adjusting the gradient values S420, and the step of converting the distribution of gradient values back into the distribution of gray values S430.

In step S410, the gray distribution data corresponding to each pixel of the synthesized image is converted into a gradient value. The gradient value refers to the variation degree of the gray value of the adjacent pixels, so the step calculates the variation degree of each pixel arranged in sequence according to the sorting direction of the rows or the columns on the image data, that is, calculates the variation degree of the gradient value of the previous row in the sequence, thereby obtaining the gradient distribution data in the sorting direction. The gradient distribution data has a value between-1 and 1.

Step S420 is to select gradient values smaller than the first gradient threshold or larger than the second gradient threshold from the gradient distribution data, and set all of the selected gradient values to 0. This step is a qualified normalization process, which is added to the gradient profile data to further assist in highlighting the defects. The first gradient threshold and the second gradient threshold may be two thresholds that have the same absolute value. For example, for a measured region with an arithmetic average roughness Ra value of 1.6(μm) or less on the surface of the object, the first gradient threshold may be selected to be-0.5, and the second gradient threshold may be selected to be 0.5, and further, may be applied even to a measured region with an arithmetic average roughness Ra value of 0.8(μm) or less.

In step S430, the gray values of 0-255 are distributed to the adjusted gradient distribution data so as to be converted back to the gray distribution data. This step corresponds to the reverse process of step S410.

The gray values of 0-255 are distributed between the first gradient threshold and the second gradient threshold to generate an image to be analyzed with new gray distribution for subsequent defect detection. Since the gradient values in the range other than the range between the first gradient threshold and the second gradient threshold are all 0, the step S430 is equivalent to distributing the gray values of 0 to 255 between the first gradient threshold and the second gradient threshold, and generating the to-be-analyzed image with new gray distribution. Wherein, the new gray value corresponding to the gradient value 0 can be set to 128 as the basis of the conversion.

Referring to fig. 3 and fig. 4, fig. 3 is a schematic diagram of a detection system according to an embodiment of the invention; fig. 4 is a schematic top view of the inspection system of fig. 3, showing the relative positions of the light source devices 31 and the image capturing device 20. The detection system comprises: the image capturing device includes a stage 10, an image capturing device 20, a light source module 30, and a control host 40. The carrier 10 is used for placing the object 50. The image capturing device 20 is disposed above the stage 10. The light source module 30 has a plurality of light source devices 31 surrounding the image capturing device 20. The control host 40 is coupled to the image capturing device 20 and the light source module 30, and is configured to control the image capturing device 20 and the light source module 30 and execute the aforementioned image processing method.

Under the harsh conditions of some machines, the space for disposing the inspection system may be limited, especially, the image capturing device 20 and the light source module 30 must be arranged compactly and must be located above the carrier 10, even if the image capturing device 20 must be located directly above the carrier 10 and the light source module 30 is located close to the image capturing device 20, so as to further reduce the occupied space.

Therefore, under these narrow space conditions, there is no arrangement for smooth surface detection, and there is no way to arrange the image capturing device 20 and the light source module 30 on two opposite sides of the carrier 10 respectively, or even to change their relative positions, so that the reflected light can easily enter the image capturing device 20. Therefore, the technical problems can be solved by using the known stereo photometry and matching with the image processing method disclosed by the invention to adjust the relevant image data, the convex or concave tiny flaws on the metal object to be detected can be easily detected, and the flaw detection accuracy of the object to be detected with a smooth surface is also improved.

The control host 40 may further execute a defect determining procedure, wherein the pixels are defined as convex defects when the gray-level values of the pixels in the image to be analyzed are greater than a convex defect threshold, and the pixels are defined as concave defects when the gray-level values of the pixels in the image to be analyzed are less than a concave defect threshold. Based on the gray value representation of each pixel after adjustment on the image, the defect position and condition can be judged on the smoother surface of the object to be detected by setting the defect threshold. In general, the convex defect threshold of the smoother surface of the object to be tested can be set to any value between 155 and 160, and the concave defect threshold can be set to any value between 95 and 100. In other cases, the adjusted gray level representation of each pixel on the image may be used to set the concave defect threshold and the convex defect threshold according to the concave degree and the convex degree (more stringent or looser conditions) to be detected. Accordingly, even when the angle θ between the average light irradiation direction of each light source device 31 and the optical axis of the image capturing device 20 is only within 30 degrees, even 0-10 degrees, the related image data can be adjusted according to the image processing method disclosed by the present invention to highlight the concave or convex defect area.

In summary, the inspection system based on the stereoscopic optical method disclosed in the present invention can be applied to the surface defect inspection of the object to be inspected having a smooth surface after the corresponding image processing, and can be not limited by the uniform parallel illumination light, thereby solving the problem that the stereoscopic optical method is limited by the application object to have a rough surface or to have a surface that is not easily reflected.

The present invention has been disclosed in the foregoing in terms of preferred embodiments, but it will be understood by those skilled in the art that the embodiments are merely illustrative of the present invention and should not be construed as limiting the scope of the invention. It should be noted that all changes and substitutions equivalent to the embodiments are intended to be included within the scope of the present invention. Therefore, the protection scope of the present invention should be determined by the claims.

Reference numerals

10 carrying platform

20 image taking device

30 light source module

31 light source device

40 control host

50 test substance

S100 to S400

S410 to S430

The angle theta.

Claims

1. An image processing method for optical inspection of an object to be inspected having a smooth surface, comprising: sequentially acquiring each original image corresponding to each illumination direction based on each illumination direction different from the object to be detected, wherein the number of the original images is at least 3;

an image data preprocessing step, respectively carrying out nonlinear adjustment on a pre-adjustment gray value of each pixel in each original image to generate a corresponding post-adjustment gray value, wherein when a gray threshold is smaller than a reversal threshold, the gray change ratios of a plurality of pixels of which the pre-adjustment gray values are smaller than the gray threshold are all larger than the gray change ratios of the rest pixels of which the pre-adjustment gray values are not smaller than the gray threshold, and when the gray threshold is larger than the reversal threshold, the gray change ratios of a plurality of pixels of which the pre-adjustment gray values are smaller than the gray threshold are all smaller than the gray change ratios of the rest pixels of which the pre-adjustment gray values are not smaller than the gray threshold; and

a synthesizing step, based on the stereo optical method, synthesizing each original image processed by the image data preprocessing step into a synthesized image for subsequent flaw detection.

2. The image processing method as claimed in claim 1, further comprising an image data post-processing step after the synthesizing step, wherein based on the gray scale distribution data of the synthesized image, the gradient values of the corresponding gradient distribution data having gradient values smaller than a first gradient threshold or larger than a second gradient threshold are all adjusted to 0, and gray scale values of 0 to 255 are distributed between the first gradient threshold and the second gradient threshold to generate an image to be analyzed having a new gray scale distribution for subsequent defect detection.

3. The image processing method as claimed in claim 2, wherein the absolute value of the first gradient threshold is the same as the absolute value of the second gradient threshold.

4. The image processing method as claimed in claim 2, wherein the first gradient threshold is-0.5, and the second gradient threshold is 0.5.

5. The image processing method as claimed in claim 4, wherein the gray-level value of the new gray-level distribution corresponding to the gradient value 0 is 128.

6. The image processing method as claimed in one of claims 1 to 5, wherein an Ra value of a surface roughness of a measured region of the object is less than or equal to 1.6(μm).

7. An inspection system using the image processing method as claimed in any one of claims 1 to 6, comprising: a carrying platform for placing an object to be detected;

an image capturing device disposed above the carrier;

a light source module including a plurality of light source devices arranged around the image capturing device; and

a control host coupled to the image capturing device and the light source module, the control host performing the image processing method according to any one of claims 1 to 6.

8. The detecting system according to claim 7, wherein an angle between a light irradiation direction of each of the light source devices and an optical axis of the image capturing device is 0 to 30 degrees.

9. An inspection system using the image processing method of claim 5, comprising:

a carrying platform for placing an object to be detected;

an image capturing device disposed above the carrier;

a control host coupled to the image capturing device and the light source module, the control host performing the image processing method according to claim 5, wherein the control host is further configured to execute a defect determining procedure, and define a pixel on the image to be analyzed having a gray value greater than a convex defect threshold as a convex defect, and define a pixel on the image to be analyzed having a gray value less than a concave defect threshold as a concave defect.

10. The inspection system according to claim 9, wherein an angle between a light emitting direction of each of the light source devices and an optical axis of the image capturing device is 0 to 30 degrees.