CN115335688A - Image processing systems and computer programs - Google Patents

Abstract

The image processing system includes: a processor; a memory; and a storage device for storing data of at least one non-defective item differential image generated from at least one non-defective item image obtained by imaging a non-defective item. The processor executes the following processing according to the program: acquiring data of an inspection image obtained by imaging an object to be inspected; generating a differential image of the acquired inspection image; calculating a difference or a ratio between a pixel value of the generated differential image and a pixel value of the non-defective differential image for each pixel at the same position of the non-defective differential image and the generated differential image; and detecting, when the difference or the ratio is larger than a predetermined value, a pixel of the inspection image to which the difference or the ratio is given as a candidate of a pixel of an image including a defect of the object.

Description

Image processing system and computer program

technical field

The present disclosure relates to image processing systems and computer programs.

Background technique

Conventionally, an appearance inspection for judging whether the inspected object has defects, that is, whether the inspected object is a good product or not, has been performed using images captured by photographing the inspected object.

For example, Japanese Patent Application Laid-Open No. 2000-132684 discloses an appearance inspection method capable of detecting defects generated on the surface of an object to be inspected having a stripe pattern differently from a stripe pattern. In Japanese Patent Application Laid-Open No. 2000-132684, the edge of a stripe is detected by differential processing of an image, and the presence or absence of a defect is determined focusing on the disorder of the detected edge.

Japanese Patent Application Laid-Open No. 2000-329538 discloses a method of performing differential processing on an image, and further performing binarization using a binarization threshold to determine a defect. In addition, in Japanese Patent Application Laid-Open No. 2000-329538, even if there is image non-uniformity, by calculating the binarization threshold for each predetermined region, it is possible to uniformly improve the performance of extracting defects in the entire captured image.

Japanese Unexamined Patent Publication No. 2005-265661 discloses a method of specifying a range of good products for each pixel from among a plurality of good product image groups, and performing a pass/fail judgment.

prior art literature

patent documents

Patent Document 1: Japanese Patent Laid-Open No. 2000-132684

Patent Document 2: Japanese Patent Laid-Open No. 2000-329538

Patent Document 3: Japanese Patent Laid-Open No. 2005-265661

Contents of the invention

The problem to be solved by the invention

According to any of the techniques described above, it is impossible to detect the entire defect, and it is impossible to detect pixels with low differential values or pixels within the range of good products. As a result, the defect may be determined to be small, or may be divided into a plurality of defects.

Embodiments of the present disclosure provide a new image processing system and computer program that can be used for visual inspection.

means to solve the problem

In an exemplary embodiment, the image processing system of the present disclosure has: a processor; a memory storing a program for controlling the operation of the processor; and a storage device storing at least one The data of at least one differential image of a qualified product generated from one qualified product image, the processor performs the following processing according to the program: acquire the data of the inspection image obtained by shooting the inspected object; generate the acquired obtained A differential image of the inspection image; calculating the pixel value of the generated differential image and the differential image of the good product for each pixel at the same position in the differential image of the good product and the generated differential image. a difference or a ratio of pixel values of an image; and when the difference or the ratio is greater than predetermined, detecting the pixel of the inspection image to which the difference or the ratio is given as including the inspected Candidates for the pixels of the image of the object defect.

In an exemplary embodiment, the computer program of the present disclosure is executed by a computer of an image processing system, wherein the image processing system has: a processor as the computer; a memory storing information for controlling the processor; A program of action; and a storage device, which saves the data of at least one differential image of a qualified product generated according to at least one image of a qualified product obtained by shooting a qualified product, and the computer program causes the processor to perform the following processing: Acquiring data of an inspection image obtained by photographing the object to be inspected; generating a differential image of the acquired inspection image; pixels, calculate the difference or the ratio between the pixel value of the generated differential image and the pixel value of the qualified product differential image; and when the difference or the ratio is greater than a predetermined threshold, the detection award The pixel of the inspection image having the difference or the ratio is a candidate for a pixel of an image including a defect of the inspection object.

Invention effect

According to the exemplary embodiment of the present disclosure, it is possible to appropriately detect candidates for defects.

Description of drawings

FIG. 1 is a diagram illustrating a configuration example of an appearance inspection system 1000 of the present disclosure including an image processing system 100 .

FIG. 2 is a diagram mainly schematically showing a configuration example of the image processing system 100 .

3 is a diagram for explaining the flow of image processing when detecting a thin scratch (line damage) in the first example.

4 is a diagram for explaining the flow of image processing when detecting a thin scratch (line damage) in the second example.

FIG. 5 is a diagram showing a comparison between the original image and the detection result after the damage detection process is performed.

FIG. 6 is a diagram schematically showing changes in pixel values of line damage.

FIG. 7A is a diagram for explaining the generation process of the differential image in the first example.

FIG. 7B is a diagram for explaining the generation process of the differential image in the second example.

FIG. 8 is a flowchart showing the procedure of processing performed in the image processing system 100 of the present embodiment.

FIG. 9 is a diagram for explaining detection results of the present embodiment.

FIG. 10 is a diagram for explaining detection results of the present embodiment when a good product image has a gentle gradient of pixel values.

FIG. 11 is a diagram for explaining detection results of the present embodiment when a good product image has a steep gradient and a gentle gradient of pixel values.

FIG. 12 is a diagram for explaining still another method for suppressing overdetection.

Detailed ways

Hereinafter, embodiments of the visual inspection system of the present disclosure will be described with reference to the drawings. In this specification, necessary detailed explanations may be omitted. For example, detailed descriptions of already well-known items and repeated descriptions of substantially the same configurations may be omitted. This is to prevent the following description from becoming unnecessarily lengthy and to facilitate understanding by those skilled in the art. In addition, the present inventors provide the drawings and the following description in order that those skilled in the art can fully understand the present disclosure, and do not intend to limit the subject matter described in the claims by these. In the following description, the same reference numerals are attached to the same or similar components.

The image processing system of the present disclosure can be suitably used for, for example, preprocessing of an appearance inspection system for articles or components in a manufacturing site such as a factory. Hereinafter, the configuration of the image processing system will be described based on the relationship with the visual inspection system.

FIG. 1 is a diagram illustrating a configuration example of an appearance inspection system 1000 of the present disclosure including an image processing system 100 . FIG. 2 is a diagram mainly schematically showing a configuration example of the image processing system 100 .

In the illustrated example, the appearance inspection system 1000 has an imaging device 30 , an image processing system 100 , and a monitor 130 . The image processing system 100, the input device 120, and the monitor 130 can be realized by a general-purpose digital computer system such as a PC. In addition, the imaging device 30 may be a digital camera. The imaging device 30 is communicably connected to the image processing system 100 via a wired communication cable or a wireless communication line (not shown). The monitor 130 is communicably connected to the image processing system 100 via a wired communication cable (not shown).

The imaging device 30 photographs the object to be inspected, and sends the image data obtained by photographing to the image processing system 100 . The image processing system 100 performs "preprocessing" for defect detection on the image data received from the imaging device 30 in accordance with a procedure described later. Through the preprocessing, it is determined whether or not each pixel constituting the image data can be a candidate for a pixel of an image including a defect of the object to be inspected (defective pixel candidate). This "preprocessing" is the main processing of the image processing system 100 of this embodiment.

Using all defective pixel candidates, defects are then extracted from the image data. In the visual inspection system 1000, it is determined whether or not there is a defect based on the size and/or range of defective pixel candidates that continuously exist. All the processing is collectively referred to as "appearance inspection". In addition, the image processing system 100 of this embodiment may perform the determination process of the presence or absence of a defect after detecting all the defect pixel candidates.

Referring to FIG. 1 , the procedure of the visual inspection of the object to be inspected by the visual inspection system 1000 will be described. The objects to be inspected are various items such as various products and parts that are the object of visual inspection. Hereinafter, the object to be inspected may be referred to as a "workpiece".

The workpiece 70 is placed on the transfer table 62 and fixed to the transfer table 62 by a gripping mechanism. The workpiece 70 is, for example, a finished hard disk drive, a housing of a hard disk drive incorporating a motor, or the like.

The transfer table 62 can move in the horizontal direction by the transfer table 64 in a state where the workpiece 70 is placed. The imaging device 30 is supported by the support member 60 above the conveyance table 64 so that the conveyance table 64 is included in the field of view. The imaging device 30 images the workpiece 70 within the field of view. The workpiece 70 may also be held by a robot arm and placed at an imaging position. In addition, a light source (not shown) may illuminate the workpiece 70 at the time of imaging.

Image data acquired by shooting is sent from the imaging device 30 to the image processing system 100 . An example of the size of an image acquired by one shot is 1600 pixels in width and 1200 in height, and another example is 800 pixels in width and 600 in height.

Next, refer to FIG. 2 .

The image processing system 100 has an arithmetic circuit 10 , a memory 12 , a storage device 14 , and an image processing circuit 16 . The constituent elements are communicably connected to each other via a bus 18 . In addition, the image processing system 100 includes an interface device 22 a for communicating with the imaging device 30 , an interface device 22 b for communicating with the input device 120 , and an interface device 22 c for outputting video data to the monitor 130 . An example of the interface device 22a is a video input terminal or an Ethernet terminal. An example of the interface device 22b is a USB terminal. An example of the interface device 22c is an HDMI (registered trademark) terminal. Instead of these examples, other terminals may be used as the interface devices 22a to 22c. In addition, the interface devices 22a to 22c may be wireless communication circuits for performing wireless communication. As such a wireless communication circuit, for example, a wireless communication circuit compliant with the Wi-Fi (registered trademark) standard that performs wireless communication using frequencies such as 2.4 GHz/5.2 GHz/5.3 GHz/5.6 GHz is known.

The arithmetic circuit 10 may be, for example, an integrated circuit (IC) chip such as a central processing unit (CPU) or a processor for digital signal processing. The memory 12 stores a computer program 12p for controlling the operation of the arithmetic circuit 10 .

The memory 12 does not need to be a single recording medium, but may be a collection of multiple recording media. The memory 12 may include, for example, semiconductor volatile memories such as RAM and semiconductor nonvolatile memories such as flash ROM. At least a part of the memory 12 may also be a removable recording medium.

The storage device 14 stores data of at least one differential image of a qualified product. The "defective product differential image" is a differential image generated from at least one good product image (an image obtained by photographing a good product). Each pixel value of the differential image is, for example, a difference value between pixel values of at least two pixels of the non-defective image.

The image processing circuit 16 may be an integrated circuit (IC) chip also called a GPU (Graphics Processing Unit: Graphics Processing Unit). The image processing circuit 16 generates image data for display on the monitor 130 .

In this embodiment, by using the differential image of the inspection image obtained by photographing the inspection object and the differential image of the good product stored in the storage device 14 in advance, it is determined whether or not the inspection image contains the inspection object for each pixel. Candidates for defective image pixels. In addition, the derivation method of the differential image will be described later.

The imaging device 30 is a device that outputs an image signal for generating image data of an object to be inspected. The image signal is sent to the arithmetic circuit 10 by wire or wirelessly. A typical example of the imaging device 30 is a camera including an area sensor such as a CMOS image sensor or a CCD image sensor in which a plurality of photodiodes are arranged in a matrix. The imaging device 30 generates data of a color image or a monochrome image of the object to be inspected. Various cameras for visual inspection can be used for the imaging device 30 .

The input device 120 is an input device that accepts an input from a user including designation of a threshold value described later and supplies it to the arithmetic circuit 10 . An example of the input device 120 is a touch panel, a mouse, and/or a keyboard.

The monitor 130 is a device that displays the result of discrimination performed by the image processing device 100 , the result of the visual inspection by the visual inspection system 1000 , and the like. The monitor 130 is also capable of displaying images captured by the imaging device 30 .

The monitor 130 and the input device 120 do not always need to be connected to the arithmetic circuit 10 by wire, and may be connected by wireless or wire via a communication interface only when necessary. The combination of the monitor 130 and the input device 120 can also be realized by a terminal device or a smart phone carried by the user.

In the description of the exemplary embodiment of the present disclosure, the case where the arithmetic circuit 10 executes image processing will be mainly described. However, this image processing may be performed by either one of the arithmetic circuit 10 and the image processing circuit 16 . In this specification, the arithmetic circuit 10 and the image processing circuit 16 are sometimes collectively referred to as a "processor".

Next, specific problems in conventional visual inspection and the purpose of developing an exemplary image processing system 100 will be described with reference to FIGS. 3 to 6 .

3 is a diagram for explaining the flow of image processing when detecting a thin scratch (line damage) in the first example. In the order of the arrows, the processing is performed sequentially from the upper left image (1st image) to obtain the lower right image (6th image). The image on the upper left (first image) is the original image captured by the camera. Next, an image (second image) obtained by applying an edge-enhancing filter to the original image. Next, damage detection processing (third sheet), expansion processing and contraction processing (fourth sheet), labeling processing (fifth sheet), and line connection processing (sixth sheet) are sequentially performed.

Although the human eye can see a thin line of damage, it is actually discontinuous on the image. In the 3rd image, the line damage roughly consists of a collection of 9 line segments (labels). This state is described as "number of labels: 9".

In the 4th image, the line damage roughly consists of a set of 5 labels (number of labels: 5). In the fifth image, numbers are assigned to each line segment. The process of assigning a number is called "marking process".

In the 6th image, 3 tags are connected and integrated into 1 tag. This processing is called "line connection processing". In the line linking process, the tags are compared, and when it is determined that a plurality of tags constitute one thread damage (thread defect), the plurality of tags are combined into one tag. That is, it is determined that there are three tags in the sixth image. In the figure, it is shown that the length of the longest label is 9.43 mm and the area of the label is 1.66 square millimeters. Each value of the length and the area is calculated by the arithmetic circuit 10 of the image processing system 100 .

When the length and/or area of the label exceed a predetermined threshold, the arithmetic circuit 10 determines that the detected label is a line defect. That is, it is determined that the original image contains line damage.

In addition, the further specific content and description of the above-mentioned edge enhancement filtering, damage detection processing, expansion processing and contraction processing, labeling processing, and line connection processing are omitted.

4 is a diagram for explaining the flow of image processing when detecting a thin scratch (line damage) in the second example. The description is based on the example in Figure 3.

As a result of performing the same processing as in the first example related to FIG. 3 , for example, in the labeling processing of the fifth sheet, it is determined that there are an infinite number of line segments, and a very large number of numbers must be assigned. As a result, in the sixth image, although there were many line damages of relatively short length and relatively small area, it was determined that there was no defect. Therefore, it is determined that no defect is contained in the original image of FIG. 4 .

The inventors of the present invention believe that the results of the processing from the original image (first image) to the damage detection process (third image) have a large influence on the final determination of the presence or absence of defects. FIG. 5 shows a comparison between the original image and the detection result after the damage detection process is performed. When compared again with the original image in the upper stage which appears to be continuous, a plurality of discontinuous damages appear to exist in the image after the detection process of the damage in the lower stage. Therefore, if the length of the damage is identified from the image in the lower row, there is no longer damage that is identified as a defect. The line damage at this time will be described from the viewpoint of pixel values with reference to FIG. 6 .

Fig. 6 schematically shows changes in pixel values of line damage. The horizontal axis represents the length of the line damage along the direction of the line damage by the number of pixels, and the vertical axis represents the pixel value of each pixel by 256 gray levels.

Pixel values larger than the upper limit value (UL) and pixel values smaller than the lower limit value (LL) are surrounded by solid lines. These are candidates for defects identified in units of pixels. Since candidates for defects exist discretely at a distance, as described above, in the conventional visual inspection, the overall defect was not recognized as one.

The present inventors developed a process capable of appropriately identifying the presence of line damage by using a differential image to detect, as line damage, part of a pixel group in a non-defective product range surrounded by a dotted line that is not originally a defect candidate.

Hereinafter, first, the processing of generating a differential image will be described, and then the processing using the differential image will be described.

FIG. 7A is a diagram for explaining the generation process of the differential image in the first example. The upper row of Fig. 7A shows the image I, and the lower row shows the differential image I'. The direction from left to right of the image is set as the positive direction of the X coordinate, and the direction from top to bottom of the image is set as the positive direction of the Y coordinate. Image I is typically an original image.

The expressions used in the specification of this application are shown below. "Position P" represents an arbitrary position of the image. "Pixel value" is represented by, for example, 256-level grayscale. P(x, y): the coordinates of the position P within the image, I(x, y): the pixel value of the pixel at the coordinate (x, y) of the image I, I'(x, y): the differential image I' The pixel value of the pixel at the coordinates (x, y), I _x (x, y): the differential value in the X direction in the coordinates (x, y) of the image I, I _y (x, y): the coordinates of the image I ( The differential value in the Y direction in x, y).

As shown in the upper part of FIG. 7A , attention is paid to the position P, the pixel Q at the adjacent position in the +X direction, and the pixel R at the adjacent position in the +Y direction. The coordinates of Q are (x+1, y), and the coordinates of R are (x, y+1). In the first example, each differential value in the X direction and the Y direction at the position P is defined as follows.

【Mathematical formula 1】

I _x (x, y) = I (x+1, y) - I (x, y)

I _y (x, y) = I (x, y+1) - I (x, y)

That is, the differential value I _x (x, y) in the X direction is the difference between the pixel value of the pixel at position Q and the pixel value of the pixel at position P. The differential value I _y (x, y) in the Y direction is the difference between the pixel value of the pixel at position R and the pixel value of the pixel at position P. In addition, the differential value in the X direction can be used to detect the contour of the image I in the Y direction. In addition, the differential value in the Y direction can be used for detection of the contour in the X direction.

When the differential value in each direction is obtained by Mathematical Expression 1, the pixel value I'(x, y) of the position (x, y) of the differential image I' is obtained by Mathematical Expression 2.

【Mathematical formula 2】

The differential image I' can be generated by calculating all the pixel values I'(x, y) using the above-mentioned Mathematical Expression 2. In addition, in this example, the number of pixels in the horizontal direction (X direction) and the number of pixels in the vertical direction (Y direction) of the image I and the image I' are the same. In order to make the number of pixels of the image I and the image I' equal, pixels with a pixel value of 0, for example, are added over the entire periphery of the differential image I'.

FIG. 7B is a diagram for explaining the generation process of the differential image in the second example. The upper row of Fig. 7B shows the image I and the kernel K, and the lower row shows the differential image I'. The direction from left to right of the image is set as the positive direction of the X coordinate, and the direction from top to bottom of the image is set as the positive direction of the Y coordinate. Image I is typically an original image.

In the second example, a convolution operation is performed using an input image I and a kernel K. A "kernel" is a collection of coefficients used in a convolution operation, a so-called filter. The convolution operation is performed on an area L including a pixel at a certain position P and a plurality of (for example, 8) adjacent pixels surrounding the pixel. Such operations are also known as spatial filtering.

To simplify the description, the pixel values of the pixels within the area L of the input image I and the kernel K are given as follows.

【Mathematical formula 3】

The (2, 2) element of the input image I shown in Mathematical Expression 3 corresponds to the pixel at the position P. At this time, the pixel value of the pixel at the position P of the differential image I' is obtained as follows.

【Mathematical formula 4】

I'(x,y)=(k1·a)+(k2·b)+(k3·c)+(k4·d)+(k5·e)+(k6·f)+(k7·g)+ (k8·h)+)+(k9·i)

When the number of elements of the input image I is larger, the kernel K is applied to the pixel value of the pixel at the position P and the pixel values of the eight pixels surrounding the pixel.

In this embodiment, a so-called differential filter is used as the kernel K. In this embodiment, the differential value I _x (x, y) in the X direction at the coordinate (x, y) of the image I and the differential value I _y in the Y direction at the coordinate (x, y) of the image I are obtained. (x, y). Therefore, the kernel K also uses a kernel Kx for obtaining a differential value in the X direction and a kernel Ky for obtaining a differential value in the Y direction. Mathematical Expression 5 respectively shows the kernels Kx and Ky of the Prewitt filter as an example.

【Mathematical formula 5】

According to Mathematical Formula 5, it can be seen that the kernel Ky is the transpose of the kernel Kx. The Pruitt filter is known as a filter that applies smoothing processing to primary differential filtering. Therefore, the description of the meaning of the processing of each row or each column is omitted. In addition, the Pruitt filter is only an example. Other differential filters such as Sobel filters may also be used.

As the kernel K of Mathematical Expression 3, by using the kernels Kx and Ky shown in Mathematical Expression 5, respectively, the differential value I _x (x, y) and the image Ix in the X direction at the coordinates (x, y) of the image I can be obtained. The differential value I _y (x, y) in the Y direction at the coordinate (x, y) of I. Then, similarly to the first example, the differential image I' can be generated by calculating all the pixel values I'(x, y) using the above-mentioned Mathematical Expression 2. In addition, in this example, the number of pixels in the horizontal direction (X direction) and the number of pixels in the vertical direction (Y direction) of the images I and I′ are also the same. In order to make the number of pixels of the image I and the image I′ the same, pixels with a pixel value of 0, for example, are added over the entire periphery of the differential image I′.

When the arithmetic circuit 12 generates the differential image I' by the method of the first example or the second example, it executes the processing of this embodiment described below.

FIG. 8 is a flowchart showing the procedure of processing performed in the image processing system 100 of the present embodiment. This processing is executed by the arithmetic circuit 10 . Before executing the processing, at least one differential image (hereinafter referred to as "defective product differential image") is generated based on at least one good product image obtained by photographing a good product, and the data of the generated good product differential image is stored in the memory device 14. When using a plurality of good product images, it is only necessary to generate an average image of the plurality of good product images and obtain a differential image of the average image. Alternatively, a differential image may be generated from a plurality of good product images, and the processing described later may be performed on each of the obtained plurality of differential images. A qualified product image may include one qualified product or multiple qualified products. Whether it is a good product can be judged by a person or the appearance inspection system 1000 . The differential image of a good product can be created by the arithmetic circuit 10, the image processing circuit 16, or a computer not shown, for example, by the above-mentioned first example or second example.

In step S2, the arithmetic circuit 10 acquires data of an inspection image obtained by imaging the object to be inspected. The arithmetic circuit 10 can directly receive an inspection image captured by the imaging device 30 from the imaging device 30 via the interface device 22a. Alternatively, the arithmetic circuit 10 may read out from the storage device 14 the inspection image transmitted from the imaging device 30 and temporarily stored in the storage device 14 . Alternatively, the arithmetic circuit 10 may acquire an inspection image stored in a storage device (not shown) on the network from the imaging device 30 via a network.

In step S4, the arithmetic circuit 10 generates a differential image of the acquired inspection image. As a method for creating a differential image, for example, the method of the above-mentioned first example or second example can be used.

In step S6, the calculation circuit 10 calculates the difference or the ratio between the pixel value of the generated differential image and the pixel value of the non-defective differential image for each pixel at the same position of the differential image of the non-defective product and the generated differential image. . The significance of this processing is to digitize the degree of deviation between the differential image of the good product and the differential image of the inspection image for each pixel. The closer the difference is to 0, or the closer the ratio is to 1, it means that the pixels of the differential image of the non-defective product and the pixels of the differential image of the inspection image that exist at the same position have pixel values with similar degrees of variation.

In step S8, when the calculated difference or ratio is greater than a predetermined threshold, the arithmetic circuit 10 detects the pixel of the inspection image to which the difference or ratio is given as a candidate for a pixel of an image including a defect of the inspection object. In this process, when the above-mentioned difference or ratio is larger than the threshold value, it means that the difference between the two is large, that is, the change in the pixel value is large. Therefore, the pixel of the inspection image is detected as a candidate for a pixel of an image including a defect.

The difference or ratio is calculated for all the pixels of the differential image of the inspection image and all the pixels of the non-defective product differential image, and the candidate of the pixel of the image including a defect is detected, thereby completing the damage detection process in FIGS. 3 and 4 (p. 3) up to the processing. Thereafter, the arithmetic circuit 10 only needs to perform the rest of the processing shown in FIG. 3 and FIG. 4 , such as expansion processing and contraction processing (fourth sheet), labeling processing (fifth sheet), and line connection processing (sixth sheet).

FIG. 9 is a diagram for explaining detection results of the present embodiment. (a) of FIG. 9 shows changes in pixel values of the inspection image along the line damage direction. (b) of FIG. 9 shows the magnitude of change in the pixel value of the differential image of the inspection image along the line damage direction. In addition, since "change" includes a positive change and a negative change, it expresses using "the magnitude|size of a change" which shows the absolute value of a change.

The change in the pixel value of the good product image stays within the "defective product range" shown in (a) of FIG. 9 , that is, changes between the upper limit value (UL) and the lower limit value (LL). Generally speaking, the change of the pixel value of the good product image is slow, and it can be assumed that it does not change rapidly within the "defective product range". Therefore, the change in the pixel value of the differential image of the good product is also gentle. To simplify the description, in (a) of FIG. 9 , on the assumption that the pixel values of the non-defective product image coincide with the average value of the non-defective product range, the pixel values of the non-defective product image are indicated by dotted lines. In addition, in (b) of FIG. 9 , under the above assumption, the pixel value of the differential image of the good product is 0, so the pixel value indicated by the one-dot chain line coincides with the horizontal axis.

The results of the existing detection method are shown in the uppermost section of FIG. 9 . In the conventional processing, the pixel group surrounded by the dotted line 50 in (b) of FIG. 9 was not detected as a candidate for a pixel of an image including a defect. However, according to the processing shown in FIG. 8 described above, the pixel groups 50a, 50b, and 50c are detected as candidates for defects. As a result, by detecting a pixel group having a pixel value larger than the upper limit value (UL) and a pixel group having a pixel value smaller than the lower limit value (LL) as candidates for defects, it is possible to detect The relatively short line damage of the defect acts as the relatively long line damage. As a result, wire damage that should be detected as a defect can be appropriately detected. (b) of FIG. 9 shows a range 52 of a series of pixel groups constituting a line damage.

In addition, the arithmetic circuit 10 holds predetermined upper limit values (UL) and lower limit values (LL). The upper limit value and the lower limit value can be changed according to the position of the pixel, and can be determined according to the pixel value of the good product image. Two specific examples are given below.

As a first specific example, when one good product image is used, the first upper limit value and the first lower limit value are determined for each pixel position. The upper limit value is determined by (pixel value of the pixel)+(predetermined value). The lower limit value is determined by (pixel value of the pixel) - (predetermined value).

As a second specific example, in the case of using a plurality of good product images, for each position of a pixel, the upper limit value is determined by (the average pixel of the pixel)+(k×σ), and the lower limit value is determined by (the pixel The average pixel)-(k×σ) decision. "k" is a predetermined value, for example, 3. "σ" is the standard deviation of pixel values calculated for each position of the pixel. Such processing is described in, for example, JP-A-7-113758 and JP-A-2013-224833.

Depending on the product, even if it is a good product, there may be an area where the pixel value changes due to the shape, the irradiation method of the illumination light, etc. Changes in pixel values can be gradual or steep. However, according to the method described above, candidates for defects existing in the image for inspection can be appropriately detected regardless of how the pixel values of the good product image change.

FIG. 10 is a diagram for explaining detection results of the present embodiment when a good product image has a gentle gradient of pixel values. The dotted line in (a) of FIG. 10 shows the pixel values of the non-defective product image, and it was confirmed that there is a gradient in some parts.

On the other hand, the pixel values of the image of the object to be inspected have a larger gradient. As shown in (b) of FIG. 10 , in the differential image of the inspection image, the gradient is expressed as a relatively large pixel value of the differential image. The pixel value of the good product differential image at this time is smaller as indicated by the dashed-dotted line. The pixel group 50d is detected as a candidate for a defect by setting a threshold that can strictly distinguish between the pixel value of the differential image of the non-defective product and the pixel value of the differential image of the inspection image where the gradient exists. A pixel group 50c corresponding to a portion having a gradient is also detected together as a candidate for a defect. Accordingly, it is possible to appropriately detect wire damage that should be detected as a defect. (b) of FIG. 10 shows a range 52 of a series of pixel groups constituting a line damage.

FIG. 11 is a diagram for explaining detection results of the present embodiment when a good product image has a steep gradient and a gentle gradient of pixel values. The gentle gradient is as described with reference to FIG. 10 .

On the other hand, at the position S, the pixel values of the good product image and the inspection image change rapidly. Since the steep change at the position S is known in advance, from the left end of the horizontal axis of (a) of FIG. up to position S. Then, after passing the position S, it is changed to a predetermined value larger than zero.

At the position S, the pixel value of the nondefective product image changes rapidly, so the pixel value of the nondefective product differential image also greatly changes at this position (one-dot chain line in FIG. 11( b )).

At the position S, when there is no defect in the inspection image, the pixel value of the inspection image changes similarly to the pixel value of the good product image. Then, the pixel values of the differential image of the good product and the pixel values of the differential image of the inspection image take substantially the same value. Therefore, when the difference or the ratio is calculated between the pixels of the differential image of the inspection image near the position S and the pixels of the non-defective differential image, the difference is approximately zero, or the ratio is approximately 1. If only the pixel values of the differential image of the inspection image are observed, they are detected as defect candidates. That is, overdetection may occur. However, according to the method of the present embodiment, since defect candidates are detected based on the relationship with the differential image of good products, overdetection can be suppressed.

As described above, according to the method of the present embodiment, defect candidates can be appropriately detected for each pixel from the inspection image even if the non-defective product image has a gentle gradient or a steep gradient.

FIG. 12 is a diagram for explaining still another method for suppressing overdetection.

With respect to the existing good product range defined by the upper limit value and the lower limit value, it is possible to set a good product range narrower than the range (“narrowed good product range” in FIG. 12 ). That is, the upper limit that defines the range of conventional good products is called "first upper limit", and the lower limit is called "first lower limit". The arithmetic circuit 10 holds a second upper limit smaller than the first upper limit and a second lower limit larger than the first lower limit. When the difference or ratio calculated in step S6 of FIG. 8 is greater than a predetermined threshold value, the arithmetic circuit 10 further determines whether the pixel value is smaller than the second lower value for each pixel of the inspection image to which such a difference or ratio is assigned. Whether the limit value and the pixel value are greater than the second upper limit value. Then, when the pixel value is smaller than the second lower limit value or when the pixel value is larger than the second upper limit value, the pixel of the inspection image is detected as a candidate for a pixel of an image including a defect. By narrowing the range of good products, the accuracy of judging good products is high, and excessive detection is suppressed.

In addition, the processing of the present embodiment may be applied within a range of m pixels around a pixel detected by good product learning. In this way, it is possible to eliminate the influence of small damages that suddenly exist at distant positions.

In the description of the above embodiment, wire damage was exemplified as a defect, but other defects other than the wire damage, such as dirt and foreign matter, may exist. In addition, in FIGS. 6 , 9 to 11 , the processing has been described using changes in pixel values along the line damage, but for example, changes in pixel values along the X direction may also be used.

Industrial availability

The image processing system and computer program of the present disclosure can be suitably used in a visual inspection system for articles or components in manufacturing sites such as factories.

Label description

10: arithmetic circuit; 12: memory; 14: storage device; 16: image processing circuit; 30: imaging device; 100: image processing system; 1000: appearance inspection system 1000.

Claims (8)

1. An image processing system having:

a processor;

a memory in which a program for controlling the operation of the processor is stored; and

a storage device for storing data of at least one non-defective product differential image generated based on at least one non-defective product image obtained by imaging a non-defective product,

the processor executes the following processing in accordance with the program:

acquiring data of an inspection image obtained by imaging the object to be inspected;

generating a differential image of the acquired image for inspection;

calculating a difference or a ratio between a pixel value of the generated differential image and a pixel value of the non-defective differential image for each pixel at the same position of the non-defective differential image and the generated differential image; and

when the difference or the ratio is larger than a predetermined value, a pixel of the inspection image to which the difference or the ratio is given is detected as a candidate of a pixel of an image including a defect of the inspection object.

2. The image processing system according to claim 1,

in the processing of generating the differential image, the differential image of the inspection image is generated by calculating a differential value of pixel values of at least two pixels adjacent to each other of the inspection image as each pixel value of the differential image, and in the processing of detecting, when the difference or the ratio is larger than the predetermined threshold value, one pixel of the at least two pixels of the inspection image to which the difference or the ratio is given is detected as the candidate.

3. The image processing system according to claim 2,

in the processing for calculation, a difference between a pixel value of the differential image and a pixel value of the non-defective differential image is calculated for each pixel.

4. The image processing system according to claim 2,

in the processing for calculation, a ratio of a pixel value of the differential image to a pixel value of the non-defective differential image is calculated for each pixel.

5. The image processing system according to any one of claims 1 to 4,

the processor maintains a 1 st upper value and a 1 st lower value of predetermined pixel values,

the processor determines, for each pixel of the inspection image, whether or not the pixel value is smaller than the 1 st lower limit value and whether or not the pixel value is larger than the 1 st upper limit value,

when the pixel value is smaller than the 1 st lower limit value or the pixel value is larger than the 1 st upper limit value, the processor detects a pixel of the inspection image having the pixel value as a candidate of a pixel of an image including a defect of the inspection object.

6. The image processing system according to claim 5,

the 1 st upper limit value and the 1 st lower limit value may be changed according to a position of a pixel, and the 1 st upper limit value and the 1 st lower limit value are determined according to a pixel value of the at least one non-defective image.

7. The image processing system according to claim 5 or 6,

the processor maintains a 2 nd upper limit value that is less than the 1 st upper limit value and a 2 nd lower limit value that is greater than the 1 st lower limit value,

the processor determines, for each pixel of the inspection image to which the difference or the ratio is given, whether or not the pixel value is smaller than the 2 nd lower limit value and whether or not the pixel value is larger than the 2 nd upper limit value, when the difference or the ratio is larger than the predetermined threshold value in the processing for detection,

when the pixel value is smaller than the 2 nd lower limit value or the pixel value is larger than the 2 nd upper limit value, the processor detects a pixel of the inspection image having the pixel value as a candidate of a pixel of an image including a defect of the inspection object.

8. A computer program to be executed by a computer of an image processing system, wherein,

the image processing system has:

a processor as the computer;

a memory in which a program for controlling the operation of the processor is stored; and

a storage device for storing data of at least one non-defective product differential image generated based on at least one non-defective product image obtained by imaging a non-defective product,

the computer program causes the processor to execute:

acquiring data of an inspection image obtained by imaging the object to be inspected;

generating a differential image of the acquired image for inspection;

calculating a difference or a ratio between a pixel value of the generated differential image and a pixel value of the non-defective differential image for each pixel at the same position of the non-defective differential image and the generated differential image; and

when the difference or the ratio is larger than a predetermined threshold value, a pixel of the inspection image to which the difference or the ratio is given is detected as a candidate of a pixel of an image including a defect of the inspection object.

Images