JP2017107541A - Information processing apparatus, information processing method, program, and inspection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus, an information processing method, a program, and an inspection system.SOLUTION: A control part 501 of an inspection system 500 comprises: a preprocessing part for aligning an object image with a reference image; an arithmetic part that defines an attention area and a peripheral area surrounding the attention area in the object image and calculates, by using the feature quantity related to colors and the specificity on an image for the respective areas, an outlier that numerically indicates the specificity on the image in the attention area; and a determination part that sums up the outliers and gives an index for a defect.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an information processing apparatus, an information processing method, a program, and an inspection system.

  Techniques for detecting characteristic points (pixels) and regions in an image, such as a product appearance inspection process and an abnormality detection by a monitoring camera, are already known. In the conventional technique for detecting characteristic pixels and regions in an image, a normal reference image group is prepared in advance. And it is often used to identify changes in the image by calculating the difference using the feature amount such as the luminance value of the corresponding pixel in the reference image in the inspection target image.

  For example, in Japanese Patent Laid-Open No. 2005-265661 (Patent Document 1), in the appearance inspection of a product, an image inspection that does not depend on the operator to be inspected is performed by statistically determining the normal range of the luminance value of the inspection target image. suggest. In JP 2013-160629 A (Patent Document 2), the average and standard deviation of luminance values for each pixel are determined in advance for a reference image. At the time of inspection, the deviation value obtained by subtracting the average value from the luminance value of the image to be inspected and dividing by the standard deviation is calculated, and pixels whose deviation value for each pixel is larger than the set threshold are identified as abnormal pixels. Suggest a way to do it.

  However, the conventional method has a problem that an abnormality cannot be sufficiently detected when there is an error in positioning of the inspection object or when the shape of the inspection object slightly changes. Furthermore, in the method of obtaining the deviation value by using the average of the entire image, there is a problem that an abnormality that exists locally acts as a noise component on the entire image and cannot detect the local abnormality. .

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a technique capable of efficiently detecting an abnormality in an image.

That is, according to the present invention, an information processing apparatus for inspecting a target image including an image to be inspected,
Means for performing preprocessing for comparing the target image with a reference image;
Means for defining a target region and a peripheral region surrounding the target region in the target image, and calculating a feature amount of the target region and the peripheral region for each region;
Means for calculating an outlier that numerically indicates the specificity of the image in the region of interest from comparison with corresponding regions of the plurality of reference images for each region;
And an information processing apparatus including means for providing an index to be used for an inspection based on the outlier.

  According to the present invention, it is possible to provide an information processing apparatus, an information processing method, a program, and an inspection system that can efficiently detect an abnormality in an image.

The conceptual diagram explaining the quality determination method of this embodiment and the conventional method. The conceptual diagram explaining the quality determination method in this embodiment. The figure which shows embodiment of the test | inspection system 300 by which the processing method of this embodiment is mounted. The conceptual diagram of the process of AutoEncoder of this embodiment. The figure which shows embodiment of the test | inspection system 500 by which the processing method of this embodiment is mounted. The figure which shows embodiment of the hardware block 600 of the control part 301 of this embodiment. The figure which shows embodiment of the software block 700 of the control part 301 of this embodiment. The flowchart explaining the processing method of this embodiment. The conceptual diagram of other embodiment which obtains the amount of defects of this embodiment. The figure which shows other embodiment of this embodiment. The graph which plotted determination performance (AUC: Area Under Curve) with respect to the threshold value which should be set to an outlier. The figure which shows the experimental example which matched the defect amount at the time of performing image evaluation with respect to an actual test | inspection object, the quality goods of a test object, and a defective product. The figure which shows the defect amount with respect to the same experiment example as shown in FIG. 8 when it is set as threshold-value = 22 (sigma). The figure which shows the change of the recognition performance (AUC) with respect to the magnitude | size (neighborhood number) of the surrounding area to refer.

  Hereinafter, although this invention is demonstrated with embodiment, this invention is not limited to embodiment mentioned later. FIG. 1 is a conceptual diagram of pass / fail judgment in a conventional inspection method given as an embodiment and a comparative example.

  In the present embodiment, as shown in FIG. 1A, an inspection is performed using an object image in which an inspection object 111 is reflected in the inspection area 110. In the present embodiment, the quality of the inspection target reflected in the target image is compared with the target image or the group of target images when determined as non-defective. Hereinafter, a target image or a target image group determined as a non-defective product is referred to as a reference image in the present disclosure. This reference image is shown in FIG. Compared with FIG. 1B, the inspection object 111 in the target image of the embodiment shown in FIG. 1A is displaced or deformed within the inspection region 110 by the allowable range Δ on the right hand side of the drawing. It is shown. The allowable range Δ is below the positioning accuracy by the robot arm and the accuracy allowed for the product. For this reason, the inspection object 111 is an inspection object that should be determined as a non-defective product.

  As shown in FIGS. 1A and 1B, in the present embodiment, a region of interest 113 a is set in the image region that constitutes the inspection region 110. The attention area 113a of the reference image in FIG. 1B indicates that the image of the inspection object 111 is not included. However, in the embodiment illustrated in FIG. 1A, the attention area 113 a includes an image of the inspection target 111. In the conventional method, the quality of the inspection object 111 is determined from the difference in the feature amount of the attention area 113a. This is shown in FIG.

  FIG. 1C is a diagram in which the feature amount of the attention area 113a in the target image is plotted assuming that the feature amount can be expressed in two dimensions. In FIG. 1C, in the target image, a pixel including the inspection target 111 is indicated by hatching “●”, and a pixel not including the inspection target 111 image is indicated by “◯”. On the other hand, the reference image shown in FIG. 1B does not include the image of the inspection object 111, and therefore, ● does not exist in FIG.

  The image of ● indicates that the feature value is different from the feature value of the reference image and is unique. For this reason, in the prior art, the inspection target 111 in the current target image is processed as being defective. However, the inspection object 111 should be determined as a non-defective product within the allowable range Δ. Such misjudgment results in a decrease in product yield and a decrease in productivity.

  Therefore, in the present embodiment, the attention region 113a and the peripheral region 113b adjacent to the attention region 113a are defined in the inspection region 110, and the attention portion 113 including the attention region 113a and the peripheral region 113b is formed. Then, using this attention portion 113, the specificity of the attention region 113a is determined. FIG. 1B shows the target portion 113 and target regions 113a and 113b constituting the target portion 113. In FIG. 1B, the attention portion 113 of the present embodiment is illustrated as being configured as a square region including the attention region 113a and a peripheral region 113b set adjacent to the attention region 113a. .

  FIG. 1D shows a plot of the feature quantity of the inspection object 111 reflected in the target portion 113 in the feature quantity space in this embodiment, as in FIG. 1C. In FIG. 1D, a pixel including the inspection object 111 in the reference image is indicated by ●, and a pixel not including the image of the inspection object 111 is indicated by ○ which is not hatched as in FIG. It is the same. In addition, a peripheral region 113b partially including an image of the inspection object 111 is indicated by hatching with a halftone. Since the inspection target is reflected in the attention area 113a, the feature amount of the attention area 113a has the same feature amount difference as that in FIG. 1C in comparison with the reference image.

  On the other hand, in the present embodiment, as shown in FIG. 1D, the attention portion 113 is composed of a plurality of pixels, so that the peripheral region 113b having the reference image feature amount around the feature amount of the attention region 113a. The feature amount is added. Therefore, it is possible to determine the feature amount of the attention area 113a in consideration of the image information of the peripheral area 113b. For this reason, the conventional non-defective product determination which depends only on the specificity of the attention area 113a is improved. That is, the present embodiment makes it possible to inspect the feature amount of the attention area 113a using the feature amount of the entire attention portion 113.

  Here, FIG. 2 is used to explain the influence of defining the target portion 113 on the determination of defective products. The inspection object 111 shown in FIG. 2 has a burr 114 in a part of the product. As shown in FIG. 2A, the attention area 113a corresponding to the burr 114 includes an image of the inspection object 111. Therefore, FIG. 2B is a graph in which the feature values in the vicinity of the burr 114 are plotted in the feature value space as in FIG.

  Here, the difference in the feature quantity in the feature quantity space shown in FIG. 2B is assumed to be the same as that in FIG. On the other hand, FIG. 2C shows a reference image determined as a non-defective product and a target portion 113 in the present embodiment. As shown in FIG. 2C, the feature amount close to the feature amount of the target portion 113a of the target image does not exist in the corresponding target portion 113 of the reference image.

  For this reason, when the burr 114 shown in FIG. 2A is detected, the pass / fail judgment can be made by using the judgment method of FIG.

  Note that the attention area 113a and the peripheral area 113b can be defined in units of pixels according to the size of the inspection object 111 and the required accuracy. Moreover, in other embodiment, it can define as a polygonal area | region which consists of a some pixel and can fill a plane. In addition, the size, the number of constituent pixels, and the shape of the attention portion 113, the attention region 113a, and the peripheral region 113b can be appropriately set according to a specific application.

  Further, the attention part can be automatically determined from statistical information or the like. For example, the variance for each pixel is obtained from the reference image group in advance, and the total variance of the target portion is set. According to this aspect, it is possible to determine that the attention part is large in the region where the variance is low and the attention part is small in the region where the dispersion is high. In addition, it is possible to use a mode in which a gradient direction is obtained for each pixel in advance for the reference image group and the target image, and the aspect ratio of the target portion is changed so that the gradient direction is preferentially viewed.

  As described above, in the present embodiment, the pass / fail judgment is made by improving the robustness of the inspection by reducing the detection sensitivity for a change within a normal range without reducing the detection sensitivity for a defect. Accuracy can be improved.

Hereinafter, for the convenience of description, unless otherwise specified, the attention area 113a and the peripheral area 113b are described as being in units of pixels. In the present embodiment, as a feature amount of an image, a feature amount related to a color such as a luminance value or a color value can be used. As long as the characteristic value regarding the color can be identified on the image such as a luminance value or a color value and can be quantified, any type and formulation such as L ^* a ^* b ^* , L ^* uv, and HSV can be used.

  When the object such as a color value is three-dimensional, defect detection can be realized by finally merging those processed for each dimension. For example, when an RGB image is a target, after performing the outlier calculation shown below for each of RGB, the average value and the maximum value may be obtained and merged as one image to detect defects.

  In addition, more specifically, in the present embodiment, edge information representing shape information and spatial frequency information can be used as the image feature amount. In addition, examples of the local image feature amount include HLAC, SHIFT, SURF, and the like. In addition, since an image texture is used as an image feature amount, a feature amount automatically acquired by a method using a density co-occurrence matrix, Auto-Encoders using deep learning, or the like can be used. All values quantified by these image characterization methods can be used as feature amounts in the present embodiment. In the following, feature quantities that can be used in addition to the above-described feature quantities relating to colors will be described in detail.

(1) Edge feature amount As the edge feature amount, for example, gradient strength and gradient direction can be used, and the gradient strength s (u, v) at the pixel position (u, v) is expressed by, for example, the following formula ( 1) (Non-patent Document 1).

  Further, the gradient direction at the pixel position (u, v) in the image L can be obtained by the following equation (2).

(2) Spatial frequency information The spatial frequency information can be digitized using a power spectrum, and the power spectrum can also be used as a feature amount (Non-patent Document 2).

(3) Texture feature amount Using the spatial frequency information, the texture feature amount can be obtained from the feature amount by the following equation (4).

  The texture feature amount defined by the above equation (4) can be used as the feature amount in the present embodiment. Note that p (r) is the sum of energy in a donut-shaped region centered on the origin in the power spectrum space. q (θ) is the sum of energy in the sector area. The texture information can be used as a feature amount using the histogram position, height, p (r), q (θ) average and / or variance of p (r) and q (θ).

  Although p (r) and q (θ) described above cannot be obtained for a single pixel, p (r) and q (θ) obtained for a local region centered on the target pixel. As an image feature amount of the target pixel, an outlier can be calculated in the same manner as a color feature amount.

(4) Local feature amount The local feature amount is an HALC value (higher-order local autocorrelation feature) x (a ₁ , a ₂ ,..., _An ) expressed by the following formula (5 (Non-Patent Documents 3 and 4).

  Although the second and higher order HALC feature values cannot be obtained for a single pixel, the HLAC feature value obtained for the region centered on the target pixel is used as the HLAC feature value of the target pixel. As with the feature quantity, outliers can be defined and calculated. Note that SIFT (Non-Patent Document 1, Non-Patent Document 5) and SURF (Non-Patent Document 6) can also be used as feature amounts in this embodiment.

(5) Feature Quantity Using Run-Length Matrix As a technique for characterizing a texture, a method using a run-length matrix has been proposed (Non-Patent Document 2). In the present embodiment, the feature amount of each pixel is calculated using the run-length matrix shown in FIG. In the run-length matrix shown in FIG. 3, the run length is associated with the row direction, and the density is associated with the column direction.

  Using the run-length matrix shown in FIG. 3, any one of the five-dimensional feature amounts given by the following formula (6) or any combination can be used as the feature amount of the present embodiment.

  None of the texture feature amounts can be obtained for a single pixel, including the method using a run-length matrix. However, outliers can be calculated as in the case of the color feature amount by using the texture feature amount obtained for the region centered on the target pixel as the feature amount of the target pixel. be able to.

(6) Feature amount by deep learning As a feature amount by deep learning, for example, a feature amount extraction method by Autoencoder (auto encoder) (Non-patent Documents 7 and 8) can be used. Autoencoder is a neural network that reconstructs input after mapping the input to a low dimension, and can automatically obtain a low-dimensional representation that is less susceptible to noise.

  FIG. 4 shows an example of AutoEncoder. A low-dimensional representation of the target image can be acquired by learning that output = input via a hidden (hidden layer) that is lower in dimension than Input (in this case, a vectorized image). In the present embodiment, among the values shown in FIG. 4, the hidden value can be used as the feature amount in the present embodiment. Here, learning can be performed using a reference image.

  In FIG. 4, the input layer is x, the input to hidden is y, and the input to output is z. Encode and decode can be calculated using the formula defined in Fig. 4. W is a weight, and b is a parameter called a bias. In the embodiment using the AutoEncoder, the feature amount can be obtained using pixel color information (RGB three-dimensional) as an input. Note that the input and output in FIG. 4 can be set to three dimensions and the hidden can be set to one or two dimensions. Further, if the hidden value obtained for the region including the target pixel is the feature amount of the target pixel, outlier calculation can be performed in the same manner as the feature amount of the color.

  As described with reference to FIGS. 1 to 4, in the present embodiment, the edge 111a of the inspection object 111 is determined by determining the specificity of the attention region 113a on the image in consideration of the state of the peripheral region. This makes it possible to numerically determine the specificity of the neighborhood more efficiently.

  FIG. 5 shows an embodiment of an inspection system 500 in which the processing method of this embodiment is implemented. The inspection system 500 shown in FIG. 3 includes a control unit 501, an operation unit 502 for giving a command to the control unit 501, and an output I / O 503.

  The control unit 501 can be configured to include an information processing apparatus such as a computer, and corresponds to the control means in the present embodiment. The operation unit 502 includes a liquid crystal display device, a keyboard, a mouse, a touch panel, and the like, and provides an interface between the control unit 501 and an operator. The operation unit 502 corresponds to an operation unit in the present embodiment. The output I / O 503 generates an output for moving the inspection object 506 based on the calculation result of the control unit 501 and controls the position of the transfer arm 505, and corresponds to an output unit in this embodiment.

  The inspection target 506 is placed on the transport unit 507, held by the transport arm 505, transported to the inspection position, and an image thereof is acquired by an imaging device 508 such as a digital camera. The transport unit 507 corresponds to the transport unit in the present embodiment. The imaging device 508 corresponds to the imaging unit in this embodiment. The acquired image is sent to the control unit 501, and the specificity of the inspected object 111 is inspected imagewise. The control unit 501 controls the movement of the transfer arm 505 in accordance with the result of the inspection, and moves the inspection object to the accommodating unit 504 that accommodates the non-defective product and the defective product.

  FIG. 6 shows an embodiment of a hardware block 600 of the control unit 501 of the present embodiment. The control unit 301 can include a CPU 601, a RAM 602, and an image RAM, a ROM 603, a display device 604, and a communication device 605, which are interconnected by a system bus 606. Furthermore, an I / O bus 607 is connected to the system bus 606 via a bus bridge such as PCI or PCI Express. In addition, an external drive 610 such as an external drive and a DVD drive is connected to the I / O bus 607 via an appropriate protocol. Furthermore, the control unit 501 is connected to an imaging device 608 such as a digital camera, a storage device 609, and an external drive 610, and enables storage of programs and data that enable image acquisition and processing of the present embodiment. It is possible.

  More specifically, the CPU used by the control unit 501 is, for example, PENTIUM (registered trademark) to PENTIUM IV (registered trademark), CORE i (registered trademark) series, ATOM (registered trademark), or PENTIUM (registered trademark). Examples include, but are not limited to, compatible CPUs, POWER PC (registered trademark), and MIPS.

  As the operating system (OS) to be used, MacOS (registered trademark), iOS (registered trademark), Windows (registered trademark), CHROME (registered trademark), ANDROID (registered trademark), Windows (registered trademark) 200X Server, UNIX ( (Registered trademark), AIX (registered trademark), LINUX (registered trademark), or other appropriate OS. Further, the control unit 301 stores and executes an application program written in a programming language such as C, C ++, Visual C ++, Visual Basic, Java (registered trademark), Perl, or Ruby, which operates on the OS described above. be able to.

  FIG. 7 shows an embodiment of the software block 700 of the control unit 501 of the present embodiment. The control unit 501 receives an inspection target image from the imaging device 308 using an appropriate transfer method such as USB (universal serial bus), HDMI (registered trademark), or Ethernet (registered trademark) in other embodiments. The control unit 501 includes an input I / O 701, a preprocessing unit 702, and a calculation unit 703. The input I / O 701 corresponds to an input unit in the present embodiment. The input I / O 701 can be configured to include an appropriate bus interface, NIC, and the like. The input I / O 701 enables the received inspection target image to be stored in, for example, the image RAM 602 which is an appropriate storage unit.

  The pre-processing unit 702 positions the target image to be inspected currently acquired by the imaging device 608 with respect to the inspection region 110, thereby enabling subsequent processing. In addition, the preprocessing unit 702 performs noise reduction filter processing and masking processing. The preprocessing unit 702 corresponds to preprocessing means in the present embodiment. For example, the pre-processing unit 702 can perform alignment by performing processing for associating at least one pixel or region of the acquired inspection target image with a predetermined position coordinate of the inspection region 110.

  The computing unit 703 provides a function for calculating an outlier according to the present embodiment, and corresponds to a means for calculating a feature quantity and a means for calculating an outlier according to the present embodiment. In order to perform the processing, the calculation unit 703 accesses the storage unit 706 storing the reference image and the second storage unit 707 storing the setting parameter, and acquires an outlier between the reference image pixels. Calculate using the specified parameters. The parameters at this time include, for example, a value for specifying the range of the peripheral region, an identification value of the reference image to be acquired, a standard deviation (σ value) calculated in advance for the reference image, and a value calculated in advance for the reference image. The average value of the feature amount, the value of std (i, j), and the like can be mentioned, but are not limited thereto. These values and parameters will be described later in more detail.

  The control unit 501 further includes a determination unit 704 and an output unit 705. The determination unit 704 corresponds to a determination unit in the present embodiment, receives the outlier generated by the calculation unit 703, applies the set threshold value, and determines whether or not the inspection target is defective. In response to the determination, the output unit 705 calculates control information for controlling the transport arm 505 and outputs the control information to the output I / O 503 so that the transport arm 505 can be controlled.

  The control unit 501 of this embodiment further includes a user interface unit 708. The user interface unit 708 has a function of allowing the user to input various settings to the control unit 501 and notifying the user of the result of the control unit 501. Specifically, for example, it is possible to have a function that allows the user to set the attention part 113 and a function that allows the number of regions constituting the attention part 113 to be set. Further, the user interface unit 708 calculates the outlier based on the function that allows the range of the peripheral area where the outlier is to be calculated to be changed / set, the function that enlarges / reduces the inspection image, and the inspection image that is enlarged / reduced It has a function to execute. Furthermore, the user can add a new reference image or exchange a reference image via the user interface unit 708.

  Hereinafter, the processing method of the present embodiment will be described with reference to FIG. The process starts from step S800, and an image to be inspected is acquired in step S801. Thereafter, preprocessing is applied to the inspection target image with respect to the inspection region 110. Preprocessing can include, for example, registration of images. In the alignment, an appropriate reference image corresponding to the inspection object is read from the storage unit 706 and set as a template. Note that the reference image can be added or deleted by the user according to the necessity of processing as appropriate.

  Then, the matching of the feature amount is determined between the target image to be inspected and the template, the region having the highest feature amount matching is determined as the same region, and the processing is performed in association with the inspection region 110. The template matching method can be performed using a so-called ZNCC (normalized cross correlation method). In addition, the preprocessing can include image enlargement / reduction and control of the amount of information of one pixel or one area depending on the application.

  Thereafter, in step S802, the calculation unit 703 defines a region with pixels, areas, shapes, and the like assigned to the inspection region, and calculates a feature amount for each region. The processing unit corresponds to means for calculating the feature amount in the present embodiment. As described above, there is no limitation on the type and formulation of the feature amount as long as it can be extracted from the image at any scale, such as luminance value, color value, and the like. Hereinafter, for the sake of more specific explanation, it is assumed that a luminance value is used as a feature amount. However, the present embodiment is not limited to processing using a luminance value.

  In step S <b> 803, the calculation unit 703 further compares the acquired feature amount with the feature amounts of the corresponding region and the peripheral region of the reference image, and calculates an outlier. The outlier calculation method is not particularly limited, and a deviation value can be used as in this embodiment, and a box plot, grubbs test, Mahalanobis distance, LOF (LOCAL OUTLIER FACTOR), and the like are used. Can do. The processing unit corresponds to means for calculating an outlier in the present embodiment. Hereinafter, the outlier calculation processing executed by the calculation unit 703 will be described in detail. In the following description, a case will be described in which the position of the attention area is position (i, j) and the peripheral area surrounding the attention area is position (i ± 1, j ± 1). This embodiment corresponds to an embodiment in which the peripheral region 113b is defined as a region surrounding and surrounding the attention region 113a. The values of i and j can be arbitrarily selected from positive integers including 0.

  First, the calculation unit 703 reads a reference image group (for example, about 50 non-defective images), and calculates an average value n (i, j) and standard deviation std (i, j) of luminance values in all regions. And store it in an appropriate storage means. Note that the feature amount of the reference image is calculated in advance as described above. At this time, depending on the reference image group, std (i, j) may be very close to 0, and it can be assumed that the calculation generates an overflow exception. Therefore, in another embodiment, it is possible to set a threshold value std_min for std (i, j) and replace std (i, j) below it with the value of std_min. Also, std_min can be set from the outside according to the application.

  The outlier is calculated from the luminance value v (i, j) at the pixel position (i, j) of the attention area 113a and the corresponding pixel position (i, j) of the reference image group and the inspection object currently determined. The absolute value of the residual with the average value m (i, j) of the luminance value of the reference image calculated without including the above information is calculated. If necessary, the residual square can be calculated. The reference image is defined as an inspection image or a set thereof in which the inspection object 111 acquired and registered in advance is determined to be non-defective. Then, the residual absolute value is divided by the standard deviation std (i, j) in the reference image of the region of interest 113a. The division result is digitized as an outlier. Note that this processing is a statistical standardization method, and so-called outliers are values corresponding to deviation values in the reference image group of the region of interest 113a.

  A similar operation is calculated using all the mean values and standard deviations of the peripheral region 113b, i.e., pixel location (i ± 1, j ± 1), and in certain embodiments, the smallest of the calculated outliers. The value is set as an outlier of the attention area 113a. In addition, it cannot be overemphasized that it can quantify as an outlier of the attention area | region 113a with a maximum value depending on a specific embodiment. The above processing is shown in the following formula (12).

  By performing this calculation over the entire region of the inspection image, the outlier is quantified by the above equation (1) for the entire image. The attention portion 113 is set or changed by user designation using the values of (i, j) and k and l. Furthermore, the range of (i, j) corresponding to the image range for calculating outliers can be changed as appropriate by user designation.

  After calculating the outlier in step S803, the determination unit 704 performs determination processing. In the present embodiment, the determination process is specifically performed as follows. An outlier for each position of the region is used, and a defect amount that is an index for determining whether or not the inspection image is a non-defective product is calculated. In the present disclosure, the index is referred to as a defect amount below.

  In the present embodiment, a predetermined threshold is set using a standard deviation (hereinafter referred to as σ) of an outlier of the entire image as a measure for which image specificity is not remarkable. For example, an outlier of 3σ or less is set to a numerical value = 0 and cut off, and the outlier is integrated for the entire image. As a result, the non-defective image can give a lower defect amount and can improve the difference from the defective image. It should be noted that the value σ is a standard deviation acquired as a normal distribution of the outlier distribution of the attention area acquired for the entire image, and is different from std (i, j) described above. Note that the values and parameters used for these calculations can be updated and learned as new values and parameters as the process proceeds.

  In the preferred embodiment, for the purpose of improving accuracy, the above-described integration processing is performed by integrating the outliers below the set threshold value, that is, the outliers of the image area that can be determined as having less noticeable image specificity. Can do. The reason for this is that the outlier obtained by the above equation (1) has a non-zero value due to the influence of various noise components even when the image area to be processed is normal. This is because the component may reduce the S / N of the image specificity of a defective product and may reduce the detectability.

  The amount of defects in the present embodiment can be further determined by various methods. For example, in the exemplary embodiment, as shown by the following formula (8), the value is defined as a value obtained by integrating the above-described outliers over the target region in the image.

In this embodiment, the defect amount can be obtained by another method. Hereinafter, an embodiment for further calculating the defect amount will be described. In another embodiment, the defect amount can be determined using the following equation (9) as the maximum outlier.

  In still another embodiment, as shown in FIG. 9A, the defect amount is determined to obtain an area where one or more outliers exist, and the area is defined as the number of pixels included in the area. ) Can be determined using.

  In addition, as shown in FIG. 9B, the area to be used as the defect amount is determined by the following equation (11) using circumscribed rectangular coordinates, and the number of pixels included in the area is used as the defect amount. Can do. In the following formula (11), k is a pixel identification value of a region where an outlier in the image is a certain value or more.

  Note that the value to be set as the threshold value can be set according to the application from the shape of the target, the accuracy of alignment, and the like. Furthermore, for example, it can be determined to be specific because there is a portion where high outliers are dense.

  In step S804, a defect to be inspected is determined using the defect amount obtained from the above-described outlier, the determination result is output to control the operation of the transfer arm 505, and the process ends in step S805.

  In the above-described embodiment, the defect amount and the like are calculated using the vicinity of the target pixel. In addition, in another exemplary embodiment, as illustrated in FIG. 10, when the control unit 501 includes a plurality of the same modules 1011 and 1012 in the inspection image, the user can set the inspection region. With reference to the attention area 1016 of the module 1011, the defect amount of the attention area of the module 1012 can be calculated. In this embodiment, the outlier calculation can be performed without reading the reference image group each time, so that memory saving and reference data loading time can be shortened.

  Further, the control unit 501 includes an area that is not to be inspected in the inspection image (an area on which a manufacturing number is printed, an area on which dust or dirt may be attached, a robot arm that holds an inspection target part, or the like). If it exists, the user can use a so-called mask process that excludes the corresponding area. Further, it is possible to control the amount of information of one pixel or one area by allowing the user to perform enlargement / reduction processing and enlarging / reducing the image by preprocessing.

  This is because the calculation cost can be reduced by reducing the reference area. For this reason, it is possible to reduce the resolution within a range where there is no problem in accuracy. For this purpose, the amount of information of one pixel or one region can be increased. As a result, the calculation cost can be reduced and high-speed inspection can be performed.

  In addition, if the resolution is small for a small specific area, the specific area also becomes small and the determination accuracy may decrease. For this reason, in another exemplary embodiment, the control unit 501 can be expected to perform control so as not to reduce the determination accuracy while speeding up the determination by not reducing only a specific area by user designation. Taking electronic board inspection as an example, the resolution is not reduced only in areas where fine parts are densely packed, and the accuracy is required for inspection by performing outlier calculation with lower resolution in other areas. Even in this case, the calculation cost can be reduced.

  In yet another exemplary embodiment, the control unit 501 may be updated by a user adding or exchanging reference images. For this reason, for example, it is assumed that the reference image includes the influence of the production lot, the data change due to the aging deterioration of the illumination, the change in the definition of normality / abnormality due to the product type correction. In this case, there is a case where it is not possible to expect an inspection with sufficient accuracy with the parameters at that time. In such a case, the parameter is updated by adding or changing the reference image, and the determination intended by the user can be performed using the updated parameter.

  FIG. 11 is a graph in which determination accuracy (AUC: Area Under Curve) is plotted against a threshold value to be set as an outlier. Note that AUC is an index value that gives a scale of inspection accuracy that gives 1 when there is no misjudgment. It is shown that as the threshold value is increased from 0σ, the AUC is improved substantially linearly up to around 5σ. On the other hand, when the threshold value reaches 6 to 7σ, the improvement effect of AUC is saturated. This is because the defect amount is calculated by ignoring the small outlier value of a normal pixel by setting the outlier below the threshold to 0σ, so that the AUC is not affected unless there is a very large defect amount. .

  FIG. 12 shows an experimental example in which a defect amount when an image evaluation is performed on an actual inspection target is associated with a non-defective product and a defective product to be inspected. FIG. 12A shows the result of evaluating the defect amount for about 70 inspection images when the threshold for outliers = 0σ, FIG. 12B is the threshold = 6σ, and FIG. 12C is the threshold = 9σ. . As shown in FIG. 12, when the threshold applied to the outlier is set to 0σ, the difference in the defect amount between the non-defective sample and the defective sample cannot be said to be remarkable due to the accumulation of noise.

  However, in the experimental example in which the threshold value = 6σ, it is recognized that the defect amount of the non-defective sample is released from the influence of noise accumulation and is significantly reduced. Further, in the case of threshold = 9σ, it can be understood that the defect amount of most good samples is numerical value = 0, and the difference from the defective product is recognized as clearer.

  FIG. 13 shows defect amounts for the same experimental example as shown in FIG. 12 when threshold = 22σ. In FIG. 13, it is the same as FIG. 12 that the defect amount for the non-defective sample is improved to number = 0. However, some defective samples have a defect amount of 0. This is considered to be because the defect amount of a defective sample having a relatively low specificity is also reduced as a result of increasing the threshold value.

  In the inspection system of the present embodiment, it is premised that the selection of the non-defective product / defective product is performed based on the determination of the control unit 501, and therefore it is preferable to eliminate the possibility of erroneously recognizing the defective product as a non-defective product. From this point of view, it is preferable that the threshold value applied to the outlier is between 5σ and 20σ based on the result of FIG.

  FIG. 14 shows a change in recognition performance (AUC) with respect to the size (number of neighbors) of the surrounding area to be referred to. The results shown in FIG. 14 show the results when the region of interest is in units of pixels as a line graph for five different samples 1401, 1402, 1403, 1404, and 1005, and the average value thereof as a bar graph 1410. .

  The number of neighbors = 0 indicates that the surrounding area is not referred to. When the number of neighbors = 1, it indicates that the nearest neighboring pixels of the target pixel shown in FIG. When the number of neighbors referring to the neighboring pixels = 1 recently, it is shown that the average value (blue bar graph) of AUC indicating the recognition performance is higher than when the neighboring region is not referenced. It is also shown that the number of neighbors = 2 is increased compared to the case where no reference is made.

  It was confirmed that the AUC was lower when the number of neighbors = 3 than when the number of neighbors = 2. One reason for this is considered to be that the sensitivity to defects decreases because the probability of statistically including locations having values close to defects is increased as the number of areas to be referenced increases. When the number of neighbors = 4, a further decrease in AUC was confirmed.

  In terms of individual samples, there are samples 1401 and 1403 in which the number of neighbors = 0 and AUC = 1 (maximum performance). Even in these samples, even when the number of neighbors = 1 and the number of neighbors = 2, the AUC does not decrease and tends to maintain a high value. For this reason, it is preferable that the number of neighbors referred in this embodiment be 1-2.

  As described above, in the present embodiment, an information processing apparatus, an information processing method, and an information processing apparatus that can efficiently detect specificity in an image regardless of the positioning, shape, and abnormality distribution of an inspection target. A program and an inspection system can be provided.

  The present invention has been described with the embodiment. However, the present invention is not limited to the embodiment, and other embodiments, additions, modifications, deletions, and the like can be conceived by those skilled in the art. Any of the embodiments is included in the scope of the present invention as long as the operations and effects of the present invention are exhibited.

110: inspection area 111: inspection object 113: attention area 113a: attention area 113b: peripheral area 113c: peripheral area 301: control section 308: imaging device 500: inspection system 501: control section 502: operation section 503: output I / O
504: storage unit 505: transfer arm 506: inspection object 507: transfer unit 508: imaging device 600: block 601: CPU
602: RAM
602: Image RAM
603: ROM
604: Display device 605: Communication device 606: System bus 607: I / O bus 608: Imaging device 609: Storage device 610: External drive 700: Block 701: Input I / O
702: Pre-processing unit 703: Calculation unit 704: Determination unit 705: Output unit 706: Storage unit 707: Second storage unit

JP 2005-265661 A JP2013-160629A

Image Local Features and Specific Object Recognition-SIFT and Recent Approaches-Fujiyoshi Lab. Chubu University Faculty of Engineering Department of Information Engineering http: // //www.vision.cs.chubu.ac.jp/cvtutorial/PDF/02SIFTandMore.pdf 2015 November 24 download file Image analysis with various texture features http: ///www.design.kyushu-u.ac.jp/lib/doctor/1999/k032/k032-05.pdf November 24, 2015 download file Download file of AIST, http://www.isi.imi.i.u-tokyo.ac.jp/en/pattern/hlac.html, November 24, 2015 Facial feature extraction and human facial expression recognition based on the amount of feature movement, HLAC features and facial expression recognition using the k-Nearest-Neighbor method http: //www.cst.nihon-u.ac.jp/research/gakujutu/53 Download file of /pdf/G-1.pdf on November 24, 2015 Feature extraction based on Gradient -SIFT and HOG-, Hironobu Fujiyoshi, http: //www.hci.iis.u-tokyo.ac.jp/~ysato/class14/supplements/sift_tutorial-Fujiyoshi.pdf March 24 download file Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. "SURF: Speeded Up Robust Features." Computer Vision and Image Understanding (CVIU). Vol. 110, No. 3, pp. 346-359 , 2008. Geoffrey E. Hinton; R. R. Salakhutdinov (2006-07-28). "Reducing the Dimensionality of Data with Neural Networks". Science 313 (5786): 504-507. Feature extraction with AutoEncoder, download file on November 24, 2015 at http://www.slideshare.net/lewuathe/auto-encoder-v2

Claims (14)

An information processing apparatus for inspecting a target image including an image to be inspected,
Means for performing preprocessing for comparing the target image with a reference image;
Defining a region of interest in the target image and a peripheral region adjacent to the region of interest, and calculating a feature amount of the region of interest;
Means for calculating an outlier that numerically indicates the specificity of the image in the region of interest from a comparison of the feature quantities of the image corresponding to the region of interest and the peripheral region in the plurality of reference images;
Means for providing an index to be used for inspection based on the outlier.   The information processing apparatus further includes means for reducing a noise component included in the outlier when the outlier is obtained, and the means for providing the index uses the outlier after cutting the noise component. The information processing apparatus according to claim 1, which generates   The information processing apparatus according to claim 1, wherein the feature amount is a luminance value, a color value, edge information, spatial frequency information, texture information, or a value generated using a neural network.   The information processing apparatus according to claim 1, further comprising means for digitizing the specificity of the image as a deviation value related to a corresponding area of the plurality of reference images.   The information processing apparatus according to claim 1, wherein the information processing apparatus includes a unit that allows a change in a spread of the peripheral area.   The information processing apparatus according to claim 1, wherein the information processing apparatus includes means for enlarging or reducing a part or all of the target image.   The information processing apparatus according to claim 1, wherein the means for calculating the outlier calculates an outlier using the enlarged or reduced target image.   The information processing apparatus enables addition of the reference image, and the information processing apparatus updates a parameter for calculating the outlier including the added reference image, and the updated parameter is updated after the update. The information processing apparatus according to claim 1, further comprising: a unit that calculates the outlier using a computer; and a unit that allows a threshold for the outlier to be externally set. An apparatus-executable method for digitizing a target image including an image to be inspected, wherein the information processing apparatus
Performing pre-processing for comparing the target image with a reference image;
Defining a region of interest and a peripheral region adjacent to the region of interest in the target image, and calculating a feature amount of the region of interest;
Calculating an outlier that numerically indicates the specificity of the image in the region of interest from a comparison of feature values of the image corresponding to the region of interest and the peripheral region in the plurality of reference images;
Providing an index to be used for inspection based on the outlier. The method according to claim 9, further comprising reducing a noise component included in the outlier when obtaining the outlier, and generating the indicator using an outlier after the noise component is cut. Method.   The method according to claim 9 or 10, wherein the feature amount is a luminance value, a color value, edge information, spatial frequency information, texture information, or a value generated using a neural network.   The method according to any one of claims 9 to 11, further comprising quantifying the specificity of the image as a deviation value related to a corresponding region of the plurality of reference images.   An apparatus-executable program for causing an information processing apparatus to function as the means according to any one of claims 1 to 8. An inspection system for inspecting an inspection object,
Imaging means for acquiring a target image including the image to be inspected;
Means for performing preprocessing for comparing the target image with a reference image;
Defining a region of interest in the target image and a peripheral region adjacent to the region of interest, and calculating a feature amount of the region of interest;
Means for calculating an outlier that numerically indicates the specificity of the image in the region of interest from a comparison of the feature quantities of the image corresponding to the region of interest and the peripheral region in the plurality of reference images;
Means for providing an index to be used for inspection based on the outlier;
Means for reducing a noise component included in the outlier when obtaining the outlier;
An inspection system comprising: means for providing the index using the outlier after cutting the noise component; and means for determining a defect of the inspection object from the target image using the index.