JP5637373B2 - Image classification method, appearance inspection method, and appearance inspection apparatus - Google Patents

Description

  The present invention relates to a technique for classifying images based on their feature amounts, for example.

  For example, as a method of classifying an image into several classification groups according to the contents, a method is known in which feature amounts are calculated from images to be classified, and classification is performed by a classifier based on the feature amounts. In this technology, the classification accuracy of the classification target greatly affects the classification accuracy. However, many feature quantities are known, and classification based on such many feature quantities is not realistic in terms of processing time. Techniques for selecting a suitable one have been conventionally proposed.

  For example, in the technique described in Patent Document 1, the significance of each feature amount is quantitatively obtained by solving a minimization problem corresponding to the SVM (support vector machine) quadratic programming format, and the value is obtained in advance. An effective feature amount is set to a value larger than a predetermined threshold value, and learning and classification are performed using only the effective feature amount.

Japanese translation of PCT publication No. 2008-524675 (for example, FIG. 4)

  In the above prior art, how to set a threshold value for distinguishing whether or not a feature amount is effective is important for classification accuracy. However, there is no specific description in Patent Document 1 regarding the setting method. Even if this is artificially set based on experience, for example, the type and number of feature values that are valid may change depending on the setting method, which adversely affects the stability of the results of subsequent classification. There is still a fear.

  The present invention has been made in view of the above problems, and in the classification technique based on the feature amount, only the feature amount effective for classification is selected efficiently and without variation, and the effective feature amount thus selected is used. An object of the present invention is to provide a technique that makes it possible to obtain a stable classification result in a short processing time.

One aspect of this inventions is a method of classifying images of classifying an image based on feature amount, in order to achieve the above object, the machine learning based on a plurality of kinds of feature amounts of the image to be teaching data From the result, a learning step for obtaining an index value for each feature quantity as an index of the effect on the classification result among the plurality of feature quantities, and based on the index value, the plurality of kinds and effective feature quantity selection step of selecting RaYu effective characteristic quantity or in the feature amount, a re-learning step of performing re-learning of the teacher data based on the effective characteristic quantity elected, the image to be classified, the A feature amount calculating step of calculating an effective feature amount; and a classification step of classifying the image based on the result of the relearning and the effective feature amount. In the effective feature amount selecting step, the plurality of types of feature amounts Of which, the index value is the best To N-th (N is a natural number) are selected as the effective feature quantities, and the indices for each of the N effective feature quantities with respect to the sum of the index values of all the plurality of feature quantities It is characterized in that the ratio of the sum of the values becomes a predetermined specified ratio.

Another aspect of this inventions is a method of classifying images of classifying an image based on feature amounts performs machine learning based on a plurality of kinds of feature amounts of the image to be teacher data, from the result A learning step for obtaining, for each feature quantity, an index value indicating the magnitude of the effect on the classification result among the plurality of feature quantities, and based on the index value, from among the plurality of feature quantities The effective feature amount is calculated for an effective feature amount selecting step for selecting the effective feature amount , a re-learning step for re-learning the teacher data based on the selected effective feature amount, and an image to be classified. A feature amount calculating step; and a classification step for classifying the image based on the result of the relearning and the effective feature amount. In the effective feature amount selecting step, the index value of the plurality of feature amounts is large. From top to bottom When the index value corresponding to the feature amount is cumulatively added, and the ratio of the cumulative added value to the sum of the index values of all the plurality of types of feature amounts reaches a predetermined specified ratio, A feature amount corresponding to the index value cumulatively added is set as the effective feature amount.

In these inventions, feature quantities that are obtained as a result of machine learning using images serving as teacher data and that have a large effect on the classification result are selected as effective feature quantities. In addition, the number of feature values to be used as effective feature values and the threshold value of the index value are not uniquely determined, but these are dynamically determined according to the value of the index value representing the effectiveness of each feature value obtained by machine learning. And an effective feature amount is selected based on the objective decision. Therefore, there is no variation in the number and type of effective feature quantities to be selected. Therefore, according to the present invention, a feature amount effective for classification can be efficiently selected from many feature amounts, and the result does not vary. Then, by performing classification using the effective feature amount selected in this way, it is possible to obtain a stable classification result in a short processing time. Further, by performing relearning using teacher data based on the effective feature amount selected as described above, and performing classification based on the relearning result, the classification accuracy of the images to be classified can be further improved. Is possible.

  In the present invention, in the learning step, for example, an optimization problem corresponding to SVM (support vector machine) for a plurality of types of feature quantities can be solved, and the value of the optimum solution can be used as an index value. According to the theory of SVM, it is known that the value of the optimal solution of the optimization problem directly becomes a value indicating the effectiveness of each feature quantity, and this is set as the “index value” of the present invention. Can effectively select effective features.

  Alternatively, a neural network may be constructed for a plurality of types of feature values, and a value based on the weight of each neuron branch in the neural network may be used as the index value. In a neural network constructed by learning, a value based on the weight of a neuron branch corresponding to a certain feature value, for example, the sum of the absolute values of the branch weights becomes an index value representing the effectiveness of the feature value. By making this an “index value” of the present invention, it is possible to efficiently select an effective feature amount.

  Further, an appearance inspection method according to the present invention is an appearance inspection method for inspecting the appearance of an inspection object, and in order to achieve the above object, an image acquisition step of acquiring an image obtained by imaging the appearance of the inspection object; And the step of classifying the acquired image by the above-described classification method. Furthermore, in order to achieve the above object, the appearance inspection apparatus according to the present invention acquires an image obtained by capturing an image of the appearance of the inspection object, and classifying means for classifying the acquired image by the classification method described above. It is characterized by having. According to these inventions, since the image of the inspection object can be classified based on the effective feature quantity selected by the selection method having no variation as described above from among many feature quantities, Appearance inspection can be performed stably in a short processing time.

  According to the present invention, effective feature quantities effective for classification can be obtained efficiently and objectively based on the results of machine learning, so that classification based on feature quantities can be performed stably in a short processing time. Can be done.

It is a figure showing the schematic structure of the inspection system as one embodiment of the appearance inspection device concerning this invention. It is a flowchart which shows the automatic defect classification | category process in this inspection system. It is a flowchart which shows the learning process in this embodiment. It is a figure which shows the basic concept of the selection method of the effective feature-value in this embodiment. It is a flowchart which shows an example of the process for selecting an effective feature-value. It is a figure which shows an example of a neural network. It is a figure which shows an example of the experimental result at the time of using SVM. It is a figure which shows an example of the experimental result at the time of using a neural network.

  FIG. 1 is a diagram showing a schematic configuration of an inspection system as an embodiment of an appearance inspection apparatus according to the present invention. This inspection system 1 is an inspection system that performs inspection of defects such as pinholes and foreign matters appearing on the appearance of a semiconductor substrate (hereinafter referred to as “substrate”) S to be inspected, and automatically classifies detected defects. . The inspection system 1 performs the defect inspection based on the image data from the image pickup device 2 and the image pickup device 2 that picks up the inspection target area on the substrate S, and when the defect is detected, the defect belongs to the category to which the defect should belong. A host computer 5 having an inspection / classification function for automatically classifying (ADC) and a function for controlling the entire operation of the inspection system 1 is provided. The imaging device 2 is incorporated in the production line of the substrate S, and the inspection system 1 is a so-called inline system.

  The imaging device 2 moves the stage 22 relative to the imaging unit 21 that acquires image data by imaging the inspection target region on the substrate S, the stage 22 that holds the substrate S, and the imaging unit 21. The imaging unit 21 includes a stage driving unit 23, and an imaging unit 21 forms an image by an illumination unit 211 that emits illumination light, an optical system 212 that guides the illumination light to the substrate S and receives light from the substrate S, and an optical system 212. An imaging device 213 that converts the image of the substrate S thus converted into an electrical signal is included. The stage drive unit 23 includes a ball screw, a guide rail, and a motor. The apparatus control unit 501 provided in the host computer 5 controls the stage drive unit 23 and the imaging unit 21, so that the inspection target region on the substrate S is changed. Imaged.

  The host computer 5 implements each functional block shown in FIG. 1 by software by executing a control program read in advance. In addition to the apparatus control unit 501, the host computer 5 includes functional blocks such as a defect detection unit 502, a defect classification unit (ADC) 503, an effective feature amount specifying unit 504, a teaching unit 505, and a feature amount calculator 506. ing. Further, the host computer 5 displays a storage device 510 for storing various data, an input receiving unit 511 such as a keyboard and a mouse for receiving operation inputs from the user, and a display for displaying visual information for the user such as operation procedures and processing results. Part 512 and the like. Although not shown in the figure, a reading device for reading information from a computer-readable recording medium such as an optical disk, a magnetic disk, a magneto-optical disk, etc. Are suitably connected via an interface (I / F) or the like.

  The details of each functional block will be described later, but here, the function of each functional block will be briefly described. The defect detection unit 502 performs defect detection by finding a unique area in the inspection target area while processing the image data of the inspection target area. When the defect detection unit 502 detects a defect from the inspection target area, image data of the defect and various data used for the inspection are temporarily stored in the storage device 510.

  On the other hand, the defect classification unit 503 performs, in software, a process of classifying detected defects using learning algorithms such as SVM (Support Vector Machine), neural network, decision tree, and discriminant analysis. To do. A teaching unit 505 gives teaching data for causing the defect classification unit 503 to machine-learn the algorithm. The feature amount calculator 506 calculates a feature amount characterizing the defect based on the detected defect image data. The effective feature amount specifying unit 504 specifies some feature amounts suitable for defect classification among many feature amounts as effective feature amounts, and gives the information to the teaching unit 505.

  FIG. 2 is a flowchart showing an automatic defect classification process in this inspection system. This process is an embodiment of the “appearance inspection method” of the present invention. First, an image of the inspection object is acquired (step S101). Specifically, the imaging device 2 images the surface of the substrate S to be inspected. Then, the defect detection unit 502 performs defect detection on the captured image (step S102). Hereinafter, an image regarded as including a defect is referred to as a “defect image”.

  When the defect image is detected, the defect detection unit 502 displays an image of the inspection target area (defect image) and data obtained at the time of the inspection process (data such as coordinate information of the inspection target area, difference image, reference image, and the like). These data are sent to the defect classification unit 503 and also written to the storage device 510. Based on the image data stored in the storage device 510, the feature amount calculator 506 calculates the feature amount of the defect image (step S103).

  Subsequently, the automatic classification of the detected defect is performed by the defect classification unit 503 (step S104). The defect classification unit 503 can execute a machine learning algorithm, and classifies a defect image based on a learning result performed in advance based on teacher data given from the teaching unit 505. Specifically, based on the feature amount of the defect image obtained by the feature amount calculator 506, it is determined which of the plurality of classification categories set in advance according to the feature of the defect corresponds to the defect image. To do. Note that the feature amount refers to a value obtained by processing the pixel value according to a predetermined rule, and most of the feature amount is obtained by applying some kind of filter to the image. For example, an index value indicating some feature of the image such as the average luminance of the image, texture information, the size of an area satisfying a predetermined condition, the amount of an edge to be extracted, and the like is used as the feature amount. A feature amount based on higher-order local autocorrelation using a plurality of autocorrelation filters may be used.

  The classification result is displayed on the display unit 512 by an appropriate display method (step S105) and presented to the user. In the inspection system 1, every time a defect is detected by the defect detection unit 502, the operations shown in steps S103 to S105 in FIG. 2 are performed in real time, and a large number of images indicating defects are classified at high speed. The defect images detected in this way are classified into corresponding classification categories.

  Next, the learning process by the defect classification unit 503 will be described. In this embodiment, prior to the defect classification process of FIG. 2, the defect classification unit 503 and the like perform a learning process described below. In the learning process, a newly detected unknown is obtained by receiving a teaching from a user about a typical defect image for each classification category corresponding to the type of defect and learning as a typical example of such a defect image as teacher data. It is possible to classify the defect image into several classification categories based on the feature.

  FIG. 3 is a flowchart showing the learning process in this embodiment. First, an image serving as teacher data is acquired (step S201). This image may be newly acquired by, for example, imaging a sample substrate by the imaging device 2, or may be an image obtained from a library as a typical example of a defect image from an external device. Then, a defective portion is detected from the image thus obtained (step S202), and the feature amount of the image data is calculated by the feature amount calculator 506 (step S203). At this point in time, it is unclear what feature values should be used to classify defects accurately, so it is desirable that the feature values to be calculated be as rich as possible.

  Next, the user is allowed to set and input how many and what classification categories the defect image is classified into, and the teaching unit 505 creates classification categories corresponding to them (step S204). Then, the detected defect images are sequentially displayed on the display unit 512, and a user's teaching input as to which classification category the displayed defect should be classified is received (step S205). When at least one defect image is designated for each classification category by the teaching input, the teaching unit 505 gives the teaching input result and the feature amount of each defect image to the defect classification unit 503, and the defect classification unit 503 A learning operation based on the above is performed (step S206). As a learning process at this time, for example, a well-known process such as an SVM (Support Vector Machine), a discriminant analysis method, or a neural network can be applied. The learning data obtained in this way is represented by reference numeral S1.

  In this learning, since many feature quantities are used, classification accuracy is high, but calculation processing takes time. For this reason, there is a problem in terms of processing time when applied to defect classification in real time as it is. On the other hand, as a result of the learning work, it becomes clear how much each feature amount contributes to the classification result. From this, among the many feature quantities, those having a large influence on the classification result are selected as effective feature quantities, and the subsequent learning / classification process is performed using only these effective feature quantities. In this way, the processing time can be shortened by reducing the types of feature values used in the calculation.

  On the other hand, reducing feature quantities to be used may lead to a decrease in classification accuracy, so how to select effective feature quantities is important to maintain classification accuracy. In this embodiment, the following is performed. That is, the effectiveness for each feature amount is quantitatively obtained from the result S1 of the learning work (step S207), and the effective feature amount is specified based on the value W of the effectiveness (step S208). Specifically, an effective feature amount having a high effectiveness value W is set.

FIG. 4 is a diagram showing the basic concept of the effective feature amount selection method in this embodiment. Consider the case where the effectiveness corresponding to each feature quantity obtained in this way is arranged in descending order of the value W, as indicated by a circle in FIG. Here, m feature values (m is a natural number) are assumed to be used for learning, and X1, X2,..., Xm are assigned in order from those having the highest effectiveness W to the feature values. . Also, the effectiveness values W corresponding to the feature amounts X1, X2,..., Xm are represented by symbols W1, W2,. Furthermore, the sum of the effectiveness W of each feature amount is represented by a symbol Sw. That is,
Sw = W1 + W2 + ... + Wm
It is.

  Among the feature amounts X1, X2,..., Xm, the feature amount having a larger value of the effectiveness W naturally has a larger influence on the classification result, and cannot be ignored in the classification operation. On the other hand, a feature amount having a low effectiveness W has little influence on the classification result, and even if these are excluded from the classification work, it is considered that the degradation of classification accuracy is small. If such a feature quantity having a small influence is excluded and classification is performed using only a feature power having a large influence, that is, a high effectiveness W, the processing time required for the calculation of the feature quantity and the classification work based thereon is reduced. It can be greatly shortened.

  Therefore, a feature quantity having a large effectiveness W may be selected as an effective feature quantity. However, there is a problem of how many of m feature quantities are effective feature quantities. According to the technical idea described in Patent Document 1 described above, a threshold value is set for the effectiveness W, and a feature amount having an effectiveness higher than the threshold is set as an effective feature amount. However, the number of effective feature quantities selected depending on how the threshold value is determined varies greatly, and the effect on the classification accuracy is unknown. Actually, Patent Document 1 does not specify how to set the threshold value.

On the other hand, in this embodiment, an effective feature amount is selected as follows. In the array W1, W2,..., Wm of the effectiveness W obtained as described above, the effectiveness values W are cumulatively added in descending order. This cumulative addition value is represented by symbol Σw. As shown in FIG. 4, it is assumed that the order of effectiveness at which the cumulative added value Σw reaches a specified ratio, for example, 50%, with respect to the total sum Sw of effectiveness is n. In other words, when the value W is cumulatively added from the highest value to the nth value (indicated by white circles in FIG. 4), the cumulative added value Σw:
Σw = W1 + W2 + ... + Wn
However, it is 50% or more of the total sum Sw of effectiveness.

  At this time, the feature values corresponding to the value of the effectiveness value W from the highest to the nth, that is, feature values X1, X2,..., Xn are used as effective feature values for the classification. The subsequent feature quantities (indicated by hatched circles in FIG. 4) are excluded from the processing target. As will be described in detail later, this makes it possible to reduce the number of feature quantities used for classification from m to n without reducing the classification accuracy. By reducing the types of feature values to be used, the processing time required for the classification work can be shortened.

  As described above, in this embodiment, the effectiveness W of each feature amount is cumulatively added in order from the largest, and the cumulative addition value Σw is added until the proportion of the total sum Sw of the effectiveness reaches a specified proportion. Further, n feature amounts (X1, X2,..., Xn) corresponding to the n effectiveness levels (W1, W2,..., Wn) are set as effective feature amounts. For this reason, the value of the effectiveness corresponding to the feature amount regarded as the effective feature amount and the number of effective feature amounts to be selected are not determined in advance, and these values are dynamically determined depending on the distribution of the effectiveness level of each feature amount. It is determined. As described above, since the effective feature amount is selected according to the distribution state of the effectiveness level of each feature amount, in this embodiment, the number of feature amounts necessary for the classification is reduced without reducing the accuracy of the classification. Can do. The selection of the effective feature amount based on this principle can be realized by various algorithms. For example, it can be realized by the following software processing.

  FIG. 5 is a flowchart showing an example of processing for selecting an effective feature amount. First, the sum Sw of m feature quantity effectiveness W (W1, W2,..., Wm) is calculated (step S301), and the cumulative effectiveness value Σw is reset to 0 (step S302). . Next, among the m feature quantities, the one having the largest value of the effectiveness W corresponding to the feature quantity is selected from unselected feature quantities not yet selected for the following processing (step S303). Naturally, in the first process, the feature amount X1 having the largest effectiveness W1 is selected.

  The effectiveness W corresponding to the selected feature quantity is cumulatively added (step S304), and the feature quantity is selected until the cumulative addition value Σw becomes 50% or more of the total sum Sw of the effectiveness (step S305). Repeat cumulative addition of effectiveness. If the cumulative addition value Σw becomes 50% or more of the total sum Sw of the effectiveness, the feature quantity selected so far can be set as the effective feature quantity (step S306). As described above, when the effective feature amount is obtained, the step of rearranging the feature amount according to the effectiveness as shown in the principle diagram of FIG. 4 is not essential.

  Here, when the learning algorithm of the defect classification unit 503 is SVM, when the optimization problem for a large number of feature quantities is solved, the optimum solution becomes an index indicating the effectiveness W of each feature quantity as it is. Therefore, in this case, it is possible to easily select an effective feature amount using the value of the optimal solution. On the other hand, when the learning algorithm uses a neural network, for example, a value corresponding to the effectiveness W can be obtained as follows, for example.

FIG. 6 is a diagram illustrating an example of a neural network. As a result of learning with the classification categories Y1, Y2, and Y3 given to the feature quantities X1, X2, and X3, the neural network shown in FIG. 5 is constructed, and the weights (W111 to W223) of the branches are given. Of the branch weights thus obtained, a value obtained by adding the absolute values of the branch weights in the first layer on the input layer side for each feature amount can be used as a value indicating the effectiveness. For example, for the feature quantity X1, the sum of the absolute values of the weights W111 and W112 of each branch of the first layer:
W1 = | W111 | + | W112 |
Can be a value indicating the effectiveness of the feature amount X1. Then, by applying the method of FIG. 4 or FIG. 5, an effective feature amount can be selected similarly. Similarly, in other learning algorithms, an appropriate value that indicates the degree of influence of each feature amount on the classification result in a relative or absolute and quantitative manner can be used as a value indicating the effectiveness.

  Returning to FIG. 3, when n effective feature amounts are specified from among the m feature amounts, the re-learning is performed on the previously acquired defect image based on the specified n effective feature amounts. This is performed (step S209). The learning data obtained at this time is represented by reference numeral S2. The learning data S2 is stored in the storage device 510 (step S210), and when the defect image is classified (step S104) in the automatic defect classification process (FIG. 2) described above, the learning data stored in the storage device 510 is stored. Data S2 is used. Therefore, at the time of calculating feature amounts in the automatic defect classification process (step S103), it is only necessary to calculate n effective feature amounts. For this reason, the number of feature quantities used in the calculation of feature quantities and the classification work based thereon is reduced, and automatic defect classification can be performed efficiently in a short time.

  Next, the reason why the “specified ratio” for obtaining the effective feature amount is set to 50% in this embodiment will be described. The inventors of the present application collect a large number of image samples, calculate various feature amounts for them, select effective feature amounts by the above method while changing and setting the specified ratio in multiple stages, and reduce the feature amount An experiment was conducted to verify the classification accuracy.

  Two types of sample images were used: a set of biological tissue images captured by a biological microscope and a set of defect images captured by a semiconductor inspection apparatus. In addition, as the feature quantity set, a high-order local autocorrelation feature quantity (Texture) using a 3 × 3 matrix, a feature quantity (Structure) indicating a defect structure such as an area of a defect region, a perimeter, and a circularity, Rotationally Symmetric Texture, which is the result of merging the same high-order local autocorrelation features using a 3 × 3 matrix of feature texture, and a 5 × 5 matrix of the above feature texture Prepare high-order local autocorrelation features (Coarse Texture) and Hu moment, which is an amount that is not affected by parallel movement, rotational movement, and scale change of objects in the image. The feature amount appropriately selected according to the above was used. Two types of learning algorithms, SVM (linear kernel method) and three-layer neural network, were used.

  In the experiment, the effective feature quantity is selected by executing the processing shown in FIG. 3 using the prepared sample image and feature quantity set, and classified by the Cross-Validation from the learning data S2 obtained by learning based on the effective feature quantity. In addition to predicting performance, the feature reduction effect was determined from the number of selected effective features. Since almost similar results were found in experiments performed on various image data, only a part of the experimental results are shown below.

  FIG. 7 is a diagram showing an example of an experimental result when SVM is used as a learning algorithm. FIG. 8 is a diagram showing an example of an experimental result when a neural network is used as a learning algorithm. In these figures, the classification performance after the selection of effective features and the number of selected effective features are normalized and displayed with the classification performance and the number of features when the feature amount is not reduced as 1, respectively. Yes.

  As shown in FIG. 7A and FIG. 8A, in any learning algorithm, when the specified ratio is set to 20%, the variation in the classification performance becomes large, and the specified ratio is set to 50% and 80%. As the percentage increases, the variation decreases. In this respect, it can be said that the larger the specified ratio, the better the classification performance. However, in the cases where the specified ratio is 50% and 80%, the performance is nearly 100%, and it can be said that there is no great difference in the classification performance and its variation. The classification performance may exceed 1, but this is known as a “dimensional curse” in which the classification performance decreases as the number of feature quantities used increases. The use of feature quantities is not necessarily the best, and it is suggested that there are cases where the classification performance is improved by excluding feature quantities with little influence.

  On the other hand, looking at the number of effective feature amounts shown in FIGS. 7B and 8B, when the specified ratio is 50%, the number of effective feature amounts is further increased as compared to the case where the specified ratio is 80%. It is possible to reduce to about half. Thus, in order to obtain a large feature quantity reduction effect while maintaining the classification performance, it is desirable to set the specified ratio to about 50%. Experiments were performed with various experimental conditions changed, and almost the same tendency was observed.

  As described above, in this embodiment, the effective feature amount is calculated with the specified ratio being 50%. In this way, in this embodiment, it is possible to greatly reduce the number of feature quantities used for classification while sufficiently maintaining practical classification performance. Then, by classifying using the effective feature quantity selected in this way, it is possible to classify the image efficiently in a short time and with excellent classification accuracy.

  As described above, in this embodiment, steps S206 and S207 in FIG. 3 correspond to the “learning process” of the present invention, and each step in step S208 and FIG. This corresponds to the “selection process”. In the above embodiment, the effectiveness W (W1, W2, etc.) corresponds to the “index value” of the present invention. In the automatic defect classification process (FIG. 2) of the above embodiment, steps S101, S103, and S104 correspond to the “image acquisition step”, “feature amount calculation step”, and “classification step” of the present invention, respectively. In the inspection system 1 shown in FIG. 1, the imaging device 2 functions as the “image acquisition unit” of the present invention, and the defect classification unit 503 functions as the “classification unit” of the present invention.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, although the SVM and the neural network are shown as examples of the learning algorithm in the above embodiment, the learning algorithm is not limited to these, and various known algorithms can be applied.

  In the above embodiment, each functional block is realized on software by a control program executed on the host computer 5, but at least a part of these functional blocks may be realized by dedicated hardware. Good.

  In the above embodiment, the “specified ratio” in the present invention is 50%. However, the value is not limited to this value, and this value is set according to the property of the image to be classified and the processing capability of the system. You may change suitably. When the specified ratio is reduced, the feature amount reduction effect is large, but the classification performance tends to be low. Also, if the specified ratio is increased, the feature amount reduction effect is reduced, but the classification performance is improved. However, as described above, increasing the number of feature quantities does not necessarily improve the accuracy, so it is desirable to determine the prescribed ratio by judging the trade-off between the feature quantity reduction effect and the classification performance.

  In the above embodiment, the present invention is applied to an in-line type inspection system, but a so-called offline type in which a real-time inspection function and a function for performing a teaching operation for learning a defect classification unit are separated. The present invention can also be applied to other classification devices. The inspection system 1 includes the imaging device 2 as an “image acquisition unit”, but does not include the imaging device, and receives and classifies image data from an external device or an external recording medium. The present invention can also be applied to. In this case, an interface unit, a disk drive, or the like that receives image data from an external device functions as the “image acquisition unit” of the present invention.

  Moreover, although the said embodiment has applied this invention to the apparatus for inspecting the defect which appears on the surface appearance of a semiconductor, the application object of this invention is not limited to this, The image of a target object is taken. The present invention can be applied to various techniques for processing and classification. For example, the present invention can also be applied to a technical field in which a tissue image acquired from a living tissue of a patient is image-processed to support pathological diagnosis.

  The present invention is applied to various technical fields to which a technique for classifying images according to their feature quantities, such as a technique for detecting and classifying defects in various substrates such as a semiconductor substrate and a printed wiring board and a pathological diagnosis support technique. It can be suitably applied.

1 Inspection system (Appearance inspection device)
2 Imaging device (image acquisition means)
5 Host computer 503 Defect classification part (classification means)
S substrate S101 image acquisition step S103 feature amount calculation step S104 classification step S206, S207 learning step S208 effective feature amount selection step Sw (effectiveness) sum Σw (effectiveness) cumulative addition value W effectiveness (index value)

Claims (8)

A method of classifying images of classifying an image based on feature amounts,
Machine learning is performed based on a plurality of types of feature values of the image serving as the teacher data, and from the result, an index value that uses the magnitude of the influence on the classification result as an index between the plurality of types of feature values is determined for each feature amount. Learning process
Based on the index value, the effective characteristic quantity selection step of selecting RaYu effective characteristic quantity or among the plurality of kinds of feature quantities,
A re-learning step of re-learning the teacher data based on the selected effective feature amount;
A feature amount calculating step for calculating the effective feature amount for images to be classified;
A classification step of classifying the image based on the result of the relearning and the effective feature amount ,
In the effective feature quantity selection step, the index values of the plurality of types of feature quantities from the highest to the Nth (N is a natural number) are selected as the effective feature quantities, the proportion of the sum of the N effective feature amounts respectively of the index value to the sum of said plurality of kinds of characteristic amounts of all of the index value, characterized in that to ensure a defined predetermined proportion, image Classification method. A method of classifying images of classifying an image based on feature amounts,
Machine learning is performed based on a plurality of types of feature values of the image serving as the teacher data, and from the result, an index value that uses the magnitude of the influence on the classification result as an index between the plurality of types of feature values is determined for each feature amount. Learning process
An effective feature amount selection step of selecting the effective feature amount from the plurality of types of feature amounts based on the index value ;
A re-learning step of re-learning the teacher data based on the selected effective feature amount;
A feature amount calculating step for calculating the effective feature amount for images to be classified;
A classification step of classifying the image based on the result of the relearning and the effective feature amount ,
In the effective feature quantity selection step, the index values corresponding to the feature quantities are cumulatively added in order from the highest-order index value among the plurality of types of feature quantities, and the cumulative addition values When the proportion of all the feature values of the seeds with respect to the sum of the index values reaches a predetermined specified ratio, the feature amount corresponding to the index value that has been cumulatively added is used as the effective feature amount. And image classification method. Wherein the learning step, solves the optimization problem corresponding to the support vector machine about the characteristics of the plural kinds, the classification method of an image according to claim 1 or 2 the value of the optimal solution to the index value . The image classification method according to claim 1 or 2, wherein in the learning step, a neural network is constructed for the plurality of types of feature amounts, and a value based on a weight of a branch of each neuron in the neural network is used as the index value. The image classification method according to claim 4, wherein in the learning step, a sum of absolute values of branch weights of each neuron is used as the index value. 6. The image classification method according to claim 1, wherein the specified ratio is 50%. In an appearance inspection method for inspecting the appearance of an inspection object,
An image acquisition step of acquiring an image obtained by imaging the appearance of the inspection object;
A step of classifying the acquired image by the classification method according to claim 1 . Image acquisition means for acquiring an image obtained by imaging the appearance of the inspection object;
An appearance inspection apparatus comprising: a classifying unit that classifies the acquired image by the classification method according to claim 1 .

Images