DE102017217596A1 - Method for checking an electronic component, method for generating a model and device - Google Patents

Abstract

The invention relates to a method for checking an electronic component which is based on electronic image analysis using a learned model. The invention further relates to an associated method for generating a model as well as an associated device for checking electronic components.

Description

The invention relates to a method for checking an electronic component, to a method for generating a model for electronic image analysis for checking electronic components, and to an associated apparatus.

Electronic components, such as microcontrollers or controllers, which are for example intended for automotive applications, are typically tested prior to shipment or installation in consumer products. For example, it should be ensured that solder joints work reliably.

To check such components, an X-ray system is used in the prior art, for example, in which the respective components are X-rayed and the images taken are subjected to a pattern analysis. Such a pattern analysis can be used to detect whether certain errors have occurred, for example, whether defective solder joints are present.

Although previously known embodiments typically have only a very low rate of false negative messages, i. only a few defective products are not recognized as such, but there is a high rate of false positives, i. Numerous components are sorted out as defective, which are in fact not defective at all. Such components are therefore typically manually checked, which means a considerable amount of work.

It is therefore an object of the invention to provide a method of verifying an electronic device which, in comparison to the prior art, operates alternatively, for example with a lower rate of false positives. It is a further object of the invention to provide an associated method for generating an electronic image analysis model for verifying electronic components. In addition, it is an object of the invention to provide an associated apparatus.

This is achieved according to the invention by methods and a device according to the respective main claims. Advantageous embodiments can be taken, for example, the respective subclaims. The content of the claims is made by express reference to the content of the description.

• - Aufnehmen einer Anzahl von Bildern des elektronischen Bauelements, und
• - Durchführen einer elektronischen Bildanalyse mit den Bildern, wobei als Ergebnis der Bildanalyse das Bauelement als gut oder schlecht klassifiziert wird,
• - wobei die Bildanalyse auf einem Modell basiert, welches mittels einer Mehrzahl von Lernbildern erzeugt wurde, wobei jedes Lernbild vorab als gut oder schlecht klassifiziert wurde.

The invention relates to a method for checking an electronic component, the method comprising the following steps:

• Taking a number of images of the electronic component, and
• Performing an electronic image analysis on the images, whereby as a result of the image analysis the device is classified as good or bad,
• - wherein the image analysis is based on a model that has been generated by means of a plurality of learning images, wherein each learning image was previously classified as good or bad.

The method according to the invention allows the use of an image analysis which is based on a learning process. This has proved to be advantageous for typical applications in the inspection of electronic components, in particular in the automotive sector. In particular, the inventors of the present application have found that a significantly lower rate of false positives can be achieved without increasing the rate of false false reports.

For example, the classification as good or bad may be stored in a parameter attached to the image, a database, a flag, or otherwise. As a result, in each case a status can be assigned both for components and for learning images, wherein the status can be "good" or "bad".

It should be understood that the status of a learning image as good or bad typically refers to a device represented in the learning image. Accordingly, however, then typically the learning image is also classified. In each case only a part of a component can be displayed on a respective image or learning image. In particular, this can be a part which is to be monitored or is prone to an error.

It should be noted that the classification can be implemented as good or bad also so that not only a binary decision space is available, but a numerical probability is given. For example, the probability of "good" or "bad" between 0% and 100% can be given. Such embodiments are encompassed by the embodiment described herein. For example, such probabilities are then compared to a threshold value, and if the threshold is undershot or exceeded, the component can be classified as "good" or "bad".

The method is performed continuously repeatedly according to an embodiment for a plurality of components. It should be understood that this applies in particular to the recording steps a number of images and performing an electronic image analysis. For example, they can use an otherwise consistent model. However, the model may also change during the execution of the method.

Preferably, a plurality of images of the device are taken and the image analysis is performed on the plurality of images. This can increase the reliability. However, it is also conceivable that only an image of a respective component is taken and used accordingly.

The image analysis can in particular use an artificial intelligence, a machine learning algorithm, a support vector machine and / or an artificial neural network. It should be understood that this list does not have to be non-overlapping. It should be further understood that other techniques or procedures may be used for image analysis. For example, a k-nearest neighbor method can be used, which has proven to be advantageous for typical applications.

• - Bereitstellen einer Anzahl von ersten Lernbildern, wobei jedes erste Lernbild ein jeweiliges Bild eines Bauelements ist, welches als gut klassifiziert wurde, und
• - Erzeugen des Modells basierend auf den ersten Lernbildern.

The invention further relates to a method for generating an electronic image analysis model for checking electronic components, the method comprising the following steps:

• Providing a number of first learning images, each first learning image being a respective image of a device classified as good, and
• Generating the model based on the first learning images.

Such a procedure in the generation of a model has proved to be advantageous, in particular for checking electronic components. In this case, learning images can be used which have been recorded by actually manufactured and correspondingly classified components and are therefore very close to reality.

• - Bereitstellen einer Anzahl von zweiten Lernbildern, wobei jedes zweite Lernbild ein jeweiliges Bild eines Bauelements ist, welches als schlecht klassifiziert wurde,
• - wobei das Modell basierend auf den ersten Lernbildern und den zweiten Lernbildern erzeugt wird.

According to a preferred embodiment, the method further comprises the following step:

• Providing a number of second learning images, each second learning image being a respective image of a device classified as bad,
• - The model is generated based on the first learning images and the second learning images.

Thus, not only learning images can be included in the model, which show well-classified components, but also learning images, which show as poorly classified components. This can further improve the model. However, it should be mentioned that it is also possible to use only learning images with well-classified components, whereby a classification as a bad component can then be recognized, for example, by a striking deviation from known learning images.

According to an embodiment, the components whose images are the first learning images and / or the second learning images have been classified, in whole or in part, manually as good or bad. In this case, the learning images can be displayed, for example, to a user who manually performs the classification and provides the respective learning image with a corresponding parameter.

The components whose images are the first learning images and / or the second learning images may also have been completely or partially electronically classified as good or bad. For this purpose, it is possible, for example, to resort to methods known from the prior art.

It should be understood that the manual and the electronic classification of the learning images can also be combined so that, for example, the model can be generated with manually classified learning images and with electronically classified learning images.
Preferably, at least a part of the components whose images are the first learning images have been manually classified as good after being previously classified as bad electronically. For this purpose, for example, a known from the prior art approach can be used to first classify images electronically, then classified as poorly classified components are subsequently classified as good manually. As a result, precisely those situations or features can be incorporated into the model in which a high rate of false positive messages arises in embodiments according to the prior art. Such erroneous classifications can thus be advantageously avoided by a corresponding generation of the model.

For example, the devices whose images are the first learning images and / or the second learning images may be electronically classified as good or bad by pattern recognition.

The invention further relates to a method for checking an electronic component, which is carried out as described above, wherein the model was produced by means of a method according to the invention for producing a model. Both in terms of A method for checking an electronic component as well as with respect to the method for generating a model can be used for all embodiments and variants described herein. It has been found that the model produced according to the invention is particularly suitable for the inventive verification of electronic components.

The image analysis works preferably without pattern recognition. As a result, the problems described above according to the prior art can be avoided.

According to one embodiment, a device is classified as good if one, some or all of the solder joints or conductive bridges between two solder joints of the device are intact. These may in particular be the monitored solder joints or conductive bridges. Such conductive bridges span, for example, two solder joints in each case.

A device may be classified as bad in one embodiment if at least one solder joint or a conductive bridge between two solder joints is not intact. In this case, a malfunction of the device is expected.

The classifications described above may refer to all classifications described herein.

The images may be, for example, optical images and / or x-ray images. The use of optical images can also be combined with the use of x-ray images.

• - Ein Bildaufnahmegerät zum Aufnehmen einer jeweiligen Anzahl von Bildern jedes Bauelements, und
• - eine elektronische Steuerungsvorrichtung, welche dazu konfiguriert ist, mittels der Bilder ein Verfahren zum Überprüfen elektronischer Bauelemente gemäß der Erfindung auszuführen. Hinsichtlich dieses Verfahrens kann auf alle hierin beschriebenen Ausführungen und Varianten zurückgegriffen werden.

The invention further relates to a device for checking electronic components, the device comprising:

• An image pickup device for picking up a respective number of images of each device, and
• - An electronic control device which is configured to perform by means of the images, a method for checking electronic components according to the invention. With regard to this method, all embodiments and variants described herein can be used.

By means of the device according to the invention, the method according to the invention can be carried out particularly advantageously. For example, optical images or X-ray images can be taken with the image capture device.

The electronic control device may comprise, for example, processor means and memory means, wherein the memory means stores program code which causes the processor means to carry out a method according to the invention.

The invention further relates to a non-transitory computer-readable storage medium on which program code is stored, by means of which a processor carries out a method according to the invention. With regard to the method, all embodiments and variants described herein can be used.

The invention further relates to the use of an image analysis, in particular based on a learned model, for checking electronic components. In this case, all methods and procedures described herein can be used.

In general, one idea may be seen to replace algorithms known in the art with classification models that are replaced by techniques such as artificial intelligence or machine learning algorithms such as support vector machines or deep artificial neural networks. Such algorithms are known in principle, but they are used here in the case of automated verification of electronic components, in particular for the automotive sector.

For example, a plurality of images can be taken from different perspectives and / or focusing on different parts of each component which is to be checked. These can be stored, for example, as raster image files. They can be stored as unique identifiers with respective serial numbers of the components.

In order to train the machine learning algorithm, components from past production can be used.

For example, a machine learning algorithm may generate a model of such data sets. This model can be used to classify new devices as good or bad, for example, based on the same set of images.

For example, a new algorithm or a method according to the invention can be introduced in two stages. First of all, it can be implemented in parallel with an already known from the prior art approach to it to compare with it. If it is confirmed that it works well, the decision processes known from the prior art, ie in particular pattern-based test algorithms and manual checking, can be eliminated. Overall, fewer false negative messages (ie, devices that are defective, but not recognized as such), fewer false positives (ie, devices that are not defective, but recognized as defective) and a reduction or avoidance of manual inspection effort can be achieved ,

The model can be continually improved, especially if appropriate evaluation data is available.

• 1: eine Vorrichtung zum Überprüfen von elektronischen Bauelementen in einem Einsatzzustand, und
• 2: eine grundsätzliche Vorgehensweise zum Überprüfen elektronischer Bauelemente.

Further features and advantages will be apparent to those skilled in the embodiment described below with reference to the accompanying drawings. Showing:

• 1 a device for checking electronic components in an operating condition, and
• 2 : a basic procedure for checking electronic components.

1 shows a production device shown only schematically 10 for electronic components 20 , By way of example only, here are two electronic components 20 shown schematically. These are supposed to be checked after they are produced and before they are installed in an end user product.

For this purpose, a device is used 30 to check the electronic components 20 , The device 30 has an image capture device 40 on, which in the present case is designed as an X-ray machine. The image capture device 40 is configured to take a number of X-ray images of each of the components 20 take.

The device 30 further comprises an electronic control device 50 which is configured to perform a method according to an embodiment of the invention. This is done by the image capture device 40 generated images are checked by an image analysis, which is based on a learned model. By means of image analysis it is recognized, whether it is a good or a bad device 20 is. If it is a good component 20 it may be further processed or sold immediately. Unless it is a bad component 20 it may be sorted out or, if necessary, subjected to further manual verification.

One possible basic procedure for generating a model and evaluating it is in 2 shown. For this purpose, good parts and bad parts are provided, ie parts or components that have already been classified before the start of a learning process as good or bad. This can be done for example by evaluations according to the prior art or by manual classification. Corresponding pictures are taken of the good parts and the bad parts and fed to a data preparation.

The data generated is stored in a database as training data. They serve to generate a model ("Model"), which is based on a machine learning algorithm. The corresponding generated model is updated so that a corresponding update can be generated. This is then used for the classification of actually generated components 20 used. In order to classify components, corresponding test data are recorded. These can, for example, by means of the image pickup device 40 as described above. By means of the model, an image analysis recognizes whether it is a good or a bad component 20 is. Accordingly, a classification result is generated.

Particularly in an introductory phase, the results of a prior art review (SdT) can be compared with the results of a machine learning model (ML Model) review. This allows the reliability of the newly created model to be checked. Likewise, parameters may be checked, such as the rate of devices that have been erroneously classified as good or bad. For this purpose, for example, a manual comparison can be used to assess the particular model in an introductory phase.

Mentioned steps of the method according to the invention can be carried out in the order given. However, they can also be executed in a different order. In one of its embodiments, for example with a specific set of steps, the method according to the invention can be carried out in such a way that no further steps are carried out. However, in principle also further steps can be carried out, even those which are not mentioned.

The claims belonging to the application do not constitute a waiver of the achievement of further protection.

If, in the course of the procedure, it turns out that a feature or a group of features is not absolutely necessary, it is already desired on the applicant side to formulate at least one independent claim which no longer has the feature or the group of features. This may, for example, be a subcombination of a claim present at the filing date or a subcombination of a claim limited by further features of a claim present at the filing date. Such newly formulated claims or feature combinations are to be understood as covered by the disclosure of this application.

It should also be noted that embodiments, features and variants of the invention, which are described in the various embodiments or embodiments and / or shown in the figures, can be combined with each other as desired. Single or multiple features are arbitrarily interchangeable. Resulting combinations of features are to be understood as covered by the disclosure of this application.

Recoveries in dependent claims are not to be understood as a waiver of obtaining independent, objective protection for the features of the dependent claims. These features can also be combined as desired with other features.

Features that are disclosed only in the specification or features that are disclosed in the specification or in a claim only in conjunction with other features may, in principle, be of independent significance to the invention. They can therefore also be included individually in claims to distinguish them from the prior art.

Claims (15)

Method for checking an electronic component (20), the method comprising the following steps: - Taking a number of images of the electronic component (20), and Performing an electronic image analysis on the images, whereby as a result of the image analysis the device (20) is classified as good or bad, - wherein the image analysis is based on a model that has been generated by means of a plurality of learning images, wherein each learning image was previously classified as good or bad. Method according to Claim 1 , - which is continuously repeated for a plurality of components (20). Method according to one of the preceding claims, - In which a plurality of images of the device (20) are recorded and the image analysis is performed with the plurality of images. Method according to one of the preceding claims, - wherein the image analysis uses an artificial intelligence, a machine learning algorithm, a support vector machine and / or an artificial neural network. Method for generating an electronic image analysis model for checking electronic components (20), the method comprising the steps of: Providing a number of first learning images, each first learning image being a respective image of a device (20) classified as good, and Generating the model based on the first learning images. Method according to Claim 5 further comprising the step of: providing a plurality of second learning images, each second learning image being a respective image of a device (20) classified as bad, the model being generated based on the first learning images and the second learning images , Method according to one of Claims 5 or 6 , Wherein the components whose images are the first learning images and / or the second learning images have been classified, in whole or in part, manually as good or bad. Method according to one of Claims 5 to 7 , Wherein the components whose images are the first learning images and / or the second learning images have been classified, in whole or in part, electronically as good or bad. Method according to one of Claims 5 to 8th in which at least a part of the components whose images are the first learning images have been manually classified as good after having been previously classified electronically as bad. Method according to one of Claims 8 or 9 in which the components whose images are the first learning images and / or the second learning images have been electronically classified as good or bad by pattern recognition. Method according to one of Claims 1 to 4 , wherein the model by means of a method according to one of Claims 5 to 10 was generated. Method according to one of the preceding claims, - where the image analysis works without pattern recognition. Method according to one of the preceding claims, - wherein a device (20) is classified as good, if one, some or all solder joints or conductive bridges between two solder joints of the device (20) are intact, and / or - A component (20) is classified as bad if at least one solder joint or a conductive bridge between two solder joints is not intact. Method according to one of the preceding claims, - The images are optical images or X-ray images. Apparatus (30) for inspecting electronic components (20), the apparatus (30) comprising: - an image capture device (40) for receiving a respective number of images of each device (20), and - an electronic control device (50) which is configured to, by means of the images a method Claim 1 or any claim dependent thereon.

Images