JP6973742B2 - Inspection method for metal processed surface, inspection device for metal processed surface - Google Patents

Description

The present invention relates to a method for inspecting a metal processed surface and an apparatus for inspecting a metal processed surface.

Inside the rough-shaped material made by die-casting or aluminum casting, cavities (hereinafter referred to as cavities) formed when the molten aluminum is hardened are formed. The causes of these cavities are the entrainment of air, the gas generated when the mold release material is exposed to high temperatures, and the sink marks when the aluminum solidifies. In the subsequent process, a part of the rough shape material is cut by machining to become a final product, but these cavities may appear on the cutting surface of the machining and chipping may occur during cutting. The cavities and chips that appear on the machined surface cause leaks of oil, air, and cooling water when the final assembly is assembled with other parts. That is, the cavities and chips that appear on the machined surface cause defective products in the final assembly.

Therefore, after machining, the machined surface is inspected. As shown in FIG. 17, a general machined surface inspection is performed visually by an operator. The main reason why automation is difficult is that cutting marks, coolant liquid, cutting fluid, cutting oil, and other misidentified elements generated during machining remain on the machined surface. The cutting marks are not a problem in terms of quality, and the coolant, facets, and cutting oil can be wiped off later. That is, it should be judged that the product is defective if it has cavities or chips as shown in FIG. 18, but it should be judged that the product is defective only if cutting marks, cut pieces, or coolant are attached. No. Therefore, at the time of inspection, it is necessary to classify only defects from these misidentified elements and defects such as cavities and chips (collectively referred to as discomfort points).

In general, the misidentified elements are convex upwards, and the defects such as cavities and chips are concave downwards, but in the image from directly above, the difference between the two can be distinguished very small. difficult. Therefore, in order to distinguish these on the conveyor of the production line within a limited time, a skilled worker was required.

In addition, since the continuous confirmation work for hours on the production line is a heavy labor that requires concentration, inspection by mechanization is desired. However, even now that image shooting and image processing technologies have advanced, it is difficult to automate this difference instantly and in a wide range of the entire processed surface of the product. In addition, although there are cases of automation, it is set to lower the threshold value on the safety side in order to prevent NG products from flowing to the subsequent process, so even if there is no problem originally, even if it is a product that has no problem. Due to the presence of misidentification factors, so-called liar judgments that are judged as defective products often occur.

By the way, if a three-dimensional measuring machine using a slit laser as described in Patent Document 1 is used, unevenness can be measured with high accuracy, so that it is possible to detect cavities and defects on the machined surface. However, high resolution is required to find minute cavities and defects at the 0.5 mm level, and the width of the slit laser is limited. Therefore, in order to inspect the entire machined surface, it is necessary to repeat measurements in a narrow range many times, which requires a long inspection time, which is a bottleneck in the production process. As can be understood from FIG. 19, when a three-dimensional measuring instrument equipped with a camera 102 and a slit laser 103 is used, the three-dimensional data is measured, then the three-dimensional data is processed, and further defects are obtained. After calculating the detection, it is determined whether the product is defective or not.

Japanese Unexamined Patent Publication No. 2014-23506

As described above, with the conventional technique, it is difficult to quickly and accurately determine the presence or absence of defects on the metal machined surface.

The inventor of this case tried to solve this problem by diligently examining this point. An object of the present invention is to make it possible to quickly and accurately determine the presence or absence of defects on a metal machined surface.

In order to solve the above problem, a set image including the first image and the second image taken so that the irradiation direction of the illumination or the position of the camera is different from that of the first image is used as an inspection target. Information obtained from a large number of set images acquired for a plurality of reference metalworked surfaces and having defects to be referred to, and a plurality of reference metalworked surfaces having no defects. Acquired for the metalworked surface to be inspected using a classifier that learned the characteristics of defects using deep learning from a group of information including information obtained from a large number of set images acquired for. Based on the information obtained from the set image, the method for inspecting the metalworked surface is to determine whether or not the metalworked surface to be inspected contains defects.

In addition, the set image acquired for the metalworked surface to be inspected is illuminated by irradiating the image obtained by irradiating the image from a specific direction and the illumination from a direction different from the previous direction. It shall include the obtained image.

Further, it is preferable that the color of the illumination emitted from a specific direction and the color of the illumination emitted from a direction different from the previous direction are different from each other.

In addition, the set image acquired for the metal machined surface to be inspected shall be obtained by distorting a plurality of images, correcting the magnification, etc., and then superimposing them in alignment or arranging them in parallel. ..

In addition, the set image acquired for the metal machined surface to be inspected includes the image acquired by the camera that captures the inspection target from a specific direction and the inspection target from a direction different from the previous direction. It shall include images acquired by the second camera.

Further, as a set image including the first image and the second image taken so that the irradiation direction of the illumination or the position of the camera is different from the first image, the metal processed surface to be inspected. A calculation unit that classifies the information obtained from the acquired information using a classifier that has been learned in advance from other information, and the results derived by the calculation unit include defects in the metalworked surface to be inspected. The calculation unit includes a determination unit for determining whether or not the result is obtained, and the calculation unit includes information obtained from a large number of set images acquired for a plurality of reference metal processed surfaces having defects to be referred to, and defects. Using a classifier that learned the features of the defect using deep learning from a group of information containing information obtained from a large number of set images acquired for multiple reference metalworked surfaces that do not have the feature. Based on the information, it is an inspection device for the metal processing surface that classifies the set images acquired for the metal processing surface to be inspected.

In addition, this inspection device includes a first illumination that irradiates from a specific direction and a second illumination that irradiates from a direction different from the previous direction, and is acquired for the metal processed surface to be inspected. The set image shall include an image obtained by irradiating the first illumination and an image obtained by irradiating the second illumination.

Further, in this inspection device, it is preferable that the color of the first illumination and the color of the second illumination are different from each other.

In addition, the set image acquired for the metal machined surface to be inspected shall be obtained by distorting a plurality of images, correcting the magnification, etc., and then superimposing them in alignment or arranging them in parallel. ..

In addition, this inspection device is equipped with a first camera that shoots the inspection target from a specific direction and a second camera that shoots the inspection target from a direction different from the previous direction, and is provided on the metal processed surface to be inspected. On the other hand, the set image acquired shall include the image acquired by the first camera and the image acquired by the second camera.

In the present invention, it is possible to quickly and accurately determine the presence or absence of defects on the metal machined surface.

It is a perspective view which showed that it is possible to irradiate a metal machined surface to be inspected from a plurality of directions by a plurality of illuminations. FIG. 3 is a perspective view showing that the metalworked surface to be inspected can be irradiated from a plurality of directions by a plurality of illuminations in an example different from FIG. 1. It is the figure which showed the discomfort point from the side. It is a figure which showed the example of the photographing result which appears when the irradiation to the discomfort point is changed to right above, front, and right beside. It is a perspective view which shows that it is possible to take a picture from a plurality of directions with a plurality of cameras with respect to a metal machined surface to be inspected. It is a figure which showed the difference of the shooting result by the state of shooting a metal processed surface, and by switching the lighting to be used. It is a figure showing the cutout of a discomfort point and the creation of a set image. It is a figure which shows the judgment example by the analysis of a disagreement point. It is a figure showing that the irradiation for a defect using white illumination is changed to right above, front, and right beside. It is a figure which showed the example of the photographing result obtained by the change of the irradiation direction of FIG. It is a figure showing that the irradiation for a defect is changed to right above, front, and side by different color lighting. It is a figure which showed the example of the photographing result obtained by the change of the irradiation direction of FIG. It is a figure which shows the example which makes the cut-out image into a set image. It is a figure which showed that the photograph for a defect is performed by the camera located directly above, in front, and right beside. It is a figure which showed the example of the photographing result obtained by the change of the photographing direction of FIG. It is a figure which showed the example of the photographing result at the time of irradiating the discomfort point from directly above. It is a figure which shows the state which a worker is visually inspecting the presence or absence of a defect. It is a photograph showing an example of a discomfort that appears on a metal processed surface. It is a figure which showed the example when the metal processing surface is analyzed by the three-dimensional measuring instrument.

The embodiment for carrying out the invention is shown below. The method for inspecting the metal processed surface 9 of the present embodiment includes a first image and a second image taken so that the irradiation direction of the illumination 3 or the position of the camera 2 is different from the first image. The set image is acquired for the metal processed surface 9 to be inspected. Then, "from a large number of set images acquired for a plurality of reference metal processed surfaces having defects and a large number of set images acquired for a plurality of reference metal processed surfaces having no defects". The metal to be inspected based on the "set image acquired for the metalworked surface 9 to be inspected" using the "classifier that learned the characteristics of the defect from the obtained information using deep learning". Determine if the machined surface contains defects. Therefore, it is possible to accurately determine the presence or absence of defects on the metal machined surface 9. The set image includes a third image taken so that the irradiation direction of the illumination 3 or the position of the camera 2 is different from that of the first image, and further includes a fourth image. It goes without saying that the above images may be included.

Further, the inspection device for the metal processed surface 9 of the present embodiment is "a first image and a second image taken so that the irradiation direction of the illumination 3 or the position of the camera 2 is different from the first image. As a set image including,, it is equipped with a calculation unit that classifies using a classifier that has been learned in advance from "information obtained from the metalworked surface to be inspected" and "other information". ing. Further, it is provided with a determination unit for determining whether or not the metal machined surface 9 to be inspected contains a defect based on the result derived from this calculation unit. The arithmetic unit has acquired information obtained from a large number of set images acquired for a plurality of reference metal processed surfaces having defects to be referred to, and acquired for a plurality of reference metal processed surfaces having no defects. The classifier is trained to learn the characteristics of the defect using deep learning from a group of information containing information obtained from a large number of set images. Further, the defect is included in the metal processed surface 9 to be inspected based on the information obtained from the set image acquired for the metal processed surface 9 to be inspected by using the classifier that has learned the characteristics of this defect. Classify whether it is. Therefore, it is possible to accurately determine the presence or absence of defects on the metal machined surface 9.

Here, the embodiment will be described in detail. In this embodiment, cavities and defects generated when the surface of a die-cast or cast rough-shaped material is processed are detected by using an image. In addition, learning by deep learning is used to detect these cavities and defects. If machine learning by deep learning is used, if information such as the presence or absence of a selection target is attached, artificial intelligence will extract the feature from the learning data and specify it even if the feature of the selection target is not defined by a person. The result of can be obtained.

In the embodiment, since it is determined that the metalworked surface 9 to be inspected contains or does not contain defects, the information in the database as a learning element is the reference metalworking surface having defects and the reference metalworking without defects. Both faces are needed. Information such as whether or not defects are included in these set images stored in the database is attached.

If the images to be learned by deep learning have distinctive characteristics of OK and NG, learning will be easy, and defects can be detected with higher accuracy with a small number of images. However, the cavities and defects on the machined surface are very similar to dirt, coolant, cutting marks, etc., and the feature difference between ON and NG is very small just by taking an image taken from directly above, even for humans. Difficult to identify. Therefore, high-precision defect detection requires a huge number of images for learning, which is extremely difficult.

Therefore, when the image is acquired, the image is acquired by using the following method in which the characteristics of the discomfort are conspicuous. As shown in FIGS. 1 and 2, the first method is photography using a plurality of illuminations 3. This is done for the shadows that appear on the image obtained by irradiating the illumination 3 from a specific direction and on the image obtained by irradiating the illumination 3 from a direction different from the previous direction. This is because the difference in the shadows that appear in the image may emphasize the characteristics of the discomfort. In order to make this possible, in the embodiment, the first illumination 3 that irradiates from a specific direction and the second, third ... Illumination 3 that irradiates from a direction different from the previous direction are provided. It is an inspection device. Further, the set image acquired for the metal processed surface 9 to be inspected is irradiated with the image obtained by irradiating the first illumination 3 and the second, third ... Illumination 3. It is assumed that the image obtained by the above is included. If a plurality of images obtained by irradiating the illumination 3 from a direction different from the previous direction are included, it may be easy to make the feature portion stand out.

To do this, for example, the camera 2 installed in the direction perpendicular to the machined surface is illuminated with a coaxial illumination 3 and another illumination 3 from two or more different directions. It suffices to take a plurality of images having different appearances of shadows on the same part of the processed surface. Learning and determination by deep learning can be performed using the set image obtained by this.

This technique will be described in more detail. First, the camera 2 is installed in the direction perpendicular to the machined surface of the workpiece to be inspected, and a plurality of lights 3 are installed on the camera 2. The first lighting 3 is a ring-shaped or plate-shaped lighting 3 that is coaxial with the camera 2, or a lighting 3 placed at a very close position. The second and subsequent lighting 3s are installed at positions of about 30 to 75 degrees with respect to the axis of the camera 2 in the X and Y directions of the surface, or both. The number of lights 3 to be used may be two or three or more, but it is desirable to use a total of three lights equal to the number of color information channels of the image, and each light 3 is white light or red different for each light 3. Uses three colors of green and blue.

In the conventional shooting method, since one camera and one coaxial lighting are mainly used, the discomfort point is judged from one direction by the image. With this, it is difficult to make a difference in the unevenness of the discomfort point, and when the discomfort point as shown in FIG. 3 is photographed, as shown in FIG. 16, defects such as concave cavities and convex facets and coolant liquid are present. , It is difficult to distinguish cutting oil, etc.

Image classification by deep learning requires many images for learning, but if there is no difference between the images to be classified, a huge number of images are required to learn the features, and the automatic inspection device It's difficult to use.

On the other hand, regarding the discomfort as shown in FIG. 3, the first method is used to irradiate the illumination 3 from different directions from directly above, front, and sideways, and the first image, the second image, and the third image are used. Once the image is acquired, the result shown in FIG. 4 can be obtained. That is, it is possible to obtain a plurality of images in which the difference in the substance of the discomfort and the difference in the degree of unevenness are greatly shown.

For example, in the case of coolant, facets, and cutting oil that are convex with respect to the machined surface, the shape of the shadow, the degree of reflection on the surface, and the transparency differ depending on the material. Therefore, it is possible to obtain different images in which the characteristics are largely exhibited by the difference in the angle of the illumination 3.

On the other hand, defects such as cavities and chips in the concave direction with respect to the machined surface have a shadow shape and a degree of surface reflection different from those in the convex direction. Therefore, the characteristics of the illumination 3 are greatly different depending on the angle of the illumination 3, and different images can be obtained.

Further, the cutting marks having almost no unevenness on the machined surface show almost no difference in shadows or the like due to the difference in the angle of the illumination 3, and an image having the same pattern can be obtained with any illumination 3. Therefore, it is possible to highlight the difference between defects such as cavities and defects and other stains. Therefore, learning by deep learning becomes easy, and defects can be detected with higher accuracy with a small number of images.

The second method is shooting using a plurality of cameras 2. For example, as shown in FIG. 5, by using different cameras 2 from two or more different directions for the camera 2 and the illumination 3 installed in the direction perpendicular to the surface of the machined surface, the same part of the machined surface can be used. Take multiple images with different angles. These images are corrected, and learning and judgment by deep learning are performed using a set image having the same angle, direction, and size. Even with such a method, the difference between defects such as cavities and defects and other stains can be highlighted, which facilitates learning in deep learning and enables more accurate defects with a small number of images. Detection is possible.

This technique will be described in more detail. First, one camera 2 is installed in the direction perpendicular to the machined surface of the workpiece to be inspected. Further, a ring-shaped or plate-shaped lighting 3 that is coaxial with the camera 2 is installed, or the lighting 3 is installed at a position very close to the camera 2. The second and subsequent cameras 2 are installed at positions of about 30 to 75 degrees with respect to the axis of the first camera 2 in the X and Y directions of the surface, or both. The number of cameras 2 to be used may be two or three or more, but it is desirable to use a total of three cameras, which is equal to the number of color information channels of the image.

In both the first method and the second method, it is possible to use an area camera, a line camera, or the like as the type of camera 2 used for shooting. The lighting 3 uses a type corresponding to each camera 2.

Here, the image processing process will be described. As shown in FIG. 6, first, the machined surface is photographed (STEP 1). Then, as shown in FIG. 7, a set image is created by cutting out the discomfort points from the obtained plurality of images (STEP2). Then, as shown in FIG. 8, it is used for learning and determination of deep learning (STEP3). The image processing process is generally such a flow. Next, each step will be described.

First, a case where the photographed surface of step 1 is photographed using a plurality of illuminations 3 will be described. FIG. 9 shows the irradiation direction and the positional relationship of the illumination 3 when three white light illuminations are used. After installing the work, first, only the illumination 3 directly above is irradiated to photograph the processed surface. Next, only the front illumination 3 is irradiated to photograph the machined surface. Finally, only the horizontal illumination 3 is irradiated to photograph the machined surface. As a result, the images of the three processed surfaces are acquired. The order of shooting is not limited to this.

The three images taken in this way are shown in FIG. Due to the difference in the position of the illumination 3, it is possible to obtain an image in which the direction of the shadow or the like is different for each image at the same discomfort point.

It is also possible to configure the color of the illumination 3 emitted from a specific direction and the color of the illumination 3 emitted from a direction different from the previous direction to be different from each other. In this way, it is possible to identify based on the difference in color. In order to make this possible, the color of the first illumination and the color of the second illumination may be different from each other. For example, FIG. 11 shows the irradiation direction and the positional relationship of the illumination 3 when three color illuminations are used. In this figure, the three color lights of red, green, and blue are arranged directly above, in front, and sideways, respectively. After installing the work, all the lights 3 are irradiated to photograph the machined surface, and an image of one machined surface is acquired.

FIG. 12 shows one image taken by simultaneously irradiating three lightings 3 with three color lightings having different colors. By decomposing this into three RGB channels for each color, it is possible to obtain three images in which the direction of shadows differs for each channel due to the difference in the color of the illumination 3. Since these three images are taken by the same camera 2, the positional relationship, the magnification, the inclination, and the like are the same.

In step 2, defects such as cavities and chips, and misidentified elements such as cutting marks, coolant, facets, and cutting oil are collectively found as discomfort points from these images, and the found discomfort points are cut out. Here, the discomfort points are cut out for each of the three images, and the discomfort points found in any one image are cut out at the same position for all three images to generate a discomfort point cutout image of a set of three images. It should be noted that the description of the method of finding and cutting out the discomfort is omitted.

After that, as shown in FIG. 13, a set image can be generated by selecting one of the following two methods from the acquired three cut-out images. The lower left side of FIG. 13 is the first method, in which three cut-out images are assigned to RGB channels to form one image. By doing so, it is possible to obtain a set image by superimposing the position information of each of the plurality of images together using a calculation unit or the like. Further, what is shown on the lower right side of FIG. 13 is the second method, in which a plurality of cut-out images are arranged so as not to overlap each other to form a single set image. Here, three cut-out images are arranged in a row so that they do not overlap each other to form one set image.

In step 3, learning and classification by deep learning are performed on the generated set image. In the set image of the discomfort point used, the difference between the cavities and defects and the misidentified element is more remarkable than the image obtained by taking the discomfort point from directly above. Therefore, the number of images required for learning to classify them in deep learning is reduced, and learning becomes easy.

Next, a case where the processed surface of step 1 is photographed using a plurality of cameras 2 will be described. The reason for doing this is that the difference between the image acquired by the camera 2 that captures the inspection target from a specific direction and the image acquired by the camera 2 that captures the inspection target from a direction different from the previous direction is a difference. This is because it may make the feature stand out. In order to make this possible, the first camera 2 that shoots the inspection target from a specific direction and the second, third ... Camera 2 that shoots the inspection target from a direction different from the previous direction. It is an inspection device equipped with. Further, the set image acquired for the metal processed surface 9 to be inspected includes an image acquired by the first camera 2 and an image acquired by the second, third ... Camera 2. It is supposed to be. If a plurality of images acquired by the camera 2 that captures the inspection target from a direction different from the previous direction are included, it may be easy to make the feature portion stand out.

FIG. 14 shows the relationship between the shooting direction of the cameras 2 and the installation position when three cameras 2 are used. After installing the work, illuminate the light 3 directly above, and the total of the first camera 2 installed directly above, the second camera 2 installed in front, and the third camera 2 installed next to it. The machined surface is photographed at the same time with three cameras. As a result, the images of the three processed surfaces are acquired.

FIG. 15 shows three images taken at the same time using three cameras 2 installed at different positions. Images with different shadow orientations, magnifications, and tilts can be obtained at the same discomfort point depending on the position and angle of the camera 2. Next, these images are corrected for enlargement ratio, inclination, etc. by image correction to obtain an image having the same positional relationship, enlargement ratio, and inclination as the image taken from directly above. This is done by a known calibration method, such as a method using marker dots. As a result, it is possible to obtain three images having the same positional relationship, magnification, and inclination, but different orientations such as shadows depending on the shooting direction. Since step 2 and subsequent steps are the same as those described above, the description thereof will be omitted.

Although the above description has focused on some embodiments, the present invention is not limited to the above embodiments and can be various embodiments. For example, in the embodiment, fixed lighting and a camera are used, but the lighting and the camera may be movable.

Alternatively, the camera may be fixed and the metalworked surface to be inspected may be moved to acquire different images.

In addition, the information attached to the image stored in the database may be simply "defective" or "no defect" so as to be directly linked to the determination result, but it is a casting cavity that is a factor for determining the presence or absence of a defect. It is also possible to attach information such as whether or not there is a coolant, or whether or not there is a coolant.

Further, the set image may be created by using two or more images. That is, the image to be used as a base when creating a set image may be two, three, or four or more.

2 Camera 3 Lighting 9 Metalworked surface

Claims (10)

A set image including the first image and the second image taken so that the irradiation direction of the illumination or the position of the camera is different from that of the first image is the metalworked surface to be inspected. Get against
Information obtained from a large number of set images acquired for multiple reference metalworked surfaces with defects to be referenced, and a large number of set images acquired for multiple reference metalworked surfaces without defects. Using a classifier that learned the characteristics of defects using deep learning from a group of information containing information obtained from
A method for inspecting a metalworked surface to be inspected, based on information obtained from a set image acquired for the metalworked surface to be inspected, to determine whether or not the metalworked surface to be inspected contains defects. The set image acquired for the metalworked surface to be inspected is obtained by irradiating the image obtained by irradiating the illumination from a specific direction and the illumination from a direction different from the previous direction. The method for inspecting a metal processed surface according to claim 1, which includes an image. The method for inspecting a metal processed surface according to claim 2, wherein the color of the illumination emitted from a specific direction and the color of the illumination emitted from a direction different from the previous direction are different from each other. The set image acquired for the metal processed surface to be inspected is described in claim 3 obtained by distorting a plurality of images, correcting the magnification, and then superimposing the images in alignment or arranging them in parallel. How to inspect metal processed surfaces. The set image acquired for the metal machined surface to be inspected includes the image acquired by the camera that captures the inspection target from a specific direction and the image acquired by the camera that captures the inspection target from a direction different from the previous direction. The method for inspecting a metal processed surface according to claim 1, which includes the image. A metal to be inspected, which is generated as a set image including the first image and the second image taken so that the irradiation direction of the illumination or the position of the camera is different from the first image. An arithmetic unit that classifies information obtained from what is acquired about the machined surface using a classifier that has been learned in advance from other information.
A determination unit for determining whether or not the metal machined surface to be inspected contains a defect based on the result derived from the calculation unit is provided.
The calculation unit includes information obtained from a set image acquired for a large number of reference metal processed surfaces having defects to be referred to, and a set acquired for a large number of reference metal processed surfaces having no defects. Using a classifier that learned the characteristics of defects using deep learning from a group of information including information obtained from images,
An inspection device for metalworked surfaces that classifies information obtained from set images acquired for metalworked surfaces to be inspected. It is equipped with a first light that illuminates from a specific direction and a second light that irradiates from a direction different from the previous direction.
A claim that the set image acquired for the metalworked surface to be inspected includes an image obtained by irradiating the first illumination and an image obtained by irradiating the second illumination. 6. The device for inspecting a metal processed surface according to 6. The device for inspecting a metal processed surface according to claim 7, wherein the color of the first illumination and the color of the second illumination are different from each other. The set image acquired for the metal processed surface to be inspected is described in claim 8 obtained by distorting a plurality of images, correcting the magnification, and then superimposing the images in alignment or arranging them in parallel. Inspection equipment for metal processed surfaces. It is equipped with a first camera that shoots the inspection target from a specific direction and a second camera that shoots the inspection target from a direction different from the previous direction.
The device for inspecting a metal processed surface according to claim 6, wherein the set image acquired for the metal processed surface to be inspected includes an image acquired by the first camera and an image acquired by the second camera. ..

Images