JP2024092565A - Visual Inspection System - Google Patents

Abstract

The present invention provides a method for easily inspecting the appearance of a product based on an image of the product captured by a camera, the method enabling inspection in a plurality of directions to be easily realized.
[Solution] An appearance inspection system according to one embodiment of the present invention comprises a mounting unit 20 on which a product 10 to be subjected to appearance inspection is placed, a camera 30 which images the product 10 placed on the mounting unit 20 from a specific direction, and an appearance inspection device 40 which performs appearance inspection processing of the product 10 based on the image taken by the camera 30. The mounting unit 20 has a mirror 23 arranged to reflect an image of the placed product 10 seen from a direction other than the specific direction towards the camera 30, and the camera 30 captures an image including an image portion which shows the product 10 seen from the specific direction and an image portion which shows the product 10 seen from the other direction via the mirror 23.
[Selected Figure] Figure 1

Description

The present invention relates to an appearance inspection system that inspects the appearance of a product based on an image of the product captured by a camera.

Conventionally, visual inspection of products is performed to check whether manufactured products have defects such as scratches or distortions. Visual inspection of products is achieved, for example, by placing the product to be inspected on a dedicated stand, photographing it with a camera, and analyzing the image. Conventional stands generally only have the function of fixing the product in place. Furthermore, the camera for photographing the product is often fixed so that it photographs the product on the stand from a specific position and direction.

When inspecting the appearance of a product using the method described above, in order to inspect the product's appearance from multiple directions, it was necessary to change the orientation of the product placed on the stand and take photos multiple times, which took a long time for the inspection. Also, while it was possible to increase the number of cameras to take photos from multiple directions, this would increase equipment costs and require the work of installing each camera in an appropriate position, which was tedious. There was also a concern that the process would become complicated, as each image of the product taken from multiple directions had to be analyzed separately and the results of these analyses had to be combined to obtain the final inspection results.

The following are examples of prior art in the technical field related to this invention. For example, Patent Document 1 discloses an invention in which, in an image analysis system that uses a camera to photograph and inspect objects moving along a production line in a food factory or the like, an operator can arbitrarily set items to be inspected on a GUI analysis screen, and the system automatically reads and compares information about the set items and records the results together with an image.

Patent Publication 2013-205182

The present invention was made in consideration of the above-mentioned conventional circumstances, and aims to easily perform inspection of the appearance of a product in multiple directions based on an image of the product captured by a camera.

In order to achieve the above object, a visual inspection system according to one aspect of the present invention is configured as follows.
In other words, the visual inspection system of the present invention comprises a mounting unit on which a product to be visually inspected is placed, a camera that images the product placed on the mounting unit from a first direction, and a visual inspection device that performs visual inspection processing of the product based on the image captured by the camera, wherein the mounting unit has a mirror positioned to reflect an image of the placed product viewed from a second direction different from the first direction towards the camera, and the camera captures an image including a first image portion that reflects the product viewed from the first direction and a second image portion that reflects the product viewed from the second direction via the mirror.

Here, the appearance inspection system according to the present invention further includes a first detection unit that detects that a product has been placed on the placement unit, and a second detection unit that detects that an operating body that performs the product placement operation on the placement unit has moved out of the camera's shooting range, and the appearance inspection device can be configured to execute an appearance inspection process based on an image captured by the camera when the first detection unit detects that a product has been placed on the placement unit and when the second detection unit detects that the operating body has moved out of the camera's shooting range.

In addition, in the visual inspection system according to the present invention, an inspection frame for each of the areas of the first image portion and the second image portion is determined in advance, and the visual inspection device can be configured to perform visual inspection processing for the inspection frame of the first image portion and visual inspection processing for the inspection frame of the second image portion, respectively, for an image captured by the camera.

In addition, in the visual inspection system according to the present invention, the visual inspection device can be configured to determine the position of the product in the placement unit, adjust the image or inspection frame according to the product position, and then perform the visual inspection process.

In addition, in the visual inspection system according to the present invention, the visual inspection device can be configured to determine the orientation of the product in the mounting unit, adjust the image or inspection frame according to the orientation of the product, and then perform the visual inspection process.

The present invention makes it possible to easily inspect the appearance of a product in multiple directions based on images of the product captured by a camera.

1 is a diagram showing an example of the configuration of a visual inspection system according to an embodiment of the present invention; 2 is a plan view of an exemplary mounting unit used in the visual inspection system of FIG. 1; 3 is a vertical cross-sectional view taken along line A-A' of the exemplary mounting unit shown in FIG. 2. FIG. 2 is a diagram showing an example of an image captured by a camera in the appearance inspection system of FIG. 1 . 2 is a diagram showing an example of an inspection frame indicating a target range of the appearance inspection process by the appearance inspection system of FIG. 1 ; 2 is a diagram showing an example of a screen display of a result of an appearance inspection process by the appearance inspection system of FIG. 1 . 2 is a diagram showing an example of a screen display of a result of an appearance inspection process by the appearance inspection system of FIG. 1 . FIG. 2 is a diagram showing an example of a flowchart of an appearance inspection process performed by the appearance inspection system of FIG. 1 . FIG. 11 is a vertical sectional view of a modified example of the mounting unit.

An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows an example of the configuration of an appearance inspection system according to an embodiment of the present invention. The appearance inspection system shown in FIG. 1 includes a mounting unit 20 on which a product 10 to be inspected is placed, a camera 30 that captures an image of the product 10 placed on the mounting unit 20 from a specific direction (in this example, from the top surface), an appearance inspection device 40 that performs an appearance inspection process on the product 10 based on the image captured by the camera 30, and a server 45 that stores data on the results of the appearance inspection process performed by the appearance inspection device 40, etc.

In the appearance inspection system of this example, the mounting unit 20 has a mirror 23 arranged to reflect an image of the mounted product 10 seen from a direction other than the specific direction toward the camera 30. The camera 30 is also set to capture an image including an image portion showing the product 10 seen from the specific direction (the direction toward the camera) and an image portion showing the product 10 seen from another direction via the mirror 23. In other words, the camera 30 can capture an image including multiple image portions of the product 10 seen from multiple directions in a single capture. Therefore, the appearance inspection device 40 can perform not only appearance inspection processing of the product 10 seen from a specific direction, but also appearance inspection processing of the product 10 seen from another direction based on a single image captured by the camera 30.

Figure 2 shows a plan view of an exemplary mounting unit 20 used in the visual inspection system of Figure 1. Also, Figure 3 shows a vertical cross-sectional view of the exemplary mounting unit 20 shown in Figure 2 along line A-A'. The mounting unit 20 of this example comprises a base portion 21, a mounting stand 22, four mirrors 23A-23D, four unwanted light prevention plates 24A-24D, and two sensor portions 25A and 25B.

The stand 22 is a member that serves as a setting position for the product 10, and is provided in the center of the upper surface of the base portion 21. As an example, the stand 22 has an upper surface with a recess formed therein that corresponds to the shape of the lower portion of the product 10. By using such a stand 22, the position and orientation of the product 10 on the stand 22 can be fixed simply by placing the lower portion of the product 10 so that it fits into the recess on the upper surface of the stand 22, making it possible to simplify subsequent processing.

Mirrors 23A-23D are arranged around the four sides of the stand 22, with an inclination of about 45 degrees from the horizontal. In addition, unwanted light prevention plates 24A-24D are arranged on the back of each of the mirrors 23A-23D to prevent background reflections and light from entering. In this example, unwanted light prevention plates 24A-24D that are slightly larger than the mirrors 23A-23D are used, but there are no particular limitations on size, shape, material, etc. as long as they can adequately prevent background reflections and light from entering.

The sensor unit 25A is an example of a first detection unit according to the present invention, and detects that the product 10 has been placed on the placement unit 20. For example, a pressure sensor built into the surface of the placement table 22 is used as the sensor unit 25A. The sensor unit 25B is an example of a second detection unit according to the present invention, and detects that the operating object (for example, the hand of a worker or the arm of a machine) performing the placement operation of the product 10 on the placement unit 20 has moved out of the shooting range of the camera 30. For example, the sensor unit 25B is a set of an infrared emitting unit that emits infrared rays horizontally into the space above the product 10 on the placement table 22, which is provided diagonally on the upper surface of the base unit 21, and an infrared sensor that detects the infrared emitting unit.

The camera 30 is positioned above the mounting unit 20 so that the mounting table 22 and the surrounding mirrors 23A to 23D are within the shooting range (angle of view). Therefore, as shown in FIG. 4, the image 50 captured by the camera 30 includes not only an image portion 51 showing the top surface of the product 10 placed on the mounting table 22, but also an image portion 52 showing the image of the front of the product 10 reflected by the mirror 23A, an image portion 53 showing the image of the left surface of the product 10 reflected by the mirror 23B, an image portion 54 showing the image of the right surface of the product 10 reflected by the mirror 23C, and an image portion 55 showing the image of the back of the product 10 reflected by the mirror 23D. In this way, by using the mounting unit 20, it is possible to obtain an image 50 showing each surface of the product 10 except for the bottom surface (i.e., the top surface, front surface, left surface, right surface, and back surface) with just one shooting.

The image 50 captured by the camera 30 is supplied to the appearance inspection device 40. The appearance inspection device 40 executes an appearance inspection process based on the image 50 supplied from the camera 30. In this example, there are one or more inspection items for each of the top, front, left, right, and back surfaces of the product 10, and an appearance inspection process is executed for each inspection item.

Figure 5 shows an example of an inspection frame indicating the target range for each inspection item. In the example of Figure 5, inspection frames 61 and 62 for the area of the image portion showing the top surface of product 10, inspection frame 63 for the area of the image portion showing the front surface of product 10, inspection frames 64 and 65 for the area of the image portion showing the left surface of product 10, inspection frame 66 for the area of the image portion showing the right surface of product 10, and inspection frame 67 for the area of the image portion showing the back surface of product 10 are shown. It is assumed that these inspection frames 61 to 67 are set in advance before starting operation of the visual inspection system.

The visual inspection device 40 also receives the detection results from the sensors 25A and 25B. The visual inspection device 40 controls the operation of the visual inspection process based on the detection results from the sensors 25A and 25B. In this example, the visual inspection device 40 controls to execute the visual inspection process based on the image captured by the camera 30 in state X where the sensor 25A detects that the product 10 has been placed on the placement unit 20, and state Y where the sensor 25B detects that the operating body has moved out of the shooting range of the camera 30. By performing such control, not only can the visual inspection process be automatically executed on the image captured of the product 10 placed on the placement unit 20, but also it can prevent the operating body, such as the hand of a worker or the arm of a machine, from being captured in the image.

Here, the image to be subjected to the visual inspection process (i.e., the image taken in state X and state Y) can be obtained in various ways. As one example, the camera 30 may repeat taking images at a predetermined time interval and transmit the images to the visual inspection device 40, and the visual inspection device 40 may identify the image taken at the timing of state X and state Y and use that image as the subject of the visual inspection process. As another example, the camera 30 may be basically in a standby state for taking images, and the visual inspection device 40 may instruct the camera 30 to take an image at the timing of state X and state Y, and the camera 30 may transmit the image taken by the camera 30 in response to this instruction to the visual inspection device 40 as the subject of the visual inspection process.

The appearance inspection device 40 inspects the appearance of the product 10 by performing AI analysis of each of the areas of the inspection frames 61 to 67 in the image captured by the camera 30 using a learning model that has been trained in advance. If the inspection results for all the inspection frames are OK (no defects), the overall inspection result is determined to be OK. On the other hand, if the inspection result for any of the inspection frames is NG (defects present), the overall inspection result is determined to be NG. Note that the appearance inspection device 40 may perform the appearance inspection process using a method other than AI analysis. For example, the appearance inspection device 40 may compare each of the areas of the inspection frames 61 to 67 in the image captured by the camera 30 with a reference image that has been prepared in advance by capturing a product without defects, calculate the difference, and determine that the difference is OK if it is less than a predetermined threshold, and determine that the difference is NG if it is not.

In the above description, the conditions X and Y are detected using sensors 25A and 25B, but this is merely one example, and conditions X and Y may be detected by other methods. For example, the camera 30 may repeatedly take images at predetermined time intervals and transmit the images to the appearance inspection device 40, and the appearance inspection device 40 may detect conditions X and Y by analyzing each image at the predetermined time intervals. In other words, the appearance inspection device 40 may be configured to be provided with the first detection unit and the second detection unit according to the present invention.

When the appearance inspection device 40 executes the appearance inspection process as described above, it displays the results of the appearance inspection process on a screen. The screen display of the results of the appearance inspection process may be displayed on a display possessed by the appearance inspection device 40, or may be displayed on a display of another device capable of receiving the results of the appearance inspection process from the appearance inspection device 40.

FIGS. 6 and 7 show examples of screen displays of the results of the visual inspection process. FIG. 6 is an example of a screen display when all inspection items are OK (no defects). FIG. 7 is an example of a screen display when any inspection item is NG (defects present). The display screen 70 of the results of the visual inspection process has a first display area 71 that displays the image captured by the camera 30, a second display area 72 that displays the overall inspection results, and a third display area 73 that displays the inspection results for each inspection item (inspection frames 61 to 67 in FIG. 5) in list format.

In the first display area 71, not only are images captured by the camera 30 displayed, but also figures that visualize the inspection results for each inspection item are overlaid. In this example, white frame displays 81-87 are superimposed on the range of inspection items that have an OK inspection result, as shown by the dashed line in FIG. 6. Red frame displays 81', 83' are superimposed on the range of inspection items that have an NG inspection result, as shown by the thick solid line in FIG. 7. Therefore, by simply looking at the display in the first display area 71, the inspector can determine which part of the product 10 has a defect in the inspection item.

The appearance inspection device 40 not only displays the results of the appearance inspection process on the screen, but also transmits the data to the server 45 for storage. The server 45 stores, for example, data on the inspection results (overall inspection results and inspection results for each inspection item) associated with a product ID that identifies the product 10 that has been appearance inspected, images captured by the camera 30, and images in which the inspection results for each inspection item are visualized and superimposed on the images. Therefore, by accessing the server 45, it is possible to utilize the results of the appearance inspection process after the fact.

The flow of the appearance inspection process by the appearance inspection system of this example will be described with reference to the example flowchart shown in FIG. 8. Here, the product 10 to be inspected has a barcode (including a two-dimensional barcode) indicating the product ID attached thereto, and the product ID is detected by reading the barcode with a barcode reader before (or after) the product 10 is placed on the placement unit 20, and stored in the server 45. Note that instead of reading with a barcode reader, the appearance inspection device 40 may analyze the barcode portion of an image of the product 10 to detect the product ID. Also, the product ID itself may be attached to the product 10 instead of the product ID barcode, and the product ID may be detected by OCR (Optical Character Reader) processing of the image of the product 10.

When the product 10 to be inspected is placed on the mounting unit 20, this is detected by the sensor unit 25A, and the detection result of the sensor unit 25A is input to the visual inspection device 40 (step S11). In addition, when the object that placed the product 10 moves out of the shooting range of the camera 30, this is detected by the sensor unit 25B, and the detection result of the sensor unit 25B is input to the visual inspection device 40 (step S12).

The camera 30 repeats photographing at predetermined time intervals, and the images obtained by photographing are sequentially supplied to the appearance inspection device 40. The appearance inspection device 40 judges whether the product 10 is in state X, where the sensor 25A detects that the product 10 has been placed on the mounting unit 20, and whether the sensor 25B detects that the operating body has moved out of the photographing range of the camera 30, and performs an appearance inspection process for the product 10 using the image when the judgment was successful (step S13). The appearance inspection device 40 then displays the results of the appearance inspection process on a screen (step S14). The appearance inspection device 40 also transmits the result data of the appearance inspection process and image data to the server 45, where they are stored in association with the product ID (step S15).

As described above, the appearance inspection system of this example includes a mounting unit 20 on which the product 10 to be inspected is placed, a camera 30 that images the product 10 placed on the mounting unit 20 from a specific direction, and an appearance inspection device 40 that performs an appearance inspection process on the product 10 based on the image captured by the camera 30. The mounting unit 20 has a mirror 23 arranged to reflect an image of the placed product 10 viewed from a direction other than the specific direction toward the camera 30, and the camera 30 is configured to capture an image that includes an image portion that shows the product 10 viewed from the specific direction and an image portion that shows the product 10 viewed from the other direction via the mirror 23.

Therefore, with the appearance inspection system of this example, it is possible to capture an image of the product to be visually inspected just once, including multiple image portions of the product viewed from multiple directions. Furthermore, with just one captured image, it is possible to perform visual inspection processing of the product for multiple directions. In this way, with the appearance inspection system of this example, it is possible to easily perform visual inspection of the product for multiple directions.

The above description assumes that the position and orientation of the product 10 on the platform 22 of the mounting unit 20 is fixed by forming a recess in the upper surface of the platform 22, but it is also possible to use a platform 22 that has a flat upper surface without such a recess, so that the position and orientation of the product 10 can be freely changed. That is, the product 10 may be positioned slightly shifted forward/backward or left/right, the product 10 may be positioned slightly tilted relative to a reference direction, or the front of the product 10 may be positioned facing upward.

However, when the position and orientation of the product 10 are allowed to be changed in this way, it is necessary to adjust the image captured by the camera 30 or the inspection frame (inspection frames 61 to 67 shown in FIG. 5) according to the position and orientation of the product 10 placed on the placement table 22. In this case, for example, the contour of the product 10 is detected in each direction when the product 10 is placed on the placement table 22 in a reference position and orientation, and the inspection frame is set in correspondence with the reference contour in each direction. Then, the appearance inspection device 40 detects the contour of the product 10 from the image portion in each direction included in the captured image of the product 10 to be inspected, and adjusts the image portion in each direction by moving, rotating, transforming, etc. so that the contour in each direction matches the reference contour, and then performs the appearance inspection process. Alternatively, the reference contour is adjusted by moving, rotating, transforming, etc. so that the reference contour matches the contour in each direction, and the same adjustment is made to the inspection frame, and then the appearance inspection process is performed. Note that the above-mentioned adjustment method is merely an example, and adjustment may be performed by other methods.

In the above explanation, images of each surface of the product 10 except for the bottom surface (i.e., the top, front, left, right, and back) are captured and the appearance inspection process is performed, but if there are directions that do not require inspection, capturing images of those directions may be omitted. In this case, of the mirrors 23A to 23D, the mirrors in the directions that do not require inspection are unnecessary and may be removed.

In addition, in the above description, it is not possible to perform a visual inspection of the underside of the product 10, but by modifying or expanding the mounting unit 20, it becomes possible to perform a visual inspection of the underside of the product 10 all at once. The mechanism behind this will be described with reference to Figure 9. Figure 9 shows a vertical cross-sectional view of a modified mounting unit 20.

The mounting unit 20 in FIG. 9 has an opening 26 that penetrates the center of the base 21 and the stand 22 in the vertical direction so that the underside of the product 10 can be photographed. In addition, an extended base 27 equipped with two mirrors 28A and 28B is attached to the underside of the base 21. Mirror 28A is positioned directly below the opening 26, with an inclination of about 45 degrees from the horizontal plane. Mirror 28B is positioned with an inclination of about 45 degrees in the opposite direction to mirror 28A with respect to the horizontal plane so that the image of the underside of the product 10 reflected on mirror 28A through the opening 26 is reflected toward the camera 30. By using such an extended base 27, it is possible to obtain an image that shows all sides of the product 10, including the underside, with just one photograph.

Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

The present invention can be provided not only as the devices described above or as systems composed of these devices, but also as methods executed by these devices, programs for implementing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable format.

The present invention can be used in an appearance inspection system that inspects the appearance of a product based on images of the product taken by a camera.

10: Product, 20: Mounting unit, 21: Base section, 22: Mounting stand, 23A to 23D: Mirror, 24A to 24D: Unwanted light prevention plate, 25A, 25B: Sensor section, 26: Opening, 27: Extension base section, 28A, 28B: Mirror, 30: Camera, 40: Appearance inspection device, 45: Server

Claims (5)

a placement unit on which a product to be inspected is placed;
a camera that captures an image of the product placed on the placement unit from a first direction;
an appearance inspection device that performs an appearance inspection process for the product based on the image captured by the camera;
the mounting unit has a mirror arranged to reflect an image of the mounted product seen from a second direction different from the first direction toward the camera;
An appearance inspection system characterized in that the camera captures an image including a first image portion showing the product viewed from the first direction and a second image portion showing the product viewed from the second direction via the mirror. 2. The visual inspection system according to claim 1,
a first detection unit that detects that the product has been placed on the placement unit;
a second detection unit that detects that an operating body that performs an operation of placing the product on the placement unit has moved out of a shooting range of the camera,
The appearance inspection system is characterized in that the appearance inspection device performs the appearance inspection process based on an image captured by the camera when the first detection unit detects that the product has been placed on the placement unit and when the second detection unit detects that the operating body has moved out of the camera's shooting range. 3. The visual inspection system according to claim 1,
An inspection frame for each of the first image portion and the second image portion is previously determined;
An appearance inspection system characterized in that the appearance inspection device performs the appearance inspection processing on the inspection frame of the first image portion and the appearance inspection processing on the inspection frame of the second image portion for the image captured by the camera. 4. The visual inspection system according to claim 3,
An appearance inspection system characterized in that the appearance inspection device determines the position of the product in the placement unit, adjusts the image or the inspection frame according to the position of the product, and then performs the appearance inspection process. 4. The visual inspection system according to claim 3,
An appearance inspection system characterized in that the appearance inspection device determines the orientation of the product in the placement unit, adjusts the image or the inspection frame according to the orientation of the product, and then performs the appearance inspection process.

Images