KR20220059091A - System and method for inspecting product appearance using mobile collaborative robot, and a recorcing medium having computer readable program for executing the method - Google Patents

Abstract

Disclosed are a mobile cooperative robot-based article appearance inspection system, a method, and a recording medium recording a computer-readable program for executing the method. The article appearance inspection system includes a loading unit, a communication unit, a cooperative robot unit, a mobile robot unit, an image acquisition unit, and an appearance state determination unit. The goods are loaded in the loading unit, the communication unit receives a transfer request of the goods, the cooperative robot unit moves the goods to the loading unit, the mobile robot unit moves the loading unit and the cooperative robot unit according to the transfer request, and the image acquisition unit moves the loading unit and the cooperative robot unit to the loading unit An image of the loaded article is acquired, and the appearance state determination unit determines the appearance state of the article from the acquired image. According to such a configuration, it is possible to automatically inspect the appearance of the article during the transport process of the article using the robot, thereby preventing continuous input of manpower or errors caused by the operator, and reducing the equipment and time required for the inspection be able to do

Description

SYSTEM AND METHOD FOR INSPECTING PRODUCT APPEARANCE USING MOBILE COLLABORATIVE ROBOT, AND A RECORCING MEDIUM HAVING COMPUTER READABLE PROGRAM FOR EXECUTING THE METHOD}

The present invention relates to an inspection system, and more particularly, to a system and method for inspecting the appearance of an article using a robot.

In general, in the machining industry, the external state (scratches, contamination, etc.) of the material before processing or the workpiece after processing is inspected. In small businesses, there are many cases where visual inspection is mainly performed, and there are cases where expensive vision-based inspection machines are introduced and inspected.

However, in the case of visual inspection, human error may occur, continuous input of human resources is required, and if the quantity increases, it is difficult to complete the inspection. In addition, in the case of vision-based inspection, it is necessary to build a logistics transfer line from processing equipment to inspection equipment, and there is a problem in that production time is delayed by the time required for inspection.

US 1020130035826 A

The present invention has been devised to solve the above-mentioned problems of the prior art, and it is possible to prevent continuous input of manpower or errors by workers, and to provide a system and method for inspecting the appearance of an article, which can reduce equipment and time required for inspection. intended to provide

In order to achieve the above object, an article appearance inspection system according to the present invention includes a loading unit, a communication unit, a cooperative robot unit, a mobile robot unit, an image acquisition unit, and an appearance state determination unit. The goods are loaded in the loading unit, the communication unit receives a transfer request of the goods, the cooperative robot unit moves the goods to the loading unit, the mobile robot unit moves the loading unit and the cooperative robot unit according to the transfer request, and the image acquisition unit moves the loading unit and the cooperative robot unit to the loading unit An image of the loaded article is acquired, and the appearance state determination unit determines the appearance state of the article from the acquired image.

According to this configuration, it is possible to automatically inspect the appearance of the goods during the transport process of the goods using the robot, it is possible to prevent continuous input of manpower or errors due to the operator, and to reduce the equipment and time required for the inspection be able to do

In this case, the image acquisition unit may be mounted on the distal end of the cooperative robot unit, and the cooperative robot unit may include an image acquisition position control unit for controlling the image acquisition position of the image acquisition unit. According to such a configuration, it is possible to easily acquire images of various positions of the article through position control of the end of the cooperative robot.

In addition, the cooperative robot unit may further include a loading direction changing unit for changing the loading direction of the article. According to this configuration, it is possible to easily acquire an image of the position of the article, which cannot be obtained only by controlling the position of the end of the cooperative robot.

In addition, when it is determined by the external state determination unit that the external state of the article is good, the mobile robot unit can move the loading part to a preset position of the post-processing facility, and when the external state of the article is judged to be defective, the preset defective article The load can be moved to the retrieval position. According to such a configuration, it is possible to immediately change the transfer destination of the article according to the state of the article determined during the transfer of the article.

Also, the image acquisition unit may acquire an image of the product after the product is loaded and the mobile robot unit operates. According to this configuration, it is possible to further reduce the time required for the inspection by performing the external inspection of the article while the moving part is moving.

In addition, the appearance state determination unit may further determine the effect of lighting on the acquired image, and the image acquisition position control unit may control the image acquisition position of the image acquisition unit according to the determination of the lighting effect. According to this configuration, by changing the image acquisition position using the cooperative robot unit, it is possible to change the lighting effect irradiated to the article.

In addition, the mobile robot unit may change the position of the loading unit according to the determination of the lighting effect. According to such a configuration, it is possible to easily change the effect of lighting by using the mobile robot unit even for lighting that cannot be coped with only by controlling the posture of the cooperative robot.

In addition, it may further include a lighting state determination unit for determining the lighting state around the loading unit, the mobile robot unit may move the loading unit according to the determination of the lighting state determination unit. According to such a configuration, it is possible to use a lighting effect suitable for an appearance inspection by further using not only the lighting state of the current location but also the lighting state of a nearby area.

In addition, the invention implementing the system in the form of a method and a recording medium recording a computer readable program for executing the method are disclosed together.

According to the present invention, it is possible to automatically inspect the appearance of the article during the transport process of the article using the robot, thereby preventing continuous input of manpower or errors caused by the operator, and reducing the equipment and time required for inspection. be able to

In addition, it is possible to easily acquire images of various positions of the article by controlling the position of the end of the cooperative robot unit.

In addition, it is possible to easily acquire an image of the position of the article, which cannot be obtained only by controlling the position of the end of the cooperative robot unit.

In addition, it is possible to immediately change the transfer destination of the article according to the state of the article determined during the transfer of the article.

In addition, by performing the external inspection of the article while the moving part is moving, it is possible to further reduce the time required for the inspection.

In addition, by changing the image acquisition position using the cooperative robot unit, it is possible to change the lighting effect irradiated to the article.

In addition, it is possible to easily change the effect of lighting by using the mobile robot unit even for lighting that cannot be coped with only by controlling the posture of the cooperative robot.

In addition, it is possible to use a lighting effect suitable for an exterior inspection by further using the lighting condition of the current location as well as the lighting condition of a nearby area.

1 is a schematic block diagram of an article appearance inspection system according to an embodiment of the present invention.
FIG. 2 is a diagram schematically illustrating an embodiment of the article appearance inspection system of FIG. 1 .
3 is a diagram in which numbers are displayed on each side of a six-sided polyhedral workpiece to explain an example of inspection of the six-sided polyhedral workpiece.
4 is a view showing the movement motion of the camera in the 150 ° ~ 180 ° section of the hexahedral target.
5 is a view showing a camera viewpoint (a surface formed by a camera viewpoint through robot motion) and an angle thereof;
6 is a schematic flowchart of an article appearance inspection method according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a schematic block diagram of an article appearance inspection system according to an embodiment of the present invention, and FIG. 2 is a diagram schematically illustrating an embodiment of the article appearance inspection system of FIG. 1 . In FIG. 1 , the product appearance inspection system includes a loading unit 110 , a communication unit 120 , a cooperative robot unit 130 , a mobile robot unit 140 , an image acquisition unit 150 , an appearance state determination unit 160 , and a lighting state determination unit 170 , and the cooperative robot unit 130 again includes an image acquisition position control unit 132 , and a loading direction change unit 134 .

An article is loaded in the loading unit 110 , and the communication unit 120 receives a transfer request of the article from the outside. In FIG. 2, it can be seen that the loading part is implemented as a top plate of the mobile base. More specifically, the mobile cooperative robot is a type of robot equipped with an AMR (Autonomous Mobile Robot) as a base and a cooperative robot on the upper part.

The mobile robot unit 140 moves the loading unit 110 and the cooperative robot unit 130 according to the transfer request transmitted through the communication unit. In FIG. 2 , the cooperative robot unit 130 and the mobile robot unit 140 are implemented in the form of a cooperative robot mounted on a mobile base, and the image acquisition unit 150 is mounted at the distal end of the cooperative robot unit 130 . can check that

The cooperative robot unit 130 moves the goods to the loading unit 110 , and the image acquisition unit 150 acquires an image of the goods loaded in the loading unit 110 . In this case, the image acquisition unit 150 may acquire an image of the product after the product is loaded and the mobile robot unit 140 operates. According to this configuration, it is possible to further reduce the time required for the inspection by performing the external inspection of the article while the moving part is moving.

The image acquisition position control unit 132 controls the image acquisition position of the image acquisition unit 150 . According to such a configuration, it is possible to easily acquire images of various positions of the article by controlling the position of the end of the cooperative robot unit.

The loading direction changing unit 134 changes the loading direction of the article. According to this configuration, it is possible to easily acquire an image of the position of the article, which cannot be obtained only by controlling the position of the end of the cooperative robot unit. In addition, since a large amount of images can be generated in a minimum amount of time while changing the external light intensity and shooting angle, image data for AI and the like can be effectively collected.

3 is a diagram showing numbers on each side of a hexahedral workpiece to explain an example of a six-sided polyhedral workpiece inspection. As shown in FIG. 4 , the image acquisition unit 150 may acquire an image through the following process.

1) If the workpiece is a 6-sided shape, take 1 shot while moving 1, 2, and 3 sides by 150~180 degrees

2) Pick it up with a gripper and turn it over so that the 4, 5, 6 sides are on top.

3) One shot while moving the 4, 5, 6 sides of the workpiece by 150~180 degrees

The image acquisition time/position change may be performed as follows. 5 is a view illustrating a camera viewpoint (a surface formed by a camera viewpoint through robot motion) and an angle thereof.

- Changed shooting standard plane: Top side shooting standard, changed from 1,2,3 plane to 4,2,6 plane

- Change of camera viewpoint (refer to Figure 3): The surface formed by moving the camera viewpoint is changed from 90 degrees to oblique angles (30 to 60 degrees, 120 to 150 degrees)

- Robot position movement: Move the robot position to a point with less interference from external light (measurement of ambient light quantity)

The appearance state determination unit 160 determines the appearance state of the article from the image. In this case, the lighting effect determination unit 162 further determines the lighting effect on the image, and the image acquisition position control unit 132 may control the image acquisition position of the image acquisition unit according to the determination of the lighting influence determination unit 162 . . According to this configuration, by changing the image acquisition position using the cooperative robot unit 130, it is possible to change the lighting effect irradiated to the article.

In addition, when it is determined by the appearance state determination unit 160 that the appearance of the article is good, the mobile robot unit 140 may move the loading unit 110 to a preset post-processing facility position, and the appearance of the article When the state is determined to be defective, the loading unit 110 may be moved to a preset defective product recovery position. According to such a configuration, it is possible to immediately change the transfer destination of the article according to the state of the article determined during the transfer of the article.

In addition, the mobile robot unit 140 may change the position of the loading unit according to the determination of the lighting effect determination unit 162 . According to such a configuration, it is possible to easily change the effect of lighting by using the mobile robot unit even for lighting that cannot be coped with only by controlling the posture of the cooperative robot.

The lighting state determination unit 170 determines the lighting state around the loading unit 110 , and the mobile robot unit 140 moves the loading unit 110 according to the determination of the lighting condition determination unit 170 . This is to measure the amount of ambient light at the point where the amount of light is suitable by measuring the amount of ambient light when acquiring an image while moving. According to such a configuration, it is possible to use a lighting effect suitable for an appearance inspection by further using not only the lighting state of the current location but also the lighting state of a nearby area.

6 is a schematic flowchart of a method for inspecting the appearance of an article according to an embodiment of the present invention. 6 shows an example of a metal appearance inspection process based on a mobile cooperative robot, and a detailed description of the process is as follows.

1. Receive logistic transport request

- The robot receives a request for post-process logistics transfer from a specific facility where the current process has been completed.

2. Workpiece movement: equipment → robot

- The robot moves to the relevant facility and moves the finished workpiece to its own mobile base top plate using the collaborative robot and gripper part

3. Start of post-process logistics transfer

- Logistics transfer of the workpiece to the post-process starts after the workpiece movement is completed

- Edge device starts the metal appearance inspection by triggering the logistics transport status signal from the mobile robot controller.

4. Multifaceted Image Acquisition: Top Side

- Acquire the surface image of a multi-faceted object using the camera of the end effector between transfers

- According to the shape of the workpiece, the first image of the center of the top and side surfaces of the workpiece is acquired by moving the camera point of view by 150 to 180 degrees 1-2 times through the cooperative robot controller and pre-programmed motion.

4-1. Normal image judgment

- Judging whether the image acquired in the AI metal appearance inspection program of Edge Device is valid (whether it is an image that cannot be identified due to diffuse reflection)

- If the image is normal, proceed to the next step 5., if the image is abnormal, proceed to 4-2.

4-2. Image acquisition time/location change

- Adjust the image acquisition direction, the camera's point of view on the object, and the robot's position to reduce the effect of external lighting when acquiring an image of a workpiece

Return to -4. Rerun image acquisition

5. Collaborative Robot Operation: Bottom up

- Flip the workpiece so that the bottom side is facing up to acquire an image of the exterior surface of the bottom side

6. Multifaceted Image Acquisition: Bottom Side

- After turning over the workpiece, move the camera point of view by 150-180 degrees 1-2 times to acquire images of the uncovered area when acquiring images of the bottom and top of the workpiece

6-1. Normal image judgment

- Judging whether the image acquired in the AI metal appearance inspection program of Edge Device is valid (whether it is an image that cannot be identified due to diffuse reflection)

- If the image is normal, proceed to the next step 7., if the image is abnormal, proceed to 6-2.

6-2. Image acquisition time/location change

- Adjust the image acquisition direction, the camera's point of view on the object, and the robot's position to reduce the effect of external lighting when acquiring an image of a workpiece

Go back to -6. Rerun image acquisition

7. AI-based metal exterior image identification

- Determining whether the photographed image is acceptable or defective through the AI metal appearance inspection program of Edge Device

8. Determination of defects

- In case of defect, it is moved to and unloaded from the defective product collection point in the factory.

- Upon acceptance, the final transfer to the post-processing facility for unloading

In summary, the present invention is to solve the transport of logistics and the visual inspection of the transported products using a mobile cooperative robot as a parallel process within one system. In the prior art, when a mobile cooperative robot is used, another sequential process is performed after logistics transport or logistics transport, whereas the present invention is different in that the external inspection of the transported product is performed in parallel with the logistics transport.

In the case of appearance inspection, there is usually a separate appearance inspection facility, but in the present invention, there is a difference in that it is merged into a mobile cooperative robot, which is a logistics system, without a separate appearance inspection facility. In addition, the production time can be shortened by merging the two processes of 'logistics transfer -> external inspection' or 'appearance inspection -> logistic transfer' of the prior art into one process of 'visual inspection during logistics transfer'.

In addition, conventionally, there are more sampling inspections than total inspections due to delay in production time, but according to the present invention, complete inspections are possible. In addition, in the case of the conventional exterior inspection technology, cross-sectional inspection should be performed several times (6 times in the case of a hexahedron) as many as the number of surfaces for a multi-sided inspection with a single-sided inspection. The number of inspections can be significantly reduced (2 times for hexahedron).

Although the present invention has been described with reference to some preferred embodiments, the scope of the present invention should not be limited thereto, but should also extend to modifications or improvements of the above embodiments supported by the claims.

110: loading unit
120: communication department
130: cooperative robot unit
132: image acquisition position control unit
134: loading direction change part
140: mobile robot unit
150: image acquisition unit
160: Appearance state determination unit
170: lighting state determination unit

Claims (14)

a loading unit in which the goods are loaded;
a communication unit for receiving a request for transport of the article;
a cooperative robot unit for moving the article to the loading unit;
a mobile robot unit for moving the loading unit and the cooperative robot unit according to the transfer request;
an image acquisition unit for acquiring an image of the goods loaded in the loading unit; and
and an appearance state determination unit configured to determine an appearance state of the article from the image.
The method according to claim 1,
The image acquisition unit is an article appearance inspection system, characterized in that mounted on the end of the cooperative robot unit.
3. The method according to claim 2,
The cooperative robot unit includes an image acquisition position control unit for controlling the image acquisition position of the image acquisition unit.
4. The method according to claim 3,
The cooperative robot unit is an article appearance inspection system, characterized in that it further comprises a loading direction changing part for changing the loading direction of the article.
5. The method according to claim 4,
When it is determined by the external state determination unit that the external appearance of the article is good, the mobile robot unit moves the loading part to a preset post-processing facility position.
6. The method of claim 5,
When the external appearance state of the article is determined to be defective by the appearance state determination unit, the mobile robot unit moves the loading unit to a preset defective article collection position.
7. The method of claim 6,
The image acquisition unit acquires an image of the product after the product is loaded and the mobile robot unit operates.
8. The method of claim 7,
The appearance state determination unit is an article appearance inspection system, characterized in that it further determines the effect of lighting on the image.
9. The method of claim 8,
The image acquisition position controller controls the image acquisition position of the image acquisition unit according to the determination of the lighting effect.
10. The method of claim 9,
The article appearance inspection system, characterized in that the mobile robot unit changes the position of the loading part according to the determination of the influence of the lighting.
11. The method of claim 10,
Item appearance inspection system, characterized in that it further comprises a lighting state determining unit for determining the lighting state around the loading unit.
12. The method of claim 11,
The mobile robot unit is an article appearance inspection system, characterized in that it moves the loading unit according to the determination of the lighting state determination unit.
An article appearance inspection method performed by the article appearance inspection system, comprising:
receiving, by the communication unit, a transfer request of the article;
moving the mobile robot unit to a position according to the transfer request;
loading the article by the cooperative robot unit mounted on the mobile robot unit to the mobile robot unit;
acquiring an image of the article by an image acquisition unit mounted at a distal end of the cooperative robot unit;
determining, by an appearance state determination unit, an appearance state of the article from the image; and
When it is determined that the external state is good, the mobile robot unit moves to a preset post-processing facility location, and when it is judged that the external state is bad, the mobile robot part moves to a preset defective product recovery position. Way.
A recording medium in which a computer readable program for executing the method of claim 13 is recorded.

Images