WO2020043362A1 - System and method for assisting a surface treatment - Google Patents

Abstract

A system for assisting a surface treatment of a workpiece (1) comprises – a scanner (3) for recognizing and localizing defects at the surface of a workpiece (1), – an output instrument, movable relative to the scanner (3), in the form of VR glasses (11) for outputting the positions at which the scanner (3) localizes a defect, – means (6, 9) for capturing the position and orientation of the VR glasses (10) relative to the workpiece (1) and – a graphics processor (12) that is configured to display, in an image displayed on the VR glasses (10), a notification about a defect on or adjacent to a line of sight (15) extending between the eye of a user (11) of the VR glasses (10) and the defect.

Description

 System and method for supporting surface processing

Description

The present invention relates to a system and a method for supporting a surface treatment, in particular the repair of

Painting errors on a vehicle body. Detecting such faults for a long time required the critical eye of an experienced employee who had to walk around the entire body in order to inspect it from different directions. Despite the high time involved, the reliability of error detection was heavily dependent on the current state of the employee.

WO 01/96829 A1 describes a method for testing a product and

especially for repairing painting errors on one

Known vehicle body that uses VR glasses to provide a worker with information about errors to be repaired. At the same time, a VR glasses camera is the only way to find such errors at all. Therefore, to find paint defects, a user of the glasses must continue to circle the body so that the camera can view them from all directions.

An object of the present invention is to provide a system and a method for supporting surface processing on a workpiece, which enable a quick and reliable detection and correction of errors.

The task is solved on the one hand by a system for supporting surface processing on a workpiece, which

- A scanner to detect and locate defects on the

Surface of a workpiece, and an output instrument movable relative to the scanner for outputting the positions at which the scanner has located a defect,

where the output instrument is VR glasses and the system

 Means for detecting the position and orientation of the VR glasses relative to the workpiece and

 comprises a graphics processor which is set up to display, in an image displayed in the VR glasses, an indication of a defect on or adjacent to a line of sight extending between the eye of a user of the VR glasses and the defect.

The detection of errors is no longer restricted to the direction of view of the wearer of the VR glasses, and there is in particular no need for the wearer of the VR glasses to circle the workpiece in order to enable error detection from different directions of view.

In order to capture the workpiece from all relevant directions in a short time, the scanner can have several cameras, which are aligned from different directions to a detection region in which the workpiece can be placed. Furthermore, drive means can be provided in order to move the workpiece and the cameras relative to one another. In particular, a variant in which the workpiece is moved past stationary cameras of the scanner is particularly suitable for integration in an assembly line production.

The cameras can be mounted on a portal, under which the workpiece is transported, and thus capture it from various directions that may be difficult for an employee to access.

The graphics processor can be set up to display an indication of a defect even if the defect is on a side of the workpiece facing away from the VR glasses. This ensures that a user of the VR glasses, when he approaches the workpiece, knows immediately whether there are defects to be repaired and does not have to go around the workpiece in a time-consuming manner.

In order to prevent the user of the VR glasses from looking in vain for a defect for which a notice is displayed, the graphics processor should furthermore be set up to use different forms of information, depending on whether the defect is located on a side of the workpiece facing the VR glasses or on a side facing away from the VR glasses.

If a defect is remedied, a continued indication of the defect can irritate the user of the VR glasses. For this reason, means are preferably also provided for detecting a gesture by the user of the VR glasses in order to

Detection of the gesture to delete a displayed message.

The gesture to be detected can be a pointing at the defect, a look at a predetermined object in the vicinity of the workpiece or a processing of the defect by the user of the VR glasses.

If the VR glasses, e.g. B. as the above-mentioned means for detecting the position and orientation of the VR glasses, a camera, an image evaluation unit can be connected to the camera to assess whether a defect is fixed or not, and the indication of the defect only to be deleted if the defect is assessed as having been remedied.

The graphics processor is also preferably set up to outline the

Play workpiece in the image displayed in the VR glasses, for example, to make it easier for the user of the VR glasses among several workpieces in his environment to identify the workpiece to be machined. This outline should coincide with the outline of the workpiece visible from the user's perspective.

The indication of a defect can be displayed exactly on the line of sight

While making it easier for users to find a defect, it can make it difficult to take a close look at the defect that is required for the

User can decide how best to fix the defect. It is therefore advisable to only display such a message if the distance between the VR glasses (and their user) and the workpiece exceeds a limit, i.e. is especially too large for the user to reach and edit the defect. If, on the other hand, the defect is within the user's reach, then the use of an indication adjacent to the line of sight is necessary

More convenient, the inspection of the defect through the VR glasses not hindered by it. Such an indication can include, for example, a circle extending around the line of sight and the defect or an arrow pointing at it.

The graphics processor can also be set up to insert details of the workpiece in alphanumeric form into the image displayed in the VR glasses. This information can be technical information about the workpiece, such as a type designation or a workpiece-specific identifier, information about the defect or also general information on handling the workpiece and in particular how to remedy the defect.

The object is further achieved by a method for supporting an

Surface processing on a workpiece, with the following steps:

a) scanning the surface of a workpiece to detect and localize defects in the surface,

b) outputting the positions at which a defect was located to a graphics processor,

c) Detecting the position and orientation of a VR glasses that the

Graphic processor for receiving image data is connected, relative to the workpiece,

e) in an image displayed in the VR glasses by the graphics processor, displaying an indication of a defect on or adjacent to a line of sight extending between the eye of a user of the VR glasses and the defect.

Preferably, the defects to which the method is applied are paint defects and the workpiece is a vehicle body.

Further features and advantages of the invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. Show it:

Fig. 1 is a schematic representation of the system according to the invention;

Fig. 2 shows an example of the image that a user of the VR glasses in

Figure 1 presents the system shown; and. Fig. 3 shows an alternative, displayed in the VR glasses.

Fig. 1 shows schematically a system according to the invention, which is used in motor vehicle production for the detection of paint defects on vehicle bodies 1, which are conveyed on a conveyor belt or by other suitable drive means 2 from a painting system, not shown, known per se. The

Vehicle bodies 1 are transported by the drive means 2 through a scanner 3, which comprises a portal 5 equipped with various cameras 4. The cameras 4 are aligned with an interior of the portal 5 in order to detect the outer surfaces of the bodies 1 transported through.

An image evaluation unit 6 connected to the cameras 4 analyzes the images supplied by the cameras and generates a data record for each body 1 that passes through the portal 3, which contains an individual identifier of the body 1 and, if applicable, locations on the body 1 at which a Painting error has been recognized, and may contain information about the nature of the painting error.

The data records thus obtained are stored in a database 7.

A process computer 8 accesses the data records in the database 7. By further communicating with a controller of the drive means 2 and recording or specifying their speed, the process computer 8 is able at any time to calculate at which location along the assembly line a particular body 1 is located.

The process computer 8 is also coupled to a plurality of cameras 9, around which

Detect location and the orientation of a VR glasses 10, which is worn by an employee involved in the elimination of the painting errors 11. One or more of these cameras 9 are fixed in the vicinity of the assembly line; a further camera 9 can be built into the VR glasses 10 and be movable therewith in order to monitor the field of view of the employee 11 wearing the VR glasses 10. On the basis of images provided by the employee 11 of the fixed cameras 9, the process computer 8 is able, for example by triangulation, to determine the location of the Employee 11 (and, albeit with limited accuracy, his direction of view). A comparison of the image supplied by the camera of the VR glasses 10 with those of the fixed cameras 9 enables the determination of the viewing direction of the employee 11 or the orientation of the VR glasses 10 worn by him with high accuracy.

Fig. 2 shows schematically the image that the employee 1 1 presents through the VR glasses 10. The VR glasses 10 comprise one in a manner known per se

Display screen, the transparency and color of which can be controlled locally, so that the user can perceive the real environment, here a sequence of the bodies 1 on the assembly line, through the transparent parts of the screen of the VR glasses 10, and at the same time graphic displays displayed on the screen Can perceive elements.

Based on the known positions of the bodies 1 and the position and

Orientation of the VR glasses 10, a graphics processor 12 of the process computer 8 is capable of calculating the outline of each of the bodies 1, as presented from the perspective of the employee 11, and one of them

resultant image to the VR glasses 10, preferably via radio interfaces 19, 20 of the process computer 8 and the VR glasses 10. From the bodies 1, the process computer 8 uses the information in the database 7 to select the one that is most advanced on the assembly line, by one

To calculate body 1 surrounding frame 13 and reproduce on the screen of VR glasses 10 that it surrounds the body 1 in question. The frame 13 may be a line extending around the body 1 in question; in the case shown here, the selected body 1 is additionally emphasized in relation to the others in that the transparency of the screen of the VR glasses 10 outside the frame 13 is reduced, so that the bodies not selected are darker and / or with a reduced contrast,

reduced sharpness or with a pattern that partially hides them. In this way, the employee's attention is intuitively drawn to the body 1 to be machined.

On the basis of the information in the database 7, the graphics processor 12 calculates where, from the point of view of the employee 1, 1 paint defects on the body 1 must be visible, and places an indication 14 in VR glasses 10 of each such defect on one Line of sight 15, which runs from the eye of the employee 11 to this paint defect. 2, these notices 14 have the form of crosses, the point of intersection of which corresponds to the position of the paint defect on the body 1, ie lies on the line of sight that the eye of the user 11 through the screen of the VR glasses 10 connects with the paint defect 16. Note 14 thus hinders the view of the paint defect, but this is irrelevant as long as

Employee 11 has not yet come close enough to the paint defect to be able to reach and process it.

As the employee 11 approaches the paint defect, the notice 14 on the screen of the VR glasses 10 can maintain its size or grow less than inversely proportional to the distance, so that with increasing

Approaching the paint defect its position becomes more clearly recognizable.

As can be seen from the example of a paint defect detected on the roof of the body 1, the information 14 can cause perspective distortion

according to the orientation of the surface on which the

There is a paint defect to which they refer; this makes it even easier for the employee 11 to find the paint defect.

A note 14 'of a shape that deviates even without perspective distortion, here on the rear fender of the body 1, e.g. in the form of a cross from dashed lines, indicates a paint defect that depends on the location of the

Employee 10 is not yet visible because he is on an opposite side of the body 1. As soon as the employee 10 walks around the body 1 and reaches a location from which this paint defect is visible, the graphics processor 12 changes the appearance of the relevant note, e.g. into a cross of solid lines like the reference 14 in FIG. 2, so that the employee 10 can judge at the appearance of an instruction 14, 14 'at any time on which side of the body 1 the associated paint defect is to be found.

If the distance to the paint defect has fallen below, in particular if the employee 10 has come close enough to the paint defect to be able to work on it, the graphics processor 12 replaces the note 14 with a note 14 “of another shape, which is not immediately in the (to the plane 3 vertical line of sight 15 between the eye of the employee 11 and the one here Lacquer flaw 16, shown schematically as a cross-hatched area, lies in order not to obstruct the view of the latter during the repair.

Based on the images supplied by the camera of the VR glasses 10

Process computer 8 capable of processing each one of the employees 1 1

Identify paint defects. The process computer s can also recognize when the employee 11 turns away from the paint defect after successful repair, and then delete the paint defect in question from the database 7 or note it there as remedied.

So that the paint defect is not deleted or noted as corrected before the employee 11 has finished repairing it, e.g. Because the employee 1 1 temporarily turns his head in a different direction, it can be provided that the paint defect is only deleted or noted as having been corrected if the

A predetermined object 17 appears in the field of view of the camera of the VR glasses 10. The object 17 can in principle be selected as desired; For an intuitive use of the VR glasses 10, it is expedient if the object 17 has a meaning that is linked to the successful repair, e.g. one with OK

labeled sign, which is mounted adjacent to the assembly line at a position where there is a low probability that the employee 1 1 looks at it involuntarily, e.g. clearly above eye level, or a correspondingly labeled card that the employee 1 1 carries along with repair tools and holds in front of the eye or in front of the camera of the VR glasses 10 when he is using the

Repair of the paint defect is finished. It is also conceivable that the object 17 is a finger of the employee 11, who, pointing to the finished paint defect, appears in the field of view of the camera 9 of the VR glasses 10.

It is also conceivable that the process computer 8 monitors the progress of the repair on the basis of the images supplied by the camera of the VR glasses 10, decides whether the paint defect has been remedied or not and, if necessary, adjusts the database 7 accordingly.

If a paint defect is deleted in the database 7 or noted as having been remedied, no reference to it is displayed in the VR glasses 10. Alternatively, a differentiated representation can take place, with the location of the rectified paint defect in the the image displayed on the VR glasses 10 is marked in color or with the addition of "OK" etc.

Information 18 on the type and identity of the body, such as a serial number, can be displayed adjacent to this by the graphics processor 12 in the image displayed in the VR glasses 10, preferably outside the frame 13, so as not to obscure the body 1 to be processed

affect. At least the information on the identity of the body 1 can also be read on the body itself, so that by comparing the information from

The graphic processor 12 displayed and the readable information of the employees 11 can verify that the indications 14 displayed in the VR glasses 10 regarding paint defects actually relate to the body 1 which is currently in front of them.

It is understood that the above detailed description and the drawings represent certain exemplary embodiments of the invention, but that they are only intended for the purpose of illustration and should not be interpreted as restricting the scope of the invention. Various modifications of the described configurations are possible without leaving the scope of the following claims and their range of equivalents. In particular go out of this

Description and the figures also features of the embodiments that are not mentioned in the claims. Such features can also occur in combinations other than those specifically disclosed here. The fact that several such characteristics are mentioned in the same sentence or in a different kind of context does not therefore justify the conclusion that they can only occur in the specifically disclosed combination; instead, it can generally be assumed that several of such features can also be omitted or modified, provided that this does not call into question the functionality of the invention.

Reference list

I body

2 drive means

 3 scanners

 4 camera

 5 portal

 6 image evaluation unit

7 database

 8 process computers

 9 camera

 10 VR glasses

 II employees

12 graphics processor

 13 frames

 14 Notice

 15 line of sight

 16 paint defects

17 object

 18 Information

 19 radio interface

Claims

Patent claims 1. System to support surface processing on one  Workpiece (1) which  - A scanner (3) for detecting and locating defects on the surface of a workpiece (1), and  comprises an output instrument which is movable relative to the scanner (3) for outputting the positions at which the scanner (3) has located a defect,  the output instrument being VR glasses (10) and the system  Means (6, 9) for detecting the position and orientation of the VR glasses (10) relative to the workpiece (1) and  comprises a graphics processor (12), which is set up, in an image displayed in the VR glasses (10), an indication (14, 14 ', 14 ") of a defect on or adjacent to one between the eyes of one  User (11) of the VR glasses (10) and the defect-extending line of sight (15). 2. System according to claim 1, wherein the scanner (3) a plurality of cameras (4), which are aligned from different directions to a detection region, and drive means (2) for moving the workpiece relative to  Coverage region includes. 3. System according to any one of the preceding claims, wherein the  Graphic processor (12) is set up to display an indication (14 “) of a defect even if the defect is on a side of the workpiece (1) facing away from the VR glasses (10). 4. System according to claim 3, wherein the graphics processor (12) is set up to use different forms of information (14, 14 ', 14 "), depending on whether the defect is facing one of the VR glasses (10) or a side of the workpiece (1) facing away from the VR glasses (10). 5. System according to one of the preceding claims, further comprising means (9) for detecting a gesture of the user of the VR glasses (10) in order to Detection of the gesture to delete a displayed message (14 '). 6. The system of claim 5, wherein the gesture to be detected is a pointing to the defect, a look at a predetermined object (16) in the vicinity of the workpiece (1) or a processing of the defect. 7. System according to claim 6, wherein the VR glasses (10) a camera (9)  and an image evaluation unit (7) is connected to the camera (9) in order to assess whether a defect has been remedied or not. 8. System according to any one of the preceding claims, wherein the  Graphic processor (12) is set up to reproduce an outline (13) of the workpiece (1) in the image displayed in the VR glasses (10). 9. System according to any one of the preceding claims, wherein the  Graphic processor (12) is set up to display the notice (14) on the line of sight (15) at least when the distance to the workpiece (1) is above a limit value. 10. System according to any one of the preceding claims, wherein the  Graphic processor is set up to display the notice (14 ') at least adjacent to the line of sight (15) when the distance to the  Workpiece (1) is below a limit. 11. System according to any one of the preceding claims, wherein the  Graphic processor (12) is set up to insert details (18) of the workpiece (1) in alphanumeric form into the image displayed in the VR glasses (10). 12. A method of supporting surface processing on a  Workpiece, with the steps:  a) scanning the surface of a workpiece (1) in order to identify and locate defects in the surface,  b) outputting the positions at which a defect was located to a graphics processor (12), c) detecting the position and orientation of VR glasses (10), which are connected to the graphics processor (12) for receiving image data, relative to the workpiece (1), e) in an image displayed in the VR glasses (10) by the graphics processor (12), displaying an indication (14, 14 ', 14 ") of a defect on or adjacent to a between the eye of a user (11) the VR glasses (10) and the defect-extending line of sight (15). 13. The method of claim 11, wherein the defects are paint defects (16). 14. The method according to claim 11 or 12, wherein the workpiece (1) a  Vehicle body is.