JP7138303B1 - Fastening work support program - Google Patents

Abstract

An object of the present invention is to enable an easy-to-understand and error-free fastening manual operation by navigating fastening positions and fastening sequences. SOLUTION: A fastening navigation program 100 provides a plurality of target two-dimensional barcodes B to a first calculation unit 13 provided in a fastening navigation device 10, and a target workpiece placed on a workbench WT. A target real still image data generating procedure for capturing an image W by the imaging unit 11 and generating corresponding real target still image data RD, and a target real still image data display for displaying the target real still image data RD on the first display unit 12 While superimposing the virtual video data including the navigation mark NM on the displayed target real still image data RD, the display position with the target two-dimensional barcode B as the positioning reference and the actual separation distance RL1. is displayed on the first display unit 12 according to the display magnification with . [Selection drawing] Fig. 2

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fastening work assistance program for navigating manual fastening by a worker using a fastening tool for a target work piece.

2. Description of the Related Art Conventionally, there has been known a tightening work assisting device that assists tightening work of bolts, screws, etc. in an assembly factory (see, for example, Patent Document 1). This prior art tightening work support device displays the tightening order of bolts, screws, etc. to the operator. Specifically, the work support device includes a light emitter, a camera, and a computer. This camera is designed to photograph the workpiece. On the display of this computer, if the tightened bolts and the like are correct, the image of the workpiece being photographed by the camera is overlaid with the image of the bolt that is painted over.

Japanese Patent Application Laid-Open No. 2010-42491 (Fig. 1 etc.)

In the prior art tightening work support device, a support program executed by a computer displays a message such as "Please tighten bolt 1" below the work image on the display. However, this tightening work support is limited to message display and does not provide navigation of the tightening position on the screen. There is a demand for a technique that enables manual fastening that is easy to understand and error-free.

It is an object of the present invention to provide a program that enables easy-to-understand and error-free manual fastening by navigating the fastening position and fastening order using animated marks superimposed on actual image data. be.

In order to achieve the above object, the fastening navigation program of the present invention has an imaging unit, a first display unit, and a first calculation unit, and has a plurality of required fastening points separated by a known reference distance. generated to be displayed superimposed on the display state of the reference real still image data obtained by photographing the reference work piece provided with a plurality of reference two-dimensional barcodes, and for the plurality of fastening required locations A fastening navigation device for displaying, on the first display unit, virtual moving image data including navigation marks for navigating the manual work order and manual work position when fastening manual work is sequentially performed using a fastening work tool one by one. The imaging unit captures an image of a target work piece provided with a plurality of target two-dimensional barcodes and placed on a workbench, and generates corresponding target real still image data. a target real still image data generation procedure, a target real still image data display procedure for displaying the generated target real still image data on the first display unit, and the virtual moving image data including the navigation mark, a display position using the plurality of target two-dimensional barcodes in the target real still image data as a positioning reference while being superimposed on the target real still image data displayed in the target real still image data display procedure; displayed on the first display unit by a display magnification based on the actual separation distance between the plurality of target two-dimensional barcodes in the target real still image data, and the fastening to the target work piece by the worker and a virtual animation data display procedure for navigating the fastening manual work using the work tool.

A worker sequentially performs manual fastening using a fastening tool for each of a plurality of locations requiring fastening on the target workpiece. At that time, the target real still image data obtained by imaging the target work placed on the workbench in front of one's eyes is displayed on the first display unit, and a virtual moving image including a navigation mark is displayed on the still image data. Data is displayed superimposed, thereby dynamically navigating the manual work sequence and manual work positions to be performed for the plurality of locations to be fastened during the manual fastening work.

At this time, the virtual moving image data is a display of reference actual still image data obtained by previously photographing a reference work piece (for example, the same type, same size, and same shape as the target work piece) having a plurality of fastening points. It is generated by displaying, for example, superimposed on the state. The reference workpiece is provided with a plurality of reference two-dimensional barcodes separated by a known reference distance. That is, the virtual video data includes in advance navigation marks whose positions in the data are set so as to indicate a plurality of fastening points while using reference two-dimensional barcodes separated from each other by a known distance as positioning references. is

On the other hand, when the worker actually executes the fastening work described above, the target workpiece is also provided with a plurality of two-dimensional barcodes (target two-dimensional barcodes). Contains spaced target two-dimensional barcodes. As a result, according to the length ratio between the distance between the plurality of target two-dimensional barcodes in the target real still image data and the distance between the reference two-dimensional barcodes included in the virtual video data, the target Navigation marks in the virtual video data can be positioned relative to the real still image data. As a result, the positions of the navigation marks superimposed and displayed as moving images on the target real still image data are accurately matched so as to indicate the positions of a plurality of fastening points of the target work object in the target real still image data. be able to.

As described above, according to the present invention, the tightening position and the tightening order can be displayed dynamically using marks in the virtual moving image that are accurately positioned, so that the operator can easily navigate. Therefore, the operator can perform the manual fastening operation without any mistakes.

Further, the fastening navigation program of the present invention further provides the first calculation unit with a position of the fastening tool when the fastening manual operation is performed on the target work piece, based on the imaging result of the imaging unit. and the position of the fastening tool detected by the position detection procedure matches the position of the navigation mark in the virtual video data displayed by the virtual video data display procedure. and a position determination procedure for determining whether or not to perform.

Further, the fastening navigation program of the present invention further determines that the position of the fastening work tool does not match the position of the navigation mark in the virtual moving image data in the position determination procedure for the first calculation unit. It is characterized by executing a warning notification procedure for executing a corresponding warning notification when the warning notification is issued.

Further, in the fastening navigation program of the present invention, in the position detection procedure, based on the detection result using a plurality of working tool two-dimensional barcodes provided in advance on the fastening work tool as a positioning reference, and detecting a position of a mounting portion to which a bolt or nut to be fastened to a fastening portion of the target work piece is attached.

A program for generating navigation video data according to the present invention is provided with a plurality of required fastening points for a terminal device having a second display section, a second calculation section, and an operation section. a reference actual still image data generating procedure for photographing a reference work on which a plurality of reference two-dimensional barcodes are provided at intervals of a distance and generating corresponding reference actual still image data; In the state displayed on the second display unit, a manual fastening operation using a fastening work tool for each of the plurality of locations to be fastened, in order to superimpose and display the displayed reference actual still image data. an instruction reception procedure for receiving an instruction to generate a navigation mark for navigating a manual work order and a manual work position when sequentially performing the steps, and receiving the navigation mark generation instruction in the instruction receiving procedure. In this case, a plurality of target two-dimensional barcodes are provided and displayed superimposed on the target real still image data obtained by imaging the target work placed on the workbench, and the fastening work for the target work by the worker is displayed. and a virtual moving image data generation procedure for generating virtual moving image data including the navigation mark for navigating the fastening manual operation using the tool.

According to this, the virtual moving image data is generated so as to be superimposed on the display state of the reference actual still image data captured in advance by receiving the instruction to generate the navigation mark at the terminal device. Therefore, on the terminal device, for example, the navigation marks can be easily generated so as to correspond to the positions of the fastening required parts in the reference actual still image data, and furthermore, in the order of the fastening required parts in the reference actual still image data. Correspondingly, it is possible to easily generate a moving image in which the navigation marks move in this order and are displayed sequentially in the reference real still image data. In addition, virtual video data can be generated in advance by a manager or the like who is different from the operator, apart from the fastening manual work. Therefore, for example, virtual moving image data generated in advance can be efficiently deployed in a large number of fastening navigation devices.

The fastening work assistance program of the present invention provides a fastening navigation device equipped with an imaging unit and a terminal device equipped with an operation unit, which are included in a fastening work assistance system, with a plurality of required fastening points for the computing units respectively. a reference real still image data generation procedure for photographing a reference workpiece provided with a plurality of reference two-dimensional barcodes spaced apart by a known reference distance and generating corresponding reference real still image data; In a state where the image data is displayed, manual fastening is sequentially performed using a fastening work tool one by one for the plurality of locations to be fastened, in order to superimpose and display the displayed reference real still image data. An instruction receiving procedure for receiving an instruction to generate a navigation mark for navigating a manual work order and a manual work position at the time of performing the manual work via the operation unit; a virtual moving image data generating procedure for generating virtual moving image data including the navigation mark for navigating the manual fastening of the target workpiece by the worker using the fastening tool; and a plurality of target two-dimensional bars. a target real still image data generating procedure for capturing an image of the target work piece provided with a code and placed on a workbench by the imaging unit and generating corresponding real target still image data; a target real still image data display procedure for displaying image data; and the target real still image while superimposing the virtual video data on the target real still image data displayed in the target real still image data display procedure. display position with the plurality of target 2D barcodes in the data as a positioning reference, and display magnification with the actual separation distance between the plurality of target 2D barcodes in the target real still image data as the scaling reference, and a virtual moving image data display procedure to be displayed by the virtual moving image data display procedure.

As with the fastening navigation program described above, it is possible to dynamically display the fastening position and fastening order using marks in the virtual animation that are positioned accurately. The operator can perform the manual fastening operation without any mistakes.

According to the present invention, an easy-to-understand and error-free fastening manual operation becomes possible.

A diagram showing the configuration of a terminal device to which the navigation video data generation program according to the embodiment of the present invention is applied, and how virtual video data is generated in a state in which reference real still image data is displayed on the terminal device. is. Overall configuration when manual fastening is performed using a fastening navigation device to which a fastening navigation program according to an embodiment of the present invention is applied, and a state in which target real still image data is displayed on the fastening navigation device 10 is a diagram showing how virtual moving image data is displayed in a superimposed manner. FIG. FIG. 3 is a diagram showing how virtual moving image data is superimposed and displayed while target real still image data is displayed on the fastening navigating device shown in FIG. 2 ; FIG. 3 is a diagram showing how virtual moving image data is superimposed and displayed while target real still image data is displayed on the fastening navigating device shown in FIG. 2 ; FIG. 3 is a functional block diagram of the fastening navigation device shown in FIG. 2; FIG. 10 is an overall configuration diagram in the case where the conclusion navigating device to which the conclusion navigating program according to the embodiment of the present invention is applied is equipped with a first display section of another aspect; FIG. 3 is a configuration diagram of a fastening tool used when performing manual fastening using the fastening navigation device shown in FIG. 2 ; When manual fastening is performed using the fastening navigation device shown in FIG. 2, when it is determined that the position of the fastening tool does not match the position of the navigation mark, a corresponding warning is issued. It is a figure which shows a state. FIG. 11 is an overall configuration diagram in the case where the conclusion navigating device to which the conclusion navigating program according to the embodiment of the present invention is applied is provided with another aspect of the warning notification unit; 2 is a functional block diagram of the terminal device shown in FIG. 1; FIG. FIG. 2 is a diagram showing how the terminal device shown in FIG. 1 captures photographed data of a reference work object, generates virtual moving image data, and then transfers the virtual moving image data to a fastening navigation device; It is a part of a series of flow charts showing navigation processing of fastening manual work executed by the fastening navigation program by the first calculation unit of the fastening navigation device shown in FIG. FIG. 13 is a flowchart continuing from FIG. 12 and being a part of a series of flowcharts showing the manual fastening navigation process executed by the fastening navigation program by the first calculation unit of the fastening navigation device shown in FIG. 2 ; FIG. 14 is a flowchart continuing from FIG. 13 and being a part of a series of flowcharts showing the manual fastening navigation process executed by the fastening navigation program by the first calculation unit of the fastening navigation device shown in FIG. 2 ; 4 is a series of flowcharts showing virtual video data generation processing executed by a second calculation unit of the terminal device shown in FIG. 1 according to a navigation video data generation program;

An embodiment of the present invention will be described below with reference to the drawings.

<Overview of navigation in manual fastening>
The conclusion navigation program 100 and the navigation moving image data generation program 200 according to the embodiment of the present invention are configured as follows. The conclusion navigation program 100 is a program applied to the conclusion navigation device 10 . The fastening navigation program 100 and the fastening navigation device 10 determine the manual work order and the manual work position when performing the manual fastening work sequentially for a workpiece (for example, a tire wheel, etc.) having a plurality of parts to be fastened. configured to navigate. The fastening manual work is, for example, a task of manually fastening a bolt or nut using a fastening tool T by the worker WR (see FIG. 2). In addition to so-called bolts, other fastening members such as screws and screws may be used, and hereinafter, the term "bolt" in the specification of the present application includes these other fastening members. The navigation is performed based on the virtual moving image data. The navigation moving image data generation program 200 is configured to generate virtual moving image data.

As shown in FIG. 1, a navigation video data generation program 200 is applied to a terminal device 20. FIG. The terminal device 20 is, for example, a fixed terminal such as a personal computer, or a mobile terminal such as a smart phone or a tablet. In the terminal device 20, the navigation moving image data generating program 200 generates virtual moving image data and reference real still image data SD. The generated virtual moving image data and reference real still image data SD can be transferred from the terminal device 20 to the fastening navigation device 10 via wireless communication, a storage medium, or the like. The procedure for generating the virtual moving image data and the reference real still image data SD will be detailed later.

The virtual video data includes navigation marks NM. For example, the navigation mark NM is displayed superimposed on the reference actual still image data SD in the state where the reference actual still image data SD obtained by photographing the reference workpiece SW is displayed on the second display unit 21. be done. The positions of the superimposed navigation marks NM correspond to the positions of a plurality of fastening points BP. More specifically, the navigation mark NM may be displayed, for example, as a figure (a circle in this embodiment) surrounding the fastening point BP in the reference actual still image data SD.

In the virtual video data, navigation marks NM are used to navigate the manual work order and manual work position when manual fastening is performed one by one using a fastening work tool for a plurality of required fastening points BP. Is displayed. For example, the reference workpiece SW (that is, the target workpiece W) has fastening points BP1 to BP5 at five different positions. , BP3, BP4, and BP5 in this order, the navigation marks NM1, NM2, NM3, NM4, and NM5 are moved in this order and displayed sequentially. It has become. In this embodiment, the virtual moving image data is a moving image in which the display position of the navigation mark NM moves corresponding to the fastening points BP1 to BP5. Therefore, by navigating to the navigation marks NM of the virtual moving image data, the operator who performs manual fastening can recognize the manual work order and manual work position. That is, the virtual moving image data functions as a digital procedure manual.

A plurality of reference two-dimensional barcodes SB are provided on the reference workpiece SW at intervals of a known reference distance SL. The reference two-dimensional barcode SB is, for example, a QR code (registered trademark). For example, two reference two-dimensional barcodes SB are provided near the edge of the reference workpiece SW. When the reference actual still image data SD, which is the imaging data of the reference workpiece SW, is displayed, two reference two-dimensional barcodes SB are also displayed in the reference actual still image data SD, and the distance between them is the distance becomes SL1. The distance SL1 is a value obtained by enlarging or reducing the reference distance SL according to the magnification when the reference workpiece SW is imaged. As will be described in detail later, when displaying virtual video data on the fastening navigation device 10, the reference two-dimensional barcode SB is used for positioning the navigation marks NM1 to NM5, and the distance SL1 is displayed. It is used to determine the display magnification of virtual video data.

As shown in FIGS. 2 to 4, the worker WR sequentially performs manual fastening of a plurality of fastening required locations BP of the target workpiece W using the fastening tool T one by one. The fastening manual work is performed with the target workpiece W placed on the workbench WT. In this embodiment, the target work W is of the same type, size and shape as the reference work SW. That is, the target work W has the required fastening points BP1 to BP5 at the same five different positions as the reference work SW. A plurality of target two-dimensional barcodes B are provided on the target work W. FIG. In this embodiment, the target two-dimensional barcode B is of the same type and arranged at the same position as the reference two-dimensional barcode SB of the reference workpiece SW. That is, the target two-dimensional barcode B is, for example, a QR code (registered trademark) or the like, and two of them are provided near the edge of the target work W. FIG. The two target two-dimensional barcodes B are spaced apart by the same distance as the reference distance SL.

The fastening tool T is configured so that it can be manually operated by the worker WR and can fasten the bolt or nut at the fastening point BP. The fastening work tool T is, for example, a wrench, a screwdriver, or the like, and the tip thereof serves as a mounting portion T1 to which a bolt or nut to be fastened to the fastening point BP of the target work W is attached (Fig. 7 ).

The fastening navigation device 10 is fixed to the workbench WT via a stand or the like. More specifically, on the workbench WT, the target work W is placed on the front side of the worker WR, and the stand stands upright on the back side of the target work W. The fastening navigating device 10 is connected to the upper end of the stand and positioned above the target workpiece W placed on the workbench WT. The fastening navigation device 10 is, for example, a mobile terminal such as a smart phone or a tablet, but is not limited thereto.

As shown in FIG. 5 , the fastening navigation device 10 has an imaging section 11 , a first display section 12 , a first calculation section 13 , a first storage section 14 and a warning notification section 15 . The imaging unit 11, the first display unit 12, the first storage unit 14, and the warning notification unit 15 are electrically connected to the first calculation unit 13 via an interface, a bus, or the like. The imaging unit 11 is, for example, a camera or the like built in a mobile terminal, and captures an angle of view including the target workpiece W placed on the workbench WT and the fastening tool T during manual fastening. , may be provided below the fastening navigation device 10 so that an image can be captured from directly above. The first display unit 12 is, for example, a monitor or the like built in a mobile terminal. It may be provided above the gate device 10 . The first calculation unit 13 is composed of a CPU, an electric circuit, etc., and executes processing of various programs including imaging by the imaging unit 11, image display by the first display unit 12, and the like. The first storage unit 14 includes a memory, an electric circuit, etc., and stores various programs such as the fastening navigation program 100 in advance. In addition, the first storage unit 14 can record virtual moving image data and reference real still image data SD generated and transferred by the terminal device 20, image data captured by the imaging unit 11, and the like. . The programs and data stored in the first storage section 14 are read by the CPU of the first calculation section 13 . Note that the warning notification unit 15 will be described in detail later.

As shown in FIG. 2(a), when the manual fastening operation is started, first, the target workpiece W placed on the workbench WT is imaged by the imaging unit 11, and the corresponding target real still image data RD is captured. is generated. The generated target real still image data RD is displayed on the first display unit 12 . The target real still image data RD may be displayed on the first display unit 12 after being recorded in the first storage unit 14, or may be displayed on the first display unit 12 without going through the first storage unit 14. good.

When the target real still image data RD, which is the imaging data of the target work W, is displayed, two target two-dimensional barcodes B are also displayed in the target real still image data RD, and the distance between them is the actual distance. The separation distance becomes RL1. The actual separation distance RL1 is a value obtained by enlarging or reducing the reference distance SL according to the magnification when the target work W is imaged.

Next, as shown in FIGS. 2(b), 3, and 4, the virtual moving image data including the navigation mark NM is superimposed on the displayed target real still image data RD while the first It is displayed on the display unit 12 . Here, as the virtual moving image data to be superimposed and displayed, the virtual moving image data generated by the terminal device 20 is adjusted in display position and display magnification. The display position of the superimposed virtual moving image data is determined using a plurality of target two-dimensional barcodes B in the target real still image data RD as a positioning reference, and two target two-dimensional barcodes B in the reference real still image data SD. are adjusted by matching the two reference two-dimensional barcodes SB (see FIGS. 1 and 2).

The display magnification of the virtual moving image data to be superimposed and displayed is based on the actual separation distance RL1 between the plurality of target two-dimensional barcodes B in the target real still image data RD, and the actual separation distance RL1 and the reference real still image data. It is adjusted based on the distance SL1 between the two reference two-dimensional barcodes SB in SD. As the display magnification, for example, a value obtained by dividing the actual separation distance RL1 by the distance SL1 may be used (see FIGS. 1 and 2).

More specifically, the distance SL1 between the two reference two-dimensional barcodes SB in the reference actual still image data SD generated by imaging the reference workpiece SW at a relatively long distance is 2.5 cm. and On the other hand, it is assumed that the actual separation distance RL1 between the two target two-dimensional barcodes B in the target real still image data RD generated by imaging the target work W at a relatively short distance is 5.0 cm. . In this case, the display magnification is RL1/SL1=(5.0 cm)/(2.5 cm)=2.0 times. Therefore, the virtual moving image data generated by the terminal device 20 is aligned as described above, enlarged at a display magnification of 2.0 times, and superimposed on the target real still image data RD.

Further, when the distance SL1 is 5.0 cm (the reference workpiece SW is imaged at a relatively short distance) and the actual separation distance RL1 is 2.5 cm (the target workpiece W is imaged at a relatively long distance), In some cases, the display magnification is RL1/SL1=(2.5 cm)/(5.0 cm)=0.50. In this case, the virtual moving image data generated by the terminal device 20 is aligned as described above, reduced at a display magnification of 0.50 times, and superimposed on the target real still image data RD.

As described above, by adjusting the display position of the virtual moving image data and adjusting the display magnification, when the virtual moving image data is displayed superimposed on the target real still image data RD, the position of the navigation mark NM is , to match the position of the fastening point BP in the target real still image data RD,

As shown in FIG. 2B, when the virtual moving image data is superimposed on the target real still image data RD and displayed on the first display unit 12, first, A navigation mark NM1 is superimposed on the position of the point BP1. The superimposed display of the navigation mark NM1 continues for a predetermined period. The first display unit 12 in this state is visually recognized by the worker WR, and the worker WR navigates to perform manual fastening to the required fastening point BP1 on the target workpiece W placed on the workbench WT. According to the navigation, the worker WR fastens the bolt or nut at the fastening point BP1 at the attachment portion T1 of the fastening tool T. FIG.

Next, as shown in FIG. 3A, the navigation mark NM1 is once erased, and the navigation mark NM2 is superimposed and displayed at the position of the fastening point BP2 in the target real still image data RD. The superimposed display of the navigation mark NM2 continues for a predetermined period. The first display portion 12 in this state is visually recognized by the worker WR, and the worker WR navigates to perform the manual fastening of the fastening point BP2 on the target workpiece W placed on the workbench WT. According to the navigation, the worker WR fastens the bolt or nut at the fastening point BP2 at the attachment portion T1 of the fastening tool T. FIG.

Next, as shown in FIG. 3B, the navigation mark NM2 is temporarily erased, and the navigation mark NM3 is superimposed and displayed at the position of the fastening point BP3 in the target real still image data RD. The superimposed display of the navigation mark NM3 continues for a predetermined period. The first display unit 12 in this state is visually recognized by the worker WR, and the worker WR navigates to perform manual fastening of the fastening point BP3 on the target workpiece W placed on the workbench WT. According to the navigation, the worker WR fastens the bolt or nut at the fastening point BP3 at the attachment portion T1 of the fastening tool T. FIG.

Next, as shown in FIG. 4A, the navigation mark NM3 is temporarily erased, and the navigation mark NM4 is superimposed and displayed at the position of the fastening point BP4 in the target real still image data RD. The superimposed display of the navigation mark NM4 continues for a predetermined period. The first display unit 12 in this state is visually recognized by the worker WR, and the worker WR navigates to perform manual fastening of the fastening point BP4 on the target workpiece W placed on the workbench WT. According to the navigation, the worker WR fastens the bolt or nut at the fastening point BP4 at the attachment portion T1 of the fastening tool T. FIG.

Then, as shown in FIG. 4B, the navigation mark NM4 is temporarily erased, and the navigation mark NM5 is superimposed and displayed at the position of the fastening point BP5 in the target real still image data RD. The superimposed display of the navigation mark NM5 continues for a predetermined period. The first display unit 12 in this state is visually recognized by the worker WR, and the operator WR navigates to perform manual fastening of the fastening point BP5 on the target workpiece W placed on the workbench WT. Become. According to the navigation, the worker WR fastens the bolt or nut at the fastening point BP5 at the attachment portion T1 of the fastening tool T. FIG.

As described above, the navigation marks NM1 to NM5 are superimposed on the target real still image data RD so as to correspond to the positions of the fastening points BP1 to BP5 in the target real still image data RD. be. The navigation marks NM1, NM2, NM3, NM4, and NM5 move in this order and are displayed sequentially, so that the fastening manual operation is navigated in this order for the fastening points BP1, BP2, BP3, BP4, and BP5. . That is, the virtual moving image data including the navigation marks NM functions as a digital procedure manual.

In this embodiment, the navigation mark NM disappears after being maintained for a predetermined period of time and moves to the next position. This predetermined period may be determined, for example, based on the target completion time of the fastening manual work. In this way, the manual work order and the manual work positions when performing the fastening manual work sequentially are navigated.

In this embodiment, as shown in FIG. 2(a), the fastening navigation device 10 is a mobile terminal, and the imaging unit 11 and the display unit 12 are integrally built into the mobile terminal. It is Instead of this, as shown in FIG. 6A, for example, the fastening navigation device 10 includes a camera unit as an imaging unit 11 and a monitor device as a display unit 12, which are separate units. may be configured as follows. In this case, the imaging unit 11 may be fixed to the upper end of the stand of the workbench WT, and the display unit 12 may be spaced apart from the workbench WT and fixed at a position visible to the worker WR. Further, as shown in FIG. 6B, for example, the fastening navigation device 10 includes a camera unit as an imaging unit 11 and smart glasses as a display unit 12, which are configured to be separate units. may In this case, the imaging unit 11 may be fixed to the upper end of the stand of the workbench WT, and the display unit 12 may be attached to the eyes of the worker WR. In these cases, for example, video data displayed on the display unit 12 may be transferred from the mobile terminal to the display unit 12 in either a wireless format or a wired format.

<Warning notification based on position detection of fastening tool>
The fastening navigation program 100 and the fastening navigation device 10 perform warning notification based on the detection of the position of the fastening work tool when navigating the manual work order and the manual work position when performing the fastening manual work sequentially. It's becoming In this embodiment, for example, it is determined whether or not the position of the fastening tool T used by the worker WR matches the position of the superimposed navigation mark NM. , a warning is issued. The purpose of this is to make the worker WR recognize the improper position by issuing an alarm when the position of the fastening tool T moved by the worker WR is improper during the manual fastening work. To urge the worker WR to correct the position to a proper position. Therefore, the fastening tool T and the fastening navigation device 10 are configured as follows.

As shown in FIG. 7, the fastening tool T has a bar shape, and has a gripping portion T2 on the opposite side of the mounting portion T1 with the mounting portion T1 as the tip. The grasping portion T2 can be grasped by the worker WR. During manual fastening, a bolt or nut to be fastened to the fastening point BP is attached to the mounting portion T1, and the mounting portion T1 is used as a fulcrum and the long axis of the fastening work tool T is used as a radius vector. Rotational operation is possible. The fastening work tool T is provided with a plurality of work tool two-dimensional barcodes TB. The work tool two-dimensional barcode TB is, for example, a QR code (registered trademark) or the like, and two of them are provided at an intermediate portion between the mounting portion T1 and the grip portion T2 of the fastening work tool T along the longitudinal direction.

As shown in FIG. 5 , the fastening navigation device 10 has a warning notification section 15 . The first calculation unit 13 instructs the warning notification unit 15 to notify the worker WR of the warning visually, audibly, or the like. As a form of warning notification, for example, the warning notification unit 15 may include a speaker device and emit a warning sound, or the warning notification unit 15 may include warning display data and the first display unit 12 may display a warning. It may be displayed (for example, an image corresponding to a warning light). (See Figure 8). When the first calculation unit 13 determines that the position of the fastening tool T does not match the position of the navigation mark NM in the virtual moving image data, the first calculation unit 13 instructs the warning notification unit 15 to issue a corresponding warning notification. do.

As shown in FIG. 2(b), for example, when the mounting part T1 is positioned at the required fastening point BP1 according to the superimposed display of the navigation mark NM1, the position of the fastening work tool T detected as described later. is determined to match the position of the navigation mark NM in the virtual moving image data, and the above-described warning is not issued. In the states shown in FIGS. 3 and 4, similarly, no warning is issued.

Here, the position of the fastening work tool T at the time of performing the manual fastening operation on the target workpiece W is detected based on the imaging result of the imaging unit 11 . Specifically, the position of the mounting portion T1 is detected based on the detection result using a plurality of work tool two-dimensional barcodes TB as a positioning reference. In this embodiment, the separation distance between the work tool two-dimensional barcode TB on the side of the mounting portion T1 and the mounting portion T1 is known. The direction of the mounting portion T1 is calculated from the work tool two-dimensional barcode TB on the grip portion T2 side. These two working tool two-dimensional barcodes TB are detected by the imaging unit 11, and the position of the mounting unit T1 is detected with high accuracy.

As shown in FIG. 8, after completion of the fastening of the fastening point BP1 in the state shown in FIG. The operation when the portion T1 is moved will be described. In this case, it is determined that the position of the fastening tool T detected as described above does not match the position of the navigation mark NM in the virtual moving image data, and the above-described warning is issued. As the warning notification, a warning sound AL1 may be emitted from the fastening navigation device 10 as shown in FIG. AL2 may be displayed.

That is, when the display shifts from the navigation mark NM1 to NM2, even if the movement destination of the mounting portion T1 is the fastening required points BP1, BP3, BP4, and BP5 (that is, other than BP2), the warning is similarly notified. is executed. Also, when the display shifts from the navigation mark NM2 to NM3, even if the movement destination of the mounting portion T1 is the fastening required points BP1, BP2, BP4, and BP5 (that is, other than BP3), the same warning is issued. is executed. Also, when the display shifts from the navigation mark NM3 to NM4, even if the movement destination of the mounting portion T1 is the fastening required points BP1, BP2, BP3, and BP5 (that is, other than BP4), the same warning is issued. is executed. Also, when the display shifts from the navigation mark NM4 to NM5, even if the movement destination of the mounting portion T1 is the fastening required points BP1, BP2, BP3, and BP4 (that is, other than BP5), the warning is similarly issued. is executed.

In this embodiment, when it is determined that the detected position of the fastening tool T does not match the position of the navigation mark NM in the virtual video data, a warning is issued. Alternatively, the imaging section 11 may capture an image of the angle of view in the inconsistent state and log it in the recording section 14 . Further, in the present embodiment, as shown in FIGS. 2A, 5 and 8, the fastening navigation device 10 is provided with a warning notification unit 15, which emits a warning sound AL1 or emits a warning light. The display of AL2 is performed at the mobile terminal. Instead of this, for example, when issuing the warning sound AL1, as shown in FIG. 9A, a speaker device as the warning notification unit 15 may be provided separately from the mobile terminal. Further, for example, when displaying the warning light AL2, as shown in FIG. 9B, a warning light device as the warning notification unit 15 may be provided separately from the mobile terminal. These speaker device and warning light device may be connected to the mobile terminal either wirelessly or by wire.

<Generation of virtual video data>
As shown in FIG. 1, the terminal device 20 generates virtual video data that functions as a digital procedure manual. The terminal device 20 is a terminal different from the fastening navigation device 10 . The terminal device 20 generates virtual video data using the navigation video data generation program 200 . That is, the terminal device 20 shares the generation of the virtual moving image data, and the virtual moving image data generated by the terminal device 20 and the reference actual still image data SD are transferred to the fastening navigation device 10 and stored therein. and manual fastening is navigated.

As shown in FIG. 10 , the terminal device 20 has a second display section 21 , an operation section 22 , a second calculation section 23 and a second storage section 24 . The second display unit 21 is, for example, an attached monitor in the case of a fixed terminal or a built-in monitor in the case of a mobile terminal, and can be visually recognized by an administrator. The operation unit 22 is, for example, touch keys, buttons, etc. that can be operated by the administrator, and includes an interface for receiving instructions generated by the administrator. The second calculation unit 23 is composed of a CPU, an electric circuit, etc., and executes processing of various programs including image display on the second display unit 21, reception of instructions from the administrator via the operation unit 22, and the like. . The second storage unit 24 is composed of a memory, an electric circuit, etc., and stores various programs such as the navigation video data generation program 200 in advance. In addition, the second storage unit 24 can record the generated virtual moving image data, reference actual still image data SD obtained by separate imaging, and the like.

As shown in FIG. 11, the reference actual still image data SD is image data obtained by imaging the same reference work object SW as the target work object W. As shown in FIG. The image of the reference workpiece SW is captured by a camera separate from the terminal device 20 or by a camera built into the terminal device 20, and captured by the terminal device 20 to generate reference actual still image data SD. .

As shown in FIGS. 11 and 1, the administrator issues an instruction to generate the navigation mark NM via the operation unit 22 while the reference actual still image data SD is displayed on the second display unit 21 . For example, the navigation marks NM are generated so as to correspond to the positions of the fastening required parts BP1 to BP5 in the reference actual still image data SD displayed on the second display unit 21 . Specifically, navigation marks NM1, NM2, NM3, Generation instructions are given so that NM4 and NM5 move in this order and are displayed sequentially. This generation instruction is accepted by the second calculation unit 23, and virtual video data is generated. The virtual video data is a video in which the display positions of the navigation marks NM superimposed and displayed on the target real still image data RD in the fastening navigation device 10 move corresponding to the fastening points BP1 to BP5. Become. That is, the part displayed as virtual moving image data is only the navigation mark NM.

As shown in FIG. 11, the generated virtual video data is transferred to and stored in the fastening navigation device 10, and the fastening navigation device 10 becomes capable of navigating manual fastening operations as described above.

<Processing of fastening navigation program>
The fastening navigation program 100 applied to the fastening navigation device 10 executes each process of the first calculation unit 13, and as described above, the manual work order and the manual work when performing the fastening manual work sequentially. Positional navigation is accomplished.

As shown in a series of flowcharts of FIGS. 12 to 14, when starting navigation for manual fastening, first, in step 101, the first calculation unit 13 generates a fastening navigation device generated by the terminal device 20. 10 and the reference real still image data SD are stored in the first storage unit 14 .

Next, at step 102 , the first calculation unit 13 captures an image of the target work W on the workbench WT with the imaging unit 11 . Next, at step 103, the first calculation unit 13 generates target real still image data RD. Next, at step 104, the first calculation unit 13 displays the generated target real still image data RD on the first display unit 12 (see FIG. 2(a)). Here, steps 102 and 103 correspond to the target real still image data generation procedure, and step 104 corresponds to the target real still image data display procedure.

Next, at step 105, the first calculation unit 13 extracts the reference two-dimensional barcode SB of the stored reference actual still image data SD and the target two-dimensional bar code SB of the generated target actual still image data RD. The code B is compared to determine the display position of the virtual moving image data. Next, at step 106, the first calculation unit 13 calculates the distance SL1 between each reference two-dimensional barcode SB of the reference actual still image data SD and each target two-dimensional barcode B of the target actual still image data RD. to obtain the actual separation distance RL1. Next, at step 107, the first calculation unit 13 determines the display magnification of the virtual moving image data based on the obtained distance SL1 and the obtained actual separation distance RL1. As the display magnification, for example, a value obtained by dividing the actual separation distance RL1 by the distance SL1 may be used.

Next, at step 108, the first calculation unit 13 superimposes the navigation mark NM1 on the target real still image data RD as the virtual moving image data at the display position and the display magnification determined above to perform the first display. It is displayed on the unit 12 (see FIG. 2(b)). Next, at step 109 , the first calculation section 13 detects the position of the mounting section T<b>1 based on the image of the work implement two-dimensional barcode TB captured by the imaging section 11 . Next, at step 110, the first calculation unit 13 determines whether or not the detected position of the mounting portion T1 thus detected matches the position of the navigation mark NM1. If the determination in step 110 is "Yes", the process immediately proceeds to step 111, and the first calculation unit 13 erases the displayed navigation mark NM1 after a predetermined period of time has elapsed.

On the other hand, in step 110, if the detected position of the mounting portion T1 does not match the position of the navigation mark NM1, a determination of "No" is made in step 110, and the process proceeds to step 112. At step 112 , the first calculation unit 13 issues a warning notification by the warning notification unit 15 . Specifically, a warning sound AL1 is emitted, a warning light AL2 is displayed, and the like (see FIG. 8). Next, at step 111, the first calculation unit 13 erases the displayed navigation mark NM1 after a predetermined period of time has elapsed.

Terminal connector "1" in the flowchart shown in FIG. 12 is connected to terminal connector "1" in the flowchart shown in FIG. Therefore, after the processing at step 111 in FIG. 12, the process proceeds to step 113 in FIG.

At step 113, the first calculation unit 13 superimposes the navigation mark NM2 on the target real still image data RD as virtual moving image data at the display position and display magnification determined above, and displays the data on the first display unit 12. display (see FIG. 3(a)). Next, at step 114 , the first calculation section 13 detects the position of the mounting section T<b>1 based on the image of the work tool two-dimensional barcode TB captured by the imaging section 11 . Next, at step 115, the first calculation unit 13 determines whether or not the detected position of the mounting portion T1 thus detected matches the position of the navigation mark NM2. If "Yes" is determined in step 115, the process immediately proceeds to step 116, and the first calculation unit 13 erases the displayed navigation mark NM2 after a predetermined period of time has elapsed.

On the other hand, in step 115, if the detected position of the mounting portion T1 does not match the position of the navigation mark NM2, a determination of "No" is made in step 115, and the process proceeds to step 117. At step 117 , the first calculation unit 13 issues a warning notification by the warning notification unit 15 . Specifically, a warning sound AL1 is emitted, a warning light AL2 is displayed, and the like (see FIG. 8). Next, at step 116, the first calculation unit 13 erases the displayed navigation mark NM2 after a predetermined period of time has elapsed.

Next, at step 118, the first calculation unit 13 superimposes the navigation mark NM3 on the target real still image data RD as the virtual moving image data at the display position and the display magnification determined as described above for the first display. It is displayed on the part 12 (see FIG. 3(b)). Next, at step 119 , the first calculation section 13 detects the position of the mounting section T<b>1 based on the image of the work implement two-dimensional barcode TB captured by the imaging section 11 . Next, at step 120, the first calculation unit 13 determines whether or not the detected position of the mounting portion T1 thus detected matches the position of the navigation mark NM3. If the determination at step 120 is "Yes", the process immediately proceeds to step 121, and the first calculation unit 13 erases the displayed navigation mark NM3 after a predetermined period of time has elapsed.

On the other hand, in step 120, if the detected position of the mounting portion T1 does not match the position of the navigation mark NM3, a determination of "No" is made in step 120, and the process proceeds to step 122. FIG. At step 122 , the first calculation unit 13 issues a warning notification by the warning notification unit 15 . Specifically, a warning sound AL1 is emitted, a warning light AL2 is displayed, and the like (see FIG. 8). Next, at step 121, the first calculation unit 13 erases the displayed navigation mark NM3 after a predetermined period of time has elapsed.

Terminal connector "2" in the flowchart shown in FIG. 13 is connected to terminal connector "2" in the flowchart shown in FIG. Therefore, after the processing at step 121 in FIG. 13, the process proceeds to step 123 in FIG.

At step 123, the first calculation unit 13 superimposes the navigation mark NM4 on the target real still image data RD as virtual moving image data at the display position and display magnification determined above, and displays the data on the first display unit 12. display (see FIG. 4(a)). Next, at step 124 , the first calculation section 13 detects the position of the mounting section T<b>1 based on the imaging of the work tool two-dimensional barcode TB by the imaging section 11 . Next, at step 125, the first calculation unit 13 determines whether or not the detected position of the mounting portion T1 thus detected matches the position of the navigation mark NM4. If the determination at step 125 is "Yes", the process immediately proceeds to step 126, and the first calculation unit 13 erases the displayed navigation mark NM4 after a predetermined period of time has elapsed.

On the other hand, in step 125, if the detected position of the mounting portion T1 does not match the position of the navigation mark NM4, a determination of "No" is made in step 125, and the process proceeds to step 127. FIG. At step 127 , the first calculation unit 13 issues a warning notification by the warning notification unit 15 . Specifically, a warning sound AL1 is emitted, a warning light AL2 is displayed, and the like (see FIG. 8). Next, at step 126, the first calculation unit 13 erases the displayed navigation mark NM4 after a predetermined period of time has elapsed.

Next, in step 128, the first calculation unit 13 superimposes the navigation mark NM5 on the target real still image data RD as virtual moving image data at the display position and display magnification determined above for the first display. It is displayed on the part 12 (see FIG. 4(b)). Next, at step 129 , the first calculation section 13 detects the position of the mounting section T<b>1 based on the image of the work implement two-dimensional barcode TB captured by the imaging section 11 . Next, at step 130, the first calculation unit 13 determines whether or not the detected position of the mounting portion T1 thus detected matches the position of the navigation mark NM5. If the determination at step 130 is "Yes", the process immediately proceeds to step 131, and the first calculation unit 13 erases the displayed navigation mark NM5 after a predetermined period of time has elapsed.

On the other hand, in step 130, if the detected position of the mounting portion T1 does not match the position of the navigation mark NM5, the determination in step 130 is "No", and the process proceeds to step 132. FIG. At step 132 , the first calculation unit 13 issues a warning notification by the warning notification unit 15 . Specifically, a warning sound AL1 is emitted, a warning light AL2 is displayed, and the like (see FIG. 8). Next, at step 131, the first calculation unit 13 erases the displayed navigation mark NM5 after a predetermined period of time has elapsed. Then, the fastening navigation program 100 completes the processing of the first storage unit 14, and the fastening manual navigation ends.

Here, steps 105-107, 108, 111, 113, 116, 118, 121, 123, 126, 128, and 131 correspond to the virtual moving image data display procedure. Steps 109, 114, 119, 124 and 129 correspond to the position detection procedure. Steps 110, 115, 120, 125 and 130 correspond to the position determination procedure. Steps 112, 117, 122, 127 and 132 correspond to the warning notification procedure.

<Processing of navigation video data generation program>
The navigation moving image data generation program 200 applied to the terminal device 20 executes each process of the second calculation unit 23, thereby achieving the generation of the virtual moving image data including the navigation marks NM as described above. be.

As shown in the flowchart of FIG. 15 , when starting the navigation of the fastening manual operation, first, in step 201 , the second calculation unit 23 photographs the reference workpiece SW with the camera, and sends it to the terminal device 20 take in. Next, at step 202, the second calculation unit 23 generates reference real still image data SD from the photographed reference workpiece SW (see FIG. 11). Here, steps 201 and 202 correspond to the reference actual still image data generation procedure.

Next, at step 203 , the second calculation section 23 displays the generated reference real still image data SW on the second display section 21 . Next, at step 204, the second calculation unit 23 receives an instruction from the administrator to generate the navigation mark NM1 via the operation unit 22 while the reference actual still image data SW is being displayed. Next, at step 205, the second calculation unit 23 adds the navigation mark NM1 to the virtual moving image data according to the received generation instruction.

Next, at step 206, the second calculation unit 23 receives an instruction from the administrator to generate the navigation mark NM2 via the operation unit 22 while the reference actual still image data SW is being displayed. Next, at step 207, the second calculation unit 23 adds the navigation mark NM2 to the virtual moving image data in accordance with the received generation instruction.

Next, at step 208, the second calculation unit 23 receives an instruction from the administrator to generate the navigation mark NM3 via the operation unit 22 while the reference actual still image data SW is being displayed. Next, at step 209, the second calculation unit 23 adds the navigation mark NM3 to the virtual moving image data according to the received generation instruction.

Next, in step 210, the second calculation unit 23 receives an instruction from the administrator to generate the navigation mark NM4 via the operation unit 22 while the reference actual still image data SW is being displayed. Next, at step 211, the second calculation unit 23 adds the navigation mark NM4 to the virtual moving image data in accordance with the received generation instruction.

Next, at step 212, the second calculation unit 23 receives an instruction from the administrator to generate the navigation mark NM5 via the operation unit 22 while the reference actual still image data SW is being displayed. Next, at step 213, the second calculation unit 23 adds the navigation mark NM5 to the virtual moving image data in accordance with the received generation instruction.

Next, at step 214, the second calculation unit 23 generates virtual moving image data including navigation marks NM1 to NM5. Then, the navigation moving image data generation program 200 completes the processing of the second calculation unit 23, and the generation of the virtual moving image data ends.

As the generation instructions in steps 204, 206, 208, 210, and 212 described above, navigation is performed so as to correspond to the positions of the fastening required parts BP1 to BP5 in the reference actual still image data SD displayed on the second display unit 21. A mark NM is generated. Furthermore, navigation marks NM1, NM2, NM3, NM4, and NM5 are provided in the reference actual still image data SD corresponding to the order of the fastening required parts BP1, BP2, BP3, BP4, and BP5 in the reference actual still image data SD. are generated and instructed to move in this order and be displayed sequentially (see FIG. 1). Therefore, in the virtual video data generated in step 214, the display positions of the navigation marks NM superimposed on the target real still image data RD in the fastening navigation device 10 correspond to the fastening required points BP1 to BP5. It will be a video that moves by moving.

Here, steps 204, 206, 208, 210, and 212 correspond to an instruction acceptance procedure. Steps 205, 207, 209, 211, 213, and 214 correspond to the virtual moving image data generation procedure.

<Effects of Embodiment>
As described above, according to the fastening navigation program 100 according to the embodiment of the present invention, the worker WR connects the fastening tool T to each of the plurality of fastening points BP1 to BP5 of the target workpiece W one by one. When the manual fastening work is performed sequentially using Virtual moving image data including navigation marks NM1 to NM5 are displayed superimposed on the target real still image data RD. As a result, the order of manual work and the position of manual work to be executed for the plurality of required fastening points BP1 to BP5 during the manual fastening are dynamically navigated.

At this time, the virtual moving image data is the reference workpiece SW (in this embodiment, the same type, same size, and same shape as the target workpiece W) having a plurality of fastening points BP1 to BP5. It is generated by superimposing and displaying it on the display state of the reference actual still image data SD captured in advance. A plurality of reference two-dimensional barcodes SB separated by a known reference distance SL are provided on the reference workpiece SW. That is, in the virtual video data, positions in the data are set so as to point to a plurality of fastening points BP1 to BP5 while using the reference two-dimensional bar codes SB spaced apart from each other by a known distance as the positioning reference. Gate marks NM1 to NM5 are included in advance.

On the other hand, when the worker WR actually executes the fastening work described above, the target workpiece W is also provided with a plurality of target two-dimensional barcodes B, and as a result, the target real still image data RD are spaced apart from each other. A target two-dimensional barcode B is included. As a result, the length ratio between the separation distance RL1 between the plurality of target two-dimensional barcodes B in the target real still image data RD and the separation distance SL1 between the reference two-dimensional barcodes SB included in the virtual video data is Accordingly, the navigation mark NM in the virtual moving image data can be positioned in accordance with the target real still image data RD. As a result, the positions of the navigation marks NM superimposed and displayed as a moving image on the target real still image data RD are made to indicate the positions of the plurality of fastening points BP of the target workpiece W in the target real still image data RD. , can be matched with good accuracy.

As described above, according to the present embodiment, the fastening position and the fastening order can be dynamically displayed using the accurately positioned marks (navigation marks NM1 to NM5) in the virtual moving image. , the operator WR can navigate in an easy-to-understand manner, and the operator WR can perform fastening work without mistakes.

Further, according to the navigation moving image data generation program 200 according to the embodiment of the present invention, the terminal device 20 accepts an instruction to generate the navigation mark NM, and displays the reference real still image data SD photographed in advance. Virtual video data is generated so as to be superimposed on it. Therefore, in the terminal device 20, for example, the navigation marks NM can be easily generated so as to correspond to the positions of the fastening required parts BP1 to BP5 in the reference real still image data SD. The navigation marks NM1, NM2, NM3, NM4, and NM5 move in this order and are displayed sequentially in the reference actual still image data SD corresponding to the order of the fastening required parts BP1, BP2, BP3, BP4, and BP5. Videos can be easily generated. In addition, virtual moving image data can be generated in advance by a manager or the like other than the worker WR, separately from the fastening manual work. Therefore, for example, virtual moving image data generated in advance can be efficiently deployed in a large number of fastening navigation devices 10 .

Further, in the fastening navigation program 100 of the present embodiment, as described above, the worker WR moves the target workpiece W at the plurality of fastening points BP according to the navigation marks NM in the virtual moving image. When manual fastening is performed, the position of the fastening tool T is detected by a position detection procedure. Then, in the subsequent position determination procedure, it is determined whether or not the detected position of the fastening work tool T matches the position of the navigation mark NM. As a result, if the fastening tool T moved by the worker WR is in an inappropriate position, the worker WR can be corrected by any means, or the worker WR can be urged to do so.

Further, in the fastening navigation program 100 of the present embodiment, particularly when it is determined in the position determination procedure that the position of the fastening work tool T does not match the position of the navigation mark NM, a warning sound is issued as a corresponding warning notification. A warning notification procedure is executed to issue AL1 or to display warning light AL2. As a result, the worker WR may mistakenly move the fastening work tool T to a fastening point BP that is not the next fastening point BP, or move the fastening work tool T to a position significantly deviated from the fastening point BP. When T is moved, a predetermined warning notification is issued in the warning notification procedure, and the operator WR can be made aware of the error.

Further, in the fastening navigation program 100 of the present embodiment, the fastening work tool T is provided with the work tool two-dimensional barcode TB, and the position of the fastening work tool T is detected based on the work tool two-dimensional barcode TB. will be Therefore, the position detection of the fastening work tool T in the above-described position detection procedure can be performed quickly with relatively simple processing.

At that time, if the work tool two-dimensional barcode TB is provided on the attachment part T1 where the bolt or nut is attached and the grip part T2 gripped by the worker WR in the fastening work tool T, it is possible to There is a risk that the work implement two-dimensional bar code TB may become dirty or worn due to friction, friction by the hand of the worker WR, or the like. In order to avoid this, it is necessary to provide working implement two-dimensional barcodes on portions other than the mounting portion T1 and gripping portion T2. By providing it at the position, the mounting part T1 can be positioned with reference to the plurality of work tool two-dimensional barcodes TB. That is, during manual fastening by the worker WR, by detecting the positions of the plurality of working tool two-dimensional barcodes TB in the position detection procedure, the position of the mounting portion T1 of the fastening tool T, in other words, the worker WR can be detected. It is possible to detect the position where a bolt or nut is to be tightened. As a result, it is possible to accurately perform the determination in the above-described position determination procedure and the warning notification in the warning notification procedure based on the determination result.
If the fastening work tool T is, for example, a screwdriver, by providing only one work tool two-dimensional barcode TB at the end of a gripping portion (not shown) gripped by the worker WR, The position may be detected directly.

<Modification>
In the above-described embodiment, the first calculation unit 13 of the fastening navigation device 10 executes the fastening manual navigation process by the fastening navigation program 100 , and the navigation moving image data generation program 200 performs the The 2 calculation unit 23 executes processing for generating virtual moving image data. Instead of this, for example, the first calculation unit 13 and the second calculation unit 23 may perform the navigation processing of the fastening manual operation and the generation processing of the virtual video data, respectively, according to the fastening work support program. . In this case, the fastening work support program is applied to a fastening work support system having the fastening navigation device 10 and the terminal device 20, and the first calculation unit 13 and the second calculation unit 23 are provided with a plurality of fastening required parts. A reference actual still image data generation procedure for photographing a reference workpiece SW provided with a plurality of reference two-dimensional barcodes SB provided with a BP and spaced apart by a known reference distance, and generating corresponding reference actual still image data SD. Then, in a state where the reference actual still image data SD is displayed, a fastening work tool T is attached to each of a plurality of fastening required locations BP to be displayed superimposed on the displayed reference actual still image data SD. An instruction reception procedure for receiving, through the operation unit 22, an instruction to generate a navigation mark NM for navigating the manual work order and the manual work position when the fastening manual work is sequentially performed using Virtual moving image data generation for generating virtual moving image data including navigation marks NM for navigating the manual fastening of the target workpiece W by the worker WR using the fastening work tool T when the generation instruction of the NM is received. A procedure and real target still image data for generating corresponding real target still image data RD by imaging a target work W provided with a plurality of target two-dimensional barcodes B and arranged on a workbench WT by an imaging unit 11. a generation procedure, a target real still image data display procedure for displaying the generated target real still image data RD, and virtual video data for the target real still image data RD displayed in the target real still image data display procedure. While being superimposed, the display position using the plurality of target two-dimensional barcodes B in the target real still image data RD as the positioning reference, and the display position between the plurality of target two-dimensional barcodes B in the target real still image data RB. and a procedure of displaying virtual moving image data by display magnification with the separation distance as a scale reference.

Further, in the above embodiment, the target two-dimensional barcode B (for example, QR code (registered trademark)) functions as a positioning reference, and the actual separation distance RL1 between the target two-dimensional barcodes B is the scaling reference It is designed to function as Based on the ratio between the actual separation distance RL1 and the distance SL1 between the reference two-dimensional barcode SB (for example, QR code (registered trademark)) at the time of generating the virtual video data, navigation is performed during the target real still image data RD. Since the gate mark NM is displayed, when the magnification when the reference workpiece SW is imaged by the camera and the magnification when the target workpiece W is imaged by the imaging unit 11 are different from each other (that is, each Even if the performance of each camera is different), the position of the navigation mark NM can be accurately matched with the fastening required part BP in the target real still image data RD.

Further, predetermined information may be stored in the target two-dimensional barcode B of the target work W and the reference two-dimensional bar code SB of the reference work SW. As a result, for example, by confirming whether or not the target two-dimensional barcode B and the reference two-dimensional barcode SB are the same, the virtual video data used when navigating the manual fastening operation can be obtained. , can be checked for suitability. Further, predetermined information may also be stored in the work tool two-dimensional barcode TB (for example, QR code (registered trademark)) of the fastening work tool T. FIG. Specifically, for example, distance information from the working tool two-dimensional barcode TB to the center position of the mounting part T1 may be stored in the working tool two-dimensional barcode TB on the mounting part T1 side. Information corresponding to the distance between two work implement two-dimensional barcodes TB (for example, a number obtained by multiplying the distance between two work implement two-dimensional barcodes TB by n) may be stored. Also by this, the position of the mounting part T1 can be accurately detected.

It should be noted that, in the above description, descriptions such as "the same", "equal", and "different" regarding external dimensions and sizes are not strictly defined. That is, the terms "same", "equal", and "different" mean "substantially the same", "substantially equal", and "substantially different", with allowance for design and manufacturing tolerances and errors. .

In addition to the methods already described above, the methods according to the above-described embodiments and modifications may be appropriately combined and used.

In addition, although not exemplified one by one, the present invention can be implemented with various modifications within the scope of the invention.

REFERENCE SIGNS LIST 10 fastening navigation device 11 imaging unit 12 first display unit 13 first calculation unit 15 warning notification unit 20 terminal device 21 second display unit 22 operation unit 23 second calculation unit 100 fastening navigation program 102 step (target actual still image data generation procedure)
103 step (procedure for generating target actual still image data)
104 Step (Procedure for Displaying Object Actual Still Image Data)
105 step (virtual video data display procedure)
106 steps (virtual video data display procedure)
107 step (virtual video data display procedure)
108 Step (Virtual Video Data Display Procedure)
109 Step (position detection procedure)
110 steps (position determination procedure)
111 step (virtual video data display procedure)
112 steps (warning notification procedure)
113 Step (Virtual Video Data Display Procedure)
114 steps (position detection procedure)
115 steps (position determination procedure)
116 steps (virtual video data display procedure)
117 steps (warning notification procedure)
118 steps (virtual video data display procedure)
119 steps (position detection procedure)
120 steps (position determination procedure)
121 steps (virtual video data display procedure)
122 steps (warning notification procedure)
123 Step (Virtual Video Data Display Procedure)
124 steps (position detection procedure)
125 steps (position determination procedure)
126 steps (virtual video data display procedure)
127 steps (warning notification procedure)
128 steps (virtual video data display procedure)
129 steps (position detection procedure)
130 steps (position determination procedure)
131 steps (virtual video data display procedure)
132 steps (warning notification procedure)
200 Navigation video data generation program 204 Step (instruction reception procedure)
205 Step (virtual video data generation procedure)
206 Step (instruction reception procedure)
207 Step (virtual video data generation procedure)
208 Step (Instruction Acceptance Procedure)
209 Step (virtual video data generation procedure)
210 Step (Instruction Acceptance Procedure)
211 Step (virtual video data generation procedure)
212 Steps (Instruction Acceptance Procedure)
213 Step (virtual video data generation procedure)
214 Step (virtual video data generation procedure)
AL1 Warning sound AL2 Warning light B Target 2D barcode BP Fastening point NM Navigate mark RD Target actual still image data RL1 Actual separation distance SB Reference 2D barcode SL Reference distance SL1 Distance SD Reference actual image data SW Reference stationary object Workpiece T Tightening work tool T1 Applied part TB Work tool two-dimensional barcode W Target work piece WR Worker WT Workbench

Claims (4)

Imaging unit of the fastening work support systemand the first display unitFastening navigation device withand the second display unitA terminal device equipped with a computing unit respectivelyA fastening work support program that is executed in and navigates a worker who performs manual fastening on a target work piece placed on a workbench using virtual video data including predetermined navigation marks There is
The second display unit of the terminal device displays the virtual video data, which has a plurality of fastening points, is the same type, same size, and same shape as the target work object, and is separated by a known reference distance. a reference workpiece provided with a plurality of reference two-dimensional barcodes for positioning is superimposed on the display state of the reference actual still image data taken in advance before the fastening manual operation by the worker; The computing unit provided in each of the fastening navigation device and the terminal device of the fastening work support system against
Said Take a picture of the reference workpiece and respondSaida reference actual still image data generation procedure for generating reference actual still image data;
Said Reference actual still image dataon the second display unit of the terminal deviceIn the displayed state, superimpose it on the displayed reference real still image data.on the second displayFor displaying, using a fastening work tool for each of the plurality of fastening required locationsSaidNavigating the manual work order and manual position when performing fastening manual work sequentiallySaidan instruction reception procedure for receiving an instruction to generate a navigation mark via the operation unit;
When an instruction to generate the navigation mark is received in the instruction receiving procedure,Saidby workerSaidUsing the fastening work tool for the target workSaidincluding navigation marks for navigating the fastening handSaida virtual video data generation procedure for generating virtual video data;
for providing positioning references for the navigation marks based on the plurality of reference two-dimensional barcodes; Multiple target two-dimensional barcodes are providedand saidThe target work piece placed on the workbenchof the fastening navigation devicea target real still image data generation procedure for capturing an image by the imaging unit and generating corresponding target real still image data;
The generated target real still image dataon the first display unit of the fastening navigation devicea procedure for displaying target real still image data to be displayed;
A virtual animation including the navigation mark for navigating the manual work order and the manual work position when the worker sequentially performs the manual fastening work using the fastening work tool one by one for the plurality of required fastening locations. generated by the data generation procedure The plurality of target two-dimensional barcodes in the target real still image data are used as a positioning reference while the virtual moving image data is superimposed on the target real still image data displayed in the target real still image data display procedure. by the display position and the display magnification with the actual separation distance between the plurality of target 2D barcodes in the target real still image data as the scaling standardon the first display unit of the fastening navigation devicedisplayand navigating the fastening manual work by the worker using the fastening work tool for the target work piecea virtual video data display procedure for
A fastening work support program characterized by sharing and executing. Fastening according to claim 1Work supportIn the program
said of said fastening navigating device; In addition to the operation part,
a position detection procedure for detecting the position of the fastening tool when the manual fastening operation is performed on the target workpiece, based on the imaging result of the imaging unit;
A position for determining whether or not the position of the fastening tool detected by the position detection procedure matches the position of the navigation mark in the virtual video data displayed by the virtual video data display procedure. judgment procedure;
A conclusion characterized by executingWork supportprogram. Fastening according to claim 2Work supportIn the program
said of said fastening navigating device; In addition to the operation part,
A warning notification procedure for executing a corresponding warning notification when it is determined in the position determination procedure that the position of the fastening tool does not match the position of the navigation mark in the virtual moving image data.
A conclusion characterized by executingWork supportprogram. In the fastening work support program according to claim 2 or claim 3,
In the position sensing procedure,
Bolts or nuts to be fastened to the required fastening points of the target workpiece among the fastening work tools based on the detection result using a plurality of work tool two-dimensional barcodes provided in advance on the fastening work tools as a positioning reference. A fastening work support program characterized by detecting the position of a mounting part to which a is mounted .

Images