JP6977481B2 - Manipulator device - Google Patents

Description

The present invention relates to a manipulator device.

Conventionally, as a manipulator device, for example, as described in Japanese Patent No. 4558682, a manipulator used for a mobile robot, equipped with a camera, while checking an image captured by the camera with the manipulator. Those that can operate manipulators are known.

Japanese Patent No. 4558682

In the manipulator described above, it is possible to operate the manipulator while checking the work status of the manipulator with an image, but there is a possibility that the manipulator cannot be operated accurately depending on the work content. For example, when the manipulator receives torque from the outside, the torque cannot be recognized. For this reason, it is difficult to grasp how to operate the manipulator accurately, and the manipulator cannot be operated accurately.

Therefore, in the manipulator device, it is desired to develop a device that can accurately operate the manipulator in consideration of the torque received from the outside.

Therefore, the manipulator device according to one aspect of the present invention displays a manipulator having at least one joint, a torque detection unit that detects the torque that the manipulator receives from the outside, a camera that captures the manipulator, and an image captured by the camera. An image signal is output to the display unit to control the display, and an icon indicating the direction of torque detected by the torque detection unit during operation of the manipulator is displayed on the display unit together with the image of the manipulator . It is provided with a display control unit that displays a sphere on the display unit together with the icon to facilitate recognition of the torque direction of the icon, and the display unit displays an arrow that curves according to the torque direction as an icon. It is configured in. According to this manipulator device, an icon indicating the direction of the torque received from the outside by the manipulator is displayed on the display unit together with the image of the manipulator during the operation of the manipulator. Therefore, the manipulator operator can recognize the operation status of the manipulator by looking at the torque icon. Therefore, the manipulator can be operated accurately. Further, in the manipulator device according to one aspect of the present invention, the display unit displays an arrow that curves according to the direction of torque as an icon. As a result, an arrow that curves according to the direction of torque is displayed as an icon, so that the operator can quickly recognize the direction of torque that the manipulator receives from the outside. Therefore, the manipulator can be operated accurately by looking at the arrow displayed on the display unit.

Further, in the manipulator device according to one aspect of the present invention, the display unit may display an arrow larger as the torque is larger. In this case, the operator of the manipulator can recognize the magnitude of the torque by looking at the arrow indicating the torque. Therefore, the operator can recognize that the operation of the manipulator is appropriate by operating the manipulator and making the arrow smaller, and the operation can be performed smoothly and accurately.

Further, in the manipulator device according to one aspect of the present invention, the display unit may display an arrow longer as the torque is larger. In this case, the operator of the manipulator can recognize the magnitude of the torque by looking at the arrow. Therefore, the operator can recognize that the operation of the manipulator is appropriate by operating the manipulator and shortening the arrow, and can perform a smooth and accurate operation.

According to the present invention, the manipulator can be accurately operated even in the work of receiving torque from the outside.

FIG. 1 is a schematic configuration diagram showing a manipulator device according to an embodiment of the present invention. FIG. 2 is a block diagram showing an electrical configuration in the manipulator device of FIG. FIG. 3 is an explanatory diagram of an image display of the manipulator device of FIG. FIG. 4 is an explanatory diagram of an image display of the manipulator device of FIG. FIG. 5 is an explanatory diagram of an image display of the manipulator device of FIG. FIG. 6 is a flowchart showing a display control process in the manipulator device of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. In the following description, the case where the present invention is mounted on a traveling moving body will be described.

FIG. 1 is a schematic configuration diagram of a manipulator device 1 according to an embodiment of the present invention.

As shown in FIG. 1, the manipulator device 1 includes a mobile body 20 on which the manipulator 23 is mounted, and an operation unit 10 for remotely controlling the manipulator 23. This manipulator device 1 is used when the work is remotely controlled in a place where no one can enter. Since the manipulator device 1 can remotely control the manipulator 23, it is possible to ensure the safety of the operator of the manipulator device 1 even when performing work in a place where no one can enter.

The moving body 20 is a traveling moving body having an endless track on the main body 21. The moving body 20 is suitable for traveling on rough terrain by providing an endless track such as a crawler. As the moving body 20, a traveling moving body provided with wheels may be used. A manipulator 23 is attached to the main body 21 of the moving body 20. The manipulator 23 is attached to, for example, the front part of the moving body 20, so that work can be performed on the front part of the moving body 20. As the manipulator 23, one having at least one joint is used. In this embodiment, a 6-axis manipulator having 6 joints is used. For example, the manipulator 23 can rotate around a vertical axis with respect to the main body 21, connects two links with two joints, and can rotate each link around a horizontal axis. On the other hand, it can rotate about the vertical axis, the horizontal axis, and the longitudinal axis, respectively.

The grip portion 231 is configured by a mechanism for gripping the work object W. For example, the grip portion 231 has a plurality of finger portions 231a, and is configured to be able to grip and release the work object W by opening and closing the finger portions 231a. A force sensor 25 is provided on the base end side of the grip portion 231. The force sensor 25 is a sensor that detects the torque received from the outside by the grip portion 231. As the force sensor, for example, a sensor capable of detecting a three-axis force component (Fx, Fy, Fz) and a three-axis moment component (Mx, My, Mz) is used.

A camera 22 is attached to the main body 21 of the moving body 20. The camera 22 is an imaging device that captures an image of the manipulator 23. The camera 22 is attached toward the work area of the manipulator 23 and captures the work situation of the manipulator 23. In FIG. 1, the camera 22 is attached toward the front so that the area in front of the moving body 20 can be imaged. The main body 21 of the moving body 20 is provided with a traveling drive unit 26, a communication unit 27, and a control unit 28 (see FIG. 2). Details of these will be described later.

As shown in FIG. 1, the operation unit 10 includes a display unit 11 and a main body unit 12. The display unit 11 is a display device for displaying an image captured by a camera, and for example, a display having a rectangular display area is used. The display unit 11 displays the work status of the manipulator 23 imaged by the camera 22. The image captured by the camera 22 may be a moving image or a still image. The main body 12 is provided with an operation input unit 13 such as an operation lever of the manipulator 23. Further, a communication unit 14 and a control unit 15 are provided in the main body unit 12 (see FIG. 2). Details of these will be described later.

The operation unit 10 and the mobile body 20 can communicate wirelessly. The manipulator 23 of the moving body 20 can be operated by the operation input of the operation unit 10. The operation unit 10 and the mobile body 20 may be electrically connected by a linear member such as a cable so that wired communication can be performed between the operation unit 10 and the mobile body 20. Further, although FIG. 1 shows a case where the main body 12 of the operation unit 10 is a stationary type, the main body 12 may be a handy type that can be picked up and operated by the operator. In this case, the display unit 11 may be an installation type, or may be attached to the main body unit 12 and the entire operation unit 10 may be configured as a handy type.

FIG. 2 is a block diagram showing an outline of the electrical configuration of the manipulator device 1.

As shown in FIG. 2, the operation unit 10 includes a display unit 11, an operation input unit 13, a communication unit 14, and a control unit 15. The control unit 15 is a controller that executes each control in the operation unit 10, and is, for example, a hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and a ROM. A computer composed of software such as a stored program is used. The control unit 15 includes an operation control unit 151, a travel control unit 152, a display control unit 153, and a torque detection unit 154. The operation control unit 151 controls the operation of the manipulator 23. The operation control unit 151 generates an operation control signal based on the operation signal of the operation input unit 13, and outputs the operation control signal to the communication unit 14. The travel control unit 152 controls the travel of the moving body 20. The travel control unit 152 generates a travel control signal based on the travel operation input of the operation input unit 13, and outputs the travel control signal to the communication unit 14.

The display control unit 153 inputs image data from the communication unit 14, generates an image signal based on the image data, and outputs the image signal to the display unit 11. The display control unit 153 causes the display unit 11 to display an icon indicating the direction of the torque detected by the torque detection unit 154 during the operation of the manipulator 23 together with the image of the manipulator 23. For example, as shown in FIG. 3, the display control unit 153 causes the display unit 11 to display the icon A together with the image of the manipulator 23 based on the image data and the torque data received from the outside by the manipulator 23. Here, the torque received from the outside by the manipulator 23 is the anti-torque received from the outside by operating the manipulator 23. Therefore, by detecting and displaying the torque received from the outside by the manipulator 23, it is possible to recognize whether or not the operation and operation of the manipulator 23 are appropriate.

The icon A indicating the direction of torque is, for example, an arrow having a curvature and curving according to the direction of torque. The orientation of the icon A is determined and displayed based on the torque data detected by the force sensor 25. The icon A may be displayed based on the torque applied to the grip portion 231 of the manipulator 23 as shown in the display unit 11 of FIG. 3, or may be displayed by the grip portion 231 as shown in the display unit 11 of FIG. It may be displayed according to the torque applied to the work object W to be operated.

The operator of the manipulator 23 can recognize the torque that the manipulator 23 receives from the outside by displaying the icon A on the display unit 11. Further, the operator can smoothly perform the work by operating the manipulator 23 so that the icon A becomes smaller.

As shown in FIG. 4, the display unit 11 may display the sphere B together with the icon A. For example, the sphere B is displayed near the inside of the display area of the icon A, and is a sphere-like circle concentric with the icon A. Hereinafter, the sphere B is a circle displayed like a sphere, but is described as a sphere. The sphere B is a display body for making it easy to recognize the direction of the icon A. That is, by displaying this sphere B, the operator can easily recognize the angle of the icon A, that is, the angle of torque. Even if the sphere B is not displayed, the angle of the icon A can be recognized by the curvature of the arrow of the icon A. However, since the sphere B is displayed, even an operator who is unfamiliar with the manipulator device 1 can easily determine whether the axis of torque indicated by the icon A is vertical or slanted with respect to the screen of the display unit 11. It can be recognized, and the angle of the icon A can be easily recognized.

The sphere B may be displayed as translucent, or only the outline may be displayed. By displaying in this way, it is possible to prevent the display of the work status of the manipulator 23 from becoming difficult to see.

As shown in FIG. 5, the display unit 11 may display the icon A by changing the size according to the torque applied to the manipulator 23. For example, the icon A may be displayed larger as the torque applied to the manipulator 23 is larger, and may be displayed smaller as the torque is smaller. Specifically, the icon A may be displayed longer as the torque applied to the manipulator 23 is larger, and may be displayed shorter as the torque is smaller. Further, the icon A may be displayed thicker as the torque applied to the manipulator 23 is larger, and may be displayed thinner as the torque is smaller. In this way, by displaying the icon A larger as the torque is larger, the work can be properly performed by operating the manipulator 23 so that the icon A becomes smaller, so that the work by the manipulator 23 can be smoothly performed. can.

In FIG. 2, the torque detection unit 154 detects the torque received from the outside by the manipulator 23. For example, the torque detection unit 154 calculates the torque received from the outside by the manipulator 23 based on the torque data input from the communication unit 14. The torque data input from the communication unit 14 is torque data detected by the force sensor 25, and is communicated to the operation unit 10 side via the control unit 28 and the communication unit 27.

The display unit 11 is connected to the control unit 15 and displays an image according to an image signal output from the display control unit 153. As described above, the display unit 11 is a display device for displaying an image captured by the camera 22, and for example, a display having a display area is used. The operation input unit 13 is an input unit for operating the manipulator 23 and operating the traveling body 20, and is composed of, for example, an operation lever, an operation switch, an operation button, and the like. The communication unit 14 is a communication device for communicating with the mobile body 20. The communication unit 14 is a communication device for communicating with the communication unit 27 of the mobile body 20. The communication unit 14 transmits a control signal or the like output from the control unit 15 toward the mobile body 20. Further, the communication unit 14 receives image data, detection data, and the like transmitted from the mobile body 20 and inputs them to the control unit 15.

The moving body 20 includes a camera 22, an actuator 24, a force sensor 25, a traveling drive unit 26, a communication unit 27, and a control unit 28. The control unit 28 is a controller that executes each control in the mobile body 20, and for example, a computer composed of hardware such as a CPU, ROM, and RAM, and software such as a program stored in the ROM is used. .. The control unit 28 includes an operation control unit 281, a travel control unit 282, and an image control unit 283.

The operation control unit 281 controls the operation of the manipulator 23. The operation control unit 151 receives an operation control signal from the communication unit 27 and outputs an operation control signal to the actuator 24. As a result, the actuator 24 is driven according to the operation control signal, and the manipulator 23 operates in response to the operation input.

The travel control unit 282 controls the travel of the moving body 20. The travel control unit 282 receives a travel control signal from the communication unit 27 and outputs a travel control signal to the travel drive unit 26. As a result, the traveling drive unit 26 is driven according to the traveling control signal, and the moving body 20 travels in response to the operation input.

The image control unit 283 executes operation control and the like of the camera 22. The image control unit 283 controls the operation of the camera 22, inputs the image data output from the camera 22, and outputs the image data to the communication unit 27.

The camera 22 is connected to the control unit 28 and outputs the captured image to the control unit 28. As the camera 22, any image pickup device may be used as long as it can capture the work status of the manipulator 23 and output image data. The actuator 24 is an actuator that rotationally operates the joint of the manipulator 23, and for example, a motor is used. The actuator 24 is provided according to the number of joints installed. The actuator 24 is connected to the control unit 28 and operates in response to an operation control signal output from the control unit 28.

The force sensor 25 is connected to the control unit 28 and outputs a detection signal to the control unit 28. The force sensor 25 is a sensor that detects the torque received from the outside by the manipulator 23, and is capable of detecting moment components (Mx, My, Mz) of at least three axes. The traveling drive unit 26 is a portion for driving the moving body 20 to travel, and for example, a driving machine such as an engine or a motor is used. The travel drive unit 26 is connected to the control unit 28 and drives according to a travel control signal output from the control unit 28. The communication unit 27 transmits a control signal or the like output from the control unit 28 toward the operation unit 10. Further, the communication unit 27 receives an operation control signal, a travel control signal, and the like transmitted from the operation unit 10 and inputs them to the control unit 28.

Next, the operation of the manipulator device 1 according to the present embodiment will be described.

In FIG. 1, when the operator of the manipulator device 1 operates the operation input unit 13 of the operation unit 10, the moving body 20 travels and the manipulator 23 operates. For example, the moving body 20 travels to the working position of the manipulator 23 and stops traveling at the working position. Then, the manipulator 23 operates in response to the operation of the operation input unit 13 to execute a predetermined operation. FIG. 1 shows a case where the manipulator 23 performs a peg-in hole (pin insertion) operation for inserting a work object W into a hole. Further, the camera 22 captures the work status of the manipulator 23, and the image captured by the camera 22 is transmitted to the operation unit 10 by wireless communication and displayed on the display unit 11.

In this peg-in-hole work, it is necessary to align the direction of the work object W with the direction of the hole while aligning the opening position of the hole with the tip position of the work object W, so that the remote control of the manipulator 23 is not easy.

In the manipulator device 1 according to the present embodiment, the display unit 11 displays an icon A indicating the direction of the torque received from the outside by the manipulator 23 together with the manipulator 23. Therefore, the operator of the manipulator device 1 can easily recognize that the direction of the work object W is deviated from the direction of the hole by looking at the icon A. Therefore, the operation of the manipulator device 1 can be modified to perform proper work.

FIG. 6 is a flowchart showing a display control process of the manipulator device 1. The series of control processes of FIG. 6 is started, for example, by starting the operation of the manipulator device 1, and is executed by the control unit 15.

First, as shown in step S10 of FIG. 6 (hereinafter, simply referred to as “S10”; the same applies to the steps after S10), the image input process is performed. The image input process is a process of reading image data captured by the camera 22. For example, the control unit 15 inputs image data captured by the camera 22 via the communication unit 14, and reads the image data at any time.

Then, the process shifts to S12, and the torque detection process is performed. The torque detection process is a process of calculating the torque received from the outside by the manipulator 23. For example, the control unit 15 inputs a detection signal detected by the force sensor 25 via the communication unit 14, and calculates the torque received from the outside by the manipulator 23 based on the detection signal.

Then, the process shifts to S14, and the display control process is performed. The display control process is a process of displaying the work status of the manipulator 23 on the display unit 11 and displaying the icon A indicating the direction of the torque received from the outside. For example, the control unit 15 generates an icon A indicating the direction of the torque based on the torque calculated by the torque detection process of S12, and causes the display unit 11 to display the icon A together with an image showing the working status of the manipulator 23. .. As a result, as shown in FIG. 3, an icon A indicating the torque received from the outside by the manipulator 23 is displayed together with the work status. By looking at this image, the operator can recognize that the manipulator 23 receives torque from the outside, and can also recognize the direction of torque and the angle of torque. Therefore, the operation of the manipulator 23 can be easily modified.

At this time, as shown in FIG. 4, when the sphere B is displayed together with the torque icon A, the operator can easily recognize the direction of the torque. That is, by displaying the sphere B together with the icon A, it becomes easy to recognize at what angle the icon A is displayed, and it becomes easy to recognize the direction of the torque.

Further, as shown in FIG. 5, when displaying the size of the icon A according to the size of the torque applied to the manipulator 23, the operator can easily determine the size of the torque by looking at the size of the icon A. Can be recognized. Further, the operator can recognize that the correction operation is correct when the torque icon A becomes smaller when the operation of the manipulator 23 is corrected. Further, the operator can recognize that the correction operation is incorrect when the torque icon A becomes large when the operation of the manipulator 23 is corrected. Therefore, the correction operation of the manipulator 23 can be appropriately performed. Therefore, the work using the manipulator 23 can be smoothly performed, and the work efficiency can be improved.

Then, the process shifts to S16 of FIG. 6, and it is determined whether or not the end condition of the control process is satisfied. The end condition of the control process is an end condition of a series of control processes shown in FIG. For example, the case where the end condition of the control process is satisfied corresponds to the case where the power of the manipulator device 1 is turned off or the case where the work of the manipulator 23 is completed. If the control unit 15 determines in S16 that the end condition of the control process is not satisfied, the process returns to S10. On the other hand, in S16, when the control unit 15 determines that the end condition of the control process is satisfied, the series of control processes of FIG. 6 is terminated.

As described above, according to the manipulator device 1 according to the present embodiment, the icon A indicating the direction of the torque received from the outside by the manipulator 23 is displayed on the display unit 11 together with the image of the manipulator 23 during the operation of the manipulator 23. .. Therefore, the operator of the manipulator 23 can easily recognize the operation status of the manipulator 23 by looking at the icon A. Therefore, the manipulator 23 can be operated accurately.

For example, if the icon A is not displayed on the display unit 11, the operator may not know how to operate the manipulator 23 when the work of the manipulator 23 does not proceed well. On the other hand, in the manipulator device 1 according to the present embodiment, by displaying the icon A on the display unit 11, the operator can know the torque applied to the manipulator 23 and accurately recognize the operation status of the manipulator 23. be able to. Therefore, the operation of the manipulator 23 can be easily modified, and even if the work is difficult, the work can proceed smoothly.

Further, for example, when displaying the external force applied to the manipulator 23 instead of the torque icon A on the display unit 11, it is difficult to properly operate the manipulator 23. That is, even if the external force applied to the manipulator 23 is displayed, it is difficult to recognize in which direction the operation of the manipulator 23 should be corrected, and the manipulator 23 cannot be operated properly. On the other hand, in the manipulator device 1 according to the present embodiment, by displaying the icon A indicating the direction of torque on the display unit 11, the operation of the manipulator 23 can be easily corrected, which is a difficult task. However, the work can proceed smoothly.

Further, according to the manipulator device 1 according to the present embodiment, it is possible to display an arrow that curves according to the direction of torque as the icon A. Therefore, the operator can quickly recognize the direction of the torque and the angle of the torque that the manipulator 23 receives from the outside. Therefore, the operator can accurately operate the manipulator 23 by looking at the icon A.

Further, according to the manipulator device 1 according to the present embodiment, the larger the torque, the larger the icon A can be displayed. Therefore, the operator of the manipulator 23 can recognize the magnitude of the torque by looking at the icon A of the arrow. Therefore, the operator can operate the manipulator 23 to recognize that the operation of the manipulator 23 is appropriate by reducing the size of the arrow icon A, and the operation can be performed smoothly.

Further, according to the manipulator device 1 according to the present embodiment, the larger the torque, the longer the icon A can be displayed. Therefore, the operator of the manipulator 23 can recognize the magnitude of the torque by looking at the icon A of the arrow. Therefore, the operator can operate the manipulator 23 to recognize that the operation of the manipulator 23 is appropriate by shortening the icon A of the arrow, and the operation can be smoothly performed.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. For example, in the above-described embodiment, the case where the traveling operation of the moving body 20 is performed by the operation unit 10 has been described, but the traveling operation of the moving body 20 may be performed by an operation unit or the like different from the operation unit 10. Specifically, a traveling control signal may be transmitted to the moving body 20 by an operation unit or the like different from the operating unit 10, and the moving body 20 may be traveled according to the traveling control signal. Even in this case, the same effect as that of the manipulator device 1 described above can be obtained. That is, during the operation of the manipulator 23, by displaying the icon A of the torque received from the outside by the manipulator 23 on the display unit 11 together with the image of the manipulator 23, the working status of the manipulator 23 can be easily recognized and the operation of the manipulator 23 can be performed. You can do it accurately.

Further, in the above-described embodiment, the case where the work of the peg-in hole is performed as the work of the manipulator 23 has been described, but the work of the manipulator 23 may be applied to the work other than the work of the peg-in hole. For example, the work other than the work of the peg-in hole is the work of turning the doorknob. Even in this case, the same effect as that of the manipulator device 1 described above can be obtained. Further, in the work of turning the doorknob, it is particularly effective to display the direction of torque and the magnitude of torque. That is, when the doorknob is turned by the manipulator 23, the torque applied to the doorknob can be recognized by displaying the torque received by the manipulator 23. Therefore, by looking at the icon A displayed on the display unit 11, the torque applied to the doorknob can be grasped, and the work of turning the doorknob can be smoothly performed.

Further, in the above-described embodiment, the manipulator device 1 using the traveling moving body as the moving body 20 has been described, but the moving body 20 may be a flying body. Specifically, the moving body 20 may be a UAV (Unmanned Aerial Vehicle) such as a drone. Even in this case, the same effect as that of the manipulator device 1 described above can be obtained.

1 Manipulator device 10 Operation unit 11 Display unit 15 Control unit 20 Mobile unit 22 Camera 25 Force sensor 28 Control unit 153 Display control unit 154 Torque detection unit A Icon W Work object

Claims (3)

With a manipulator with at least one joint,
A torque detector that detects the torque that the manipulator receives from the outside,
A camera that captures the manipulator and
A display unit for displaying an image captured by the camera, and a display unit.
The display unit performs output to display control image signals to, an icon indicating the direction of torque detected by the torque detection unit during an operation of the manipulator is displayed on the display unit together with the image of the manipulator, the icon A display control unit for displaying a sphere for facilitating the recognition of the direction of torque on the display unit together with the icon is provided.
The display unit displays, as the icon, an arrow that curves according to the direction of the torque.
Manipulator device. The display unit displays the arrow larger as the torque is larger.
The manipulator device according to claim 1. The display unit displays the arrow longer as the torque increases.
The manipulator device according to claim 1 or 2.

Images