CN114767272A - Human-computer interaction system and method based on operator intention correction - Google Patents

Abstract

The invention relates to a human-computer interaction system and a human-computer interaction method based on operator intention correction, wherein the human-computer interaction system comprises a mechanical arm, a surgical tool, a six-dimensional force sensor, a force sensor and a controller; the six-dimensional force sensor is arranged at the tail end of the mechanical arm, the surgical tool is arranged on the six-dimensional force sensor through the fixing device, and the force sensor is arranged on the surgical tool; when an operator pulls a part, which is provided with a force sensor, on the surgical tool to operate, whether the reading of the force sensor reaches a preset threshold value is detected; if not, the mechanical arm does not move, otherwise, the moment read by the six-dimensional force sensor is corrected according to the reading of the force sensor and the position relation between the six-dimensional force sensor and the force sensor, and the movement of the mechanical arm is controlled according to the force signal of the six-dimensional force sensor and the corrected moment signal. Compared with the prior art, the invention solves the problem that the intention of an operator is inconsistent with the actual motion of the mechanical arm in the man-machine cooperation process by utilizing admittance control.

Description

A human-computer interaction system and method based on operator intention correction

technical field

The invention relates to the field of surgical robots, in particular to a human-computer interaction system and method based on operator intention correction.

Background technique

In the field of surgical robotics, due to the ever-changing intraoperative conditions, preoperative planning may not be applicable, which requires the operator to adjust the surgical plan based on clinical experience and directly manipulate the robotic arm for repositioning and manipulation. For example, in actual clinical scenarios, active surgical robots cannot fully meet the needs of oral and maxillofacial surgery, and passive surgical robots that doctors can continuously adjust the state of the robot during surgery are more needed.

Human-machine collaborative control is a common robot control method that combines the high-precision control of the robot with the high-dynamic decision-making ability of the human through the coordinated control based on force-position mixing to realize the robot's collaborative operation with humans. It mainly adopts admittance control or impedance control. Generally, a force sensor is installed at the end of the robot arm. When the operator applies a force at the end, the external force received at the end of the robot arm is mapped to the movement of the robot arm, which can be output by human operation. A quantitative relationship is established between the force and the output motion of the robot to solve the problem of man-machine cooperation control flexibility, so as to achieve the purpose of the doctor's active control of the robot and the maintenance of the doctor's skill level in the surgical situation.

At present, in the field of surgical robots, the common interaction methods include direct traction on the surgical tool installed at the end of the robotic arm and traction on the handle installed with the force sensor.

In the first interaction method, such as the Yomi dental implant robot navigation system developed by the American medical company Neocis, the doctor interacts by directly pulling the surgical tool installed at the end of the robotic arm. However, since the working degree of freedom of the implant drill is only 5 (the rotation of the drill does not need to be considered, only its axis and position need to be determined), the adjustment range of the rotary motion of the implant drill is not large, mainly translational motion.

The second interaction method, related to research by Yucheng He et al. in "Research on Mechanism Design and Safety Control of Assisted Robot for Nasal Endoscopic Surgery", the assistive robot for nasal endoscopic surgery uses two force sensors, one of which is installed on the surgical tool ( Nasal endoscope), which is used to sense the force on the surgical tool during operation; the other is located on the operating handle, which is used for human-machine cooperative control. The doctor controls the robot by pulling the operating handle.

However, the above methods have disadvantages. In the first case, when the surgical tool needs to perform a large rotation and the operator's hand is far away from the sensor position, it will cause the phenomenon that the robot motion is different from the operator's intention. Because there is a displacement between the force application point and the sensor at this time, according to the principle of force translation, although the force in the X, Y, and Z directions will not change, the moment received by the sensor and the force application point is different, resulting in the mechanical arm The problem is that the rotation is different from the operator's intent. For the second case, because the operator only holds the handle to operate, it can be considered that the robotic arm can be executed according to the operator's intention, but the implementation of this solution requires two six-dimensional force sensors to achieve, and the cost is high. Moreover, the six-dimensional force sensor is relatively large, and in many clinical scenarios, it is difficult to install it on the handle of the surgical tool.

To sum up, the existing interactive devices and methods have defects such as poor precision, poor stability, and high technical cost. Therefore, there is a need for a human-computer interaction solution that can realize the movement of the manipulator according to the operator's intention.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a human-computer interaction system and method for correcting based on the operator's intention in order to overcome the above-mentioned defects of the prior art.

The object of the present invention can be realized through the following technical solutions:

In a first aspect, the present invention discloses a human-computer interaction system based on operator intention correction, including a robotic arm, a surgical tool, a six-dimensional force sensor S1, a force sensor S2 and a controller;

The six-dimensional force sensor S1 is installed on the end of the mechanical arm, the surgical tool is installed on the six-dimensional force sensor S1 through a fixing device, the force sensor S2 is installed on the surgical tool, and the controller is connected to the mechanical arm and the six-dimensional force sensor. S1 communicates with the force sensor S2;

When the operator pulls the part where the force sensor S2 is installed on the surgical tool to operate, the controller detects whether the reading of the force sensor S2 reaches the preset threshold; if the reading of the force sensor S2 does not reach the preset threshold, the robotic arm does not move, otherwise, The controller corrects the torque read by the six-dimensional force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six-dimensional force sensor S1 and the force sensor S2. The torque signal controls the motion of the manipulator.

Further, the preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and holding the surgical tool stably.

Further, the positional relationship between the six-dimensional force sensor S1 and the force sensor S2 is determined according to the mechanical dimensions of the robotic arm, the surgical tool and the fixing device.

Further, correcting the torque of the six-dimensional force sensor S1 according to the reading of the force sensor S2 is as follows:

为六维力传感器S1中心O_S与力传感器S2中心O_Doc的位置向量，由六维力传感器S1与力传感器S2之间的位置关系确定，F为六维力传感器S1读到的力，T为六维力传感器S1读到的力矩，T_S为六维力传感器S1校正后的力矩。

is the position vector of the center O _S of the six-dimensional force sensor S1 and the center O _Doc of the force sensor S2, determined by the positional relationship between the six-dimensional force sensor S1 and the force sensor S2, F is the force read by the six-dimensional force sensor S1, T is the torque read by the six-dimensional force sensor S1, and T _S is the corrected torque of the six-dimensional force sensor S1.

Further, the controller converts the force signal of the six-dimensional force sensor S1 and the corrected torque signal into the motion signal of the robotic arm according to the admittance control, the robotic arm moves according to the motion signal, and the motion of the robotic arm drives the surgical tool to move. .

Further, the six-dimensional force sensor S1 is fixedly installed on the end of the mechanical arm, the fixing device is a clamp, and the surgical tool is installed on the six-dimensional force sensor S1 through the clamp.

Further, the force sensor S2 is installed on the handle of the surgical tool, and the operator pulls the handle of the surgical tool to operate.

In a second aspect, the present invention discloses a human-computer interaction method based on operator intention correction, comprising the following steps:

Check the robotic arm, the surgical tool, the six-dimensional force sensor S1, the force sensor S2 and the controller, the six-dimensional force sensor S1 is installed at the end of the robotic arm, and the surgical tool is mounted on the six-dimensional force sensor S1 through a fixing device, and the force The sensor S2 is installed on the surgical tool, and the controller is connected in communication with the robotic arm, the six-dimensional force sensor S1 and the force sensor S2;

The operator pulls the part where the force sensor S2 is installed on the surgical tool to operate, and the controller detects whether the reading of the force sensor S2 reaches a preset threshold;

If the reading of the force sensor S2 does not reach the preset threshold, the manipulator does not move, otherwise, according to the reading of the force sensor S2 and the positional relationship between the six-dimensional force sensor S1 and the force sensor S2, the torque of the six-dimensional force sensor S1 is determined Correction is performed, and the motion of the robotic arm is controlled according to the force signal of the six-dimensional force sensor S1 and the corrected torque signal.

Further, the preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and holding the surgical tool stably.

Further, the positional relationship between the six-dimensional force sensor S1 and the force sensor S2 is determined according to the mechanical dimensions of the robotic arm, the surgical tool and the fixing device.

Further, correcting the torque of the six-dimensional force sensor S1 according to the reading of the force sensor S2 is as follows:

为六维力传感器S1中心O_S与力传感器S2中心O_Doc的位置向量，由六维力传感器S1与力传感器S2之间的位置关系确定，F为六维力传感器S1读到的力，T为六维力传感器S1读到的力矩，T_S为六维力传感器S1矫正后的力矩。

is the position vector of the center O _S of the six-dimensional force sensor S1 and the center O _Doc of the force sensor S2, determined by the positional relationship between the six-dimensional force sensor S1 and the force sensor S2, F is the force read by the six-dimensional force sensor S1, T is the torque read by the six-dimensional force sensor S1, and T _S is the corrected torque of the six-dimensional force sensor S1.

Further, the controller converts the force signal of the six-dimensional force sensor S1 and the corrected torque signal into the motion signal of the robotic arm according to the admittance control, the robotic arm moves according to the motion signal, and the motion of the robotic arm drives the surgical tool to move. .

Further, the six-dimensional force sensor S1 is fixedly installed on the end of the mechanical arm, the fixing device is a clamp, and the surgical tool is installed on the six-dimensional force sensor S1 through the clamp.

Further, the force sensor S2 is installed on the handle of the surgical tool, and the operator pulls the handle of the surgical tool to operate.

Compared with the prior art, the present invention has the following beneficial effects:

(1) Through the preset threshold, when the reading of the force sensor is less than the preset threshold, the robotic arm does not move, and after reaching the preset threshold, the torque of the six-dimensional force sensor is corrected, and the motion of the robotic arm is controlled based on the corrected torque, It solves the problem of inconsistency between the operator's intention and the actual movement of the manipulator during the man-machine cooperative operation using admittance control.

(2) Install a force sensor on the surgical tool. The preset threshold is the reading of the force sensor when the operator's hand is stationary and holding the surgical tool stably. The force on the force sensor can be compared with the preset threshold to determine whether the operator needs to pull the robotic arm. , to increase the security of control. Only when the force of the force sensor reaches the preset threshold, the robot arm can move according to the force situation, so as to avoid disturbing and destroying the normal movement of the robot arm.

(3) The operation is performed by pulling the handle of the surgical tool, and the operating habits of the doctor during the operation are fully considered. The human-computer interaction is good. This surgical procedure has good versatility.

Description of drawings

1 is a schematic structural diagram of a human-computer interaction system based on operator intention correction;

Reference numerals: 1. Robot arm, 2. Surgical tool, S1, six-dimensional force sensor, S2, force sensor.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation manner and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

In the drawings, structurally identical components are denoted by the same numerals, and structurally or functionally similar components are denoted by like numerals throughout. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. Parts in the drawings have been appropriately exaggerated in some places for clarity of illustration.

Example 1:

In a first aspect, the present invention discloses a human-computer interaction system based on operator intention correction, comprising a robotic arm 1, a surgical tool 2, a six-dimensional force sensor S1, a force sensor S2 and a controller;

As shown in Figure 1, the six-dimensional force sensor S1 is installed on the end of the robotic arm 1, the surgical tool 2 is mounted on the six-dimensional force sensor S1 through the fixing device, the force sensor S2 is mounted on the surgical tool 2, the controller and the robotic arm 1, Communication connection between the six-dimensional force sensor S1 and the force sensor S2;

When the operator pulls the part where the force sensor S2 is installed on the surgical tool 2 to operate, the controller detects whether the reading of the force sensor S2 reaches the preset threshold; if the reading of the force sensor S2 does not reach the preset threshold, the robotic arm 1 does not move, Otherwise, the controller corrects the torque read by the six-dimensional force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six-dimensional force sensor S1 and the force sensor S2, according to the force signal of the six-dimensional force sensor S1 and the corrected torque The torque signal controls the movement of the robotic arm 1.

The preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and holding the surgical tool 2 stably. The readings of the force sensor S2 can be recorded several times before the operation, and the readings of the force sensor S2 when the operator's hand is stationary and stably holding the surgical tool 2 are recorded, and the average value is taken as the preset threshold value.

Correcting the torque of the six-dimensional force sensor S1 according to the reading of the force sensor S2 is as follows:

为六维力传感器S1中心O_S与力传感器S2中心O_Doc的位置向量，即操作者手握的位置相对于六维传感器S1中心的位置，由六维力传感器S1与力传感器S2之间的位置关系确定，机械臂1和手术工具2同步运动，因此操作过程中六维力传感器S1中心O_S与力传感器S2中心O_Doc是相对不变的，

不变；F为六维力传感器S1读到的力，实际为操作者施加的外力，T为六维力传感器S1读到的力矩，T_S为六维力传感器S1校正后的力矩。

is the position vector of the center O _S of the six-dimensional force sensor S1 and the center O _Doc of the force sensor S2, that is, the position of the operator's hand relative to the center of the six-dimensional sensor S1, which is determined by the position between the six-dimensional force sensor S1 and the force sensor S2. The positional relationship is determined, and the robotic arm 1 and the surgical tool 2 move synchronously, so the center O _S of the six-dimensional force sensor S1 and the center O _Doc of the force sensor S2 are relatively unchanged during the operation.

unchanged; F is the force read by the six-dimensional force sensor S1, which is actually the external force applied by the operator, T is the torque read by the six-dimensional force sensor S1, and T _S is the corrected torque by the six-dimensional force sensor S1.

When in use, the process is as follows:

(1) Install the six-dimensional force sensor S1 at the end of the robot arm 1, and the six-dimensional force sensor S1 is fixedly installed at the end of the robot arm 1 through screws;

(2) The surgical tool 2 is installed on the six-dimensional force sensor S1 through a fixing device, the force sensor S2 is installed on the surgical tool 2, the fixing device is a clamp, and the surgical tool 2 is installed on the six-dimensional force sensor S1 through the clamp, which is convenient for replacement and Adjust the surgical tool 2, the force sensor S2 is installed on the handle of the surgical tool 2, the operator holds the handle of the surgical tool 2, and the position of the operator's hand should be the center of the force sensor S2;

(3) Determine the positional relationship between the six-dimensional force sensor S1 and the force sensor S2 according to the mechanical dimensions of the robotic arm 1, the surgical tool 2 and the fixture. As shown in Figure 1, S and Doc are the six-dimensional force sensor S1 respectively With the force sensor S2 coordinate system, the position of O _S is the installation position of the six-dimensional force sensor S1, and the position of O _Doc is the installation position of the force sensor S2;

(4) The operator pulls the part where the force sensor S2 is installed on the surgical tool 2 to operate, that is, the operator pulls the handle of the surgical tool 2 to operate;

(5) Judging whether the reading of the force sensor S2 reaches the preset threshold, if not, it means that the operator's intention is not to pull the robot arm 1, then the robot arm 1 does not move, otherwise, it means that the operator's intention is to pull the robot arm 1. The controller corrects the torque read by the six-dimensional force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six-dimensional force sensor S1 and the force sensor S2, and eliminates the existence between the force application point and the six-dimensional force sensor S1. The displacement of the robot arm 1 conforms to the operator's intention. Specifically, according to the admittance control, the controller converts the force signal F of the six-dimensional force sensor S1 and the corrected torque signal T _S into the mechanical force according to the admittance control. The motion signal of the arm 1, the robotic arm 1 moves according to the motion signal, and the motion of the robotic arm 1 drives the surgical tool 2 to move.

The present invention has the following advantages:

(1) Through the preset threshold, when the reading of the force sensor S2 is less than the preset threshold, the robot arm 1 does not move, and after reaching the preset threshold, the torque of the six-dimensional force sensor S1 is corrected, and the corrected torque is used as the basis to control the mechanical The movement of the arm 1 solves the problem that the operator's intention is inconsistent with the actual movement of the robotic arm 1 during the man-machine cooperative operation using admittance control.

(2) Install the force sensor S2 on the surgical tool 2. The preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and the surgical tool 2 is stably held. The force of the force sensor S2 can be compared with the preset threshold to judge the operator Whether it is necessary to pull the robotic arm 1 to increase the security of the control, only when the force of the force sensor S2 reaches the preset threshold, the robotic arm 1 can move according to the force, so as to avoid interference and damage to the normal movement of the robotic arm 1.

(3) The operation is performed by pulling the handle of the surgical tool 2, fully considering the operating habits of the doctor during the operation, and the human-computer interaction is good. It has good versatility in a variety of surgical procedures.

In a second aspect, the present invention discloses a human-computer interaction method based on operator intention correction, comprising the following steps:

①Check the robotic arm 1, the surgical tool 2, the six-dimensional force sensor S1, the force sensor S2 and the controller, wherein the six-dimensional force sensor S1 is installed at the end of the robotic arm 1, and the surgical tool 2 is installed on the six-dimensional force sensor S1 through a fixing device On the upper side, the force sensor S2 is installed on the surgical tool 2, and the controller is connected in communication with the robotic arm 1, the six-dimensional force sensor S1 and the force sensor S2;

② The operator pulls the part of the surgical tool 2 where the force sensor S2 is installed to operate, and the controller detects whether the reading of the force sensor S2 reaches the preset threshold;

③ If the reading of the force sensor S2 does not reach the preset threshold, the robot arm 1 does not move, otherwise, according to the reading of the force sensor S2 and the positional relationship between the six-dimensional force sensor S1 and the force sensor S2 is determined. The torque is corrected, and the movement of the robotic arm 1 is controlled according to the force signal of the six-dimensional force sensor S1 and the corrected torque signal.

The human-computer interaction method based on operator intention correction provided by the present invention is based on the same inventive concept as the system embodiment. For details, please refer to the system embodiment, which will not be repeated here.

The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and changes according to the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art through logical analysis, reasoning or limited experiments on the basis of the prior art according to the concept of the present invention shall fall within the protection scope determined by the claims.

Claims (10)

1. A human-computer interaction system based on operator intention correction is characterized by comprising a mechanical arm, a surgical tool, a six-dimensional force sensor, a force sensor and a controller;

the six-dimensional force sensor is arranged at the tail end of the mechanical arm, the surgical tool is arranged on the six-dimensional force sensor through a fixing device, the force sensor is arranged on the surgical tool, and the controller is in communication connection with the mechanical arm, the six-dimensional force sensor and the force sensor;

when an operator pulls a part, which is provided with a force sensor, on the surgical tool to operate, a controller detects whether the reading of the force sensor reaches a preset threshold value; and if the reading of the force sensor does not reach the preset threshold value, the mechanical arm does not move, otherwise, the controller corrects the torque read by the six-dimensional force sensor according to the reading of the force sensor and the position relation between the six-dimensional force sensor and the force sensor, and controls the movement of the mechanical arm according to the force signal of the six-dimensional force sensor and the corrected torque signal.

2. The human-computer interaction system based on operator intention correction of claim 1, wherein the preset threshold is a reading of a force sensor when an operator's hand is stationary and holding a surgical tool stably.

3. The human-computer interaction system based on operator intention correction as claimed in claim 1, wherein the correction of the moment of the six-dimensional force sensor according to the reading of the force sensor is specifically:

is the center O of a six-dimensional force sensor_SAnd force sensor center O_DocIs determined by the position relation between the six-dimensional force sensor and the force sensor, F is the force read by the six-dimensional force sensor, T is the torque read by the six-dimensional force sensor_SThe corrected torque of the six-dimensional force sensor is obtained.

4. The system of claim 1, wherein the controller converts the force signal and the corrected torque signal of the six-dimensional force sensor into a motion signal of the robotic arm according to the admittance control, the robotic arm moves according to the motion signal, and the motion of the robotic arm moves the surgical tool.

5. The human-computer interaction system based on operator intention correction of claim 1, wherein the six-dimensional force sensor is fixedly installed at the end of the mechanical arm, the fixing device is a clamp, and the surgical tool is installed on the six-dimensional force sensor through the clamp.

6. The human-computer interaction system based on operator intention correction of claim 1, wherein the force sensor is installed on a handle of a surgical tool, and an operator pulls the handle of the surgical tool to operate.

7. A human-computer interaction method based on operator intent correction, comprising the steps of:

the device comprises an inspection mechanical arm, a surgical tool, a six-dimensional force sensor, a force sensor and a controller, wherein the six-dimensional force sensor is arranged at the tail end of the mechanical arm, the surgical tool is arranged on the six-dimensional force sensor through a fixing device, the force sensor is arranged on the surgical tool, and the controller is in communication connection with the mechanical arm, the six-dimensional force sensor and the force sensor;

an operator pulls a part, on which a force sensor is installed, of the surgical tool to operate, and a controller detects whether the reading of the force sensor reaches a preset threshold value;

and if the reading of the force sensor does not reach the preset threshold value, the mechanical arm does not move, otherwise, the moment of the six-dimensional force sensor is corrected according to the reading of the force sensor and the position relation between the six-dimensional force sensor and the force sensor, and the movement of the mechanical arm is controlled according to the force signal of the six-dimensional force sensor and the corrected moment signal.

8. The human-computer interaction method based on operator intention correction of claim 7, wherein the preset threshold is a reading of a force sensor when an operator's hand is stationary and stably holding a surgical tool.

9. The human-computer interaction method based on operator intention correction as claimed in claim 7, wherein the correction of the torque of the six-dimensional force sensor according to the reading of the force sensor is specifically:

is the center O of a six-dimensional force sensor_SAnd force sensor center O_DocIs determined by the position relation between the six-dimensional force sensor and the force sensor, F is the force read by the six-dimensional force sensor, T is the torque read by the six-dimensional force sensor_SThe corrected moment of the six-dimensional force sensor is obtained.

10. The human-computer interaction method based on operator intention correction of claim 7, wherein the controller converts the force signal and the corrected moment signal of the six-dimensional force sensor into a motion signal of the robot arm according to the admittance control, the robot arm moves according to the motion signal, and the motion of the robot arm drives the surgical tool to move.

Images