CN110355750A - Interactive control method for teleoperation hand-eye coordination - Google Patents

Abstract

The invention discloses an interactive control method oriented to hand-eye coordination of remote operation, which is used to solve the technical problem of poor practicability of the existing human-computer interaction control method. The technical solution is based on the conversion of the coordinate system. When the operator controls the movement of the scene object by switching the device, the pose of the scene object is first converted from the world coordinate system to the camera coordinate system, and then moves as expected in the camera coordinate system plus interaction The precession amount of the device is used to obtain the pose coordinates of the new camera coordinate system, and finally the newly obtained pose coordinates are converted from the camera coordinate system to the world coordinate system to control the actual movement of the scene object. The invention realizes the hand-eye coordination in the remote operation process, that is, the movement of the scene object in the interactive operation process is not affected by the operator's viewing angle transformation of the scene, and is consistent with the movement of the interactive device, reduces the difficulty of remote operation, and has good practicability.

Description

Interactive control method for teleoperation hand-eye coordination

technical field

The invention relates to a human-computer interactive control method, in particular to an interactive control method oriented to hand-eye coordination of remote operation.

Background technique

The document "Master-Slave Control Strategy for Teleoperation Robot System, Journal of Jiangsu University of Science and Technology (Natural Science Edition), 2013, Vol27(8), p643-647" discloses a control method for a master-slave teleoperation robot system. This method adopts an incremental position control method to control the movement of the slave hand with the increment of the master hand, which effectively avoids the cumbersome initial return to the origin. When the operator directly observes the operated environment information and when the operator observes the operated environment information through the image device, the position correspondence between the master and the slave is different. By adjusting the proportional control gain matrix, the master hand and the actual environment or the master hand and the image device Coordinate matching, establishing the workspace mapping of the master end and the slave end has a better visual sense of presence. The method described in the literature is only based on the difference between the operator’s direct observation of the environment information and the observation of the environment information through the image device, and the working space mapping between the master hand and the actual environment or the master hand and the image device is simply established through the proportional control gain coefficient, without giving When the operator observes the change of the viewing angle of the operating environment information, the method of establishing the master-slave workspace mapping method does not realize that the master-slave workspace mapping is not affected by the change of the observation angle, and the movement is always consistent, the scope of application is not wide, and the operation is difficult. larger.

Contents of the invention

In order to overcome the disadvantage of poor practicability of the existing human-computer interaction control method, the present invention provides an interactive control method oriented to hand-eye coordination of remote operation. This method is based on the transformation of the coordinate system. When the operator controls the movement of the scene object by exchanging the device, the pose of the scene object is first converted from the world coordinate system to the camera coordinate system, and then moves as expected in the camera coordinate system plus the interactive device The precession amount of the new camera coordinate system is obtained, and finally the newly obtained pose coordinates are converted from the camera coordinate system to the world coordinate system to control the actual motion of the scene object. The invention realizes the hand-eye coordination in the remote operation process, that is, the movement of the scene object in the interactive operation process is not affected by the operator's viewing angle transformation of the scene, and is consistent with the movement of the interactive device, reduces the difficulty of remote operation, and has good practicability.

The technical solution adopted by the present invention to solve the technical problem is: a kind of interactive control method for remote operation hand-eye coordination, which is characterized in that it includes the following steps:

Step 1. Interactive device data collection. During the movement of the hand controller, the real-time position information P _c ⁿ of the hand controller is collected at equal time intervals; the collected real-time position information P _c ⁿ of the hand controller is compared with the previous moment pose The information P _c ^n-1 is subtracted to obtain the precession ΔP of the hand controller; the precession ΔP is mapped to the initial motion command ΔP ₀ at the end of the manipulator according to ΔP ₀ =k*ΔP, where k is the operating proportional coefficient.

△P ₀ = k*△P (1)

Step 2: Convert the initial motion command ΔP ₀ in combination with the conversion matrix R _x from the interactive operation coordinate system to the camera coordinate system to obtain the motion command ΔP ₁ in the camera coordinate system.

△P ₁ ＝△P ₀ *R _x (2)

Step 3: Combining the pose matrix C of the camera coordinate system in the world coordinate system, the motion command ΔP ₁ in the camera coordinate system is converted to obtain the motion command ΔP ₂ in the world coordinate system.

During the interaction process, the view observed by the operator is determined by the position and attitude of the camera in the world coordinate system, and the position and attitude matrix of the camera depends on three elements: the direction of the camera line of sight, the position of the center of the camera, and the forward direction of the camera. Orientation, these three elements can determine the position and posture of the camera in the world coordinate system. When observing the viewing angle conversion, the essence is that the position and posture of the camera in the world coordinate system has changed, and the direction of the camera coordinate system in the world coordinate system has also changed accordingly.

To convert the motion obtained in step 2 from the camera coordinate system to the world coordinate system, the inverse matrix C ^-1 of the camera matrix is needed. In addition, since the momentum matrix for the interactive device is a 1x3 matrix, a 1x4 matrix D needs to be constructed.

D＝[△P ₁ 0] (4)

△P ₂ =DC ^-1 (5)

Step 4: The motion command ΔP ₂ in the world coordinate system is obtained through motion mapping to obtain the final motion amount ΔP ₃ of the end of the mechanical arm, and a teleoperation command for the entire interaction process is generated.

Expand the world coordinate system motion command ΔP ₂ obtained in step 3 to obtain ΔP ₂ as a 1x4 matrix, the first three elements of which are the final movement amount at the end of the scene object, take the first three elements of ΔP ₂ through motion mapping to generate ΔP ₃ , the simplified expression is as follows :

△P ₃ =[△x △y △z] (6)

Among them, Δx, Δy, and Δz respectively represent the movement amounts of the end of the scene object in the direction of the three coordinate axes in the world coordinate system. So far, they are used to realize the generation of teleoperation commands for hand-eye coordination.

Step 5. Use the teleoperation command generated in step 4 to drive the movement of the robotic arm to achieve hand-eye coordination.

The real-time position at the end of the scene object is recorded as P _j ⁿ , and the position at the next moment is P _j ⁿ⁺¹ , then

The movement of the scene object is driven by the real-time end position of the scene object, and the hand-eye coordination in the teleoperation process is realized.

The beneficial effects of the present invention are: the method is based on the conversion of the coordinate system. When the operator controls the movement of the scene object through the exchange device, the pose of the scene object is first converted from the world coordinate system to the camera coordinate system, and then in the camera coordinate system Add the precession of the interactive device to the expected motion to obtain the pose coordinates of the new camera coordinate system, and finally convert the newly obtained pose coordinates from the camera coordinate system to the world coordinate system to control the actual motion of the scene object. The invention realizes the hand-eye coordination in the remote operation process, that is, the movement of the scene object in the interactive operation process is not affected by the operator's viewing angle transformation of the scene, and is consistent with the movement of the interactive device, reduces the difficulty of remote operation, and has good practicability.

The present invention will be described in detail below in combination with specific embodiments.

Detailed ways

In order to introduce the content of the present invention in detail, it is necessary to set forth the definitions of 3 commonly used coordinate systems:

(1) World coordinate system O _world : The world coordinate system is the reference coordinate system of the entire system, which is used to describe the actual movement of the object model in the scene, which is equivalent to the base coordinate system. The movement of scene objects is described based on the world coordinate system .

(2) Camera coordinate system O _camera : The camera coordinate system is used to describe the remote operation scene. When the operator observes the viewing angle transformation, the essence is that the position and posture of the camera in the world coordinate system have changed. When the viewing angle is transformed, the pose of the camera coordinate system in the world coordinate system changes, but its orientation relative to the computer screen does not change.

(3) Interactive operation coordinate system (i.e. hand controller coordinate system) O _interaction : The interactive device coordinate system is used to describe the motion of the interactive device during teleoperation. When the viewing angle changes, the position and posture of the interactive device in the world coordinate system will change, but its orientation relative to the computer screen will not change.

In order to verify the effectiveness of the operation technology for teleoperation hand-eye coordination proposed by the present invention, the present invention combines the three-dimensional graphics development environment OSG (OpenSenceGraph) and the interactive tool NovintFalcon hand controller, based on the control of the motion of the IRB120 mechanical arm in the virtual scene Carry out simulation demonstration verification, the specific implementation method is as follows:

Step 1. Interactive device data collection. Taking the upward movement of the hand controller as an example, the real-time position and attitude information P _c ⁿ of the hand controller is collected at equal time intervals during the movement of the hand controller; the collected real-time pose of the hand controller The difference between the information P _c ⁿ and the pose information P _c ^n-1 at the previous moment is obtained to obtain the precession ΔP of the hand controller; the precession ΔP is mapped to the initial motion command ΔP ₀ at the end of the mechanical arm according to ΔP ₀ =k*ΔP, where k is the operating proportionality coefficient, the unit of movement of the end of the hand controller is meter, the unit of movement of the end of the mechanical arm in the virtual scene is millimeter, the operating proportionality coefficient k=1000, when the end of the hand controller moves upwards by 0.1 meters, the initial movement command is

△P ₀ =k*△P=[0 0 100] (1)

Step 2: Convert the initial motion command ΔP ₀ in combination with the conversion matrix R _x from the interactive operation coordinate system to the camera coordinate system to obtain the motion command ΔP ₁ in the camera coordinate system.

Through the analysis of the positional relationship between the interactive device coordinate system and the camera coordinate system, it can be seen that when the viewing angle is changed, the relative direction of the interactive device coordinate system and the camera coordinate system in the world coordinate system does not change, and the interactive device coordinate system is along the X axis. Rotate 90 degrees to be consistent with the direction of the camera coordinate system.

Step 3: Combining the pose matrix C of the camera coordinate system in the world coordinate system, the motion command ΔP ₁ in the camera coordinate system is converted to obtain the motion command ΔP ₂ in the world coordinate system.

When facing the scene, the pose matrix of the camera in the world coordinate system is C.

Since the momentum matrix used by the interactive device is a 1x3 matrix, a 1x4 matrix D needs to be constructed.

D＝[△P ₁ 0]＝[0 100 0 0] (4)

△P ₂ ＝DC ^-1 ＝[0 0 100 0] (5)

Step 4: The motion command ΔP ₂ in the world coordinate system is obtained through motion mapping to obtain the final motion amount ΔP ₃ of the end of the mechanical arm, and a teleoperation command for the entire interaction process is generated.

△P ₃ =[△x △y △z]=[0 0 100] (6)

Among them, Δx, Δy, and Δz respectively represent the movement amounts of the end of the scene object in the direction of the three coordinate axes in the world coordinate system. So far, they are used to realize the generation of teleoperation commands for hand-eye coordination.

Step 5. Use the teleoperation command generated in step 4 to drive the movement of the robotic arm to achieve hand-eye coordination. The real-time position at the end of the scene object is recorded as P _j ⁿ , and the position at the next moment is P _j ⁿ⁺¹ , then

This embodiment is based on the interactive control of the IRB120 robotic arm. To complete step five, first use 3D MAX and other tools to establish a three-dimensional model of the IRB120 robotic arm to complete the construction of the virtual scene; on the basis of the D-H coordinate system, establish the kinematics of the robotic arm Model.

Firstly, forward kinematics analysis is carried out on it. Every two adjacent coordinate systems can be transformed into each other through four homogeneous transformations. Let the transformation matrix be but

Among them, a _i , α _i , d _i , and θ _i are the length of the connecting rod, the torsion angle of both banks, the offset of the connecting rod, and the joint angle, respectively.

For a six-degree-of-freedom manipulator, when the joint angles of each joint are determined, the end pose of the manipulator is also determined. Let the end pose be T, then

Secondly, use the analytical method to analyze its kinematics, and calculate each joint angle according to the terminal pose.

θ ₁ can be obtained from the above formula, and other joint angles can be obtained sequentially by analytical method.

The initial position of the end of the arm When facing the scene squarely,

Through the above hand-eye coordination interactive control method for motion mapping, it can be obtained that when the hand controller moves upwards, the direction of movement of the robotic arm is along the Z-axis when the hand controller is moving upwards, and the direction of movement of the robotic arm is along the Y-axis when looking down at the virtual scene. It can be concluded that when the operator's viewing angle changes, the hand-eye coordination implementation method proposed in this paper can achieve the same movement of scene objects and interactive devices, thus verifying the effectiveness of the hand-eye coordination method proposed in this paper.

Claims (1)

1. a kind of interaction control method towards remote operating hand eye coordination, it is characterised in that the following steps are included:

Step 1: interactive device data acquire, in hand controller motion process, constant duration acquires the real time position of hand controller Information P_c ⁿ；By collected hand controller real-time position information P_c ⁿWith previous moment posture information P_c ^n-1Hand controller precession is obtained as difference Measure Δ P；By precession amount Δ P according to Δ P₀=k* Δ P is mapped as the initial motion instruction Δ P of mechanical arm tail end₀, wherein k is behaviour Make proportionality coefficient；

ΔP₀=k* Δ P (1)

Step 2: in conjunction with the transition matrix R of interactive operation coordinate system to camera coordinates system_x, Δ P is instructed to initial motion₀Turned Processing is changed, the movement instruction Δ P in camera coordinates system is obtained₁；

ΔP₁=Δ P₀*R_x (2)

Step 3: position auto―control C of the combining camera coordinate system in world coordinate system, to the movement instruction Δ P in camera coordinates system₁ Conversion process is carried out, the movement instruction Δ P in world coordinate system is obtained₂；

In interactive process, the what comes into a driver's that operator observes is determined by position and attitude of the camera in world coordinate system, and camera Position and attitude matrix depend on three elements: i.e. camera sight line direction, the position of image center, camera positive direction, this Three elements just can determine that position and attitude of the camera in world coordinate system；When observation visual angle conversion, essence is that camera is alive Position and attitude in boundary's coordinate system is changed, and direction of the camera coordinates system in world coordinate system also changes therewith；

Amount of exercise obtained in step 2 is transformed into world coordinate system from camera coordinates system, needs to use camera inverse of a matrix square Battle array C^-1；Further, since interactive device is 1x3 matrix with momentum matrix, it is therefore desirable to construct the matrix D of a 1x4；

D=[Δ P₁ 0] (4)

ΔP₂=DC^-1 (5)

Step 4: by the movement instruction Δ P in world coordinate system₂The final amount of exercise of mechanical arm tail end is obtained by moving mapping ΔP₃, generate the remote operating instruction of entire interactive process；

The world coordinate system movement instruction Δ P that step 3 is obtained₂Expansion obtains Δ P₂For 1x4 matrix, first three column element is field The final amount of exercise of scape object end takes Δ P by moving mapping₂First three columns Element generation Δ P₃, it is simplified to be expressed as follows:

ΔP₃=[Δ x Δ y Δ z] (6)

Wherein, Δ x, Δ y, Δ z respectively indicate the amount of exercise of scenario objects end three change in coordinate axis direction in world coordinate system, So far, it is generated to realize that the remote operating of hand eye coordination instructs；

Step 5: the remote operating order-driven manipulator motion generated using step 4, realizes hand eye coordination；

The real time position of scenario objects end is denoted as P_j ⁿ, the position of subsequent time is P_j ⁿ⁺¹, then

Scenario objects movement is driven by scenario objects real-time end position, realizes the hand eye coordination of remote operating process.