[go: up one dir, main page]

CN115469576A - A Teleoperation System Based on Hybrid Mapping of Human-Robot Arm Heterogeneous Motion Space - Google Patents

A Teleoperation System Based on Hybrid Mapping of Human-Robot Arm Heterogeneous Motion Space Download PDF

Info

Publication number
CN115469576A
CN115469576A CN202211059660.8A CN202211059660A CN115469576A CN 115469576 A CN115469576 A CN 115469576A CN 202211059660 A CN202211059660 A CN 202211059660A CN 115469576 A CN115469576 A CN 115469576A
Authority
CN
China
Prior art keywords
human
arm
mapping
mechanical arm
joint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211059660.8A
Other languages
Chinese (zh)
Other versions
CN115469576B (en
Inventor
赵永嘉
叶双休
张宁
雷小永
李卫琪
戴树岭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202211059660.8A priority Critical patent/CN115469576B/en
Publication of CN115469576A publication Critical patent/CN115469576A/en
Application granted granted Critical
Publication of CN115469576B publication Critical patent/CN115469576B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/042Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
    • G05B19/0423Input/output
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/20Pc systems
    • G05B2219/25Pc structure of the system
    • G05B2219/25257Microcontroller

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Manipulator (AREA)

Abstract

本发明公开一种基于人‑机械臂异构运动空间混合映射的机械臂遥操作系统,使用相机识别操作人员的关节位置信息并进行数据处理,得到操作人员的部分关节角度以及手臂末端坐标位姿,将人手臂末端位姿映射到机械臂末端,并对机械臂进行逆运动学求解获得机械臂的期望关节角度,针对多解和解不存在的情况,参考操作人员的关节角度进行调整,并根据期望关节角对机械臂进行运动规划,控制真实机械臂运动,使机械臂尽可能地模仿人手臂姿态运动。

Figure 202211059660

The invention discloses a manipulator teleoperating system based on hybrid mapping of human-manipulator heterogeneous motion space, which uses a camera to identify the joint position information of the operator and performs data processing to obtain part of the joint angles of the operator and the coordinate pose of the end of the arm , map the pose of the end of the human arm to the end of the manipulator, and solve the inverse kinematics of the manipulator to obtain the expected joint angle of the manipulator. For the situation where there are multiple solutions or no solution exists, adjust the joint angle of the operator with reference to the joint angle of the operator, and according to The joint angle is expected to plan the motion of the robotic arm, control the motion of the real robotic arm, and make the robotic arm mimic the posture of the human arm as much as possible.

Figure 202211059660

Description

一种基于人-机械臂异构运动空间混合映射的遥操作系统A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space

技术领域technical field

本发明属于人机交互领域,是一种基于光学姿态捕捉和姿态映射远程控制机器人的方法。The invention belongs to the field of human-computer interaction, and relates to a method for remotely controlling a robot based on optical attitude capture and attitude mapping.

背景技术Background technique

受限于当前机器人智能水平,相当长一段时间内,研制出完全自主的机器人仍是遥不可及的目标,特别是在核辐射、空间、深海等危险、远距离或非结构环境下。将人类的高级决策及灵活操作能力与机器人进行有机结合,发展人在回路的精细遥操作技术是未来相当长一段时间内的必然选择。Limited by the current level of robot intelligence, the development of a fully autonomous robot is still an unattainable goal for a long time, especially in dangerous, long-distance or unstructured environments such as nuclear radiation, space, and deep sea. Organically combining human's advanced decision-making and flexible operation capabilities with robots, and developing human-in-the-loop fine teleoperation technology will be an inevitable choice for a long time in the future.

虽然遥操作机器人领域已经取得了巨大进展,甚至其中一些已经商业化,得以应用于实际,但操作人员指挥从端机器人的方式并没有太大的改变。一般来说,操作者在对从端机器人在远程环境中的行为进行视觉监控的同时,直接向从机器人发出特定的指令,称之为直接遥操作,它经常给操作者带来沉重的认知负担。除此之外,操作者与遥控机器人之间通信的时延和带宽限制一直是制约遥操作系统性能的主要因素;在使用传统的遥操作系统时,操作者的2种重要感知(视觉和力觉)被剥夺,严重影响操作者完成遥操作任务的能力。尤其是面对非结构环境、突发的事故时,人类操作者不能够及时处理问题,会降低任务成功率,甚至给用户带来人身安全隐患,也会使得操作者处于高度紧张状态,易产生精神疲劳,降低工作效率。Although the field of teleoperated robots has made great progress, and some of them have even been commercialized and put into practical use, the way operators command slave robots has not changed much. Generally speaking, the operator directly sends specific instructions to the slave robot while visually monitoring the behavior of the slave robot in the remote environment, which is called direct teleoperation, which often brings heavy cognitive burden to the operator. burden. In addition, the delay and bandwidth limitation of the communication between the operator and the remote robot have been the main factors restricting the performance of the teleoperation system; when using the traditional teleoperation system, the operator's two important perceptions (vision and force) sense) is deprived, seriously affecting the operator's ability to complete teleoperation tasks. Especially in the face of unstructured environments and unexpected accidents, human operators cannot deal with problems in time, which will reduce the success rate of tasks, and even bring hidden dangers to users' personal safety, and will also make operators in a state of high tension and prone to accidents. Mental fatigue reduces work efficiency.

为了解决上述问题,科研人员进行了许多研究,大致可以归为2类:其一,结合基于视觉、触觉、肌肉神经等感知信号的人机交互技术的特殊性能,能提高遥操作机器人系统的工作效率和控制精度;其二,针对性地研究开发人机遥操作控制模式,充分发挥人类操作者与机器人各自的优势,在缓解人类操作者疲倦的同时提高系统的灵敏度、精确度,实现人机任务合理配置。In order to solve the above problems, researchers have conducted a lot of research, which can be roughly classified into two categories: First, combining the special performance of human-computer interaction technology based on visual, tactile, muscle nerve and other sensory signals can improve the work of teleoperated robot systems. Efficiency and control accuracy; Second, research and develop man-machine remote operation control mode in a targeted manner, give full play to the respective advantages of human operators and robots, improve the sensitivity and accuracy of the system while alleviating the fatigue of human operators, and realize man-machine Tasks are properly configured.

要对机器人进行遥操作,需要生成运动指令,基于光学的姿态捕捉技术是所有解决方案中最不受物理约束的技术。商用的基于标记的产品,如OptiTrack,旨在通过跟踪反射标记来捕捉姿势[Spanlang B,Navarro X,Normand J M,Kishore S,Pizarro R,SlaterM.Real time whole body motion mapping for avatars and robots[A].In:the 19thACM Symposium on Virtual Reality Software and Technology[C].Singapore:Association for Computing Machinery,2013:175-178.]。然而,这种方法由耗时的标记过程组成,操作符要正确地使用所有标记,这使得它的应用没有那么广泛。例如,无标记动作捕捉系统Capture Live承诺提供更好的便利性,但设备和许可证的较高价格也极大地限制了它们的普及。此外,这些商业方法都要求操作者在被摄像机阵列包围的有限室内工作空间进行操作,导致环境灵活性较低。如今,便携式深度检测设备如著名的微软Kinect、Intel RealSense和LeapMotion变得越来越可行,研究人员已经使得减少动作捕捉摄像机的数量(甚至只有一台)可以实现,并极大地简化了动作捕捉过程。To teleoperate a robot, motion commands need to be generated, and optical-based pose capture is the least physically constrained of all solutions. Commercial marker-based products, such as OptiTrack, aim to capture poses by tracking reflective markers [Spanlang B, Navarro X, Normand J M, Kishore S, Pizarro R, Slater M. Real time whole body motion mapping for avatars and robots [A] .In:the 19thACM Symposium on Virtual Reality Software and Technology[C].Singapore:Association for Computing Machinery,2013:175-178.]. However, this approach consists of a time-consuming tagging process for operators to use all tags correctly, which makes it less widely applicable. For example, the markerless motion capture system Capture Live promises better convenience, but the high price of equipment and licenses has also greatly limited their popularity. In addition, these commercial methods all require operators to operate in a limited indoor workspace surrounded by camera arrays, resulting in less environmental flexibility. Today, portable depth detection devices such as the famous Microsoft Kinect, Intel RealSense, and LeapMotion are becoming more and more feasible. Researchers have made it possible to reduce the number of motion capture cameras (or even just one) and greatly simplify the motion capture process. .

人-机器人姿态映射,即将人的姿态转换为机器人的姿态,弥合了人-机器人运动之间的最后差距。Luo等人[Luo R C,Shih B H,Lin T W.Real time human motionimitation of anthropomorphic dual arm robot based on Cartesian impedancecontrol[A].In:IEEE International Symposium on Robotic and SensorsEnvironments[C].Piscataway:IEEE,2013:25-30.]使用避免逆运动学计算过程的笛卡儿空间阻抗控制实现了对6自由度机械臂的在线示教。然而,笛卡儿空间的运动映射只关注人机末端位姿的同步,机械臂肩部和肘部关节的运动轨迹另需人为规划。Nguyen等人[NguyenV V,Lee J H,et al.Full-body imitation of human motions with Kinect andheterogeneous kinematic structure of humanoid robot[A].In:IEEE/SICEInternational Symposium on System Integration[C].Piscataway:IEEE,2012:93-98.]首先建立人臂与3自由度机械臂的关节空间映射关系,然后使用空间向量法实现了对人臂关节角度的求解并传递给机械臂作为控制指令实现实时的运动模仿。吴伟国等人[吴伟国,栗华,高力扬.人体步行捕捉下的双足机器人跟随步行与实验[J].哈尔滨工业大学学报,2017,49(1):21-29.]对人类下肢进行6自由度逆运动学计算后建立步行样本,然后使用运动相似性指标进行样本拼接来驱动机器人实现步行跟随。Human-robot pose mapping, i.e. converting human poses to robot poses, bridges the last gap between human-robot motion. Luo et al [Luo R C, Shih B H, Lin T W.Real time human motionimitation of anthropomorphic dual arm robot based on Cartesian impedance control[A].In:IEEE International Symposium on Robotic and SensorsEnvironments[C].Piscataway:IEEE,2013: 25-30.] Online teaching of a 6-DOF manipulator is realized using Cartesian spatial impedance control that avoids the inverse kinematics calculation process. However, the motion mapping in Cartesian space only focuses on the synchronization of the end pose of the man-machine, and the motion trajectories of the shoulder and elbow joints of the robotic arm need to be planned manually. Nguyen et al. [NguyenV V, Lee J H, et al.Full-body imitation of human motions with Kinect and heterogeneous kinematic structure of humanoid robot[A].In:IEEE/SICEInternational Symposium on System Integration[C].Piscataway:IEEE,2012 :93-98.] First establish the joint space mapping relationship between the human arm and the 3-DOF robotic arm, and then use the space vector method to solve the joint angle of the human arm and pass it to the robotic arm as a control command to realize real-time motion simulation. Wu Weiguo et al [Wu Weiguo, Li Hua, Gao Liyang. Biped robot following walking and experiment captured by human walking[J]. Journal of Harbin Institute of Technology, 2017,49(1):21-29.] Performed six-freedom human lower limbs After the degree inverse kinematics calculation, the walking samples are established, and then the motion similarity index is used for sample splicing to drive the robot to achieve walking follow.

在实际的基于视觉的遥操作过程中,由于摄像机噪声、人体关节识别的局限性以及人体骨骼识别的误差,使用单一的映射方法很难保证非拟人机器人能够很好地模仿人体手臂的姿态。末端位姿映射可以使机器人末端很好地跟随人的手腕,但存在多个逆运动学解和无解的问题。关节角度映射可以使机器人更好地模仿人臂的姿态,但是人臂的关节数量与机器人的关节数量并不相等,而且人臂的一些关节角度很难测量。In the actual vision-based teleoperation process, due to camera noise, limitations in human joint recognition, and errors in human bone recognition, it is difficult to ensure that non-anthropomorphic robots can well imitate human arm poses using a single mapping method. The end pose mapping can make the end of the robot follow the human wrist well, but there are many inverse kinematics solutions and no solutions. Joint angle mapping can enable the robot to better imitate the posture of the human arm, but the number of joints in the human arm is not equal to the number of joints in the robot, and some joint angles of the human arm are difficult to measure.

发明内容Contents of the invention

针对上述问题,本发明提出基于人-机械臂异构运动空间混合映射的遥操作系统,对人体关节进行识别并将其映射到机械臂,减少了对人体运动的限制,并使机械臂可以做出与人手臂相似的姿态,在遥操作中实现了较好的效果。In view of the above problems, the present invention proposes a teleoperation system based on the hybrid mapping of human-robot heterogeneous motion space, which can identify the joints of the human body and map them to the robotic arm, which reduces the restrictions on human motion and enables the robotic arm to do A gesture similar to that of a human arm has been achieved, and better results have been achieved in teleoperation.

本发明基于人-机械臂异构运动空间混合映射的遥操作系统,分为两大模块,分别为人-机械臂运动映射模块与机械臂运动规划和控制系统。The present invention is based on the teleoperation system of human-manipulator arm heterogeneous motion space hybrid mapping, which is divided into two modules, namely, the human-manipulator arm motion mapping module and the manipulator arm motion planning and control system.

其中人-机械臂运动映射模块构,建了人体关节角度及位置姿态识别系统,采用Kinect相机的人体关节识别系统获取人体关节位置数据并绘制人体骨架,对人体关节位置数据进行卡尔曼滤波,再计算输出人手臂末端位姿以及关节角度。Among them, the human-robot motion mapping module is constructed, and the human joint angle, position and posture recognition system is built. The human joint position recognition system of the Kinect camera is used to obtain the human joint position data and draw the human skeleton, and the Kalman filter is performed on the human joint position data, and then Calculate and output the pose and joint angle of the end of the human arm.

并采用人机运动空间混合映射方法,先将人手臂末端位姿映射到机械臂的末端,对机械臂进行逆运动学求解;进一步加入关节映射,获得机械臂关节角度最优解。And using the human-machine motion space hybrid mapping method, first map the end pose of the human arm to the end of the robotic arm, and then solve the inverse kinematics of the robotic arm; further add joint mapping to obtain the optimal solution for the joint angle of the robotic arm.

机械臂运动规划和控制系统使用socket建立了Windows与ROS的通信传输,将期望关节角度数据发布到ROS节点,并使用Moveit订阅并进行运动规划,最终将控制信息传输给真实机械臂,控制真实机械臂进行运动。The robot arm motion planning and control system uses socket to establish the communication transmission between Windows and ROS, publishes the expected joint angle data to the ROS node, and uses Moveit to subscribe and perform motion planning, and finally transmits the control information to the real robot arm to control the real machine arm movement.

本发明的优点:Advantages of the present invention:

1、本发明基于人-机械臂异构运动空间混合映射的遥操作系统,采用无标记动作识别,对操作人员的运动约束较小,且采集人体关节数据仅需要一台Kinect相机,硬件配置要求低。1. The present invention is based on the teleoperation system based on the hybrid mapping of human-robot heterogeneous motion space, which adopts unmarked motion recognition, which has less movement constraints on the operator, and only needs a Kinect camera to collect human joint data, and the hardware configuration requirements Low.

2、本发明基于人-机械臂异构运动空间混合映射的遥操作系统,解决了末端姿态映射导致机械臂逆运动学多解和解不存在的问题。2. The present invention is based on the teleoperation system of human-manipulator arm heterogeneous motion space hybrid mapping, which solves the problem of multiple solutions and non-existence of the inverse kinematics of the manipulator caused by the terminal attitude mapping.

3、本发明基于人-机械臂异构运动空间混合映射的遥操作系统,仅根据Kinect相机采集人体关节数据,无法计算出人手臂的所有关节角度,采用混合映射方法避免了这一问题,仅需测量三个关节角度。3. The present invention is based on the teleoperation system of human-manipulator arm heterogeneous motion space hybrid mapping. Only according to the Kinect camera to collect human joint data, it is impossible to calculate all the joint angles of the human arm. The hybrid mapping method is used to avoid this problem. Three joint angles need to be measured.

4、本发明基于人-机械臂异构运动空间混合映射的遥操作系统,机械臂可以较好地模仿人手臂的姿态,做出与人手臂姿态类似的动作,操作人员更容易控制其完成操作任务。4. The present invention is based on the teleoperation system based on the hybrid mapping of human-robot heterogeneous motion space. The robot arm can better imitate the posture of the human arm and make actions similar to the posture of the human arm. It is easier for the operator to control it to complete the operation Task.

附图说明Description of drawings

图1为本发明基于人-机械臂异构运动空间混合映射的遥操作系统整体框图;Fig. 1 is the overall block diagram of the teleoperation system based on the mixed mapping of human-robot heterogeneous motion space in the present invention;

图2为人类手臂的运动学模型;Fig. 2 is the kinematics model of human arm;

图3为ur5机械臂的运动学模型;Figure 3 is the kinematics model of the ur5 robotic arm;

图4为连杆之间的参数变换图;Fig. 4 is a parameter conversion diagram between connecting rods;

图5为25个相对于摄像机坐标的三维关节位置;Figure 5 shows 25 three-dimensional joint positions relative to the camera coordinates;

图6为人体右臂模型。Figure 6 is a human right arm model.

具体实施方式detailed description

下面结合附图对本发明做进一步详细说明。The present invention will be described in further detail below in conjunction with the accompanying drawings.

本发明基于人-机械臂异构运动空间混合映射的遥操作系统,如图1所示,分为两大模块,分别为人-机械臂运动映射模块与机械臂运动规划。The present invention is based on the teleoperation system of human-robot heterogeneous motion space hybrid mapping, as shown in Figure 1, which is divided into two modules, namely, the human-robot motion mapping module and the robot motion planning.

其中,人-机械臂运动映射模块采用Kinect相机获取人体关节信息,并绘制人体骨架;使用任意的六自由度机械臂作为遥操作控制的对象,将人手臂末端的位姿以及人手臂各个关节角度分别映射机械臂末端及机械臂各个关节,两者的映射组合到一起,形成混合映射方法,最终得到机械臂期望关节角度。Among them, the human-robot motion mapping module uses the Kinect camera to obtain human joint information, and draws the human skeleton; uses any six-degree-of-freedom robotic arm as the object of teleoperation control, and maps the pose of the end of the human arm and the angles of each joint of the human arm The end of the manipulator and each joint of the manipulator are mapped separately, and the mappings of the two are combined to form a hybrid mapping method, and finally the desired joint angle of the manipulator is obtained.

机械臂运动规划和控制模块通过ROS机器人控制系统来实现,根据机械臂期望关节角度,在Moveit中对其进行运动规划,并在Gazebo中仿真,并控制真实机械臂运动。The motion planning and control module of the manipulator is realized by the ROS robot control system. According to the expected joint angle of the manipulator, it is planned in Moveit, simulated in Gazebo, and controls the real manipulator movement.

所述人-机械臂运动映射模块中建立运动学模型,该运动学模型为手臂末端位姿和关节角度映射基础,具体建立方法采用D-H法,在机器人运动学建模的基础上,建立了5-DOF手臂模型和UR5机器人模型两个运动学模型;其中,5-DOF手臂模型包括肩部3-DOF和肘部2-DOF;Ur5机器人模型为6-DOF机械臂;可见两个模型间为异构模型,如图2和图3所示。The kinematics model is established in the human-robot motion mapping module. The kinematics model is the basis for mapping the pose and joint angle of the end of the arm. The specific establishment method adopts the D-H method. On the basis of the kinematic modeling of the robot, 5 Two kinematic models of -DOF arm model and UR5 robot model; among them, the 5-DOF arm model includes shoulder 3-DOF and elbow 2-DOF; Ur5 robot model is a 6-DOF mechanical arm; it can be seen that the two models are Heterogeneous models, as shown in Figure 2 and Figure 3.

D-H参数定义如图4所示,图4中zi与zi-1分别与机械臂第i与第i-1个关节转轴重合,xi-1沿第i-1个关节轴和第i个关节轴的公垂线ai-1方向;xi沿第i个关节轴和第i个关节轴的公垂线ai方向;αi-1为zi-1和zi之间的夹角,ai-1沿着xi-1测量的从zi-1到zi的连接长度,其中xi-1垂直于zi-1和zi;di为xi-1到xi沿zi测量的链接偏移量;θi为xi-1和xi之间的夹角。The definition of DH parameters is shown in Figure 4. In Figure 4, z i and z i-1 coincide with the rotation axes of the i-th and i-1 joints of the manipulator respectively, and x i-1 is along the i-1 joint axis and the i-th joint axis. The direction of the common vertical line a i-1 of the joint axis; x i is along the i-th joint axis and the direction of the common vertical line a i of the i-th joint axis; α i-1 is the distance between z i-1 and z i Included angle, a i-1 is the connection length from z i-1 to z i measured along x i-1 , where x i-1 is perpendicular to z i-1 and z i ; d i is x i-1 to z i The link offset of xi measured along zi ; θi is the angle between xi -1 and xi .

根据D-H法,从机械臂第i-1个关节坐标系到第i个关节坐标系的坐标变换描述如式(1)所示。According to the D-H method, the coordinate transformation description from the i-1 joint coordinate system of the manipulator to the i joint coordinate system is shown in formula (1).

Figure BDA0003826167530000041
Figure BDA0003826167530000041

进一步可由式(1)推导得到人体手臂从基本坐标系到终端坐标系的坐标变换矩阵

Figure BDA0003826167530000042
Further, the coordinate transformation matrix of the human arm from the basic coordinate system to the terminal coordinate system can be derived from formula (1)
Figure BDA0003826167530000042

Figure BDA0003826167530000051
Figure BDA0003826167530000051

同理,得到式(3)中的6自由度机械臂的变换矩阵为

Figure BDA0003826167530000052
Similarly, the transformation matrix of the 6-DOF manipulator in formula (3) is obtained as
Figure BDA0003826167530000052

Figure BDA0003826167530000053
Figure BDA0003826167530000053

上式中,[nx,ny,nz;ox,oy,oz;ax,ay,az]与[px py pz]分别表示末端执行器的姿态和位置。In the above formula, [n x , n y , n z ; o x , o y , o z ; a x , a y , a z ] and [p x p y p z ] denote the posture and position of the end effector, respectively .

所述人-机械臂运动映射模中Kinect相机获取人体关节数据并进行滤波处理的方法为:The Kinect camera in the described human-mechanical arm motion mapping model acquires the human body joint data and performs filtering processing as follows:

通过Kinect直接获取人体关节数据,其中包含25个相对于摄像机坐标的三维关节位置,如图5所示,包括:脊柱根基、脊柱中段、脖子、头部、右肩、右肘、右手腕、右手掌、左肩、左肘、左手腕、左手掌、右髋、右膝盖、右脚踝、右脚、左髋、左膝盖、左脚踝、左脚、脊柱与肩的交点、右手尖、右拇指、左手尖、左拇指。Human body joint data is obtained directly through Kinect, which contains 25 three-dimensional joint positions relative to the camera coordinates, as shown in Figure 5, including: spine foundation, mid-spine, neck, head, right shoulder, right elbow, right wrist, right Palm, left shoulder, left elbow, left wrist, left palm, right hip, right knee, right ankle, right foot, left hip, left knee, left ankle, left foot, intersection of spine and shoulder, right tip, right thumb, left hand Pointer, left thumb.

当人体保持静止时,Kinect采集的关节点的位置不是恒定的,而是在实际值附近波动;其有两个原因,一是静止的人体并不是真正静止的;二是Kinect和身体跟踪方法收集的数据有误差。同时凸起的关节点的坐标会影响对人腕部坐标的描述。为了提高系统的准确性,在使用数据前需要对采集到的数据通过卡尔曼滤波进行数据过滤,如下:When the human body remains still, the position of the joint points collected by Kinect is not constant, but fluctuates around the actual value; there are two reasons for this, one is that the static human body is not really static; the other is that Kinect and body tracking methods collect data has errors. At the same time, the coordinates of the raised joint points will affect the description of the coordinates of the human wrist. In order to improve the accuracy of the system, it is necessary to filter the collected data through Kalman filtering before using the data, as follows:

xk=Akxk-1+Bkuk+wk zk=Hkxk+vk (4)x k =A k x k-1 +B k u k +w k z k =H k x k +v k (4)

xk是状态向量,uk是状态控制向量,zk是测量向量。Ak是状态转移矩阵,Bk是控制输入矩阵,Hk是状态观测矩阵。wk代表过程噪声,vk代表测量噪声。wk和vk服从高斯分布,其协方差矩阵分别为Q和R,即wk~N(0,Q),vk~N(0,R)。x k is the state vector, u k is the state control vector, z k is the measurement vector. A k is the state transition matrix, B k is the control input matrix, and H k is the state observation matrix. w k represents the process noise and v k represents the measurement noise. w k and v k obey Gaussian distribution, and their covariance matrices are Q and R respectively, that is, w k ~N(0,Q), v k ~N(0,R).

所述人-机械臂运动映射模中在采集到人体关节数据后,对人手臂末端位姿及关节角度进行计算,方法如下:In the human-mechanical arm motion mapping module, after collecting the data of human joints, the pose and joint angle of the end of the human arm are calculated, and the method is as follows:

当人体与地面不垂直时,人体手臂的基础坐标会出错。为了解决这一问题,本发明中建立了人体右臂的几何模型,如图5所示。该模型有三个坐标系统,分别为肩部坐标系统

Figure BDA0003826167530000054
手腕坐标系统
Figure BDA0003826167530000055
和Kinect坐标
Figure BDA0003826167530000056
Figure BDA0003826167530000057
需要左肩、右肩和脊柱的平面来构建坐标系,图中A点为左肩,G点为脊柱中段,O点为右肩,C点为右手腕,D点、E点、F点分别为右拇指、右手与右手指尖;上述各点均为kinect相机直接测量获得,与相机识别的25个点相对应。When the human body is not perpendicular to the ground, the basic coordinates of the human arm will be wrong. In order to solve this problem, the geometric model of the right arm of the human body is established in the present invention, as shown in FIG. 5 . The model has three coordinate systems, namely the shoulder coordinate system
Figure BDA0003826167530000054
Wrist Coordinate System
Figure BDA0003826167530000055
and Kinect coordinates
Figure BDA0003826167530000056
Figure BDA0003826167530000057
The planes of the left shoulder, right shoulder and spine are needed to construct the coordinate system. In the figure, point A is the left shoulder, point G is the middle of the spine, point O is the right shoulder, point C is the right wrist, and points D, E, and F are the right Thumb, right hand and right fingertips; the above points are directly measured by the kinect camera, corresponding to the 25 points recognized by the camera.

令上述点O为基础坐标的原点,根据上述建立的人体右臂的几何模型,则:Let the above point O be the origin of the basic coordinates, and according to the geometric model of the right arm of the human body established above, then:

①人手臂末端位姿计算方法为:① The calculation method of the end pose of the human arm is:

Figure BDA0003826167530000061
与向量
Figure BDA0003826167530000062
方向相同,
Figure BDA0003826167530000063
与平面AGO的法向量相同。因此,
Figure BDA0003826167530000064
是由右手坐标系的规定得到的,如下:
Figure BDA0003826167530000061
with vector
Figure BDA0003826167530000062
same direction,
Figure BDA0003826167530000063
Same as normal vector for planar AGO. therefore,
Figure BDA0003826167530000064
It is obtained by the regulation of the right-handed coordinate system, as follows:

Figure BDA0003826167530000065
Figure BDA0003826167530000065

点C为末端执行器(即右手腕)坐标的原点。因此,腕关节的位置为:Point C is the origin of the coordinates of the end effector (ie, the right wrist). Therefore, the position of the wrist joint is:

Figure BDA0003826167530000066
Figure BDA0003826167530000066

Figure BDA0003826167530000067
表示平面DEF的法向量,
Figure BDA0003826167530000068
与向量
Figure BDA0003826167530000069
方向相同。
Figure BDA00038261675300000610
Figure BDA00038261675300000611
都是由右手得到的,如下:use
Figure BDA0003826167530000067
represents the normal vector of the plane DEF,
Figure BDA0003826167530000068
with vector
Figure BDA0003826167530000069
same direction.
Figure BDA00038261675300000610
with
Figure BDA00038261675300000611
are obtained by the right hand, as follows:

Figure BDA00038261675300000612
Figure BDA00038261675300000612

Figure BDA00038261675300000613
Figure BDA00038261675300000613

②人手臂关节角度计算方法② Calculation method of human arm joint angle

如图6所示,大臂与YOZ平面的夹角θ1是通过计算向量

Figure BDA00038261675300000614
在YOZ平面上的投影与
Figure BDA00038261675300000615
的夹角获得的,大臂与小臂的夹角θ2是向量
Figure BDA00038261675300000616
Figure BDA00038261675300000617
的夹角,手腕的夹角θ3
Figure BDA00038261675300000618
Figure BDA00038261675300000619
的夹角表示。As shown in Figure 6, the angle θ 1 between the boom and the YOZ plane is obtained by calculating the vector
Figure BDA00038261675300000614
The projection on the YOZ plane is the same as
Figure BDA00038261675300000615
The included angle is obtained, and the included angle θ 2 between the arm and the forearm is a vector
Figure BDA00038261675300000616
and
Figure BDA00038261675300000617
The included angle, the included angle of the wrist θ 3 is used
Figure BDA00038261675300000618
and
Figure BDA00038261675300000619
angle representation.

通过上述得到人手臂末端位姿及关节角度后,即可将人手臂末端的位姿以及人手臂各个关节角度分别映射机械臂末端及机械臂各个关节,具体映射过程如下:After obtaining the pose and joint angle of the end of the human arm through the above, the pose of the end of the human arm and the angles of each joint of the human arm can be mapped to the end of the manipulator and each joint of the manipulator. The specific mapping process is as follows:

Ⅰ、进行末端姿态映射Ⅰ. Perform terminal attitude mapping

采用点对点方法将人手臂末端位置映射到机器人的末端执行器位置,结合式(2)与式(3)得到的两个变换矩阵,通过逆运动学求解获得机械臂各个关节角度。映射公式如下:The point-to-point method is used to map the position of the end of the human arm to the position of the end effector of the robot. Combining the two transformation matrices obtained by formula (2) and formula (3), the angles of each joint of the manipulator are obtained by solving the inverse kinematics. The mapping formula is as follows:

Figure BDA00038261675300000620
Figure BDA00038261675300000620

其中,[xr,yr,zr]T是机器人末端执行器的位置;[xh,yh,zh]T是人类手臂末端执行器位置;kx、ky、kz是比例系数;lx、ly、lz是坐标的平移距离;kx、ky、kz、lx、ly、lz根据人手臂和机械臂的实际尺寸计算得到。where [x r , y r , z r ] T is the robot end effector position; [x h , y h , z h ] T is the human arm end effector position; k x , ky , k z are the ratio Coefficients; l x , ly , l z are translational distances of the coordinates; k x , ky , k z , l x , ly , l z are calculated according to the actual size of the human arm and the mechanical arm.

一般情况下,机械臂逆运动学求解会存在多个解,本发明针对最优解选择提出最优解应满足:In general, there will be multiple solutions for the inverse kinematics solution of the manipulator. The present invention proposes that the optimal solution should satisfy the following requirements for optimal solution selection:

1)为了保证机械臂运动的连续性和稳定性,当前时刻的第1个关节角度值(即机械臂基座旋转角度)选择与上一时刻差值最小的;1) In order to ensure the continuity and stability of the movement of the manipulator, the angle value of the first joint at the current moment (that is, the rotation angle of the manipulator base) is selected to have the smallest difference from the previous moment;

2)为了使机械臂做出与人手臂相似的姿态,在1)的基础上,选择与θ1差值最小的第2个关节角度值。按照以上方式最终可确定一组最优解。2) In order to make the robot arm make a posture similar to that of the human arm, on the basis of 1 ), select the second joint angle value with the smallest difference from θ1. According to the above method, a group of optimal solutions can be finally determined.

Ⅱ、加入关节角度映射Ⅱ. Add joint angle mapping

由于Kinect相机识别精度有限,存在两个问题:1)逆运动学解不存在;2)机械臂的姿态与期望的姿态存在一定的差距。Due to the limited recognition accuracy of the Kinect camera, there are two problems: 1) the inverse kinematics solution does not exist; 2) there is a certain gap between the pose of the robot arm and the expected pose.

针对逆运动学解不存在的情况,机械臂的第2~4个关节角度值无法求解,可将人手臂的关节角度θ1、θ2、θ3直接映射到机械臂的第2~4个关节。In the case that the inverse kinematics solution does not exist, the angle values of the 2nd to 4th joints of the manipulator cannot be solved, and the joint angles θ 1 , θ 2 , θ 3 of the human arm can be directly mapped to the 2nd to 4th joint angles of the manipulator. joint.

针对姿态不一致的问题,加入关节角度映射,将前述得到的最优解中的第2、3个关节角度值改为θ1、θ2Aiming at the problem of inconsistency in attitude, joint angle mapping is added, and the second and third joint angle values in the optimal solution obtained above are changed to θ 1 and θ 2 .

添加关节角度映射之后的控制效果明显优于单纯的末端位姿映射,由此实现了人-机械臂异构运动空间的混合映射。The control effect after adding the joint angle mapping is significantly better than the simple end pose mapping, thus realizing the hybrid mapping of the heterogeneous motion space of the human-robot arm.

所述机械臂运动规划和控制模块采用socket实现Windows端与Linux端ROS系统的通信,将通过前述方法的到的最终期望关节角度值发布到ROS节点上,在ROS端订阅话题消息,为了减小时延,每次都订阅话题节点上最新的消息。Moveit是ROS中一系列移动操作的功能包的组成,主要包含运动规划,碰撞检测,运动学,3D感知,操作控制等功能,可以与真实机械臂建立通信。因此用Moveit对机械臂进行正向运动学轨迹规划,最终实现对真实机械臂的控制。Described manipulator motion planning and control module adopts socket to realize the communication of Windows end and Linux end ROS system, publishes the final expected joint angle value obtained by the aforementioned method to the ROS node, subscribes topic message at ROS end, in order to reduce time Delay, subscribe to the latest news on the topic node every time. Moveit is a function package of a series of mobile operations in ROS, mainly including motion planning, collision detection, kinematics, 3D perception, operation control and other functions, which can establish communication with real robotic arms. Therefore, Moveit is used to plan the forward kinematics trajectory of the manipulator, and finally realize the control of the real manipulator.

基于上述遥操作系统,本发明提出遥操作控制具体流程如下:Based on the above-mentioned teleoperation system, the present invention proposes the specific flow of teleoperation control as follows:

A、获取由Kinect摄像头采集的人体关节坐标;A, obtain the human body joint coordinates collected by the Kinect camera;

B、对人体关节数据进行滤波,并计算人手臂末端位姿及关节角度;B. Filter the human joint data, and calculate the pose and joint angle of the end of the human arm;

C、根据人手臂和机械臂的真实长度计算映射参数kx、ky、kz、lx、ly、lz,并进行末端位姿映射;C. Calculate the mapping parameters k x , ky , k z , l x , ly , l z according to the real length of the human arm and the robotic arm, and perform terminal pose mapping;

D、根据机械臂的D-H参数构建逆运动学求解函数;D. Construct an inverse kinematics solution function according to the D-H parameters of the mechanical arm;

E、根据最优解选择原则选出最优解;E. Select the optimal solution according to the optimal solution selection principle;

F、添加关节角度映射;F. Add joint angle mapping;

G、实现通信,将期望关节角度发布到ROS节点上;G. Realize communication and publish the desired joint angle to the ROS node;

H、订阅话题消息并进行处理,用Moveit进行运动规划,并在Gazebo中进行仿真;H. Subscribe and process topic messages, use Moveit for motion planning, and simulate in Gazebo;

G、操作者正对Kinect相机做出动作,机械臂跟随操作者右手臂的运动做出相应的姿态,实时交互完成遥操作任务。G. The operator is making an action on the Kinect camera, and the robotic arm follows the movement of the operator's right arm to make a corresponding gesture, and interacts in real time to complete the teleoperation task.

Claims (7)

1. A teleoperation system based on human-mechanical arm heterogeneous motion space hybrid mapping is divided into two modules, namely a human-mechanical arm motion mapping module and a mechanical arm motion planning and control system; the method is characterized in that:
the human-mechanical arm motion mapping module is constructed, a human joint angle and position posture recognition system is built, human joint position data are obtained by adopting the human joint recognition system of the Kinect camera, a human skeleton is drawn, kalman filtering is carried out on the human joint position data, and then the tail end pose and the joint angle of the human arm are calculated and output;
a human-machine motion space mixed mapping method is adopted, the pose of the tail end of the human arm is mapped to the tail end of the mechanical arm, and inverse kinematics solution is carried out on the mechanical arm; further adding joint mapping to obtain an optimal solution of the joint angle of the mechanical arm;
the mechanical arm motion planning and control system establishes communication transmission between Windows and the ROS by using a socket, issues expected joint angle data to ROS nodes, uses Moveit subscription to perform motion planning, finally transmits control information to a real mechanical arm, and controls the real mechanical arm to move.
2. The teleoperation system based on the human-mechanical arm heterogeneous motion space hybrid mapping, as claimed in claim 1, wherein: establishing a kinematics model in a human-mechanical arm movement mapping module, wherein the establishment method adopts a D-H method, and two kinematics models, namely a 5-DOF arm model and a UR5 robot model, are established on the basis of robot kinematics modeling; wherein the 5-DOF arm model includes a shoulder 3-DOF and an elbow 2-DOF; the Ur5 robot model is a 6-DOF mechanical arm.
3. The teleoperation system based on the human-mechanical arm heterogeneous motion space hybrid mapping, as claimed in claim 1, wherein: the Kalman filtering method for the human body joint position data comprises the following steps:
x k =A k x k-1 +B k u k +w k z k =H k x k +v k
x k is a state vector, u k Is a state control vector, z k Is a measurement vector; a. The k Is a state transition matrix, B k Is a control input matrix, H k Is a state observation matrix; w is a k Representative of process noise, v k Representative of measurement noise; w is a k And v k Obeying a Gaussian distribution with covariance matrices Q and R, respectively, i.e. w k ~N(0,Q),v k ~N(0,R)。
4. The teleoperation system based on the human-mechanical arm heterogeneous motion space hybrid mapping, as claimed in claim 1, wherein: the method for calculating the end pose and the joint angle of the human arm comprises the following steps:
a geometric model of the right arm of the human body is established, and the geometric model is provided with three coordinate systems which are respectively a shoulder coordinate system
Figure FDA0003826167520000011
Figure FDA0003826167520000012
Wrist coordinate system
Figure FDA0003826167520000013
And Kinect coordinates
Figure FDA0003826167520000014
Let point A be the left shoulder, point G be the middle section of the spine, point O be the right shoulder, point C be the right wrist, point D, point E, point F be the right thumb, right hand and right finger tip respectively;
according to the geometric model of the right arm of the human body, which is established as above, the following steps are carried out:
(1) the method for calculating the pose of the tail end of the human arm comprises the following steps:
Figure FDA0003826167520000021
and vector
Figure FDA0003826167520000022
The direction is the same as that of the first and second guide rails,
Figure FDA0003826167520000023
the normal vector is the same as the planar AGO; therefore, the temperature of the molten metal is controlled,
Figure FDA0003826167520000024
is derived from the specification of the right hand coordinate system as follows:
Figure FDA0003826167520000025
thus, the positions of the wrist joints are:
Figure FDA0003826167520000026
by using
Figure FDA0003826167520000027
A normal vector of the plane DEF is represented,
Figure FDA0003826167520000028
and vector
Figure FDA0003826167520000029
The directions are the same;
Figure FDA00038261675200000210
and
Figure FDA00038261675200000211
all obtained from the right hand, as follows:
Figure FDA00038261675200000212
Figure FDA00038261675200000213
(2) human arm joint angle calculation method
Angle theta between the large arm and the YOZ plane 1 By calculating vectors
Figure FDA00038261675200000214
Projection on YOZ plane
Figure FDA00038261675200000215
Obtaining the included angle of the angle; the included angle theta between the big arm and the small arm 2 Is a vector
Figure FDA00038261675200000216
And
Figure FDA00038261675200000217
angle of the wrist theta 3 By using
Figure FDA00038261675200000218
And with
Figure FDA00038261675200000219
Is shown.
5. The teleoperation system based on the human-mechanical arm heterogeneous motion space hybrid mapping, as claimed in claim 1, wherein: the man-machine motion space hybrid mapping method comprises the following steps:
mapping the tail end position of the arm of the human hand to the position of an end effector of the robot by adopting a point-to-point method, combining a coordinate transformation matrix of the arm of the human body from a basic coordinate system to a terminal coordinate system and a transformation matrix of the arm with 6 degrees of freedom, and solving by inverse kinematics to obtain each joint angle of the arm; the mapping formula is as follows:
Figure FDA00038261675200000220
wherein, [ x ] r ,y r ,z r ] T Is the position of the robot end effector; [ x ] of h ,y h ,z h ] T Is the human arm end effector position; k is a radical of formula x 、k y 、k z Is a proportionality coefficient; l x 、l y 、l z Is the translation distance of the coordinates; k is a radical of x 、k y 、k z 、l x 、l y 、l z Calculating according to the actual sizes of the human arm and the mechanical arm;
the optimal solution should satisfy:
1) In order to ensure the continuity and stability of the motion of the mechanical arm, the difference value between the value of the 1 st joint angle at the current moment and the previous moment is the minimum;
2) In order to make the mechanical arm to make a similar posture as the human arm, on the basis of 1), theta is selected 1 The 2 nd joint angle value with the smallest difference; finally, a set of optimal solutions is determined.
6. The teleoperation system based on the human-mechanical arm heterogeneous motion space hybrid mapping, as claimed in claim 1, wherein: the joint mapping method comprises the following steps:
aiming at the condition that the inverse kinematics solution does not exist, the 2 nd to 4 th joint angle values of the mechanical arm cannot be solved, so that the joint angle theta of the human arm is determined 1 、θ 2 、θ 3 Directly mapping to the 2 nd to 4 th joints of the mechanical arm;
aiming at the problem of inconsistent postures, joint angle mapping is added, and the 2 nd and 3 rd joint angle values in the obtained optimal solution are changed into theta 1 、θ 2
7. The teleoperation system based on human-mechanical arm heterogeneous motion space hybrid mapping of claim 1, wherein: the specific process of teleoperation control is as follows:
A. acquiring human body joint coordinates acquired by a Kinect camera;
B. filtering the human joint data, and calculating the terminal pose and the joint angle of the human arm;
C. calculating mapping parameters according to the real lengths of the human arm and the mechanical arm, and mapping the end pose;
D. constructing an inverse kinematics solving function according to the D-H parameters of the mechanical arm;
E. selecting an optimal solution according to an optimal solution selection principle;
F. adding joint angle mapping;
G. communication is achieved, and the expected joint angle is issued to the ROS node;
H. subscribing and processing topic messages, performing motion planning by using Moveit, and performing simulation in a Gazebo;
G. and (5) finishing the teleoperation task in real time in an interactive manner.
CN202211059660.8A 2022-09-01 2022-09-01 A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space Active CN115469576B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211059660.8A CN115469576B (en) 2022-09-01 2022-09-01 A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211059660.8A CN115469576B (en) 2022-09-01 2022-09-01 A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space

Publications (2)

Publication Number Publication Date
CN115469576A true CN115469576A (en) 2022-12-13
CN115469576B CN115469576B (en) 2024-06-18

Family

ID=84369347

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211059660.8A Active CN115469576B (en) 2022-09-01 2022-09-01 A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space

Country Status (1)

Country Link
CN (1) CN115469576B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116214549A (en) * 2023-01-19 2023-06-06 中国科学院微小卫星创新研究院 Teleoperation system and teleoperation method for space robot
CN116602772A (en) * 2023-05-19 2023-08-18 哈尔滨思哲睿智能医疗设备股份有限公司 Surgical robot, control method, device and medium for slave-end surgical arm of surgical robot
CN116619422A (en) * 2023-06-21 2023-08-22 哈尔滨理工大学 Motion capture and wrist joint angle estimation mechanical arm mapping method based on human body characteristic signals
CN116979419A (en) * 2023-09-12 2023-10-31 国网江苏省电力有限公司常州供电分公司 Bypass cable traction system and method
CN117921677A (en) * 2024-03-06 2024-04-26 睿尔曼智能科技(北京)有限公司 Mechanical arm posture following control method and device
CN118046394A (en) * 2024-04-15 2024-05-17 帕西尼感知科技(张家港)有限公司 A remote motion control method, device, system and storage medium
WO2025043853A1 (en) * 2023-08-28 2025-03-06 东南大学 Safety shared control system based on performance of teleoperator

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108638069A (en) * 2018-05-18 2018-10-12 南昌大学 A kind of mechanical arm tail end precise motion control method
CN108828996A (en) * 2018-05-31 2018-11-16 四川文理学院 A kind of the mechanical arm remote control system and method for view-based access control model information
US20210197384A1 (en) * 2019-12-26 2021-07-01 Ubtech Robotics Corp Ltd Robot control method and apparatus and robot using the same
CN113110142A (en) * 2021-03-29 2021-07-13 航天时代电子技术股份有限公司 Follow-up remote control operation table and remote control method thereof
US20220203526A1 (en) * 2020-12-30 2022-06-30 Ubtech Robotics Corp Ltd Humanoid robot control method, humanoid robot using the same, and computer readable storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108638069A (en) * 2018-05-18 2018-10-12 南昌大学 A kind of mechanical arm tail end precise motion control method
CN108828996A (en) * 2018-05-31 2018-11-16 四川文理学院 A kind of the mechanical arm remote control system and method for view-based access control model information
US20210197384A1 (en) * 2019-12-26 2021-07-01 Ubtech Robotics Corp Ltd Robot control method and apparatus and robot using the same
US20220203526A1 (en) * 2020-12-30 2022-06-30 Ubtech Robotics Corp Ltd Humanoid robot control method, humanoid robot using the same, and computer readable storage medium
CN113110142A (en) * 2021-03-29 2021-07-13 航天时代电子技术股份有限公司 Follow-up remote control operation table and remote control method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
赵京;王鑫;张自强;龚世秋;李晓辉;: "基于肘部自运动的主从异构7自由度机械臂运动映射及其几何逆解", 机械工程学报, no. 15, 5 August 2020 (2020-08-05), pages 195 - 204 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116214549A (en) * 2023-01-19 2023-06-06 中国科学院微小卫星创新研究院 Teleoperation system and teleoperation method for space robot
CN116214549B (en) * 2023-01-19 2024-03-01 中国科学院微小卫星创新研究院 Teleoperation system and teleoperation method for space robot
CN116602772A (en) * 2023-05-19 2023-08-18 哈尔滨思哲睿智能医疗设备股份有限公司 Surgical robot, control method, device and medium for slave-end surgical arm of surgical robot
CN116619422A (en) * 2023-06-21 2023-08-22 哈尔滨理工大学 Motion capture and wrist joint angle estimation mechanical arm mapping method based on human body characteristic signals
WO2025043853A1 (en) * 2023-08-28 2025-03-06 东南大学 Safety shared control system based on performance of teleoperator
CN116979419A (en) * 2023-09-12 2023-10-31 国网江苏省电力有限公司常州供电分公司 Bypass cable traction system and method
CN117921677A (en) * 2024-03-06 2024-04-26 睿尔曼智能科技(北京)有限公司 Mechanical arm posture following control method and device
CN118046394A (en) * 2024-04-15 2024-05-17 帕西尼感知科技(张家港)有限公司 A remote motion control method, device, system and storage medium
CN118046394B (en) * 2024-04-15 2024-06-21 帕西尼感知科技(张家港)有限公司 A remote motion control method, device, system and storage medium

Also Published As

Publication number Publication date
CN115469576B (en) 2024-06-18

Similar Documents

Publication Publication Date Title
CN115469576B (en) A teleoperation system based on hybrid mapping of human-robot heterogeneous motion space
CN107943283B (en) Robotic arm pose control system based on gesture recognition
CN107363813B (en) A desktop industrial robot teaching system and method based on wearable device
CN105291138B (en) It is a kind of to strengthen the visual feedback platform of virtual reality immersion sense
CN109262609A (en) Mechanical arm tele-control system and method based on virtual reality technology
CN106313049B (en) A kind of apery mechanical arm motion sensing control system and control method
CN113829343B (en) Real-time multitasking and multi-man-machine interaction system based on environment perception
CN110815189B (en) Robot rapid teaching system and method based on mixed reality
CN107856014B (en) Mechanical arm pose control method based on gesture recognition
Luo et al. Real time human motion imitation of anthropomorphic dual arm robot based on Cartesian impedance control
CN106346485A (en) Non-contact control method of bionic manipulator based on learning of hand motion gestures
CN102350700A (en) Method for controlling robot based on visual sense
CN110045740A (en) A kind of Mobile Robot Real-time Motion planing method based on human behavior simulation
CN105014677A (en) Visual mechanical arm control device and method based on Camshift visual tracking and D-H modeling algorithms
CN115157261B (en) Flexible robotic arm remote operation human-computer interaction device and method based on mixed reality
CN118322199B (en) A teleoperation method for a five-finger dexterous arm-hand robot based on data gloves
CN111300408B (en) Motion planning control method for humanoid double-arm robot combining shape similarity and look similarity
CN116197899A (en) Active robot teleoperation system based on VR
Xiong et al. Predictive display and interaction of telerobots based on augmented reality
CN111702787A (en) Human-machine cooperative control system and control method
CN115488880A (en) Simulation test system for mechanical arm visual servo grabbing in space environment
Chen et al. Application of haptic virtual fixtures on hot-line work robot-assisted manipulation
CN117301056A (en) A robot remote control system and method based on real-time capture of arm postures
Li et al. Intelligent robotic arm for human pose recognition based on teleoperation system
Xu et al. Design of a human-robot interaction system for robot teleoperation based on digital twinning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant