CN100389013C - Reconstruction method of humanoid robot operation scene based on multi-information fusion - Google Patents

Abstract

The operating scene of the humanoid robot constructed by the present invention is a real-time video image combined with operator control commands and feedback information. In the scene, the humanoid robot model and its working environment model at the job site can be displayed in real time. The scene can receive the position information of the humanoid robot's job site and the sensing information of the robot in operation. Use the above information to drive the motion of the humanoid robot model and its environment model to display the real-time video image. At the same time, the scene is predicted and simulated according to the commands issued by the operator, and the ideal humanoid robot operation data and position data of each model are generated. Use data generated by predictive simulations to drive models when real-time feedback is missing or unavailable. The operator can change the angle of view arbitrarily to achieve full-angle observation.

Description

Reconstruction method of humanoid robot operation scene based on multi-information fusion

Technical field:

The invention belongs to the field of robots and is mainly used for three-dimensional reconstruction of a humanoid robot operation scene. It is suitable for robot teleoperation control, and can display the three-dimensional images of the humanoid robot and the objects in the operation scene in real time, providing visual presence for the humanoid robot teleoperation.

Background technique:

A humanoid robot is a robot that has human appearance characteristics and can simulate basic human actions. Teleoperation is an important technology for robotic applications. Through the teleoperation platform, operators can monitor and control remote robots to complete various tasks, so that humanoid robots can replace humans to complete various tasks in some inaccessible, even some endangering human health or life safety environments.

The image display of the working environment is a key technology of teleoperation control. At present, the image display of the working environment mainly adopts two ways of on-site two-dimensional video image and three-dimensional virtual scene based on the operator's command.

The on-site two-dimensional video image method is to install multiple video cameras on the robot's work site to take video shots of the robot and its work site. The video image is transmitted to the operator through the network and displayed on the computer. Such an image is a real scene of robot operation, which can truly reflect the situation of robot operation. However, there are the following disadvantages: 1) It cannot provide three-dimensional information, so the actual three-dimensional position operation is difficult to realize; 2) The viewing angle is limited, and the camera installed on the robot operation site is fixed in position and cannot provide a comprehensive viewing angle; 3) Large time delay , the amount of video image file data transmitted in the network is relatively large, and in the case of limited network bandwidth, there is a large delay in the transmission process.

The basic principle of the three-dimensional virtual scene based on the operator's command is that the operator sends various operating instructions to the robot, assuming that the robot executes correctly and generates the corresponding movement. Modeling software, which establishes the robot motion model and three-dimensional virtual image based on the operator's instructions, and produces the effect of predictive simulation. Such an image is a prediction of the robot's execution process, which can provide the operator with a vivid job scene. In addition, since it is made with 3D modeling software, the operator can easily change the viewing angle. But there are the following disadvantages: what such a system provides to the operator is the prediction and simulation of the robot and its job site, which cannot truly reflect the robot's job situation.

Invention content:

The invention combines operator control commands, sensor detection information of the humanoid robot itself and environment detection information to construct a three-dimensional scene of a working environment.

The technical scheme adopted in the present invention is:

The operating scene of the humanoid robot constructed by the present invention is a real-time video image combined with operation commands and feedback information. The scene can display the humanoid robot model and the environment model of the robot job site in real time.

The scene can receive the position information of the humanoid robot's job site and the sensing information of the robot in operation. Use the above information to drive the motion of the humanoid robot model and its environment model to display real-time video animation images. At the same time, the scene receives and predicts the commands issued by the operator, and generates ideal robot operation data and position data of each model. Use data generated by predictions to drive models when real-time feedback is missing or unavailable.

The operator can change the angle of view arbitrarily to achieve full-angle observation.

Its main technical solutions are:

Use 3D modeling software to make 3D models of various objects on the job site of humanoid robots. The models have the same appearance characteristics as the actual objects. At the same time, a data processing module matching the model is established. The data processing module can receive a variety of information, and drive the model movement in the job scene through matching processing.

There are two main models of humanoid robot operation scenarios: environment model and humanoid robot model.

(1) Use 3D modeling software to make an environment model in the robot workplace with a known structure. The environment model has the same appearance characteristics and positional relationship as the actual object. All-round observation can be realized.

(2) Use 3D modeling software to make a humanoid robot model. The model has the same geometric features and degrees of freedom settings as the robot, and meets the kinematic constraints of the multi-link mechanism of the humanoid robot. A humanoid robot model receives position data to orient itself in the scene. The model receives the joint angle data of each degree of freedom to drive the angle change between the connecting rods to represent the motion of the humanoid robot.

Install a position sensor on the robot's job site, measure the position of the robot and its job target in real time, and output the measured three-dimensional coordinate data. Each joint of the robot body is equipped with an angle sensor, which can measure the relative angle data between the connecting rods connected to each joint of the robot body in real time.

The data processing module mainly realizes the following functions:

(1) Receive the position data of the robot in its working environment, and position the robot model in the environment model of the working place.

(2) Receive the relative angle data between the connecting rods connected to each joint of the robot body, match it to the robot model, and drive the movement between the connecting rods of the model to express the motion of the robot.

(3) Receive the command issued by the operator and explain the command. According to the command, the ideal trajectory of the robot's operation is generated, and the operating data of the robot is predicted. The motion track that can be generated includes the change track of the joint angle data of the robot during the command execution process, and the change track of the position data of the robot. Ideally, this data would be the same as the data fed back from the robot's job site.

During the working process of the scene, the data processing module normally receives data from the robot work site in real time to drive various model movements. In the case of temporary lack of on-site data, the prediction data generated by the simulation prediction issued by the operator is used to drive the movement of various models and maintain the continuity of the model movement.

The beneficial effects of the present invention are:

1. Integrating multiple information to display animated images of the robot's job site in real time. The drivers of each model in the scene use on-site feedback data. When the on-site feedback data is temporarily missing, use the command to predict the data for display to maintain the continuity of the display.

2. Observe images from all angles of view. The operator can change the viewing angle arbitrarily and observe the details of the scene.

Description of drawings:

Figure 1 is a working principle diagram of a humanoid robot operation scene based on multi-information fusion

Detailed ways:

The working process of the whole humanoid robot working scene is as follows:

In the first step, the robot starts running and the teleoperation control starts. Start the computer program to display the established scene model. Use the initialization data to determine the initial position of the robot model and its operation target model, and determine the initial angle between the connecting rods of the robot model. What is generated in this step is the initialization interface of the scene.

In the second step, the scene data processing module receives the operation commands issued by the teleoperator in real time, interprets and generates predicted trajectory data. Use the prediction data to drive each model in the virtual scene to form a three-dimensional virtual scene. The scene generated in this step can show the ideal motion picture of the robot executing the commands issued by the operator.

In the third step, the sensor of the robot itself measures the data of each joint angle in real time, and transmits it to the data processing module of the scene through the teleoperation platform. The data processing module stops using the predicted joint angle data, and instead uses the real joint angle data to drive the model to express the motion of the robot itself. At this time, the robot model in the three-dimensional virtual scene can truly express the running state of the robot on the job site.

In the fourth step, the environmental detection sensor on the robot's job site starts to operate, and obtains the position data of the robot and its job target. The above data is transmitted to the data processing module of the scene through the teleoperation platform. The data module uses the above data to localize the robot model and its environment model. At this time, the three-dimensional scene can truly show the positional relationship between the robot and its environment in the job site.

At the same time, the robot's own sensor feedback data and the environment detection position sensor feedback data are used to drive the three-dimensional virtual scene of the model of the robot and its working environment, which can truly express the robot's operating state and its position information in the environment. Among them, in the case of temporary lack of on-site feedback, the scene data processing module can automatically select the trajectory data generated by command prediction to drive various models. After reacquiring the feedback data, switch to the 3D virtual scene where the model is driven by real data.

Claims (1)

1. the distant operation task scene reconstruction of the human emulated robot based on many information fusion method, can make the operator monitor real machine people's practical operation situation, it is characterized in that: show the environmental model in human emulated robot and the operation place thereof in the real time video image; Each joint angle data of the real-time robot measurement of the sensor of robot itself, the environmental detection sensor at robot manipulating task scene gets access to the position data of robot and operative goals thereof, use above data to come driving model to show real time video image, can express the position relation of robot in the running status of robot and the operation field and environment thereof really; The operator can change the visual angle arbitrarily to observe the details of model; Scene can also receive the order that the operator sends, make an explanation, the generation forecast data, at the scene under the situation of the temporary transient disappearance of real time data, use the model sport in the prediction data driving scene, after getting access to feedback data again, switch to three-dimensional virtual scene by the True Data driving model.

Images