CN106005086A - Leg-wheel composite robot based on Xtion equipment and gesture control method thereof - Google Patents

Abstract

The invention discloses a leg-wheel composite robot based on Xtion equipment and a gesture control method thereof, including a robot body and six robot single-leg structures; a sensing layer, a driving layer, a control layer and a power layer inside the robot body for installation and control system; the gesture control is realized through the XtionPRO LIVE camera installed on the sensing layer combined with the gesture recognition module and the motion control module installed on the control layer; wherein, the gesture recognition module is divided into a dynamic gesture control sub-module and a static gesture control sub-module, which can be Realize dynamic and static gesture control respectively. The invention has the advantages of providing more information for subsequent image processing, realizing map construction and human-computer interaction functions, improving the naturalness of human-robot interaction, enabling users to control the six-legged wheel-leg composite robot through gestures, It is beneficial to the practical application of the hexapod wheel-leg composite robot; and the gesture recognition control part of the present invention is based on ROS design, has strong portability, and can be used in other robot control systems.

Description

A leg-wheel composite robot based on Xtion device and its gesture control method

technical field

The invention belongs to the technical field of robots, and in particular relates to an Xtion-based leg-wheel composite robot and a gesture control method and device thereof.

Background technique

Robots have broad application prospects in social life and industrial production. Applications in industrial production, such as planetary detection, post-disaster rescue, etc. The application prospects in social life, such as helping the elderly and the disabled. As our country enters an aging society, the elderly need to be taken care of, while the young and middle-aged need to work to make ends meet, and have no time to take care of the elderly. Robots can act as part of the labor force to supplement the shortage of labor force, and have broad application prospects in helping the elderly, the young, the sick, and the disabled.

Legged robots can adapt to complex terrain, but the walking speed is slow, while wheeled robots walk faster, but can only walk on relatively flat ground, and the ability to cross obstacles is poor. The combination of legs and wheels It provides new convenience for the movement of the robot, and how to design a combination of wheels and legs has become a problem that needs to be solved at present.

Leg-wheeled robots usually require a specific person to input specific control commands to control the movement of the robot. Ordinary people without professional knowledge cannot directly operate it. This interactive method restricts the wide application of robots to a certain extent.

Contents of the invention

In view of the above problems, the present invention provides a leg-wheel composite robot based on Xtion equipment, which enables better interaction between the robot and humans.

The leg-wheel composite robot based on the Xtion device of the present invention includes a robot body and 6 robot single-leg structures, and the 6 robot single-leg structures are uniformly installed on the robot body in the circumferential direction. The single-leg structure is a wheel-leg structure with the functions of walking and wheeling. It has three leg sections and four driving servos; the three leg sections are respectively the first leg section, the second leg section, and the third leg section. The driving steering gears are respectively the first steering gear, the second steering gear, the third steering gear and the fourth steering gear. Wherein, one end of the first leg section is fixedly installed on the output shaft of the first steering gear to form a hip joint; the axis of the output shaft of the first steering gear is perpendicular to the horizontal plane, and the first steering gear drives the first leg section to swing laterally. The second steering gear is fixedly installed on the other end of the first leg joint, and the output shaft of the second steering gear is fixed to one end of the second leg joint to form a knee joint; the axis of the output shaft of the second steering gear is perpendicular to the axis of the output shaft of the first steering gear, by The second steering gear drives the second leg section to swing longitudinally. The other end of the second leg section is fixed to the output shaft of the third steering gear to form an ankle joint; the axis of the output shaft of the third steering gear is parallel to the axis of the output shaft of the second steering gear, and the third leg joint is driven by the third steering gear to swing longitudinally. The fourth steering gear is located in the middle of the third leg section. The axis of the output shaft of the fourth steering gear is parallel to the axis of the output shaft of the third steering gear. The output shaft of the fourth steering gear is coaxially fixed with wheels through splines. The machine drives the wheels to turn. The other end of the third leg section is equipped with a foot-ground detection mechanism, which is used for the contact between the single-leg structure and the ground, and can realize the detection of the foot-ground contact state at the same time.

The interior of the robot body is divided into four layers by an aluminum alloy plate with weight-reducing holes. From top to bottom, it is the sensing layer, driving layer, control layer and power layer, which are used to install the control system. Among them, Xtion PRO LIVE camera and IMU are installed on the sensing layer; Xtion PRO LIVE camera is used to collect environmental information, realize simultaneous positioning and map creation, and obtain human information for human-computer interaction; IMU is used to obtain robot information Own pose, assist localization and create environment map. Six steering gear drive boards are installed on the driving layer, which are used to control the movement of each steering gear on the 6 single-leg structures. A main control board is installed on the control layer to realize functions such as communication management, sensor data collection, data processing and drive management. A gesture control module and a motion control module are also arranged in the control layer to realize the gesture control of the leg-wheel composite robot. A battery box is installed on the power layer, and a robot power supply battery is installed in the battery box.

The robot body communicates wirelessly with the remote control terminal, and is controlled by the remote control terminal; the remote control terminal has a safety authentication module, an audio and video playback module, a control module and an information display module. Each module is divided into a front-end and a back-end; the front-end is a human-computer interaction interface; the back-end is responsible for network communication and data processing. Among them, the security authentication module is used to obtain the user information input by the user, which is encapsulated into a data packet through the background and sent to the main control board of the robot, and the main control board feeds back the login result to the security authentication module for display. The audio and video playback module receives the sound information returned by the Xtion PRO LIVE camera and the images collected by the camera through the background, and displays it in the audio and video playback module after being decoded in the background. The information display module receives six sets of robot sensing information returned by the IMU, such as posture, joint angle and joint torque information, and is used to display the sensing information of the hexapod robot. The control module is composed of motion controls, which can control the movement of the robot; at the same time, it can set the robot's travel mode.

The gesture recognition method for the leg-wheel composite robot based on the above-mentioned Xtion equipment is to install a gesture recognition module and a motion control module on the control layer; the gesture recognition module includes a dynamic gesture recognition submodule and a static gesture recognition submodule, wherein:

The dynamic gesture recognition sub-module is used to realize dynamic gesture recognition, and has a stability detector, a circle detector, a push detector, and a movement detector. The stability detector is used to identify whether the currently active hand is in a stable state, the circle detector is used to identify the circle movement of the currently active hand, the push detector is used to identify the forward movement of the currently active hand, and the movement detector is used to identify the current active hand. The striking action of the hand.

The specific design method of the above-mentioned dynamic gesture recognition sub-module is as follows:

A. Use the Xtion SDK of the Xtion PRO LIVE camera to initialize the environment;

B. Establish a context to create an image generator, and set the pixel and frame rate of the image generator;

C. Create and register the scene manager, and initialize the scene manager;

D. Establish and register stable detectors, circle detectors, push detectors, and motion detectors, and add the listeners in each detector to the scene manager; at the same time, set the stable detectors and circle detectors , push detector, and motion detector callback function.

The dynamic gesture recognition module designed in the above way, when performing gesture recognition, first collects RGB image information through the Xtion PRO LIVE camera; then processes the collected images based on OpenNI software, extracts the trajectory of the gesture, and passes the stable detector, Circle detectors, push detectors, and movement detectors recognize hand gestures of stillness, circles, pushes, and movements; then the stability detectors, circle detectors, push detectors, and movement detectors will be included in their respective callback functions Output the recognition result, and send the recognition result to the motion control module; the motion control module controls the robot to complete the corresponding motion according to the recognition result, combined with the mapping relationship between the dynamic gesture and the robot motion predefined in the dynamic gesture recognition module.

The static gesture recognition sub-module is based on a color image and is used to recognize static gestures; through the static gesture recognition sub-module, when performing gesture recognition, at first the RGB image information is collected by the Xtion PRO LIVE camera, and the image information passes through the robot operating system. Message implementation in Topic; then, use the skin color model and HOG features pre-stored in the static gesture recognition sub-module to detect the face part, and then detect the human body part according to the human body structure features, and segment the human body part from the body part In the hand area, finally extract the image features of the segmented hand area, conduct gesture training and recognition, and send the recognition result to the main control board. corresponding movement.

The advantages of the present invention are:

1. In the leg-wheel composite robot based on the Xtion device of the present invention, the Xtion Pro Live camera can not only obtain RGB information and depth information, but also obtain the user's bone point information, provide more information for subsequent image processing, and realize map construction , human-computer interaction function;

2. The leg-wheel composite robot based on the Xtion device of the present invention and its gesture control method have strong portability. The method is based on ROS. Specifically, the dynamic gesture recognition is based on ROS and OpenNI framework design and implementation, and the static gesture recognition is based on ROS, which can be used in other robot control systems;

3. The leg-wheel composite robot based on the Xtion device of the present invention and its gesture control method include two modes of static gesture control and dynamic gesture control, and the user can choose an appropriate control mode according to needs;

4. The present invention is based on the Xtion device-based leg-wheel composite robot and its gesture control method. The research object is a picture of the upper body of a person, and the gesture area is segmented based on the human body structure, which has strong versatility. This segmentation method can not only segment ungloved hands, but also segment gloved hands;

5. The present invention is based on the Xtion device-based leg-wheel composite robot and its gesture control method, using gestures to control the robot, which improves the naturalness of the interaction between humans and robots, and conforms to the development trend of human-computer interaction;

6. The Xtion-based leg-wheel composite robot and its gesture control method and device according to the present invention apply static gesture recognition and dynamic gesture recognition based on OpenNI software to a six-legged wheel-leg composite robot, so that users can control the six-legged wheel-leg composite robot through gestures. The caster-leg compound robot is beneficial to the practical application of the hexapod-wheel-leg compound robot.

Description of drawings

Fig. 1 is the overall structure schematic diagram of the leg-wheel compound robot based on Xtion equipment of the present invention;

Fig. 2 is the single-leg structure schematic diagram of structure one in the leg-wheel composite robot based on Xtion equipment of the present invention;

Figure 3 is a schematic diagram of the installation method between the steering gear and the leg segment in the single-leg structure;

Fig. 4 is a schematic diagram of the structure of a single leg when the leg-wheel composite robot based on the Xtion device of the present invention stands;

Fig. 5 is a schematic diagram of the single leg structure when the wheel is walking in the leg-wheel composite robot based on the Xtion device of the present invention;

Fig. 6 is a schematic diagram of the main body structure of the robot in the leg-wheel composite robot based on the Xtion device of the present invention;

7 is a schematic diagram of the layered structure inside the shell of the robot body in the leg-wheel composite robot based on the Xtion device of the present invention;

Fig. 8 is a schematic diagram of the design position of the charging interface and the power switch in the leg-wheel composite robot based on the Xtion device of the present invention;

Fig. 9 is the flow chart of the setting method of the dynamic gesture recognition submodule in the leg-wheel composite robot based on the Xtion device of the present invention;

Fig. 10 is the gesture recognition method of the dynamic gesture recognition submodule in the leg-wheel composite robot based on the Xtion device of the present invention;

Fig. 11 is a flow chart of the gesture recognition method of the static gesture recognition sub-module in the Xtion device-based leg-wheel composite robot of the present invention.

In the picture:

1-robot body 2-single-leg structure 3-rudder plate

4-Protruding shaft 5-Bearing 6-Wheel

7-Actuator connector 8-End effector 9-Xtion PRO LIVE camera

10-IMU 11-Servo driver board 12-Main control board

13-Battery box 14-Charging interface 15-Power switch

101-shell 102-perimeter cover 201-first leg section

202-Second Leg Section 203-Third Leg Section 204-First Servo

205-Second steering gear 206-Third steering gear 207-Fourth steering gear

208-foot detection mechanism 901-base 902-inverted U-shaped bracket

detailed description

The present invention will be described in detail below in conjunction with the accompanying drawings.

The leg-wheel composite robot based on Xtion equipment in the present invention includes a robot body 1 and six robot single-leg structures 2. As shown in FIG.

Generally, in order to realize the free movement of the foot end in space when the robot walks, the single-leg structure 2 degrees of freedom must be at least 3, and the more the 2 degrees of freedom of the single-leg structure of the robot, the more flexible the legs are, but the design, control difficulty and quality of the legs will gradually increase. In the present invention, the six robot single-leg structures 2 all adopt three steering gears to output three degrees of freedom to realize the basic walking function.

The single-leg structure in the present invention is a wheel-leg structure with the functions of walking and wheeling, as shown in FIG. 2 , with three leg sections and four driving steering gears. Let the three leg sections be respectively the first leg section 201, the second leg section 202, and the third leg section 203, and the four driving steering gears are respectively the first steering gear 204, the second steering gear 205, the third steering gear 206, and the third steering gear 206. Four steering gear 207. Wherein, one end of the first leg section 201 is fixedly installed on the output shaft of the first steering gear 204 to form a hip joint; the axis of the output shaft of the first steering gear 204 is perpendicular to the horizontal plane, and the first steering gear 204 drives the first leg section 201 to move horizontally swing. The second steering gear 205 is fixedly installed on the other end of the first leg section 201, and the output shaft of the second steering gear 205 is fixed to one end of the second leg section 202 to form a knee joint; The axis of the output shaft is vertical, and the second leg section 202 is driven to swing longitudinally by the second steering gear 205 . The other end of the second leg section 202 is fixed to the output shaft of the third steering gear 206 to form an ankle joint; the axis of the output shaft of the third steering gear 206 is parallel to the axis of the output shaft of the second steering gear 205, and the third leg is driven by the third steering gear 206 Section 203 swings longitudinally.

The connection mode between the steering gear at the above hip joint, knee joint and ankle joint and the first leg section 201, the second leg section 202, and the third leg section 203 is the same, as shown in Figure 3, wherein the output shaft of the steering gear The rudder plate 3 is fixed by screws, and the rudder plate 3 is embedded in the groove designed on the connection position of the end of the leg section; The part is connected with the other side connection position of the leg joint end through the bearing 5. Therefore, the steering gear transmits the power to the leg joint through the above connection method, which reduces the use of bearings and achieves the purpose of weight reduction.

The above-mentioned third leg section 203 is designed to bend toward the direction of the robot main body 1, and the bending angle is 140 degrees; the fourth steering gear 207 is located at the bend, and the axis of the output shaft of the fourth steering gear 207 is aligned with the axis of the output shaft of the third steering gear 203. In parallel, the output shaft of the fourth steering gear 207 is coaxially fixed with a wheel 6 through a spline, and the fourth steering gear 207 drives the wheel 6 to rotate.

The other end of the third leg section 203 is equipped with a foot-ground detection mechanism 208 for the contact between the single-leg structure 2 and the ground, and at the same time can realize the detection of the foot-ground contact state. The above-mentioned foot-ground detection mechanism 208 can adopt a foot-end touch-ground detection mechanism suitable for legged robots disclosed in the invention patent with publication number CN104816766A. This foot-ground contact detection device essentially installs a contact switch at the foot end.

In the single-leg structure 2 of the above structure, the first steering gear 204 is directly fixed on the robot body 1 to complete the installation with the robot body. When the robot walks forward, the support forms of the six single-leg structures 2 are also different according to different joint poses. According to the principle of bionics, the single-leg support configuration of a cockroach when walking is imitated, as shown in Figure 4. The first steering gear 201, the second steering gear 202, and the third steering gear 203 respectively drive the rotation of the hip joint, the knee joint, and the ankle joint, and adjust the poses of the three joints to realize the forward and backward swing or longitudinal movement of the single-leg structure 2. different gaits forward. When the robot is wheeled forward, it can drive the knee joints and ankle joints in at least three single-leg structures 2 to rotate to change the configuration of the single-leg structure 2, so that the wheels 6 in the single-leg structure 2 are on the ground, as shown in Figure 5 , to realize wheel motion.

The robot body 1 is composed of a shell 101 and a peripheral cover 102, as shown in FIG. 6 ; they are all manufactured by 3D printing, and the material is plastic, which is light in weight and suitable for hardness, and has a short processing cycle, which is convenient for the shape control of the robot body 1 . The peripheral cover 102 is installed on the outside of the casing 101 , and an installation opening of the single-leg structure 2 is reserved in the circumferential direction; the peripheral cover 102 is used to protect the casing 101 and the internal control system of the casing 101 . In the above-mentioned robot body 1, the shell 101 adopts a nearly cylindrical body, and the interior is divided into four layers by an aluminum alloy plate with weight-reducing holes, as shown in FIG. 7 , from top to bottom are the sensing layer, driving layer, and control layer And the power layer, used to install the control system.

Wherein, an Xtion PRO LIVE camera 9 and an IMU (inertial navigation unit) 10 are installed on the sensing layer; the Xtion PRO LIVE camera 9 is located outside the casing 101, and its base 901 is located in the sensor layer, and is fixedly mounted on an inverted U-shaped bracket 902, The inverted U-shaped bracket 902 is fixed on the upper surface of the sensing layer. The IMU 10 is fixed on the sensor layer and located in the inverted U-shaped bracket 902 , thereby achieving a compact design and reducing the volume of the robot. The above-mentioned Xtion PRO LIVE camera 9 has two functions: one is to collect environmental information to realize simultaneous positioning and map creation (Simultaneous Localization and Mapping, SLAM, simultaneous positioning and map creation); the other is to obtain human information for human-computer interaction . Xtion PRO LIVE camera 9 includes 2 cameras, one RGB camera and one depth camera; the effective distance of the depth camera is between 0.8m and 3.5m. In addition, the microphones on both sides of the Xtion PRO LIVE camera 9 form a microphone array, which can effectively obtain sound information in the environment, and can be used to realize natural human-computer interaction methods such as voice control. The IMU10 uses an inertial navigation unit to obtain the robot's own attitude, assist positioning and create an environmental map. In the present invention, the IMU10 adopts MTI-300AHRS, which can output three-axis acceleration, three-axis angular velocity, and three-axis deflection angle. The breakthrough sensor fusion algorithm XEE (Xsens Estimation Engine) is used inside, which overcomes the limitation of Kalman filter to a certain extent, and the performance close to an optical gyroscope.

Six steering gear drive boards 11 are installed on the driving layer, which are respectively used to control the movement of each steering gear on the six single-leg structures 2 . Six steering gear drive boards 11 are arranged side by side on the upper surface of the drive layer, which can realize parallel computing and has good real-time performance.

The main control board 12 is installed on the control layer, and the main control board 12 is the control center of the robot, which can realize functions such as communication management, sensor data collection, data processing, and drive management. The main control board 12 adopts MIO-2263 series embedded single-board computer, which is equipped with embedded Industrial-grade embedded single-board computer with CeleronJ1900 quad-core processor. The main control board 12 is an x86 architecture, which has good compatibility with various hardware and software, and its high performance can meet the demand for large amount of calculation, and it is developed using the ROS framework.

A battery box 13 is installed on the power layer, and a 12V lithium battery and a 7.4V lithium battery are installed in the battery box 13, so that the robot needs no external power supply, is independent and autonomous, and expands the scope of application of the robot.

The above-mentioned control layer is also designed with a charging interface 14 and a power switch 15, as shown in Figure 9; the two ends of the charging wire are respectively connected to the charging interface 14 and the two lithium batteries in the power layer; Charge the lithium battery. The power switch 15 is used to control the power-on and power-off of each servo and the main control board 12 in each single-leg structure 2 .

The present invention performs wireless communication with a remote control terminal (mobile phone terminal or tablet terminal) under the WLAN wireless network environment, and is controlled by the remote control terminal. The control terminal adopts the Android system and is designed with a control module; the control module includes a security authentication module, an audio and video playback module, a control module, and an information display module, as shown in Figure 8. Each module in the control terminal is divided into front-end and back-end. The front end is a human-computer interaction interface; the background is responsible for network communication and data processing. Wherein, the security authentication module is used to obtain the user information input by the user, which is encapsulated into a data packet through the background and sent to the main control board 12 of the robot, and the main control board feeds back the login result to the security authentication module for display. The audio and video playback module receives the sound information returned by the Xtion PROLIVE camera in the hexapod robot and the images collected by the camera through the background, and displays it in the audio and video playback module after being decoded by the background. The information display module receives six sets of robot sensing information returned by the IMU, such as attitude, joint angle and joint moment information, and is used to display the sensing information of the hexapod robot, such as attitude, joint angle and joint moment. The control module is composed of multiple controls, which can control the hexapod robot to move forward, backward, turn left, turn right, pause, stop and other actions. At the same time, you can set the robot's travel mode, such as leg travel mode or wheel travel mode.

As for the tablet computer end, since the tablet computer has a larger screen, it can accommodate more functional modules. Compared with the mobile phone terminal, a three-dimensional simulation module is also added on the tablet computer terminal in the present invention. The 3D simulation model synchronously displays the robot's movement and environmental terrain based on the robot's movement and surrounding environment signals returned by the Xtion PRO LIVE camera and IMU module. During the operation of the robot, the 3D simulation module drives the 3D model to move synchronously (low delay) with the real robot according to the joint angle information of the hexapod robot.

For the leg-wheel composite robot with the above structure, the present invention also proposes a gesture control method, adding a gesture control module and a motion control module in the control layer to realize gesture control of the leg-wheel composite robot. In the present invention, the gesture recognition module is designed based on the hydro version of the robot operating system (Robot Operating System, ROS) installed on Linux Ubuntu 12.04, and the gesture recognition module and the motion control module are abstracted into different nodes in the entire robot operating system, then the gesture recognition The communication between the module and the motion control module can be regarded as the communication between different nodes, so the communication between the modules is in the form of Topic (topic) in ROS (Robot Operating System), and the communication content is the message (message) in Topic.

The above-mentioned gesture recognition module includes a dynamic gesture recognition submodule and a static gesture recognition submodule, and the user can select an appropriate gesture recognition method as required, wherein the dynamic gesture recognition is based on OpenNI (Open Natural Interaction) software design for realizing dynamic gesture recognition; OpenNI provides developers with a multi-language, cross-platform framework, and provides its natural interactive API interface. OpenNI API (Application Programming Interface, Application Programming Interface) is a set of interfaces for writing natural interactive applications. The main purpose of OpenNI is to form a standard application programming interface to build a bridge for communication between vision and audio sensors and vision and audio perception middleware. NITE is the middle layer of OpenNI, mainly used to realize gesture recognition. NITE is mainly implemented by gesture detectors, which appear in the form of classes in the NITE program and are used to recognize specific gestures. The dynamic gesture recognition sub-module in the present invention is designed and implemented based on NITE, and the gesture detectors used include SteadyDetector, CircleDetector, PushDetector, and SwipeDetector. Among them, the stability detector is used to identify whether the currently active hand is in a stable state, the circle detector is used to identify the circle movement of the currently active hand, and the push detector is used to identify the forward motion of the currently active hand (that is, the hand is vertical The motion detector is used to identify the strike action of the currently active hand (i.e., the hand moves along the plane of the front of the body).

As shown in Figure 9, the specific design method of the above-mentioned dynamic gesture recognition sub-module is as follows:

A. Use the XML document (such as Sample-tracking.xml) in the Xtion SDK (sensing development tool on the android platform) of the Xtion PRO LIVE camera 9 to initialize the environment;

B. Create a context environment to create an image generator (ImageGenerator), and set the pixel and frame rate of the image generator;

C. Establish and register the scene manager (SessionManager), and initialize the scene manager;

D. Establish and register stable detectors, circle detectors, push detectors, and motion detectors, and add the listeners in each detector to the scene manager; at the same time, set the stable detectors and circle detectors , the callback function of push detector and mobile detector;

The dynamic gesture recognition module designed in the above way, when performing gesture recognition, as shown in Figure 10, first collects the RGB image information of the dynamic gesture through the XtionPRO LIVE camera 9; then processes the collected image based on OpenNI software to extract the gesture information Motion trajectory, and through the stability detector, circle detector, push detector, movement detector to identify the gestures of the hand standing still, circle, push and move; then the stability detector, circle detector, push detector, The mobile detectors will output the recognition results in their respective callback functions, and send the recognition results to the motion control module in the form of Message in the robot operating system. According to the recognition result, the motion control module controls the robot to complete the corresponding motion in combination with the mapping relationship between the predefined dynamic gesture in the dynamic gesture recognition module and the robot motion. The above-mentioned dynamic identification sub-module has a wide application range and can be used when the indoor light is not good.

The static gesture recognition sub-module is based on color images and is used to recognize static gestures. Through the static gesture recognition sub-module, when performing gesture recognition, as shown in Figure 11, the RGB image information of the human body is first collected through the Xtion PRO LIVE camera 9, and the image information is sent to the static gesture recognition through the message in the Topic in the robot operating system Sub-module; then, use the pre-stored skin color model and HOG features in the static gesture recognition sub-module to detect the face part, and then detect the human body part according to the human body structure features, and segment the human hand area from the body part , and finally extract the image features of the segmented hand area, conduct gesture training and recognition, and send the recognition result to the motion control module. According to the recognition result, combined with the mapping relationship between the pre-stored static gesture and the robot movement, the robot is controlled to complete the corresponding movement. . The above-mentioned static gesture recognition sub-module can be used indoors or outdoors with good lighting.

Example 1: Using dynamic gestures to control robot movement

The user waves towards the Xtion PRO LIVE camera 9. After the robot recognizes the user's waving motion, the robot walks in place. When the user's hand moves vertically upward, the robot walks straight forward. When the user's hand moves vertically downward, the robot goes straight back. , when the user's hand draws a circle clockwise, the robot turns clockwise and then walks in place. When the user's hand pushes forward, the robot stops moving.

Example 2: Using static gestures to control robot movement

The static gesture recognition module can pre-define the mapping relationship between gesture types and robot actions, for example, pre-define the user's single hand 1-9 digital gestures, as shown in Table 1:

Table 1 Predefined static gestures

By default, the hexapod walks with a legged "3+3" gait. The user makes a gesture of number 1, and the hexapod robot moves forward; the user makes a gesture of number 3, and the hexapod robot turns left; the user makes a gesture of number 4, and the hexapod robot turns right; the user makes a gesture of number 5, and the hexapod robot Stop; the user makes a gesture of number 6, and the hexapod robot switches from leg-style walking to wheel-walking; the user makes a gesture of number 2, and the hexapod robot moves backward in a wheel-like manner; the user makes a gesture of number 7, and the hexapod robot switches to wheel-walking Walking on legs; the user makes a gesture of number 4, and the hexapod robot turns right in the way of walking on legs; the user makes a gesture of number 5, and the hexapod robot stops; the user makes a gesture of number 8, and the hexapod robot lifts a leg ; The user makes a gesture of number 5, and the hexapod robot stops; the user makes a gesture of number 9, and the hexapod robot lifts two legs; the user makes a gesture of number 5, and the hexapod robot stops.

Claims (8)

1. A leg-wheel composite robot based on Xtion equipment, including a robot body and 6 robot single-leg structures, and the 6 robot single-leg structures are evenly installed on the robot body in the circumferential direction; the single-leg structure is a walking and wheel walking function The wheel-leg structure has three leg sections and four driving servos; let the three leg sections be the first leg section, the second leg section, and the third leg section, and the four driving steering gears are respectively the first steering gear and the second steering gear. The second steering gear, the third steering gear, and the fourth steering gear; wherein, one end of the first leg section is fixedly installed on the output shaft of the first steering gear to form a hip joint; the axis of the output shaft of the first steering gear is perpendicular to the horizontal plane, and is formed by the first A steering gear drives the first leg to swing laterally; the second steering gear is fixedly installed on the other end of the first leg, and the output shaft of the second steering gear is fixed to one end of the second leg to form a knee joint; the axis of the output shaft of the second steering gear It is perpendicular to the axis of the output shaft of the first steering gear, and the second leg joint is driven by the second steering gear to swing longitudinally; the other end of the second leg joint is fixed to the output shaft of the third steering gear to form an ankle joint; the axis of the output shaft of the third steering gear is aligned with the The axis of the output shaft of the second steering gear is parallel, and the third steering gear drives the third leg section to swing longitudinally; the fourth steering gear is located in the middle of the third leg section, and the axis of the output shaft of the fourth steering gear is parallel to the axis of the output shaft of the third steering gear The output shaft of the fourth steering gear is coaxially fixed with wheels through splines, and the wheels are driven to rotate through the fourth steering gear; the other end of the third leg section is equipped with a foot-ground detection mechanism, which is used for the distance between the single-leg structure 2 and the ground. At the same time, the detection of the contact state of the foot and the ground can be realized; It is characterized in that: the interior of the robot body is divided into four layers by an aluminum alloy plate with a weight-reducing hole, and from top to bottom are the sensing layer, driving layer, control layer and power layer, which are used to install the control system; among them, the sensing layer The Xtion PRO LIVE camera and IMU are installed on the layer; the Xtion PRO LIVE camera is used to collect environmental information, realize simultaneous positioning and map creation, and obtain human information for human-computer interaction; the IMU is used to obtain the robot's own posture and assist Locate and create an environment map; six steering gear drive boards are installed on the driver layer, which are used to control the movement of each steering gear on the six single-leg structures; the main control board is installed on the control layer to realize communication management and sensor data collection , data processing and drive management and other functions; a battery box is installed on the power layer, and a robot power supply battery is installed in the battery box; The robot body communicates wirelessly with the remote control terminal, and is controlled by the remote control terminal; the remote control terminal has a safety authentication module, an audio and video playback module, a control module and an information display module; each module is divided into a front end and a back end; the front end is for human-computer interaction Interface; the background is responsible for network communication and data processing; among them, the security authentication module is used to obtain the user information input by the user, which is encapsulated into a data packet by the background and sent to the main control board of the robot, and the main control board feeds back the login result to the security authentication The module displays; the audio and video playback module receives the sound information returned by the Xtion PRO LIVE camera and the images collected by the camera through the background, and displays it in the audio and video playback module after being decoded by the background; the information display module receives six groups of robot sensors returned by the IMU Information, such as posture, joint angle and joint torque information, is used to display the sensing information of the hexapod robot; the control module is composed of motion controls, which can control the movement of the robot; at the same time, the robot's travel mode can be set. 2. A kind of leg-wheel composite robot based on Xtion equipment as claimed in claim 1, is characterized in that: the steering gear at the hip joint, knee joint and ankle joint place and the first leg joint, the second leg joint, the third leg joint The connection mode between them is the same, wherein the output shaft of the steering gear is fixed with a rudder plate by screws, and the rudder plate is embedded in the groove designed on the connection position at the end of the leg joint for positioning; the output shaft of the steering gear is coaxially fixed with a A shaft is extended, and the end of the shaft is connected to the connection position on the other side of the end of the leg joint through a bearing. 3. a kind of leg-wheel composite robot based on Xtion equipment as claimed in claim 1, is characterized in that: outside the robot body, a peripheral cover is added, and the peripheral cover circumferentially reserves the installation port of the single-leg structure; for protection. 4. A kind of leg-wheel compound robot based on Xtion equipment as claimed in claim 1, it is characterized in that: Xtion PRO LIVE camera is located outside the main body of the robot, the base is located in the sensor layer, fixedly installed on the inverted U-shaped bracket, inverted U-shaped The bracket is fixed on the upper surface of the sensing layer; the IMU is located in the inverted U-shaped bracket. 5. a kind of leg-wheel compound robot based on Xtion equipment as claimed in claim 1 is characterized in that: six steering gear drive boards are arranged side by side on the upper surface of the drive layer, which can realize parallel computing and good real-time performance. 6. A kind of leg-wheel composite robot based on Xtion equipment as claimed in claim 1, is characterized in that: charging interface and power switch are also designed on the control layer; The charging lead two ends are respectively connected with charging interface and two pieces of lithium in the power layer. The battery is connected; thus, the external power supply can charge the lithium battery through the charging interface; the power switch is used to control the power-on and power-off of each servo and the main control board in each single-leg structure. 7. a kind of leg-wheel composite robot based on Xtion equipment as claimed in claim 1, is characterized in that: remote control end also has three-dimensional simulation module; Three-dimensional simulation model returns robot motion and its surroundings according to Xtion PRO LIVE camera and IMU module The environmental signal displays the movement of the robot and the terrain of the environment synchronously; during the operation of the robot, the 3D simulation module drives the 3D model to move synchronously with the real robot according to the joint angle information of the hexapod robot. 8. for the gesture control method of a kind of leg-wheel composite robot based on Xtion equipment described in claim 1, it is characterized in that: gesture recognition module and motion control module are installed on the control layer; gesture recognition module includes dynamic gesture recognition submodule and Static gesture recognition submodule, in which: The dynamic gesture recognition sub-module is used to realize dynamic gesture recognition, and has a stability detector, a circle detector, a push detector, and a movement detector; among them, the stability detector is used to identify whether the currently active hand is in a stable state, and the circle detection The detector is used to identify the circular movement of the currently active hand, the push detector is used to identify the forward movement of the currently active hand, and the movement detector is used to identify the hitting action of the currently active hand; The specific design method of the above-mentioned dynamic gesture recognition sub-module is as follows: A. Use the Xtion SDK of the Xtion PRO LIVE camera to initialize the environment; B. Establish a context to create an image generator, and set the pixel and frame rate of the image generator; C. Create and register the scene manager, and initialize the scene manager; D. Establish and register stable detectors, circle detectors, push detectors, and motion detectors, and add the listeners in each detector to the scene manager; at the same time, set the stable detectors and circle detectors , the callback function of push detector and mobile detector; The dynamic gesture recognition module designed in the above way, when performing gesture recognition, first collects RGB image information through the Xtion PRO LIVE camera; then processes the collected images based on OpenNI software, extracts the trajectory of the gesture, and passes the stable detector, Circle detectors, push detectors, and movement detectors recognize hand gestures of stillness, circles, pushes, and movements; then the stability detectors, circle detectors, push detectors, and movement detectors will be included in their respective callback functions Output the recognition result, and send the recognition result to the motion control module; the motion control module controls the robot to complete the corresponding motion according to the recognition result, combined with the mapping relationship between the dynamic gesture and the robot motion predefined in the dynamic gesture recognition module. The static gesture recognition submodule is based on a color image and is used to recognize static gestures; through the static gesture recognition submodule, when performing gesture recognition, at first the RGB image information is collected by the Xtion PRO LIVE camera, and the image information passes through the robot operating system. The message in the Topic is implemented; then, the skin color model and HOG features pre-stored in the static gesture recognition sub-module are used to detect the face part, and then the body part of the person is detected according to the structural characteristics of the human body, and the human body part is segmented from the body part In the hand area, finally extract the image features of the segmented hand area, conduct gesture training and recognition, and send the recognition result to the main control board. corresponding movement.