CN106217393B - Mobile interaction platform of distally coming personally - Google Patents

Abstract

The invention discloses a mobile remote on-site interactive platform, which is composed of a human simulation head, a human simulation body, a mobile platform load-bearing bracket, and a mobile base robot connected sequentially from top to bottom. The mobile telepresence interactive platform is a humanoid mobile robot, which can display the image of the local user on the torso and head of the robot through two projectors, and display the user's facial expression and the surrounding area in real time. Scenarios to reduce the communication gap between humans and robots. At the same time, the mobile telepresence interactive platform can collect information about its surrounding environment, including images and sounds of the surrounding environment. Through the use of these collected information, the user's sense of presence can be improved and the user can be immersed in the scene. In addition, this mobile telepresence interactive platform also has the function of automatic return and charging within a certain area.

Description

Mobile telepresence interactive platform

technical field

The invention relates to the technical field of human-computer interaction. In particular, it relates to a mobile telepresence interactive platform.

Background technique

With the acceleration of the pace of people's life, the situation that people have no means to separate themselves is increasing day by day, thus promoting the development of mobile remote interaction. At the beginning, due to the birth of the Internet, in order to keep up with people's life rhythm, people invented online video, so that people can communicate with multiple people indoors at the same time. But people are still not satisfied. After years of development, many new products have been born for remote interaction. Such as the Anybots QB mobile robot of Anybots Company and the Double robot of American Double Robotics Company. The functions of these two products are very similar. Both are mainly to realize the user's "separation" ability, allowing users to remotely control the mobile robot indoors to attend meetings, go to work and travel. However, the interactive experience of these products currently on the market is relatively poor. Basically, there is only one screen display and a rigid skeleton bracket. People interacting with robots at the remote end can only interact with cold machines. If the remote robot is designed to be more human-like, more human-like in appearance, and have real-time facial expressions of the user, this will definitely enhance the intimacy of the remote interacting person and shorten the "distance between the robot and the human." ". In addition, for local users, the presence experience is very important. If the virtual reality technology can be applied to the local end user, so that the local end user can synchronize its field of vision with the remote robot through the virtual reality technology, this will definitely enhance the presence of the local end user.

In recent years, many papers and patents related to this patent have been produced, as follows:

1. Chinese Publication Patent No.: CN104898652A, Name: Interacting with a Movable Remote Robot. This device can enable users to collect various information of the device remotely, so that users can communicate with the outside world, and its upper limbs can also make certain movements, but the appearance of the device is relatively rigid, not beautiful enough, and at the same time it cannot show the user's The situational information is poor, the interactive experience is poor, and the local user's sense of presence is poor.

2. China Public Patent No.: CN103802108A, Name: Remote Assistance Office Robot. The device described in this patent can meet the general work of human beings in the office, such as audio-visual communication, remote inspection of the work site, remote file transfer, and participation in remote video conferences, etc. But again, the shape of the device is relatively rigid, which does not fit the aesthetics of human interaction, and the degree of restoration of the user's situation is relatively low. At the same time, the sense of presence of the local end user is not considered.

3. Chinese published patent number: CN102755194A, name: a remote monitoring robot. The device can monitor the life and health of the patient through various life status detectors on it, and can interact with the patient to a certain extent, but its interactive functions are relatively few, and the appearance of the device is also relatively blunt, which does not conform to human The aesthetics of the user's situation are not restored, resulting in a poor interactive experience. More importantly, the sense of presence of its local end users is very poor.

Contents of the invention

The purpose of the present invention is to provide a mobile remote presence interaction platform in order to solve the above-mentioned defects in the prior art, such as the remote mobile robot being unable to display the user's situation, poor interactive experience, and poor presence of the user.

The purpose of the present invention can be achieved by taking the following technical solutions:

A mobile remote on-site interactive platform, the platform is composed of a human simulation head 1, a human simulation body 5, a mobile platform load-bearing bracket 8, and a mobile base robot 9 connected sequentially from top to bottom,

The human simulation head 1 includes a first translucent glass plate 2, a first projector 10 and a head shell, and the first projector 10 projects the image information of the head of the local end user on the first translucent glass plate 2. on the glass plate 2, so that the human simulation head 1 has the facial expression of the local end user;

The remote user refers to a user who communicates with the mobile remote presence interaction platform, and correspondingly, the local user refers to a user who controls the mobile remote presence interaction platform.

The human simulated body 5 includes a second translucent glass plate 7, a second projector 11 and a body shell, and the second projector 11 projects the image information of the remote user's body on the second translucent glass plate 7 Above, the human simulation body 5 has the body scene of the remote user.

Further, the first projector 10 and the second projector 11 are placed parallel to the first translucent glass plate 2 and the second translucent glass plate 7, the first projector 10 And the second projectors 11 respectively include a projector image turning device, which is used to turn the parallel images onto corresponding translucent glass plates for display.

Further, the projector image steering device includes: a first fixed plate a1, a mirror glass a2 and a second fixed plate a3, the first fixed plate a1, the mirror glass a2 and the second fixed plate a3 form A 45° angle turns the vertical image of the projector to be horizontal.

Further, the human simulation body 5 is provided with a miniature microphone 6, a 3D camera 3 and a sound box 4, the miniature microphone 6 is used to collect sound signals around the platform, and the 3D camera 3 is used to collect video signals around the platform , the speaker 4 is used to play the user's voice.

Further, the mobile platform load-bearing bracket 8 plays a load-bearing role and supports up and down height adjustment, and the positions and heights of the human simulation body 5 and the human simulation head 1 can be adjusted through the mobile platform load-bearing bracket 8 .

Further, the mobile base robot 9 includes: a sensor module, a motor and its drive module, a power supply module and an embedded computer,

Wherein, the motor and its drive module are used to provide power torque for the mobile remote presence interaction platform;

The power supply module is used to provide working voltage for each component in the mobile remote presence interactive platform, and can output 24V, 6V, 5V, 3.3V voltage;

The embedded computer is used to be responsible for the control of the entire platform, and it is equipped with a wireless network module for receiving user control signals, images, and sound signals. At the same time, the embedded computer has the data interface required by the projector, power interface.

Further, the sensor module includes: a Hall-type current sensor, a gyroscope sensor and an ultrasonic sensor,

Wherein, the Hall-type current sensor is used to detect the power condition of the mobile remote presence interactive platform;

The gyro sensor is used to measure the steering angle of the mobile base robot 9;

The ultrasonic sensor module is used to collect the distance of obstacles around the mobile remote presence interactive platform, and realize autonomous obstacle avoidance.

Further, the mobile telepresence interaction platform also includes an external charging pile 12, the charging pile 12 fits the shape of the mobile telepresence interaction platform, when the mobile telepresence interaction platform When moving to the charging pile for charging, the charging plug of the charging pile 12 just corresponds to the charging jack of the mobile base robot.

Furthermore, the mobile remote presence interactive platform will monitor the remaining power in real time, and when the power is lower than the set warning value, the platform will warn the user and remind the user to charge.

Furthermore, within a certain range of the charging pile 12, the mobile remote on-site interactive platform can start the automatic tracking return function, and return to the charging pile 12 to charge itself according to its inherent obstacle avoidance algorithm and tracking algorithm. .

Compared with the prior art, the present invention has the following advantages and effects:

The invention discloses a mobile remote on-the-spot interactive platform. The platform can allow users to realize the effect of "split" and remotely control one or more mobile remote on-the-spot interactive platforms indoors, so that users can appear in the world at the same time. Various occasions everywhere. The platform can restore the user's facial expressions and voice. Compared with other products of the same type, this platform transmits its environmental information to local end users, and restores its environment through computer equipment and virtual reality glasses, so the user's on-the-spot feeling is more real. At the same time, the platform supports automatic return charging function within a certain location range, which can facilitate the user's operation.

Description of drawings

Fig. 1 is a structural diagram of the mobile remote presence interactive platform disclosed in the present invention;

Fig. 2 is a partial cross-sectional view of the mobile telepresence interactive platform disclosed in the present invention;

Fig. 3 is a schematic structural view of the first projector in the present invention;

Fig. 4 is a schematic structural diagram of a second projector in the present invention;

Fig. 5 is a schematic structural diagram of the charging pile of the mobile remote on-site interactive platform;

Fig. 6 is a top view of the charging pile of the mobile telepresence interactive platform.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Example

This embodiment provides a mobile remote on-site interaction platform, which involves cutting-edge applications of human-computer interaction technology. This interactive platform can provide a mobile robot for the user, so that the user can contact and communicate with people and objects in the outside world in front of the computer. This interactive platform can perfectly map the upper body scene (including expression and voice) of the local end user to the mobile robot, so that the robot can have the user's facial expression in real time, and make the person interacting with it at the remote end feel intimacy. At the same time, the robot can allow users to communicate with other people through speakers and microphones. Local end users can "real" experience the environment where the mobile telepresence interactive platform is located through virtual reality glasses, with a strong sense of presence. The interactive platform supports the automatic return charging function, which can reduce the difficulty of users' operation. By using this interactive platform, users can achieve the effect of "split" and appear in various places around the world at the same time.

As shown in Figure 1 and Figure 2, the mobile remote presence interactive platform consists of a human simulation head 1, a first translucent glass plate 2, a 3D camera 3, a speaker 4, a human simulation body 5, a miniature microphone 6, a second half A transparent glass plate 7, a mobile platform load-bearing bracket 8, a mobile base robot 9, a first projector 10, and a second projector 11 are composed.

The human simulation head 1 and the human simulation body 5 are shown in FIG. 2 , the human simulation head 1 is composed of a first translucent glass plate 2 , a first projector 10 and a head shell. Its interior is hollow, and the signal line and power line of the projector go along the inner back shell of the head to the neck, then pass through the inner back shell of the torso, and finally connect to the mobile base robot 9 . The first projector 10 is at the rear of the head shell, and the first projector 10 projects the image information of the user's head onto the first translucent glass plate 2, so that the human simulation head 1 has the user's facial expression.

Likewise, the human simulation body 5 is made up of the second translucent glass plate 7, the second projector 11 and the body shell. Its interior is hollow, and the signal line and power line of the projector are connected to the mobile base robot 9 along the inner rear shell of the torso. The second projector 11 is in the rear part of the body shell, and the second projector 11 projects the image information of the user's body onto the second translucent glass plate 7, so that the human simulation body 5 has the user's body scene. The human simulated body is provided with a miniature microphone 6, a 3D camera 3 and a speaker 4, which are used to collect environmental information (image, sound signal) around the platform and play the voice of the local end user. The collected information is sent to the embedded computer for processing. For image processing, different image processing algorithms are used according to different requirements.

Since the first projector and the second projector are placed parallel to the first translucent glass plate and the second translucent glass plate, it is necessary to build a projector image turning device to turn the parallel image to the translucent glass plate on display. The projector steering device is composed of the first fixed plate a1, the mirror glass a2 and the second fixed plate a3, forming a 45° angle to turn the vertical picture of the projector to be horizontal and projected onto the first translucent glass plate 2 or the second translucent glass plate 2. 7 on the translucent glass plate. In addition, it is necessary to adjust the size of the projector image, so according to specific needs, a convex lens can be added to the steering device to achieve the image adjustment function.

In order to make the mobile telepresence interactive platform close to the body of a human, it needs to be close to the height of a human, which is 1.75m. Except for the fixed heights of the mobile base robot, the human simulation head, and the body, the remaining height needs to be provided by the load-bearing bracket 8 of the mobile platform. The mobile platform load-bearing bracket 8 raises the position of the body and the head to the position of a normal adult, and simultaneously plays a load-bearing role.

The mobile base robot 9 is mainly composed of a sensor module, a motor and its drive module, a power module and an embedded computer. Its sensor modules include Hall-type current sensors, gyroscope sensors, and ultrasonic sensors. The Hall-type current sensor is used to detect the current situation of the circuit in the entire mobile base robot, and plays the role of system protection. If the operating current of the system exceeds the rated range during the operation of the motor, the system is considered to be in an abnormal state. The gyroscope sensor is used to measure the steering angle of the mobile base robot, and its information is mainly used in its automatic return. The ultrasonic sensor module is used to collect the distance of obstacles around the platform and realize autonomous obstacle avoidance. The motor and its drive module provide power torque for the mobile telepresence interactive platform. The motor should meet the characteristics of small size and high torque. Considering that the installation space of the motor is small and the load is heavy, it is necessary to select a motor with a small size and a large torque, such as a 57BYG250C stepping motor. At the same time, in order to control the motor, a motor driver, such as a TB6600 motor driver, is required.

The power module needs to meet the different voltage requirements of each module, and can output different voltages such as 24V, 6V, 5V, and 3.3V. Choose a 24V 12AH lithium battery, and also need to be equipped with constant voltage conversion circuits with different voltages such as 6V, 5V, and 3.3V. The embedded computer is responsible for the control of the entire platform, and it is equipped with a wireless network module for receiving user control signals, video and audio signals. At the same time, the embedded computer has the data interface and power interface required by the projector.

The charging pile 12 of the mobile remote presence interactive platform is fixed on the flat ground by two screws d1 and d2. Its overall shape is a triangular prism with a hollow in the middle, and the hollow shape is a cylinder with specific grooves on the ground for fixing and limiting the position of the mobile base robot 9 . The charging pile 12 is in contact with the charging socket of the mobile base robot 9 through the d3 charging plug, thereby charging. Through its built-in sensor, monitor the power supply of the mobile telepresence interactive platform in real time.

Here, it will be described in conjunction with an embodiment of the mobile telepresence interaction platform. When the local end user controls the mobile remote presence interactive platform in front of the computer to attend a meeting in the company, the local end user can wear virtual reality glasses through the information collected by the 3D camera 3 of the mobile remote presence interactive platform, and " At the same time, the mobile telepresence interaction platform projects the facial expressions of the local end user on its human simulation head 1 . When the local end user needs to interact with the remote end user, the local end user can directly talk to the remote end user in real time, and at the same time, the local end user can put notes and the like on his upper limbs, through the first projector and the second projector Two projectors display.

Considering the functional effects of the mobile telepresence interactive platform, it is required to perform certain image processing on the images collected by users. Since two projectors are used, and the parameters of the two projectors are different, the size of the projected image will be different. Moreover, the first projector is required to project the user's head, and the second projector is required to project the user's torso, so the source image needs to be segmented. By moving the embedded computer of the base robot 9, the image information can be processed and divided into two partial images of the head and the torso. Then, modify the size and specification of the image according to the parameters of the corresponding projector, and finally send it to the corresponding projector and project it onto the translucent glass plate.

The mobile remote presence interactive platform will monitor its remaining power in real time. When the power is lower than the set warning value, the platform will warn the user and remind the user to charge. When the mobile remote on-site interactive platform needs to be charged, the local terminal only needs to control it to the vicinity of the charging pile 12. When the mobile remote on-site interactive platform is within a certain range of the charging pile, it can start the automatic tracking return function, according to Its built-in obstacle avoidance algorithm and tracking algorithm return to the charging pile to charge itself. When the charging is completed, the mobile remote presence interactive platform will send a charging completion signal to the user.

When the mobile remote on-site interactive platform needs to be charged, the local terminal only needs to control it to the vicinity of the charging pile, and then the mobile remote on-site interactive platform can return to the charging pile to charge itself according to its built-in obstacle avoidance algorithm and tracking algorithm . The charging pile fits the shape of the mobile remote presence interactive platform. When the mobile remote presence interactive platform moves to the charging pile for charging, the charging plug of the charging pile just corresponds to the charging jack of the mobile base robot, and the mobile base robot’s The wheels just move to the bottom of the wheel slots of the charging pile, limiting the position of the mobile base robot to charging within the designated area.

To sum up, the interactive platform can realize the user's "separation" function, allowing the user to remotely control the interactive platform in front of the computer or tablet, and interact with the outside world. In addition, the platform can perfectly present the real-time situation of the user, perfectly express the expression and voice of the local user, and reduce the distance between people and machines. What's more prominent is that by collecting the information of its surrounding environment on the mobile remote on-site interactive platform, and building its virtual reality environment, local end users can "immerse themselves in the scene" through virtual reality glasses to make a more humane environment. operation. In addition, the invention has an appearance that conforms to human aesthetics, making people interacting with it at the remote end feel intimacy and providing a good user interaction experience.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (10)

1. The interaction platform 1. a kind of mobile distal end is come personally, which is characterized in that the platform is by mankind's artificial head (1), Ren Leifang From top to bottom connection forms successively for true body (5), mobile platform support bracket (8), mobile base robot (9),

   Mankind's artificial head (1) includes the first translucent glass plate (2), the first projecting apparatus (10) and head shell, institute It states the first projecting apparatus (10) image information on local end user head is projected on the first translucent glass plate (2), make Obtain the facial expression that mankind's artificial head (1) possesses local end user；

   It includes the second translucent glass plate (7), the second projecting apparatus (11) and body shell, institute that the mankind, which emulate body (5), It states the second projecting apparatus (11) image information of local end user body is projected on the second translucent glass plate (7), make It obtains the mankind and emulates the body scene that body (5) possesses local end user.
2. The interaction platform 2. mobile distal end according to claim 1 is come personally, which is characterized in that first projecting apparatus (10) And second projecting apparatus (11) is each parallel to the first translucent glass plate (2) and the second translucent glass plate (7) it places, first projecting apparatus (10) and second projecting apparatus (11) respectively include a projecting apparatus image and turn to Device is shown for parallel image to be redirect to respectively on corresponding translucent glass plate.
3. The interaction platform 3. mobile distal end according to claim 2 is come personally, which is characterized in that the projecting apparatus image turns to Device includes：First fixed plate (a1), two-way mirror (a2) and the second fixed plate (a3), first fixed plate (a1), institute It states two-way mirror (a2) and second fixed plate (a3) forms a 45° angle and turns to the vertical picture of projecting apparatus at water It is flat.
4. The interaction platform 4. mobile distal end according to claim 1 is come personally, which is characterized in that the mankind emulate body (5) mini microphone (6), 3D cameras (3) and speaker (4) are provided on, the mini microphone (6) is for acquisition platform week The voice signal enclosed, the 3D cameras (3) are used for the video signal around acquisition platform, the speaker (4) for playing The sound at family.
5. The interaction platform 5. mobile distal end according to claim 1 is come personally, which is characterized in that the mobile platform load-bearing branch Frame (8) plays load-bearing effect and upper-lower height is supported to adjust, and the mankind are adjusted by the mobile platform support bracket (8) Emulate the position height of body (5) and mankind's artificial head (1).
6. The interaction platform 6. mobile distal end according to claim 1 is come personally, which is characterized in that the mobile base robot (9) include：Sensor assembly, motor and its drive module, power module and embedded computer,

   Wherein, the motor and its drive module, for for mobile distal end, interaction platform to provide kinetic moment when participating in the cintest；

   The power module can be exported for providing operating voltage for all parts in mobile distal end when participating in the cintest interaction platform 24V, 6V, 5V, 3.3V voltage；

   The embedded computer, the control for being responsible for entire platform, while it is equipped with wireless network module, is used for receiving Control signal and image, the voice signal at family, meanwhile, the embedded computer have the required data-interface of projecting apparatus, Power interface.
7. The interaction platform 7. mobile distal end according to claim 6 is come personally, which is characterized in that the sensor assembly packet It includes：Hall-type current sensor, gyro sensor and ultrasonic sensor,

   Wherein, the Hall-type current sensor is used to detect the charge condition of mobile interaction platform of distally coming personally；

   The gyro sensor is used to measure the steering angle of the mobile base robot (9)；

   The ultrasonic sensor modules are used to acquire the distance of mobile interaction platform peripheral obstacle of distally coming personally, and realize Automatic obstacle avoiding.
8. The interaction platform 8. mobile distal end according to claim 1 is come personally, which is characterized in that come personally the mobile distal end Interaction platform further includes external charging pile (12), and charging pile (12) the fitting mobile distal end is come personally interaction platform Shape, when interaction platform is moved to charging pile charging when participating in the cintest for the mobile distal end, the charging plug of the charging pile (12) Just correspond to the charging jacks of the mobile base robot.
9. The interaction platform 9. mobile distal end according to claim 8 is come personally, which is characterized in that come personally the mobile distal end Interaction platform can monitor remaining capacity in real time, and when electricity is less than setting warning value, platform can alert user, and user is reminded to carry out Charging.
10. The interaction platform 10. mobile distal end according to claim 9 is come personally, which is characterized in that the mobile distal end is faced Field interaction platform can start Automatic Track Finding backhaul function in a certain range of the charging pile (12), interior in avoidance according to it Algorithm and the algorithm that tracks return to the charging pile (12) and voluntarily charge.