CN101618280B - Humanoid-head robot device with human-computer interaction function and behavior control method thereof - Google Patents

Abstract

The invention relates to a humanoid head portrait robot device and a behavior control method with human-computer interaction function, and relates to a humanoid head portrait robot and a behavior control method thereof. It solves the problems that the existing humanoid avatar robot cannot fully realize the reproduction of human facial expressions, has limited perception function, does not have artificial emotion model and human-computer interaction function. The sensor perception system outputs the sensed information to the main control machine for processing. The control system software in the robot behavior control system obtains the relevant control quantities of the corresponding motors according to the artificial emotion model, and outputs the motion control commands through the motion control card to output PWM pulses to drive the corresponding motors. The motor moves to the designated position to realize the robot's human-computer interaction function and various emotional responses. The sensor perception system perceives the external emotional signals, and recognizes the corresponding emotional signals, and uses the artificial emotional model to realize the behavior control of the robot. The invention realizes the reproduction of human facial expressions, and has anthropomorphic multi-sensory functions such as smell, touch and vision.

Description

Human-like avatar robot device with human-computer interaction function and behavior control method

technical field

The invention relates to a humanoid avatar robot device and a behavior control method thereof, belonging to the field of robot applications.

Background technique

The research on humanoid robot began in the 1960s, and after more than 50 years of development, it has become one of the main research directions in the field of robotics. It integrates many sciences such as machinery, electronics, computers, materials, sensors, and control technology, and represents a country's high-tech development level. The meaning of "humanoid" is that the robot has human-like perception, decision-making, behavior and interaction capabilities. Humanoid avatar robot is an important direction to realize human-computer emotional interaction in the field of humanoid robot research. Emotions can enhance the convenience and trustworthiness of robots, while providing feedback to users about the robot's internal state, goals and intentions. In human-computer interaction, a machine designed to have emotions establishes a friendly interface with humans, making it capable of participating in social affairs and conducting interpersonal communication, and is more easily accepted by humans; at the same time, machines have "life" to make their subjects Thinking (or behavior) has a clear purpose and direction, thereby significantly improving the efficiency and speed of thinking (or behavior); in a dynamic, unpredictable and potentially "dangerous" environment, endow machines with creativity and creativity in thinking Consciousness in behavior improves the ability of the machine to adapt to the environment.

At present, the development of some humanoid avatar robots does not have multiple perception functions, and the realization of basic facial expressions is limited to one expression. Through literature search, it is found that the Chinese patent announcement number is CN 201088839, and the patent number is 200720189947.7. Its characteristic is that the mechanism is simple and can realize various smiling expressions. The mechanism falls short in that other facial expressions cannot be achieved. The Chinese patent announcement number is CN 101020315A and the patent number is 200710038295.1, which is called "humanoid robot head system". The system includes a six-degree-of-freedom serial mechanism, and an expandable processor network with DSPF2812 as the main control node. The body of the six-degree-of-freedom series mechanism is driven by six servos to simulate the movements of the eyes, neck and jaw. The scalable processor network is composed of video processor, voice processor, interface module and DSPF2812 main control circuit, which can realize the motion control and calculation requirements in the process of human-computer interaction. However, the humanoid robot head system does not have elastic facial skin, so it cannot realize the reproduction of human facial expressions, and does not have anthropomorphic multi-sensory functions such as smell, touch, and vision. At the same time, the robot in the above patent application does not have artificial emotion model and human-computer interaction function.

Contents of the invention

In view of the above-mentioned state of the art, the purpose of the present invention is to provide a humanoid avatar robot device and a behavior control method with human-computer interaction function, so as to solve the problem that existing humanoid avatar robots cannot fully realize the reproduction and perception functions of human facial expressions. Problems with limited and no artificial emotion models and human-computer interaction functions.

The technical solution adopted by the present invention to solve the problems of the technologies described above is:

The humanoid avatar robot device with human-computer interaction function of the present invention is composed of three parts: a humanoid avatar robot body, a robot behavior control system and a sensor perception system; the humanoid avatar robot body includes an eyeball movement unit, an upper and lower jaw movement unit , artificial lung device, facial expression and mouth shape driving mechanism, front support board, rear support board, stand, face shell and facial elastic skin; the eyeball movement unit consists of two eyeballs, eyeball transmission mechanism, two eyeball servo motors, two One eyelid, eyelid transmission mechanism, two eyelid servo motors and servo motor; upper and lower jaw movement unit is composed of upper jaw, lower jaw, motor and shaft; artificial lung device is composed of hose, cylinder, piston, nut, gas-driven guide shaft, Gas-driven lead screw, drive motor and air intake pipe; facial expression and imitation mouth shape drive mechanism consists of the first drive servo motor, the second drive servo motor, the third drive servo motor, the fourth drive servo motor, and the fifth drive servo motor 1. The sixth driving servo motor and multiple sets of facial skin driving rope wheel mechanisms; the robot behavior control system includes control system hardware and control system software, and the control system hardware includes a main control machine and a motion control card; the control system software is It is a behavior control method; the sensor perception system includes two small CCD sensors, a speech recognition single-chip microcomputer, and an olfactory sensor; the front support plate and the rear support plate are arranged in parallel and are fixedly connected with the stand to form the head skeleton of the humanoid head robot, and the eyeballs The movement unit and the upper and lower jaw movement units are installed on the stand from top to bottom; a small CCD sensor is embedded in each eyeball to form the binocular vision of the robot, each eyelid is set above the corresponding eyeball, and the two eyeball servo motors are respectively Drive the two eyeballs to rotate left and right through the eyeball transmission mechanism, and the two eyelid servo motors drive the two eyelids to move through the eyelid transmission mechanism respectively, and the servo motor simultaneously drives the two eyeballs to move up and down together; the upper jaw is set above the lower jaw, and the motor drives the shaft Drive the movement of the lower jaw; the olfactory sensor is installed on the stand between the eye movement unit and the upper and lower jaw movement unit, one end of the hose is connected to the olfactory sensor, the other end of the hose is connected to the cylinder, and the motor is connected to the gas-driven screw for rotation. The nut is installed on the gas-driven lead screw, and the nut moves along the gas-driven guide shaft to drive the piston fixedly connected to the nut to realize the olfactory function of the robot. One end of the intake pipe is connected to the cylinder; the first drives the servo motor, the second The drive servo motor, the third drive servo motor, the fourth drive servo motor, the fifth drive servo motor and the sixth drive servo motor are all installed on the humanoid head robot head skeleton composed of the front support plate, the rear support plate and the stand Above, the six driving servo motors are connected to the corresponding control points of the facial elastic skin through the facial skin driving rope wheel mechanism; It is installed on the front end of the eyeball movement unit and the upper and lower jaw movement unit from the inside to the outside to form the outer contour shape of the human head robot device; the motion control card is installed on the rear support plate, and the voice recognition single-chip microcomputer is installed on the upper end of the front support plate; the sensor perception The system outputs the perceived information to the main control machine for processing. The control system software in the robot behavior control system obtains the relevant control quantities of the corresponding motors according to the artificial emotion model, and outputs the motion control commands through the motion control card (motor controller) to PWM The pulse drives the corresponding motor to move to the specified position, so as to realize the human-computer interaction function and various emotional responses of the robot.

The above-mentioned behavior control method of the humanoid avatar robot device with human-computer interaction function is realized in the following steps:

Step 1. Perceive external emotional signals through the sensor perception system (sensor CCD and MIC), analyze, feature extract and identify the perceived emotional signals;

Step 2, passing the identified emotion signal to the artificial emotion model; the artificial emotion model mainly includes three parts: emotion generation, emotion interaction and emotion expression of the robot; emotion generation mainly includes stimulus signal collection, emotion definition, emotion drive, Four modules of emotional conversion; two thresholds α and β are defined in the emotional drive; α is the activation threshold of emotion, and β is the saturation threshold of emotion; the conversion of emotional state should take into account external factors, internal factors and the previous emotional state aspects of the impact;

On the basis of clarifying each state of the artificial emotion model and some trigger events can be used to determine the state transition relationship, the extended finite state machine is used to realize the state transition between emotions; the variable attribute set on the defined state is added as An extended finite state machine EFSM; after using the EFSM to analyze the emotional interaction model, the variable composition of each function in the artificial emotional model can be accurately determined, which can effectively avoid variable definition conflicts and provide a basis for the next step of robot control behavior;

Step 3. According to the artificial emotion model, calculate the relevant control quantity (corresponding rotation angle) of the corresponding drive motor, and then get the robot's facial expression and imitation human shape, and then obtain the behavior of the robot to express its emotion.

The invention has the following beneficial technical effects: the invention realizes the reproduction of human facial expressions, and also has anthropomorphic multi-sensory functions such as sense of smell, touch and vision. The invention can realize basic facial expressions and dynamic mouth shapes of robots through elastic facial skin. The behavior control method based on the emotional model is used to realize the behavior control of the robot. The sensor in the robot device perceives the external emotional stimulation, and the artificial emotion model is calculated and recognized. The control system realizes various behavioral responses of the robot, so that the robot has the function of human-computer interaction. . The greatest feature of the present invention is that the humanoid head robot device realizes a design with a ratio of 1:1 to the head volume of an adult, and has a compact structure.

Description of drawings

Fig. 1 a is the perspective view of robot of the present invention, and Fig. 1 b is the perspective view of robot of the present invention (for ease of expression, does not draw facial shell and facial elastic skin etc.); Fig. 2 a is the upper and lower jaw of robot of the present invention Motion unit perspective view, Fig. 2 b is the upper and lower jaw movement unit perspective view (front perspective view) of robot of the present invention; Fig. 3 is the perspective view of robot eye movement unit of the present invention; Fig. 4 a is the definition skin feature point on the face skin of robot Concrete position schematic diagram, Fig. 4 b is the schematic diagram of the forming principle diagram of robot facial expression and mouth shape of the present invention; Fig. 5 is the schematic diagram of human-computer interaction principle of robot of the present invention; Fig. 6 is the block diagram of the behavior control hardware composition of the robot of the present invention; Fig. 7a is a specific flow chart of the behavior control software of the robot of the present invention. FIG. 7b-1 is a voice signal diagram when the robot expresses "Nice to meet you", and Fig. 7b-2 is the lower jaw when the robot expresses "Nice to meet you". The corner diagram of the drive motor; Fig. 8a ~ 8c is a schematic diagram of the robot facial expression recognition fuzzy neural network method of the present invention; Fig. 9 is a schematic diagram of the robot speech recognition method of the present invention; Fig. 10 is a structural representation of the artificial emotion model of the robot of the present invention; Fig. 11 is a schematic diagram of the emotional interaction model of the robot based on the finite state machine of the present invention; Fig. 12a is a part of the basic mouth shapes that the robot of the present invention can express; Group of pictures; Figure 12c is a group of photos of the human-computer interaction experiment of the robot of the present invention.

Detailed ways

Specific embodiment one: as shown in Fig. 1a, Fig. 1b, Fig. 2a, Fig. 2b, Fig. 3, Fig. 4a, Fig. 4b and Fig. 6, the humanoid avatar robot device with human-computer interaction function described in this embodiment consists of The humanoid head robot body, robot behavior control system and sensor perception system are composed of three parts; the humanoid head robot body includes eye movement unit 1, upper and lower jaw movement unit 61, artificial lung device 71, facial expression and mouth shape driving mechanism 81, Front support plate 7, rear support plate 6, stand 51, facial shell 17 and facial elastic skin 18; An eyelid transmission mechanism, two eyelid servo motors 16 and a servo motor 29 are formed; the upper and lower jaw movement unit 61 is composed of an upper jaw 8, a lower jaw 9, a motor 27 and a rotating shaft 28; the artificial lung device 71 is composed of a hose 19, a cylinder 20, and a piston 21 , nut 22, gas-driven guide shaft 23, gas-driven screw 25, drive motor 26 and intake pipe 92 constitute; Three drive servo motors 33, the fourth drive servo motor 40, the fifth drive servo motor 41, the sixth drive servo motor 43 and multiple groups of facial skin drive rope pulley mechanisms;

The robot behavior control system includes control system hardware and control system software, and the control system hardware includes a main control machine 91 and a motion control card 5; the control system software is the behavior control method;

The sensor perception system includes two small CCD sensors 3, a voice recognition microcontroller 4, and an olfactory sensor 24;

The front support plate 7 and the rear support plate 6 are arranged in parallel and are affixed together with the stand 51 to form the head skeleton of the humanoid head robot. The eyeball movement unit 1 and the upper and lower jaw movement units 61 are installed on the stand 51 from top to bottom. Embed a small CCD sensor 3 in each eyeball 12 to form the binocular vision of the robot, each eyelid 13 is arranged on the top of the corresponding eyeball 12, and two eyeball servo motors 14 drive the two eyeballs 12 to rotate left and right through the eyeball transmission mechanism respectively The two eyelid servo motors 16 respectively drive the two eyelids 13 to move through the eyelid transmission mechanism, and the servo motor 29 simultaneously drives the two eyeballs 12 to move up and down together; the upper jaw 8 is arranged above the lower jaw 9, and the motor 27 drives the rotating shaft 28 to drive the lower jaw 9 movement; the smell sensor 24 is installed on the stand 51 between the eye movement unit 1 and the upper and lower jaw movement unit 61, one end of the hose 19 is connected with the smell sensor 24, the other end of the hose 19 is connected with the cylinder 20, and the motor 26 Rotately connected with the gas-driven lead screw 25, the nut 22 is installed on the gas-driven lead screw 25, the nut 22 moves along the gas-driven guide shaft 23, drives the piston 21 fixedly connected to the nut 22 to move, and realizes the olfactory function of the robot. One end of the air pipe 92 is connected with the cylinder 20; the first drive servo motor 35, the second drive servo motor 34, the third drive servo motor 33, the fourth drive servo motor 40, the fifth drive servo motor 41 and the sixth drive servo motor 43 All are installed on the humanoid head robot head frame that is made of front support plate 7, rear support plate 6 and stand 51, and described six driving servo motors drive the rope pulley mechanism and the corresponding corresponding parts of facial elastic skin 18 through facial skin. The control points are connected; the facial shell 17 and the facial elastic skin 18 are consistent with the contour shape of the human face, and the facial shell 17 and the facial elastic skin 18 are installed on the front ends of the eyeball movement unit 1 and the upper and lower jaw movement unit 61 from inside to outside. Imitate the external contour shape of the human head robot device; the motion control card 5 is installed on the rear support plate 6, and the voice recognition single-chip microcomputer 4 is installed on the upper end of the front support plate 7; the sensor perception system outputs the perceived information to the main control unit 91 For processing, the control system software in the robot behavior control system obtains the relevant control amount of the corresponding motor according to the artificial emotion model, and outputs the PWM pulse through the motion control command through the motion control card 5 (motor controller) to drive the corresponding motor to the specified position , so as to realize the human-computer interaction function and various emotional responses of the robot.

The humanoid avatar robot device of the present embodiment has a length of 162 mm, a width of 156 mm, a height of 184 mm, a weight of 2.8 kg, and a total of 14 degrees of freedom. The human-like facial organ movement of the robot is realized through the eyeball movement unit 1 and the upper and lower jaw movement unit 61; the motor 27 drives the rotating shaft 28 to drive the lower jaw 9 to realize the movement of the mouth of the robot. The artificial lung device drives the gas-driven lead screw 25 to rotate through the motor 26, and the nut 22 moves along the gas-driven guide shaft 23, thereby driving the movement of the piston 21 fixedly connected to the nut 22 to realize the olfactory function of the robot. Able to identify alcohol, smoke, ammonia and other odors, the artificial lung device can be placed in an appropriate position on the chest of the humanoid robot. Fig. 3 is the eye movement driving unit 1 of the robot of the present invention, the eye movement has two degrees of freedom, and the eyelid movement has one degree of freedom. Small CCD sensors 3 are respectively embedded in the eyeballs 12 to form the binocular vision of the robot. Among them, the fastest movement speed of the eyeball can reach 500deg/s, and the fastest movement speed of the eyelid is 900deg/s. Considering the specific space requirements of the mechanism in the humanoid avatar robot device, the behavior control of the robot adopts eyeball servo motor 14, eyelid servo motor 16, and servo motor 29, which are small in size, large in output torque, and easy to position control, and are driven by synchronous belts. The mechanism is used as the transmission mechanism of the motor, and considering the support of the motor drive shaft, the bearing support 15 is designed to enhance the rigidity of the mechanism. The robot's eye movement mechanism is left-right symmetrical. Wherein the eyeball servo motor 14 drives the eyeball 12 to rotate left and right through the corresponding first sheave 31 and the second sheave 11 . The servo motor 16 drives the corresponding eyelid 13 to move through the corresponding third sheave 30 and fourth sheave 33 . Servo motor 29 drives two upper eyeballs 12 to move up and down together.

Considering the communication problem between the servo control card and the main control machine and the reasonable allocation and effective use of the hardware resources of the motion control card, and the realization of the hardware integration of the humanoid head robot system, the motion control card 5 uses the SSC-32 motion control card, SSC-32 motion control card can control up to 32 servo motors to coordinate actions. The motion control hardware is the SSC-32 board 5 itself. The steering gear controller uses the RS232 serial port to communicate with the PC, and the PC operates the upper computer software to transmit the control instruction signal to the controller, and then the controller can output a certain duty cycle. The ratio of the PWM signal to achieve separate or simultaneous control of multiple servo motors. The control instructions of the control card are simplified, and the control accuracy is 0.09°/μs, which can control the position and speed of the steering gear.

Speech recognition single-chip microcomputer 4 is SPCE061A single-chip microcomputer. The humanoid avatar robot adopts the combination of the main control computer and the SPCE061A single-chip microcomputer 4 as the auditory system of the robot. SPCE061A single-chip microcomputer 4 is a 16-bit microprocessor launched by Sunplus, embedded with 32k words of flash memory, with high processing speed, and can easily complete speech recognition and speech signal processing. The communication part of the system mainly includes a 32-bit I/O communication circuit and a universal asynchronous serial interface (UART) communication circuit. Data transfer between host computers. The voice prompt used in voice recognition is realized by means of the API voice function provided by Sunplus. The template matching algorithm in recognition adopts Viterbi algorithm. It is a forward search algorithm, which can find the best state sequence from the model when the corresponding observation sequence is given, that is, select the template with the highest output probability as the output result. Template training requires a lot of calculation, which is completed with the help of the main control computer platform. The voice signal samples are collected through the SPCE061A system, and the voice signal is transmitted to the main control computer through the RS232 communication module for storage, which ensures the voice characteristics used for training. Consistency with recognition, thereby reducing errors caused by inconsistencies in hardware systems.

Smell sensor 24 adopts FIS series gas sensor.

Embodiment 2: The sensor perception system in this embodiment further includes a tactile sensor, and the tactile sensor is arranged in the middle of the forehead. Other components and connections are the same as those in the first embodiment.

Embodiment 3: The sensor perception system in this embodiment further includes two temperature sensors, and the two temperature sensors are respectively arranged on the left and right sides of the forehead. Other compositions and connections are the same as those in Embodiment 1 or 2.

Specific embodiment four: the small-sized CCD sensor 3 described in this embodiment utilizes the new generation of COM technology-based stream media processing development package DirectShow that Microsoft Corporation releases on the basis of ActiveMovie and Video for Windows to carry out video capture; Placed on the main board, the visual recognition of the external environment is completed by the windows software development platform based on the windows software development platform built in the small CCD sensor 3, which mainly refers to the identification of human face and facial expressions by the humanoid head robot; The main functions of the small CCD sensor 3 are:

(1) Static and dynamic image acquisition functions include image acquisition card parameter setting, bitmap image reading and storage, multi-channel image acquisition control, display and switching;

(2) Dynamic and static image analysis algorithms and facial expression recognition algorithms involved. Others are the same as in the first embodiment.

Specific embodiment five: as shown in Fig. 1a, Fig. 1b, Fig. 4a, Fig. 4b, Fig. 12a and Fig. 12b, the method for realizing facial expression and imitating human shape in this embodiment is:

Step 1. Define skin feature points on the facial elastic skin 18: a, a-1, b, b-1, c, c-1, d, d-1, e, e-1, g, each point is the motion control point;

Step 2, a slider is set at each motion control point, and the slider is connected with the corresponding control point of the facial elastic skin 18, and each slider is slidably connected with a corresponding guide groove, and the guide groove It is arranged on the facial shell 17, and the setting direction of the guide groove determines the corresponding force direction of each motion control point; one end of the rope of the facial skin driving the sheave mechanism is connected with the slider, and the facial skin drives the rope of the sheave mechanism. The other end is connected with the corresponding driving servo motor;

Step 3: Realize different basic facial expressions and robot mouth shapes through the combination of expression control points and displacement changes.

In order to realize expression control, a skin movement guide curved plate 10 with a guide groove can also be provided at the corresponding control point. For realizing facial expression and mouth shape of robot, define skin feature point (a, a-1, b, b-1, c, c-1, d, d-1, e, e-1, g ). The dots in the figure represent the control points set on the robot skin, and the arrows indicate the movement direction of the control points. In the robot facial expression design, the basic facial expression and mouth shape are realized by controlling the combination of these control points and the displacement and direction of the movement, simulating the bidirectional movement of human muscles. In the actual mechanism, the facial skin driving rope group is connected with the control points, and different basic facial expressions can be realized through the combination and displacement changes of the expression control points. Table 1 expresses the distribution of each control point and facial skin driving rope group and driving servo motor. In order to effectively utilize the space, the motion of each control point is left-right symmetrical (the fourth driving servo motor 40 and the fifth driving servo motor 41 are symmetrically arranged in the left middle; 38 and 38-1 are symmetrically arranged; 39 and 39-1 are symmetrically arranged).

skin control point drive servo motor Facial skin drive rope pulley a, a-1 35 2-1, 2-2 b, b-1 34 37-1, 37-2 c, c-1, 33 36-1, 36-2 d, d-1 40 38,39 e, e-1 41 38-1, 39-1 g 43 42

Specific embodiment six: as shown in Figure 1a, Figure 1b, Figure 5, Figure 6 and Figure 7a, Figure 7b-1, Figure 7b-2, Figure 10, Figure 11, Figure 12a, Figure 12b and Figure 12c, this implementation The behavior control method of the humanoid avatar robot device with human-computer interaction function described in the method is realized in the following steps:

Step 1. Perceive external emotional signals through the sensor perception system (sensor CCD and MIC), analyze, feature extract and identify the perceived emotional signals; the emotional signals include human basic facial expressions and voice signals; human basic facial expressions The recognition adopts the fuzzy neural network structure to recognize; the speech signal recognition adopts the CHMM speech recognition model structure to recognize;

Step 2, passing the identified emotion signal to the artificial emotion model; the artificial emotion model mainly includes three parts: emotion generation, emotion interaction and emotion expression of the robot; emotion generation mainly includes stimulus signal collection, emotion definition, emotion drive, Four modules of emotional conversion; two thresholds α and β are defined in the emotional drive; α is the activation threshold of emotion, and β is the saturation threshold of emotion; the conversion of emotional state should take into account external factors, internal factors and the previous emotional state Aspects of influence; in the generation of emotion, the sensor is used to realize the human perception function, which is used to feel the external events. Since vision and hearing can perceive most of the information of the external environment, the human perception function is realized through visual sensors and auditory sensors in the humanoid avatar robot. Emotional expression includes facial expression expression and voice expression.

Emotional interaction is established based on the finite state machine theory. On the basis of clarifying each state of the emotional model and some trigger events can be used to determine the state transition relationship, the finite state machine is used to realize the state transition between emotions. The purpose of establishing emotional interaction is to control the behavior of the humanoid avatar robot, so that it can make corresponding behavioral responses according to the emotional state. Therefore, after the emotional state machine is established, the main functions and variables to be used should be clearly and accurately determined. According to the requirements, the basic concept of the finite state machine is expanded, and the variable attribute set on the defined state is added as an extended finite state machine (EFSM). After using the EFSM to analyze the emotional interaction model, the variable composition of each function in the emotional model can be accurately determined, which can effectively avoid variable definition conflicts and provide a basis for the next step of robot behavior control. In Figure 8, E ₁ represents the initial emotional state of the individual, and the condition represents the input state. According to the current emotional state and input state, the emotional state of the emotional carrier changes and makes corresponding behaviors. The sound and expression in the behavior represent the variable set V on the state;

Step 3. According to the artificial emotion model, calculate the relevant control quantity (corresponding rotation angle) of the corresponding drive motor, and then get the robot's facial expression and imitation human shape, and then obtain the behavior of the robot to express its emotion.

Specific embodiment seven:, as shown in Fig. 8a, 8b and 8c, the concrete process that the present embodiment adopts fuzzy neural network structure to recognize human's basic facial expression is: the network input layer node number is 6, and facial expression characteristic value {θ ₁ , θ ₂ , θ ₃ , θ ₄ , L ₁ , L ₂ }; the number of nodes in the output layer is 7 basic facial expressions (happy, surprised, sad, angry, disgusted, scared and normal); the output of the expected network is the ith The value of the first output node is 1, and the remaining output nodes are all zero, while the actual output is a specific value around the expected value interval; according to the competition selection, the input sample category is determined to be the corresponding output node with the maximum value in the actual output of the network category; if the actual output node of the network has multiple maximum values at the same time, it will make a rejection judgment; based on individual differences and changes in expression, the relative positions of feature points are not fixed but have certain variability, and in the process of image acquisition, Factors such as changes in the distance between people and the camera cause changes in eigenvalues, so dimensionless values are used as eigenvalues for facial expression recognition, where ${\stackrel{\‾}{\mathrm{\θ}}}_i=\frac{{\mathrm{\θ}}_i}{\mathrm{\Δ\θ}},$ ${\stackrel{\‾}{L}}_i=\frac{L_i}{\mathrm{\ΔL}}.$ Others are the same as in the sixth embodiment.

Embodiment eight:, as shown in Figure 9, the specific process of using the CHMM speech recognition model structure to recognize speech signals in this embodiment is: speech signals can usually be regarded as observations produced from a series of HMM states, each The observation sequence is a frame of MFCC parameters; in the recognition process, the endpoint detection of the speech signal is realized through the short-term average energy and short-term zero-crossing rate; the classic Baum-Welch algorithm is used to realize the HMM parameter estimation problem, and the dynamic Planning algorithm - Viterbi algorithm. Others are the same as in the sixth embodiment.

The principle of the method of the present invention is to perceive external emotional signals through sensors CCD and MIC, and identify corresponding emotional signals. According to the elements in the variable set V in the extended finite state machine in the emotional model, such as facial expression control points and basic mouth shapes, the behavior of the robot to express its emotions is obtained. Some of the basic mouth shapes that the robot can express in this implementation are shown in Figure 12a, and each mouth shape of the robot in the figure is the pronunciation of the corresponding Chinese pinyin. According to the artificial emotion model theory, external stimuli affect the behavior of the robot. In the experiment, external stimuli include voice signals and visual signals. According to different external stimuli signals, the robot and human have different emotional interactions, where the emotional interaction mainly refers to the interaction between language and facial expressions of the robot. Driven by the emotional model, the robot responds accordingly—expressions and voice responses. The control method achieves the "human-like" behavior of the robot through the interaction of emotion, drive and behavior. In this system, "drive" decides "what to do", and "emotion" decides "how to do it". Perceive the external emotional signal through the sensor CCD and MIC, and obtain the behavior of the robot to express its emotion according to the elements in the variable set V in the extended finite state machine in the emotional model, such as facial expression control points and basic mouth shapes. Fig. 7a is a flow chart of the software of the behavior control system of the humanoid avatar robot device. After the robot fuses the collected facial expression information and voice information through the sensor, the control software obtains the relevant control amount of the drive motor according to the artificial emotion model, and controls the behavior expression of the robot. Figure 7b-1 and Figure 7b-2 show the rotation angle of the lower jaw drive motor when the robot expresses "Nice to meet you". In the figure, 0-11.8 seconds is the greeting "Hello" from human to the robot. The device of the invention recognizes the external voice signal through the voice recognition single-chip microcomputer, and makes a corresponding trigger response. When answering, the answering time of each isolated word and the corresponding mouth shapes are obtained according to the sentences of the answering, so as to obtain the corresponding rotation angle of the driving motor.

Claims (10)

1. humanoid-head robot device with human-computer interaction function, described humanoid-head robot device is made up of humanoid-head robot body, robot behavior control system and sensor senses system three parts; It is characterized in that:

Described humanoid-head robot body comprises eye movement unit (1), the upper jaw and the lower jaw moving cell (61), artificial lung device (71), facial expression and degree of lip-rounding driving mechanism (81), front supporting plate (7), rear carrier plate (6), stand (51), facial housing (17) and facial cutis elastica (18); Eye movement unit (1) is made of two eyeballs (12), eyeball transmission mechanism, two eyeball servomotors (14), two eyelids (13), eyelid transmission mechanism, two eyelid servomotors (16) and servomotor (29); The upper jaw and the lower jaw moving cell (61) is made of last Hubei Province (8), lower jaw (9), motor (27) and rotating shaft (28); Artificial lung device (71) is made of flexible pipe (19), cylinder (20), piston (21), nut (22), the gas-powered axis of guide (23), gas-powered leading screw (25), drive motors (26) and air inlet pipe (92); Facial expression and apery degree of lip-rounding driving mechanism (81) drive servomotor (34), three driving servomotor (33), 4 wheel driven moving servomotor (40), five driving servomotor (41), six driving servomotor (43) and organize skin of face by the first driving servomotor (35), second more and drive rope sheave mechanism formation;

The robot behavior control system comprises control system hardware and control system software, and described control system hardware comprises main control computer (91) and motion control card (5); Described control system software is the behavior control method;

The sensor senses system comprises two small ccd sensors (3), speech recognition single-chip microcomputer (4), olfactory sensor (24);

Front supporting plate (7), rear carrier plate (6) be arranged in parallel and are fixed together with stand (51) and constitute the head skeleton of humanoid-head robot, and eye movement unit (1), the upper jaw and the lower jaw moving cell (61) from top to bottom are installed in the stand (51); In each eyeball (12), embed the binocular vision that a small ccd sensor (3) forms robot, each eyelid (13) is arranged on the top of corresponding eyeball (12), two eyeball servomotors (14) drive two eyeballs (12) left-right rotation by the eyeball transmission mechanism respectively, two eyelid servomotors (16) drive two eyelids (13) motion by the eyelid transmission mechanism respectively, and it is moving up and down together that servomotor (29) drives two eyeballs (12) simultaneously; Last Hubei Province (8) is arranged on the top of lower jaw (9), and motor (27) drives rotating shaft (28) and drives lower jaw (9) motion; Olfactory sensor (24) is installed in the stand (51) between eye movement unit (1) and the upper jaw and the lower jaw moving cell (61), one end of flexible pipe (19) is connected with olfactory sensor (24), the other end of flexible pipe (19) is connected with cylinder (20), drive motors (26) is rotationally connected with gas-powered leading screw (25), nut (22) is installed on the gas-powered leading screw (25), nut (22) moves along the gas-powered axis of guide (23), driving is connected in piston (21) motion on the nut (22), realize the olfactory function of robot, an end of air inlet pipe (92) is connected with cylinder (20); The first driving servomotor (35), second drives servomotor (34), the 3rd driving servomotor (33), the moving servomotor (40) of 4 wheel driven, the 5th drives servomotor (41) and the 6th driving servomotor (43) is installed on the humanoid-head robot head skeleton that is made of front supporting plate (7), rear carrier plate (6) and stand (51), and described six driving servomotors are connected with the control corresponding point of facial cutis elastica (18) by skin of face driving rope sheave mechanism; Facial housing (17) and facial cutis elastica (18) are consistent with the contour shape of people's face, and facial housing (17), facial cutis elastica (18) are installed in the outer contoured shape of formation humanoid-head robot device on the front end of eye movement unit (1), the upper jaw and the lower jaw moving cell (61) from inside to outside; Motion control card (5) is installed on the rear carrier plate (6), and speech recognition single-chip microcomputer (4) is installed on the upper end of front supporting plate (7); The sensor senses system exports to main control computer (91) with the information that perceives and handles, and the artificial emotion model of control system software basis in the robot behavior control system obtains the relevant controlled quentity controlled variable of corresponding motor, and motion control instruction is passed through motion control card

(5) the output pwm pulse drives corresponding motor movement to appointed positions, thereby realizes the human-computer interaction function and the various emotional responses of robot.

2. the humanoid-head robot device with human-computer interaction function according to claim 1 is characterized in that: described sensor senses system also comprises touch sensor, and described touch sensor is arranged on the forehead medium position.

3. the humanoid-head robot device with human-computer interaction function according to claim 1 and 2 is characterized in that: described sensor senses system also comprises two temperature sensors, and described two temperature sensors are separately positioned on the left and right sides of forehead.

4. the humanoid-head robot device with human-computer interaction function according to claim 1 is characterized in that: described small ccd sensor (3) is to utilize a new generation that Microsoft releases on the basis of ActiveMovie and Videofor Windows to handle kit DirectShow based on the Streaming Media of COM technology to carry out Video Capture; Image pick-up card is placed on the mainboard by PCI, the vision of setting up humanoid-head robot based on the windows Software Development Platform by band in the small ccd sensor (3) is finished the identification of environment to external world, mainly refers to the identification of humanoid-head robot to people's face portion expression here; The major function of described small ccd sensor (3) is:

(1) static, dynamic image-acquisition functions comprises that parameter setting, the bitmap images of image pick-up card read in and storage, multiplex image acquisition control, demonstration and switching;

(2) relate to dynamically, still image parser and human facial expression recognition algorithm.

5. the humanoid-head robot device with human-computer interaction function according to claim 1 is characterized in that: the method that realizes the facial expression and the apery degree of lip-rounding is:

Step 1, go up definition skin characteristic point at facial cutis elastica (18): a, a-1, b, b-1, c, c-1, d, d-1, e, e-1, g, described each point are the motion control point;

Step 2, a slide block is set at each motion control point place, the corresponding control point of described slide block and facial cutis elastica (18) links together, each slide block and one are slidingly connected with corresponding gathering sill, described gathering sill is arranged on the facial housing (17), and the direction that is provided with of described gathering sill determines that each motion control point is subjected to force direction accordingly; The end that skin of face drives the rope of rope sheave mechanism is connected with slide block, and the other end that skin of face drives the rope of rope sheave mechanism is connected with the corresponding driving servomotor;

Step 3, by the expression control point combination and change in displacement, realize the different basic facial expressions and the robot degree of lip-rounding.

6. the described behavior control method of claim 1 with humanoid-head robot device of human-computer interaction function, it is characterized in that: described method realizes according to the following steps:

Step 1, the emotion signal by the sensor senses system senses external world, to the emotion signal that perceives analyze, feature extraction and identification;

Step 2, the emotion signal after will discerning pass to artificial emotion model; Described artificial emotion model mainly comprises three parts: the emotion of robot produces, emotion is mutual and emotional expression; Emotion produces and comprises that mainly stimulus signal collection, emotion definition, emotion drive, emotion conversion four module; Two threshold alpha of definition in emotion drives, β; α is the activation threshold of emotion, and β is the saturation degree threshold value of emotion; The conversion of affective state will be considered external factor, internal factor and the influence of affective state three aspects in the past;

Can determine on the basis of each state transformational relation with trigger event fully at each state of clear and definite artificial emotion model and some, realize that with the finite state machine of expanding the state between the emotion changes; Increase the variable's attribute set on the definition status, as a kind of extended finite state machine EFSM; After utilizing EFSM to analyze to know the emotion interaction models, can determine the variable formation of each function in the artificial emotion model exactly, can avoid the variable-definition conflict effectively, for next step robot control behavior provides foundation;

Step 3, the artificial emotion model of basis calculate the relevant controlled quentity controlled variable of respective drive motor, to the facial expression and the apery degree of lip-rounding of robot, and then obtain the behavior that its emotion is expressed by robot.

7. the behavior control method with humanoid-head robot device of human-computer interaction function according to claim 6 is characterized in that: described emotion signal comprises human basic facial expression and voice signal.

8. the behavior control method with humanoid-head robot device of human-computer interaction function according to claim 7 is characterized in that: human basic human facial expression recognition adopts structure of fuzzy neural network to discern; Voice signal identification adopts CHMM speech recognition modeling structure to discern.

9. the behavior control method with humanoid-head robot device of human-computer interaction function according to claim 8, it is characterized in that: adopt the detailed process of the human basic facial expression of structure of fuzzy neural network identification to be: network input layer number is 6, i.e. the facial expression feature value

Output layer node number is 7 kinds of basic facial expressions: glad, be taken aback, sad, angry, detest, fear and normally; The value that the expectation network is output as i output node is that 1 all the other output nodes are zero, and actual output is a certain concrete numerical value around the desired value interval; Select according to competition, the input sample class is judged to be in the actual output of network and has peaked output node corresponding class; Making refusal if there are a plurality of maximums simultaneously in the actual output node of network judges; Variation based on individual difference and expression, the relative position of characteristic point be not fix but have certain changeability, and in image acquisition process, the factors such as variation of the distance of people and camera cause the variation of characteristic value, so adopt the characteristic value of dimensionless number, wherein as human facial expression recognition

10. the behavior control method with humanoid-head robot device of human-computer interaction function according to claim 8, it is characterized in that: adopt the detailed process of CHMM speech recognition modeling structure recognition of speech signals to be: voice signal can be regarded the observed quantity that produces from a series of HMM states usually as, and each observation sequence is exactly a frame MFCC parameter; In identifying, realize the end-point detection of voice signal by short-time average energy and short-time zero-crossing rate; Adopt classical Baum-Welch algorithm to realize HMM parameter Estimation problem, adopt dynamic programming algorithm in the identifying---the Viterbi algorithm.

Images