JP2002269530A - Robot, behavior control method of the robot, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control the appearance probability of the learnt behavior of a robot. SOLUTION: This robot comprises a learning part 1 for learning a new behavior, a detection part 2 for detecting an external sensor signal, an evaluation part 3 for evaluating the sensor signal detected by the detection part 2, and a coordinate part 4 for performing a weighting to the behavior learnt by the learning part 1 as the evaluation by the evaluation part 3.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a robot apparatus that behaves autonomously, a method for controlling the action of such a robot apparatus, a program for controlling the action of such a robot apparatus, and a program for recording such a program. The recorded recording medium.

[0002]

2. Description of the Related Art An autonomous entertainment robot apparatus automatically reproduces data (specifically, action pattern data) stored in advance in accordance with emotions and internal states of instinct, thereby realizing various entertainment robot apparatuses. The action is appearing.

On the other hand, there has been proposed a robot apparatus which does not reproduce predetermined action pattern data but performs an action in accordance with the surrounding environment or situation. That is, there is a robot device that holds action pattern data and does not act using the action pattern data, but causes an ad-hoc action to appear according to an external environment or the like.

[0004] Specifically, Japanese Patent Application Laid-Open No. 2000-122992 discloses that by using a reward as a criterion for action motivation, an autonomous action such as selecting an action range according to an external environment or the like is disclosed. There has been proposed a technology of a robot device that performs the following.

Japanese Patent Application Laid-Open No. H11-126198 proposes a technique for learning behavior using a recurrent neural network (hereinafter, referred to as RNN). In this technology, a series of actions can be segmented and acquired by learning actions using an RNN, and a higher-level structure such as a segmented sequence of a series of segmented actions, In addition, it is possible to hierarchically acquire a higher-order structure. According to this technology, the robot device is adapted to each learning situation,
For example, "go straight to the exit", "get out of the room", "turn right in the corridor" or "go straight in the corridor"
And the like are segmented, and these actions are combined to act.

[0006] The robot apparatus capable of learning the behavior by segmentation as described above can acquire various behaviors by learning according to the user. That is, since the learning operation of the robot apparatus has a different learning environment, various operations can be obtained according to the operating environment. That is, the user (eg, owner)
Therefore, the environment in which the robot apparatus is taught is different, and the robot apparatus can acquire various operations according to such an environment.

[0007] Such a robot device behaves in accordance with its own environment as compared with a robot device that can only act by reproducing the behavior pattern data as described above. Therefore, it can be appreciated as a more autonomous behavior.

[0008]

However, there is a case where the robot device learns an undesired action.
For example, actions such as "colliding with a vase" and "going out of a window" are undesirable actions. It is necessary to suppress such undesirable behavior. However, on the other hand, if the robot device learns a favorable behavior,
We also want to do that again.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described circumstances, and has been made in consideration of the above-mentioned circumstances, and is intended to provide a robot apparatus capable of controlling the appearance probability of a learned action, a robot apparatus action control method, a program, and a recording medium. For the purpose of providing.

[0010]

In order to solve the above-mentioned problems, a robot apparatus according to the present invention has an input signal detecting means for detecting an external input signal, and an external signal detected by the input signal detecting means. Evaluating means for evaluating the input signal; associating means for associating the evaluation result by the evaluating means with the action content information; and controlling the action based on the action content information based on the evaluation associated by the associating means. Action control means for performing the action.

[0011] The robot device having such a configuration is as follows.
The external input signal detected by the input signal detection unit is evaluated by the evaluation unit, the evaluation result by the evaluation unit is associated with the action content information by the association unit, and based on the evaluation associated by the association unit, The action is controlled by the action control means based on the action content information. Thereby, the robot device evaluates the learned behavior and causes the behavior to appear based on the evaluation.

Further, in order to solve the above-mentioned problems, a behavior control method for a robot apparatus according to the present invention includes an input signal detection step in which the robot apparatus detects an external input signal from outside, and an input signal detection step. An evaluation step in which the robot apparatus evaluates the detected external input signal, an associating step in which the evaluation result obtained in the evaluation step in the robot apparatus is associated with the action content information, and an associating step. A behavior control step in which the robot device controls the behavior based on the behavior content information based on the evaluation. According to such a behavior control method for a robot device, the robot device evaluates the learned behavior and causes the behavior to appear based on the evaluation.

In order to solve the above-mentioned problems, a program according to the present invention includes an input signal detecting step of detecting an external input signal, and an external input signal detected in the input signal detecting step. An evaluation step to perform, an associating step of associating the evaluation result obtained in the evaluating step with the action content information, and controlling the action based on the action content information based on the evaluation associated in the associating step. And a behavior control step to be performed by the robot apparatus. The robot device in which the control of the action is executed by such a program evaluates the learned action and causes the action to appear based on the evaluation.

According to another aspect of the present invention, there is provided a recording medium comprising: an input signal detecting step for detecting an external input signal from the outside; and an external input signal detected in the input signal detecting step. Based on the evaluation step to evaluate, the associating step of associating the evaluation result obtained in the evaluating step with the action content information,
A program for causing the robot apparatus to execute a behavior control step of controlling behavior based on behavior content information is recorded. The robot device in which the control of the behavior is executed by the program recorded in such a recording medium evaluates the learned behavior, and causes the behavior to appear based on the evaluation.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, the present invention is applied to a robot device that acts autonomously.

The robot device to which the present invention is applied is an autonomous robot device that behaves autonomously according to the surrounding environment and internal state. By applying the present invention, the robot device learns a new behavior, weights the learned behavior, for example, as an evaluation thereof, and acts based on the weight. .

In the description of the embodiment, learning of behavior by a robot device realized by applying the present invention will be described first, and then a specific configuration of the robot device will be described.

(1) Learning of Behavior and Weighting of Learned Behavior As shown in FIG. 1, the robot apparatus implements the present invention by including a learning unit 1, a detection unit 2, an evaluation unit 3, and a correspondence unit 4. are doing. Here, the learning unit 1 functions as a learning unit that learns a new action, the detection unit 2 functions as an input signal detection unit that detects an external input signal from outside, and the evaluation unit 3 includes a detection unit. 2. The associating unit 4 functions as an evaluation unit that evaluates the external input signal detected by the learning unit 2. The associating unit 4 weights the behavior learned by the learning unit 1 to be evaluated by the evaluating unit 3. Function as For example, learning unit 1, detection unit 2, evaluation unit 3
The associating unit 4 is configured as an object or module configured by a software program in the robot device.

The learning unit 1 is configured by a learning model such as a recurrent neural network (hereinafter, RNN). Here, RNN is
The information of the action to be learned is a neural network that is input toward the input layer, the intermediate layer, and the output layer. The processing at the time of action learning in the RNN will be described later in detail.

The learning unit 1 receives the input,
Output the corresponding output and learn with the input as the learning target. The input is made as time-series data. For example, examples of the input when learning an action include a sensor input and a motor output obtained by the action of the robot apparatus. Specifically, the sensor input includes an image signal. The information input as a learning target to the learning unit 1 is not limited to this, and may be, for example, behavior information generated internally corresponding to a behavior.

The learning section 1 outputs an input to be learned. Here, the output of the learning unit 1 is action content information indicating the content of the action, and is grasped as a so-called teaching signal. This output shows the same value after learning as long as the same action appears.

On the other hand, the detector 2 detects an external input signal. For example, the detection unit 2 is a sensor. In particular,
As will be described later, it is a touch sensor arranged at the top of the robot device. The robot device detects “stroke” or “hit” by the user as an interface with the user by using the touch sensor.

The evaluation unit 3 performs an evaluation based on the signal detected by the detection unit 2. That is, for example, an action performed by the user such as “stroke” or “praise” is detected, and a weight as an evaluation value is generated.
For the evaluation by the evaluation unit 3, for example, a signal pattern to be evaluated is prepared in advance and the detection unit 2
By comparing the detected signal pattern with a previously prepared signal pattern, the action performed by the user is evaluated.

The detection section 2 is not limited to a touch sensor, but may be any as long as it can evaluate an action performed by a user. For example, the detection unit 2 may be a microphone. In this case, the evaluation unit 3 can also determine the evaluation made by the user from the voice of the user input by the microphone, for example, from the tone of the voice. That is, for example, the evaluation unit 3 identifies and evaluates a voice uttered by the user, such as “no good” or “good”.

The association unit 4 associates the weight obtained by the evaluation unit 3 with the output from the learning unit 1. The association unit 4 is, for example, a storage unit, and implements association by storing the output from the learning unit 1 and the weight in association with each other.

The association is made with the action (output) learned by the learning unit 1 simultaneously with or before or after the external input signal detected by the detection unit 2. This is usually the case for pets, etc.
This is because "stroke", "hit", or the like, which is evaluated simultaneously with or before or after the action to be taught, corresponds to such an action. However, it is needless to say that the timing of such association is not limited.

The robot apparatus having the configuration shown in FIG. 1 as described above can learn a new action, and can weight the learned action. . Thereby, the robot device can control the learned behavior according to the weighting. Specifically, by stroking the head when learning a certain action, the action can be taught as a preferable action, and as a result, the robot apparatus comes to appear the action frequently, while By hitting the head when learning, the behavior can be taught as an unfavorable behavior, and as a result, the robot device can hardly appear the behavior. That is, the user (owner) can “train” the behavior of the robot device.

More specifically, such behavior control can be realized by a behavior control unit (not shown) for controlling the behavior of the robot apparatus, which determines the appearance probability of the behavior by referring to the weights. it can. For example, when the weight is large, the probability of appearance of the action is increased, and when the weight is small, the probability of appearance of the action is reduced.

The weighting of such an action can be performed for the entire action to be completed, or for each of the actions to be completed as a series of actions. The latter example is a specific example described in detail later.

As described above, the robot device learns the behavior and determines the appearance probability of the behavior, so that "discipline" is performed on the pet, and a more biological expression is realized. It will be.

(2) Specific Configuration for Learning Behavior As described above, behavior can be learned with weighting. Techniques for learning behavior include, for example,
-126198. The robot device to which the present invention is applied learns an action, for example, by adopting this technology. Here, an outline of a technique for learning the behavior will be described.

FIG. 2 shows a configuration example of the data processing unit. The configuration shown in FIG. 2 is a specific configuration of the learning unit 1 shown in FIG. As will be described in detail later, the robot device includes a sensor that detects an obstacle and a motor that is driven to move the robot.The information is input to the data processing unit as a learning target. You.

[0033] n-number of RNN1-1～1-n are input _{x t} corresponding to the state of the sensor and the motor is input.
The RNN 1-1 is configured as shown in FIG. Although not shown, the other RNNs 1-2 to 1-n also
The configuration is the same as that of the RNN 1-1 shown in FIG.

As shown in FIG. 3, RNN 1-1 is
Has a neuronal 31 of a predetermined number of the input layer, the neuron 31, an input s _t corresponding to the state of the sensor, the input m _t corresponding to the state of the motor is input. The output of the neuron 31 is output through the neuron 32 in the hidden layer.
The signal is supplied to the neuron 33 in the output layer. From the neuron 33 in the output layer, RNN1
An output _{st + 1} corresponding to the state of the sensor of −1 and an output mt _{+ 1} corresponding to the state of the motor are output. Also, part of the output is context (contex
As t) C _t, it is adapted to be fed back to the neuron 31 in the input layer.

Outputs of the RNNs 1-1 to 1-n are input to the synthesizing circuit 3 via the corresponding gates 2-1 to 2-n.
Here, they are synthesized and the predicted output yt _{+ 1} is output.

At the time of learning, the error between the target value y ^* _{t + 1} as the teacher signal and the output of each of the RNNs 1-1 to ₁ -n controls the state of the corresponding gate 2-1 to 2-n. It has been done.

The same configuration as the above-described lower RNNs 1-1 to 1-n, gates 2-1 to 2-n, and synthesizing circuit 3 is also formed in a higher hierarchy. That is, RNNs 11-1 to 11-n and gates 12-1 to 12-1 are located at higher levels.
12-n and a combining circuit 13 are provided. Then, the RNN11-1~11-n, are adapted sequence corresponding to the conduction state of the gate 2-1 to 2-n of the lower layer (closed degree) (gating sequence) _{G t} is input . Then, outputs G ¹ _{T + 1 to} G ⁿ _{T + 1} are output from the RNNs 11-1 to 11 -n, and a prediction output G _{T + 1} is output from the combining circuit 13. At the time of learning, a target value G ^* _{T + 1} is input as a teacher signal. Note that FIG.
Shows only two hierarchies, but higher hierarchies can be provided if necessary.

FIG. 4 is a diagram showing a first RN constituting a higher hierarchy.
The configuration of N11-1 is shown. Note that other RNNs 11
-2 to 11-n have the same configuration as the RNN 11-1 shown in FIG.

As shown in FIG. 4, the upper layer RNN1
1-1 is basically the RNN of the lower hierarchy shown in FIG.
1-1, a plurality of neurons 41 are arranged in an input layer, a plurality of neurons 42 are arranged in an intermediate layer, and a plurality of neurons 43 are arranged in an output layer. Signals g ¹ _{T to} g ⁿ _T corresponding to the conduction states of the gates 2-1 to 2-n are input to the input layer,
Period that the gate conduction (opening) (Time) I _T is input. From the output layer, outputs g ¹ _{T + 1 to} g ⁿ _{T + 1} and _{IT + 1} are output corresponding to these inputs. Also, part of the output of the output layer is fed back to the input layer as a context C _T.

Here, the algorithm of the RNNs 1-1 to 1-n will be described. The conduction state of the gate is expressed as shown in equation (1) using a soft-max activation function.

[0041]

(Equation 1)

Where gⁱIs the conduction state of the i-th gate
Represents the gate coefficient corresponding to the state, s ⁱIs the i-th game
The value corresponding to the internal state of the conduction state of the switch. Obedience
Thus, the output y of the synthesis circuit 3_{t + 1}Is given by equation (2)
You.

[0043]

(Equation 2)

Here, a likelihood function represented by the equation (3), which becomes the maximum value during prediction learning, is defined.

[0045]

(Equation 3)

Here, σ represents a scaling parameter.

At the time of learning, the weight coefficients and gate coefficients g of RNNs 1-1 to 1-n are simultaneously updated so that the likelihood function is maximized. At the time of recognition, only the gate coefficient is updated.

[0048] In order to establish a rule to update these weighting coefficients and gate coefficient, and the tilt about the internal variable S ⁱ of the exponential function of the likelihood function, the tilt related to the output y ⁱ of the i-th RNN (4) equation And (5).

[0049]

(Equation 4)

[0050]

(Equation 5)

Here, g (i | x _t , y ^* _{t + 1} )
When the i-th RNN is an input _{x t,} the target output ^{y *}
_It means the post-event probability of generating _{t + 1} and is expressed by equation (6).

[0052]

(Equation 6)

[0053] In this _{^{case, || y * t + 1 -y}} j t + 1 || 2
Represents the square error of the current prediction.

[0054] Equation (4) represents a direction to update the ^{s i.} Also, (5) as shown in the formula, the tilt related to y ^{i t} _{+ 1} of the exponential function of the likelihood function, the error condition y
^* _{T +1} contains the error term of -y ⁱ _{t + 1.} This error term is weighted by the i-th RNN post-event probability.

As described above, the weight coefficients of the RNNs 1-1 to 1-n are adjusted so as to correct the error between the output of the i-th RNN and the target value in proportion to only the post-event probability. Thereby, one expert RNN out of n RNNs
Only the given training pattern (learning pattern) is exclusively learned. The error of each RNN is expressed by equation (7).

[0056]

(Equation 7)

The actual learning of the RNNs 1-1 to 1-n is executed by the back propagation method based on the error obtained by the above equation.

Thus, RNNs 1-1 to 1-n are:
Of the input x _t, so that each the experts can identify predetermined time series pattern which is different from the others, learning is performed.

The above is based on the fact that the RNN in the higher hierarchy
The same applies to 11-1 to 11-n. However, the input in this case is a gate sequence G _T,
The output is G ⁱ _{T + 1} .

With such a configuration, individual operations can be performed by RN
N1-1 to 1-n can learn individually. The RNNs 1-1 to 1-n learn and the manifestation of each operation is managed by the gates 2-1 to 2-n, and various operation sequences of the gates (that is, combinations of various operation orders) are determined. RNNs 11-1 to 11-n are learning. That is, by using such an information learning method, the behavior can be segmented and learned.

By having such a learning unit 1 composed of a plurality of RNNs, the robot device can move in a room forming a passage as shown in FIG. 5 and learn an action during the movement. Can be. For example, the action based on the distance sensor is learned. Specifically, the robot device learns the behavior by moving in the room and self-organizing the layers constituting the learning unit during the movement.

Incidentally, Japanese Patent Application Laid-Open No. 11-126198 discloses that
It is disclosed that by using the above-described RNN, it is possible to associate an action without actually performing an action. For example, when the data processing unit has a configuration as illustrated in FIG. It is possible.

The technique of the learning method outlined above is disclosed in Japanese Patent Application Laid-Open No. H11-126198, and the learning unit 1 of the robot apparatus according to the embodiment of the present invention employs such a learning technique. Can be built.

When the learning section 1 has such a configuration, the behavior is weighted as follows.

When an input such as “stroke” or “hit” is detected as a detection result of the detection unit 2 by a sensor or the like of the robot device, the action currently being executed is detected by the gate 2.
It is determined according to the situation of 1-2-n, and as shown in the following table, the corresponding action is associated with a score and stored. That is, a score is assigned to each learned motion.

[0066]

[Table 1]

For example, an act of increasing the probability of appearance,
For example, when the act of “stroke” is performed, the score is set to +1. On the other hand, when the act of lowering the appearance probability, for example, the act of “struck” is performed, the score is set to −1. I do.

When the robot device determines the next learned operation, the data processing unit as shown in FIG. 6 that enables the association of the above-mentioned behavior is performed, and a rehearsal of the behavior is performed. Then, a sum of scores is obtained for actions that are a series of actions appearing therein. The robot apparatus determines an action to actually appear (combination of a series of actions) based on the sum of the scores obtained in such a rehearsal. That is, for example,
If the sum of the scores is determined to be as large as possible, the robot device will act obediently, while if the sum of the scores is determined as small as possible, the robot device will act rebelliously. .

In this example, a description has been given of weighting a low-level operation and controlling the operation. However, as in the RNN system disclosed in Japanese Patent Laid-Open No. Since a hierarchical structure of layers can be adopted, it goes without saying that control can be performed on higher layers (a series of actions and action policies).

(3) Configuration of Robot Apparatus According to the Present Embodiment Next, a specific configuration of the robot apparatus that learns the above-described behavior will be described.

As shown in FIG. 7, a so-called pet robot imitating a “dog” is formed.
Leg units 103A, 103
B, 103C, and 103D are connected, and a head unit 104 and a tail unit 105 are connected to the front end and the rear end of the body unit 102, respectively.

As shown in FIG. 8, a CPU (Central Processing Unit) 110 and a DR
AM (Dynamic Random Access Memory) 111, Flash ROM (Read 0nly Memory) 112, PC (Perso
control unit 116 formed by connecting a card interface circuit 113 and a signal processing circuit 114 to each other via an internal bus 115.
And a battery 117 as a power source of the robot device 100 are stored. The body unit 10
2 includes an angular velocity sensor 118 and an acceleration sensor 11 for detecting the acceleration of the direction and movement of the robot apparatus 100.
9 etc. are also stored.

The head unit 104 receives a charge coupled device (CCD) camera 120 for capturing an image of an external situation, and receives a pressure applied by a physical action such as “stroke” or “hit” from the user. A touch sensor 121 for detection, a distance sensor 122 for measuring a distance to an object located ahead, a microphone 123 for collecting external sounds, and a speaker 124 for outputting a sound such as a squeal And an LED (Light Emitting Diode) (not shown) corresponding to the “eye” of the robot device 100 are arranged at predetermined positions.

Further, each leg unit 103A-103
D, joint portions of the leg units 103A to 103D and the trunk unit 102, the head unit 10
Actuators 125 _{1 to} 125 _n and potentiometers 126 _{1 to} 126 _n are provided for the number of degrees of freedom, respectively, at a connection portion between the body unit 4 and the body unit 102 and a connection portion at the tail 105 A of the tail unit 105. For example, each of the actuators 125 _{1 to} 125 _n has a servomotor. By driving the servo motor, the leg units 103A to 103D are controlled, and the state shifts to the target posture or operation.

The angular velocity sensor 118, the acceleration sensor 119, the touch sensor 121, and the distance sensor 12
2. Various sensors such as a microphone 123, a speaker 124, and each of the potentiometers 126 _{1 to} 126 _n , an LED, and each of the actuators 125 _{1 to} 125 _n are:
The hubs 127 ₁ to 127 _n are connected to the signal processing circuit 114 of the control unit 116 via the corresponding hubs 127 ₁ to 127 _n , respectively.
The CD camera 120 and the battery 117 are directly connected to the signal processing circuit 114, respectively.

The signal processing circuit 114 sequentially takes in the sensor data, image data, and audio data supplied from each of the above-mentioned sensors, and sequentially stores them at predetermined positions in the DRAM 111 via the internal bus 115. In addition, the signal processing circuit 114 sequentially takes in remaining battery power data indicating the remaining battery power supplied from the battery 117 and stores the data in a predetermined position in the DRAM 111.

The sensor data, image data, voice data, and remaining battery data stored in the DRAM 111 in this manner are used when the CPU 110 subsequently controls the operation of the robot apparatus 100.

In practice, the CPU 110 controls the robot device 10
At the initial time when the power supply of the main unit 102 is turned on, the control program stored in the memory card 128 or the flash ROM 112 inserted in the PC card slot (not shown) of the body unit 102 is read out directly or directly through the PC card interface circuit 113. Is stored in the DRAM 111.

The CPU 110 then determines the status of itself and its surroundings and the usage based on the sensor data, image data, audio data, and remaining battery data sequentially stored in the DRAM 111 from the signal processing circuit 114 as described above. Judge the instruction from the person and the presence or absence of the action.

[0080] Furthermore, CPU 110 is configured to determine a subsequent action based on the control program that is stored in the determination result and DRAM 111, by driving the actuator ₁₂₅ 1 to 125 _n as required based on the determination result, the head Actions such as swinging the unit 104 up and down, left and right, moving the tail 105A of the tail unit 105, and driving and walking each leg unit 103A to 103D are performed.

At this time, the CPU 110 generates audio data as necessary and supplies the generated audio data to the speaker 124 as an audio signal via the signal processing circuit 114, thereby outputting an audio based on the audio signal to the outside. The above-mentioned LED is turned on, turned off or blinked.

In this way, the robot device 100 is capable of acting autonomously in accordance with the situation of itself and the surroundings, and instructions and actions from the user.

(2) Software Configuration of Control Program Here, the software configuration of the above-described control program in the robot device 100 is as shown in FIG. In FIG. 9, the device driver layer 30 includes:
A device driver set 13 located at the lowest layer of the control program and including a plurality of device drivers
1 is comprised. In this case, each device driver is an object permitted to directly access hardware used in a normal computer, such as a CCD camera 120 (FIG. 8) and a timer, and receives an interrupt from the corresponding hardware. Perform processing.

The robotic server object 132 is located at the lowest layer of the device driver layer 130, and is an interface for accessing hardware such as the various sensors and actuators 125 _{1 to} 125 _n described above. Virtual robot 133, which is a software group that provides power, and a power manager 134, which is a software group that manages switching of power supply and the like.
And a device driver manager 135 which is a software group for managing various other device drivers.
And a designed robot 136 which is a software group for managing the mechanism of the robot apparatus 100.

The manager object 137 is composed of an object manager 138 and a service manager 139. The object manager 138 manages the robotic server object 13
2. A software group that manages activation and termination of each software group included in the middleware layer 140 and the application layer 141. The service manager 139 is a software group that stores the connection stored in the memory card 128 (FIG. 8). A group of software that manages the connection of each object based on the connection information between the objects described in the file.

The middleware layer 140 is located on the upper layer of the robotic server object 132.
It consists of a software group that provides the basic functions of. The application layer 141 is located above the middleware layer 140, and determines the behavior of the robot device 100 based on the processing result processed by each software group constituting the middleware layer 140. It consists of a group of software for performing

FIG. 10 shows specific software configurations of the middleware layer 140 and the application layer 141, respectively.

The middleware layer 140 corresponds to FIG.
As shown in, each of the signal processing modules 150 to 158 for noise detection, temperature detection, brightness detection, scale recognition, distance detection, attitude detection, touch sensor, motion detection, and color recognition; A recognition system 160 having an input semantics converter module 159 and the like; an output semantics converter module 168; and signal processing modules 161 for posture management, tracking, motion reproduction, walking, falling back, LED lighting and sound reproduction. And an output system 69 having 167.

Each signal processing module 15 of the recognition system 160
0 to 158 are robotic server objects 1
DRAM 11 by 32 virtual robots 133
1 (FIG. 8), the corresponding data among the sensor data, image data, and audio data read from
A predetermined process is performed based on the data, and a processing result is provided to the input semantics converter module 159. Here, for example, the virtual robot 133
It is configured as a part that exchanges or converts signals according to a predetermined communication protocol.

The input semantics converter module 159 is composed of these signal processing modules 150 to 158.
"Noisy" based on the processing result given by
"Hot", "Bright", "Detected ball", "Detected fall", "Stroked", "Slapped", "Heared Domiso scale", "Detected moving object" Alternatively, it recognizes the situation of itself and surroundings such as “detected an obstacle”, and commands and actions from the user, and outputs the recognition result to the application layer 141 (FIG. 8).

The application layer 141 is the one shown in FIG.
As shown in FIG. 1, it is composed of five modules: a behavior model library 170, a behavior switching module 171, a learning module 172, an emotion model 173, and an instinct model 174.

The behavior model library 170 has the contents shown in FIG.
As shown in, "When the battery level is low"
Independently corresponding to several pre-selected condition items such as "return to fall", "when avoiding obstacles", "when expressing emotion", "when ball is detected", etc. Behavior models 170 _{1 to} 170 _n are provided.

The behavior models 170 _{1 to} 170 ₁
0 _n are sent to the emotion model 173 as described later, as necessary, when a recognition result is given from the input semantics converter module 159 or when a certain period of time has passed since the last recognition result was given. The subsequent actions are determined with reference to the parameter values of the corresponding emotions held and the parameter values of the corresponding desires held in the instinct model 174, and the determination result is output to the action switching module 171.

In this embodiment, each of the behavior models 170 _{1 to} 170 _n uses one node (state) NODE as shown in FIG.
_{0 to} NODE _n to any other node NODE _{0 to} NOD
Finite probability automaton for determining probabilistically based on the transition probability _P 1 to P _n which is set respectively arc _ARC 1 _{~ARC n1} connecting between whether a transition to E _n each node NODE ₀ ~NODE _n An algorithm called is used.

More specifically, each of the behavior models 170 _{1 to} 170 _1
n has a state transition table 180 as shown in FIG. 14 for each of the nodes NODE _{0 to} NODE _n corresponding to the nodes NODE ₀ to NODE _n forming their own behavior models 170 _{1 to} 170 _n , respectively. ing.

In this state transition table 180, the node N
Input events (recognition results) as transition conditions in ODE _{0 to} NODE _n are listed in order of priority in the row of “input event name”, and further conditions for the transition conditions are described in the rows of “data name” and “data range”. It is described in the corresponding column.

Therefore, the node NODE ₁₀₀ represented by the state transition table 80 in FIG.
ALL) ", the size of the ball (SIZ) given together with the recognition result is given.
E) is in the range of “0 to 1000”, or when a recognition result of “obstacle detected (OBSTABLE)” is given, the “distance (DISTANCE)” to the obstacle given together with the recognition result is given. )) Is in the range of “0 to 100”, which is a condition for transitioning to another node.

In the node NODE ₁₀₀ , even when the recognition result is not input, the behavior model 170
₁ to 170 _n is out of the parameter values of the emotions and the desire held respectively in the emotion model 173 and the instinct model 74 refers periodically, held in the emotion model 73 "joy (JOY)", "surprise ( When the parameter value of either “SURPRISE” or “Sadness” is in the range of “50 to 100”, transition to another node can be made.

In the state transition table 180, the names of nodes that can transition from the nodes NODE ₀ to NODE _n are listed in the column of “transition destination node” in the column of “transition probability to another node”. Input event name ",
Other nodes NOD that can transition when all the conditions described in the rows of “data value” and “data range” are met
The transition probabilities from E _{0 to} NODE _n are respectively described in corresponding portions in the column of “transition probability to another node”, and the action to be output when transitioning to the node NODE _{0 to} NODE _n is “other It is described in the row of “output action” in the column of “transition probability to node”. Note that the sum of the probabilities of each row in the column of “transition probability to another node” is 100
[%].

Therefore, in the node NODE ₁₀₀ represented by the state transition table 180 in FIG. 14, for example, “ball is detected (BALL)”, and the “SIZE” of the ball is in the range of “0 to 1000”. Is given, the probability of “30 [%]” and “node N
ODE ₁₂₀ (node 120) "and then" A
The action of “CTION1” is output.

Each of the behavior models 170 _{1 to} 170 _n is formed by connecting a number of nodes NODE ₀ to NODE _n described as such a state transition table 180, and is recognized from the input semantics converter module 159. When a result is given, the next action is determined stochastically using the state transition table of the corresponding nodes NODE ₀ to NODE _n , and the determined result is output to the action switching module 171. .

The action switching module 171 shown in FIG.
Is the behavior model 170 of the behavior model library 170
Among the behaviors output from _{1 to} 170 _n, behavior models 170 _{1 to} 170 having a predetermined high priority
_n, and outputs a command to execute the action (hereinafter referred to as an action command) to the output semantics converter module 168 of the middleware layer 140. In this embodiment, the priority order is set higher for the behavior models 170 _{1 to} 170 _n shown on the lower side in FIG.

When reproducing the learned behavior, the behavior switching module 171 selects the specified desired behavior and sends a command to execute the behavior to the output semantics converter module 168. By the command from the action switching module 171,
The robot device 100 can output the learned behavior.

Furthermore, the action switching module 171
After the action is completed, the learning module 172, the emotion model 173, and the instinct model 174 are notified of the completion of the action based on the action completion information provided from the output semantics converter module 168.

On the other hand, the learning module 172 determines whether the recognition result given by the input semantics converter module 159 is “strapped” or “stroked”.
The recognition result of the instruction received as an action from the user is input.

Then, based on the recognition result and the notification from the action switching module 171, the learning module 172 lowers the probability of occurrence of the action when “beaten (scorched)” and “strokes ( In some cases, the corresponding behavior model 17 in the behavior model library 170 is increased so as to increase the probability of occurrence of the behavior.
Changing the 0 ₁ to 170 _n corresponding transition probability.

For example, the learning section 1 as described above is configured and realized by such a learning module 172 in the actual robot apparatus 1.

On the other hand, the emotion model 173 indicates “joy (jo
y) "," sadness "," anger ",
For a total of six emotions, “surprise”, “disgust” and “fear”, a parameter indicating the intensity of the emotion is stored for each emotion. Then, the emotion model 173 converts the parameter values of each of these emotions into “strapped” and “stroke” given from the input semantics converter module 159, respectively.
The update is periodically performed based on a specific recognition result such as, for example, an elapsed time and a notification from the action switching module 171.

Specifically, emotion model 173 is based on a recognition result given from input semantics converter module 159, the behavior of robot device 100 at that time, the elapsed time since the last update, and the like. The variation amount of the emotion at that time calculated by the arithmetic expression is ΔE [t], and the current parameter value of the emotion is E
[T], the coefficient representing the sensitivity of the emotion as _{k e,}
The parameter value E [t + 1] of the emotion in the next cycle is calculated by the equation (8), and the parameter value of the emotion is updated by replacing the parameter value E [t] with the parameter value E [t] of the emotion. The emotion model 173 is
Similarly, the parameter values of all emotions are updated.

[0110]

(Equation 8)

It is determined in advance how much each recognition result and the notification from the output semantics converter module 168 affect the variation ΔE [t] of the parameter value of each emotion. Is the amount of change in the parameter value of the emotion of “anger” △ E
[T] is greatly affected, and the recognition result such as “stroke” is the variation amount of the parameter value of the emotion of “joy” 喜 び E
[T] is greatly affected.

Here, the notification from the output semantics converter module 168 is so-called action feedback information (action completion information), information on the appearance result of the action, and the emotion model 173 also uses such information. Change emotions. This is, for example, a behavior such as "barking" that lowers the emotional level of anger. Note that the notification from the output semantics converter module 168 is also input to the learning module 172 described above, and the learning module 1
72 is an action model 170 _{1 to} 17 based on the notification.
Change the corresponding transition probabilities of 0 _n .

On the other hand, the instinct model 174 indicates that “the desire to exercise (ex
ercise), “affection”, “appet”
ite) "and" curiosity ", each of which has a parameter indicating the strength of the desire for each of the four independent desires. Then, the instinct model 174 periodically updates these parameter values of the desire based on the recognition result given from the input semantics converter module 159, the elapsed time, the notification from the action switching module 171 and the like.

More specifically, the instinct model 174 determines, based on the recognition result, the elapsed time, the notification from the output semantics converter module 168, and the like, for “exercise desire”, “affection desire”, and “curiosity”. The change amount of the desire at that time calculated by the arithmetic expression is ΔI
[K], the current parameter value of the desire I [k], as the coefficient k _i which represents the sensitivity of the desire, in a predetermined cycle (9)
Using the equation, the parameter value I of the desire in the next cycle
[K + 1] is calculated, and the calculation result is replaced with the current parameter value I [k] of the desire to update the parameter value of the desire. Instinct model 174
Updates the parameter values of each desire except “appetite” in the same manner.

[0115]

(Equation 9)

Note that the degree to which the recognition result and the notification from the output semantics converter module 168 affect the variation ΔI [k] of the parameter value of each desire is determined in advance. For example, the output semantics converter The notification from the module 168 has a large influence on the variation ΔI [k] of the parameter value of “fatigue”.

In the present embodiment, the parameter values of each emotion and each desire (instinct) are regulated to fluctuate in the range of 0 to 100, and the coefficient k
_e, the value of k _i is also set individually for each emotion and each desire.

On the other hand, the output semantics converter module 168 of the middleware layer 40
As shown in the above, an abstract action command such as "forward", "pleasure", "scream" or "tracking (chasing the ball)" provided from the action switching module 171 of the application layer 141 is output as described above. Corresponding signal processing module 161 of system 169
~ 167.

The signal processing modules 161 to 161
167, given the behavior command based on the action command, and servo command value to be supplied to the actuator ₁₂₅ 1 to 125 _n (FIG. 8) corresponding to perform that action, the output from the speaker 124 (FIG. 8) The audio data of the sound to be played and / or the driving data to be given to the LED of the "eye" are generated, and these data are sequentially processed through the virtual robot 133 of the robotic server object 132 and the signal processing circuit 114 (FIG. 8). Actuator 125 _{1 to} 125 _n or speaker 124 or LE
D.

In this way, the robot apparatus 100 can perform autonomous actions according to its own (internal) and surrounding (external) conditions and instructions and actions from the user based on the control program. It has been made possible.

The robot apparatus 100 as described above can learn a new action, and can weight the learned action.
Thereby, the robot device 100 can control the learned behavior according to the weighting.

In the above-described embodiment, the learning of the behavior has been described as the learning by the RNN, the learning of the behavior by the segmentation using the RNN, and the like. However, the present invention is not limited to this, and it goes without saying that the behavior can be learned by other learning means. In this case, the detecting unit 2, the evaluating unit 3, and the associating unit 4 as shown in FIG. 1 are configured according to the learning means.

[0123]

The robot apparatus according to the present invention has an input signal detecting means for detecting an external input signal from the outside, an evaluation means for evaluating the external input signal detected by the input signal detecting means, and an evaluation by the evaluation means. An input signal detection unit that includes an association unit that associates the result with the activity content information; and an activity control unit that controls the activity based on the activity content information based on the evaluation associated with the association device. The external input signal detected by the means is evaluated by the evaluation means, the evaluation result by the evaluation means is associated with the action content information by the association means, and the action content information is determined based on the evaluation associated by the association means. The behavior can be controlled by the behavior control means based on the. Accordingly, the robot device can evaluate the learned behavior and cause the behavior to appear based on the evaluation.

Further, in the behavior control method for a robot device according to the present invention, the robot device detects an external input signal from the outside, and outputs the external input signal detected in the input signal detection process to the robot device. A robot apparatus based on the action content information based on the evaluation step evaluated by the user, the associating step of assigning the evaluation result obtained in the evaluation step to the action content information, and the evaluation associated in the associating step. Having a behavior control step of controlling the behavior, the robot device whose behavior is controlled by such a behavior control method of the robot device evaluates the learned behavior, and performs the behavior based on the evaluation. Can appear.

The program according to the present invention includes an input signal detecting step for detecting an external input signal from the outside, an evaluating step for evaluating the external input signal detected in the input signal detecting step, and an evaluating step. An associating step of associating the obtained evaluation result with the action content information, and an action control step of controlling an action based on the action content information based on the evaluation associated in the associating step, to the robot apparatus. By executing the program, the robot apparatus in which the control of the action is executed by such a program can evaluate the learned action and cause the action to appear based on the evaluation.

Further, the recording medium according to the present invention includes an input signal detecting step for detecting an external input signal from the outside, an evaluation step for evaluating the external input signal detected in the input signal detecting step, and an evaluation step. Robot apparatus comprising: an associating step of associating the evaluation result obtained with the action content information; and an action control step of controlling an action based on the action content information based on the evaluation associated in the associating step. The robot device in which the program to be executed is recorded, and the control of the behavior is executed by the program recorded in such a recording medium, evaluates the learned behavior and causes the behavior to appear based on the evaluation. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a main part for realizing the invention in a robot device according to an embodiment.

FIG. 2 is a diagram showing a specific configuration of the above-described learning unit, which is hierarchically configured by a plurality of RNNs.

FIG. 3 is a diagram illustrating a configuration of an RNN in a lower layer of a learning unit configured as the above-described hierarchical structure.

FIG. 4 is a diagram illustrating a configuration of an RNN in an upper layer of a learning unit configured as the above-described hierarchical structure.

FIG. 5 is a diagram used to explain behavior learning by the learning unit described above.

FIG. 6 is a diagram illustrating a configuration of a data processing unit that implements rehearsal for operation.

FIG. 7 is a perspective view illustrating an external configuration of the robot device according to the embodiment.

FIG. 8 is a block diagram showing a circuit configuration of the robot device described above.

FIG. 9 is a block diagram showing a software configuration of the robot device described above.

FIG. 10 is a block diagram showing a configuration of a middleware layer in a software configuration of the robot device described above.

FIG. 11 is a block diagram showing a configuration of an application layer in the software configuration of the robot device described above.

FIG. 12 is a block diagram showing a configuration of an action model library of the application layer.

FIG. 13 is a diagram used to explain a finite probability automaton that is information for determining an action of a robot device.

FIG. 14 is a diagram showing a state transition table prepared for each node of the finite probability automaton.

[Explanation of symbols]

1 learning unit, 2 detection unit, 3 evaluation unit, 4 association unit,
100 robot device

Claims (9)

[Claims] 1. An input signal detecting means for detecting an external input signal from outside, an evaluating means for evaluating an external input signal detected by the input signal detecting means, and an evaluation result by the evaluating means as action content information. A robot device comprising: a matching unit for making a correspondence; and an action control unit that controls a behavior based on the behavior content information based on the evaluation associated with the correspondence unit. 2. The robot apparatus according to claim 1, further comprising learning means for learning a new action, wherein the learning means makes the new action correspond to the action content information by learning. 3. The method according to claim 1, further comprising learning means for learning a new action, wherein the learning means segments and learns the time-series data to be the action to be learned. Robotic device. 4. The robot apparatus according to claim 1, wherein the evaluation result is a probability or a weight for causing an action to appear. 5. The robot apparatus according to claim 1, wherein the association unit is a storage unit that stores the action content information and the evaluation result as a pair. 6. An internal state is changed according to input information,
The robot device according to claim 1, wherein the robot device acts based on an internal state. 7. An input signal detecting step in which the robot apparatus detects an external input signal from the outside; an evaluation step in which the robot apparatus evaluates the external input signal detected in the input signal detecting step; In the associating step of associating the evaluation result obtained in the above evaluating step with the action content information, based on the evaluation associated in the associating step,
A behavior control step of controlling the behavior of the robot apparatus based on the behavior content information. 8. An input signal detection step of detecting an external input signal from the outside, an evaluation step of evaluating the external input signal detected in the input signal detection step, and an evaluation result obtained in the evaluation step. Based on the associating step of associating with the action content information,
A program for causing a robot apparatus to execute a behavior control step of controlling behavior based on the behavior content information. 9. An input signal detecting step of detecting an external input signal from outside, an evaluating step of evaluating the external input signal detected in the input signal detecting step, and an evaluation result obtained in the evaluating step. Based on the associating step of associating with the action content information and the evaluation associated in the associating step,
A recording medium on which a program for causing a robot apparatus to execute an action control step of controlling an action based on the action content information is recorded.