JP2023149322A - Method for recovering from failure in interaction and computer program - Google Patents

Abstract

[Problem] To enable a robot to recover in a natural way when it fails to communicate with a human. The recovery method includes a failure detection step 366 of detecting a failure in communication with a dialogue partner, and in response to the detection of the failure, the robot is configured to perform a dialogue with the dialogue partner that involves the expression of emotions. The control includes steps 370-394 for returning using information obtained in the interaction. The failure detection step includes a process of making a first utterance to confirm the result of the identification process with a first attitude according to the reliability of the identification process, and a process of starting the process of identifying the dialogue partner with a second attitude that appears lacking in confidence. selectively performing a process of performing a second utterance, and performing the identification procedure with a second attitude in response to the interaction partner's response to the first utterance indicating an error in the result of the identification process; and controlling the robot to make a third utterance to initiate. [Selection diagram] Figure 3

Description

The present invention relates to robot control technology, and in particular to improvements in communication technology with humans using humanoid robots.

Recent developments in robot technology have been remarkable, and robots are now able to perform a variety of tasks that were previously difficult. In particular, robots are expected to take over the tasks of humans in daily life and perform tasks that require communication with humans. Therefore, research and development on robots that blend into society and perform daily activities that interact with humans is actively being conducted.

Robots that perform daily activities are required to have conversational capabilities. Various robots that are said to have conversational functions have been developed. Dialogue is originally thought of as an exchange of words between two people. Therefore, even when a human interacts with a robot, it is considered desirable to have a robot that resembles a human as the conversation partner. There are humanoid robots that look exactly like humans as beings that pursue such human-like characteristics. Hereinafter, the humanoid robot will be simply referred to as a robot.

In human-robot communication, there is published literature showing that many people cannot tell the difference between a robot and a human if they only look at their behavior over a short period of time. Therefore, it is believed that robots can become communication media that can closely interact with humans in society.

When such robots interact with humans, a problem that arises is the collision of utterances. Conflicts in utterances often occur in conversations between people. In dialogue between people, when a conflict of utterances occurs, the conflict is resolved by various responses depending on the situation. However, such flexibility is not possible when the conversation partner is a robot. Therefore, interactions between humans and robots can be quite different from interactions between humans.

One proposal related to these problems is proposed in Patent Document 1 mentioned below. The technology disclosed in the patent document is a technology for estimating who will be the next speaker in a conversation in which multiple speakers participate. In this estimation, the next speaker is estimated based on information regarding changes in the shape of each speaker's mouth. If this method is used, it may be possible to prevent utterance collisions in, for example, remote conferences.

Japanese Patent Application Publication No. 2018-77791

The technique disclosed in Patent Document 1 may be effective for estimating the next speaker in a conversation between humans. However, the technique disclosed in Patent Document 1 cannot solve the problem that when a robot participates in a conversation, the robot's own utterances collide with the utterances of the conversation partner. Furthermore, since the conversation partner is a human, there is always a possibility that a conflict of utterances will occur between the robot and the human. There is also the problem that the technique disclosed in Patent Document 1 cannot be applied to the problem of how to resolve such a conflict of speech without causing a sense of discomfort to the conversation partner.

In short, these problems can be seen as the question of how to recover from a failure in communication with a human without causing discomfort to the other person when a robot fails to communicate with a human. For example, if the robot cannot recognize the conversation partner or makes a mistake in recognizing the conversation partner at the beginning of a conversation, the robot and the conversation partner will no longer be able to proceed with the conversation. Even in such cases, it is necessary to recover from a communication failure in a natural way.

Therefore, it is an object of the present invention to provide a method and computer program for recovering from a failure in dialogue, which allows a robot to recover in a natural manner when it fails to communicate with a human.

A method for recovering from a failure in dialogue according to a first aspect of the present invention includes a failure detection step in which a computer detects a failure in communication between a robot and a dialogue partner; In response to this, the robot is controlled to have a dialogue that involves the expression of emotion with the dialogue partner according to a predetermined procedure, and the information obtained during the dialogue is used to recover from the failure. step.

Preferably, in the failure detection step, the computer confirms the result of the identification process to the conversation partner with a first attitude prepared in advance, depending on whether the reliability of the conversation partner identification process is higher than a predetermined threshold. a process for controlling the robot to make a first utterance to perform a conversation; and a second utterance for starting a dialogue partner identification procedure with a second attitude prepared in advance so as to appear less confident than the first attitude. selectively performing a process of controlling the robot to perform a second utterance in response to the interaction partner's response to the first utterance indicating an error in the result of the identification process; and controlling the robot to make a third utterance for starting the identification procedure with the attitude of the robot.

More preferably, in the step of performing the return, the computer classifies the interaction partner as an acquaintance of the robot in response to the interaction partner's response to the first utterance indicating that the identification result is correct. and a step in which the computer controls the robot to initiate a dialogue according to a previously prepared scenario for dialogue with an acquaintance.

More preferably, the second utterance and the third utterance are the same utterance.

Preferably, the second utterance is an utterance in which the robot asks whether the conversation partner is meeting the robot for the first time.

More preferably, the step of performing the return further includes a step in which the computer determines whether or not the dialogue partner's response to the second utterance affirms that the dialogue partner is meeting the robot for the first time; In response to a negative response from the dialogue partner, the user adopts a third attitude prepared in advance so as to appear even less confident than the second attitude, regarding whether or not the dialogue partner is the person identified by the identification process. the computer controlling the robot to make a fourth utterance; and the computer, in response to the interaction partner's response to the fourth utterance being affirmative, classifying the interaction partner as an acquaintance of the robot; controlling the robot to start the dialogue with a fourth attitude prepared in advance so as to appear relieved; and in response to the negative response of the dialogue partner to the fourth utterance, the robot appears disappointed. the computer controlling the robot to display a pre-prepared fifth attitude to be visible and to perform an additional identification process.

More preferably, the additional identification process includes a step in which the computer controls the robot to utter a question asking the interaction partner to ask the interaction partner a name, and a step in which the computer controls the robot to utter a question asking the interaction partner to ask the interaction partner a name included in the response to the interaction partner's name question. a step of generating a determination result by determining whether the name matches a person's name registered in a person information database prepared in advance; controlling the robot to classify the other party as an acquaintance for the robot and to start a dialogue according to a scenario for dialogue with an acquaintance while displaying a fifth attitude prepared in advance so as to appear happy; In response to a negative determination result, the computer classifies the conversation partner as a person unknown to the robot, and starts a conversation with the conversation partner according to a scenario prepared in advance as a conversation with an unknown person. and controlling the robot.

Preferably, the additional identification process includes a step in which the computer controls the robot to utter a question asking the interaction partner to ask his or her name, and a step in which the computer determines in advance the name included in the interaction partner's response to the name asking question. a step of generating a determination result by determining whether the name matches the name of the person registered in the prepared person information database; a step of confirming whether or not the person is the same as a person registered in a person information database, and classifying the conversation partner into an acquaintance or an unknown person for the robot according to the confirmation result; In response to the conversation partner being classified as an acquaintance for the robot, the robot starts a conversation according to a scenario for a conversation with an acquaintance while showing the fifth attitude prepared in advance so as to appear happy. A step of controlling the robot, and a scenario prepared in advance as a dialogue with an unknown person in response to the determination result being negative or the dialogue partner being classified as an unknown person to the robot. controlling the robot to initiate a dialogue with the dialogue partner according to the method.

More preferably, the step of returning further includes the computer controlling the robot to make a fifth utterance for identifying the dialogue partner in response to the dialogue partner's response to the fourth utterance being affirmative. a step in which the computer generates a determination result regarding whether or not the information identifying the dialogue partner included in the dialogue partner's response to the fifth utterance matches the result of the identification process; In response to the affirmative, the computer controls the robot to make a sixth utterance for confirming that the conversation partner corresponds to an acquaintance of the robot; In response to an affirmative response from the conversation partner, the robot classifies the conversation partner as an acquaintance, and while displaying the fifth attitude prepared in advance so as to appear happy, the robot enters into a scenario for a conversation with an acquaintance. controlling the robot to initiate the compliant interaction.

More preferably, the step of performing the return further includes, in response to the negative determination result, the computer classifying the conversation partner as a person unknown to the robot, and creating a scenario prepared in advance as a conversation with an unknown person. controlling the robot to initiate a dialogue with the dialogue partner according to the method.

Preferably, the step of performing the return further comprises: in response to the interaction partner's response to the sixth utterance being negative, the computer classifies the interaction partner as a person unknown to the robot, and the computer classifies the interaction partner as a person unknown to the robot, The method includes the step of controlling the robot to start a dialogue with a dialogue partner according to a scenario prepared in advance.

A computer program according to a second aspect of the invention causes a computer to function to execute any of the methods described above.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention, understood in conjunction with the accompanying drawings.

FIG. 1 is a block diagram showing the hardware configuration of a robot system according to a first embodiment of the present invention. FIG. 2 is a schematic diagram showing an example of a description format of a program executed by the robot system shown in FIG. 1 to control a robot. FIG. 3 is a flowchart showing the control structure of a program that the robot system shown in FIG. 1 executes when starting a conversation with a dialogue partner. FIG. 4 is a schematic diagram showing a pattern of speech conflicts for which the robot is responsible. FIG. 5 is a schematic diagram showing a pattern of utterance conflicts for which a person is responsible. FIG. 6 is a diagram showing the timing of detecting a speech collision for which the robot is responsible. FIG. 7 is a diagram showing the timing of detecting a speech conflict for which a person is responsible. FIG. 8 is a flowchart showing the control structure of a program that the robot system according to the first embodiment executes when it detects a speech conflict for which the robot is responsible. FIG. 9 is a flowchart showing the control structure of a program that the robot system according to the first embodiment executes when it detects an utterance conflict for which the conversation partner is responsible. FIG. 10 is a flowchart showing a control structure of a program executed by the robot system according to the first embodiment when detecting and classifying speech collisions. FIG. 11 is a diagram for explaining the range to be excluded from speech collision during speech. FIG. 12 is a flowchart showing the control structure of a program that the robot system according to the second embodiment executes when starting a conversation with a conversation partner. FIG. 13 is a flowchart showing the control structure of a program that the robot system according to the third embodiment executes when starting a conversation with a conversation partner. FIG. 14 is a flowchart showing the control structure of a program executed by the robot system according to the fourth embodiment when detecting and classifying speech collisions. FIG. 15 is an external view of an example of a computer for realizing the robot system according to each embodiment. FIG. 16 is a block diagram of an example of the computer shown in FIG. 15.

In the following description and drawings, identical parts are provided with the same reference numerals. Therefore, detailed description thereof will not be repeated.

1st Embodiment 1. Configuration FIG. 1 shows, in block diagram form, the hardware configuration of a robot system 100 that communicates with humans according to a first embodiment of the present invention. Referring to FIG. 1, robot system 100 includes a camera 60, a microphone 66, a speaker 62, and a robot 110. The robot 110 is a humanoid robot, and has actuators at least in parts corresponding to the joints of the upper body, and can take various postures by driving the actuators. Further, the head of the robot 110 is provided with a plurality of actuators that control the positions of control points defined on the face of the robot 110, and by driving these actuators, it is possible to give the robot various facial expressions.

The robot system 100 further includes a face image recognition PC (Personal Computer) 116 connected to receive the output of the camera 60 and for performing face image recognition on a face image of a person in the video output by the camera 60, and a microphone 66. and a voice recognition PC 118 for receiving signals from the conversation partner and performing voice recognition on the utterances of the conversation partner. The facial image recognition PC 116 is composed of a neural network, and has a function of outputting information indicating which of a plurality of predetermined persons the inputted facial image belongs to. More specifically, the neural network that constitutes the facial image recognition PC 116 has outputs equal to the number of recognizable people, and calculates the likelihood that the input facial image is that of those people for each person. Output. The facial image recognition PC 116 determines the person with the highest likelihood as the facial image recognition result, and outputs information corresponding to the identifier. In the following explanation, the likelihood regarding the person selected as a recognition result will be referred to as the certainty of face image recognition. Note that the face image recognition PC 116 outputs not only the recognition result of the person but also a predetermined number of other identifiers with a high likelihood together with their likelihood.

The robot system 100 is further connected to a network 114 to which a face image recognition PC 116 and a voice recognition PC 118 are connected, and controls each actuator of the robot 110 according to the control commands given to control the posture and motion of the robot 110. and an action control PC 112 for controlling facial expressions. Several motions and facial expressions of the robot 110 are defined in advance, and programs for realizing them are prepared in advance. Facial expressions include a confident facial expression, a non-confident facial expression, a happy facial expression, a disappointing facial expression, and the like. The motion control PC 112 controls each actuator so that the robot 110 performs a desired motion with a desired facial expression by giving control information such as the duration of the motion and the magnitude of the motion as arguments to the program.

In order to realize each facial expression, the robot 110 is made to make facial expressions using various parameters in advance, and a plurality of subjects are asked to determine what kind of emotion the robot 110 is feeling based on the facial expressions. Based on the results, parameters for making the robot 110 make each facial expression may be determined.

The robot system 100 further includes a personal information DB (Database) 92 that manages information regarding a person to be recognized by the facial image recognition PC 116 (name, affiliation, etc.) so that the information can be accessed using the person's identifier as a key, and a network 114. and integration for controlling the overall motion of the robot 110 by calculating the motion and speech content of the robot 110 based on information from other PCs connected to the personal information DB 92 and connected to the network 114. It is connected to the control PC 122 and the network 114, and in response to the speech command output from the integrated control PC 122, performs speech synthesis to perform specified speech, and provides it as an audio signal to the speaker 62 to generate speech. includes a speech synthesis PC 120. The integrated control PC 122 stores a program for causing the robot 110 to behave according to a predetermined scenario in a storage device (not shown). In this embodiment, this program is created as a script expressed in a graph format, as will be described later.

In this way, the face image recognition PC 116, the voice recognition PC 118, the voice synthesis PC 120, the integrated control PC 122, and the motion control PC 112 work together to control the robot 110 so as to perform an action to communicate with a person.

With reference to FIG. 2, the format of the scenario stored in the storage device (not shown) of the integrated control PC 122 will be described. In this embodiment, the scenario is described in graphical form, such as graph 150 shown in FIG. Note that FIG. 2 is merely for explaining the description format of the scenario, and is not for causing the robot 110 to perform the operations described below.

The graph 150 shown in FIG. 2 is made up of blocks representing individual movement units of the robot and directed edges connecting each block. Graph 150 includes a start block 160, an utterance block 162, a question block 164, and a speech recognition block 166. These blocks are serially connected by directed edges and are executed sequentially along the edges. Also, the same goes for each block described later, but each block describes information for the robot to perform a group of actions. This information includes what the robot should say in that block, the action it should take, and its emotional state when performing the action. When the robot reaches each block, it operates based on the information written in that block.

The graph 150 further includes a facial expression block 168 and a motion block 170 that are connected in parallel to each other after the speech recognition block 166, and a termination block 172 that is connected next to the facial expression block 168 and the motion block 170. In speech recognition block 166, robot 110 speech recognizes the interlocutor's answer to the question in block 164 and, based on the results, executes block 168 or block 170 using different parameters. In this example, the questions asked in the question block 164 are basically questions that can be answered with a yes/no response, or questions for which the answer category can be predicted (for example, a question asking the other party's name). In the voice recognition block 166, the conversation partner's answers are classified into yes, no, don't know, can't say, no answer, and so on. If the conversation partner's answer is yes or no, block 168 operates using the content as a parameter. For example, when the answer matches the answer predicted by the robot, the robot makes a happy expression, and when it does not match, it makes a surprised expression. If there is an answer of "I don't know" or "I can't say," or if there is no answer, the corresponding parameters are passed to block 170, and the process of block 170 is executed. At block 170, the robot moves its hands or moves its head depending on the parameters.

In the following embodiment, it is assumed that the robot 110 is, for example, accepting visitors to a research institute, and recovery when communication with the visitor fails in the process will be described. Communication with a visitor as a conversation partner begins with the process of identifying the conversation partner. The embodiment described below is an example of recovery (recovery) from a communication failure, and describes a recovery process when a robot fails to identify a conversation partner and a speech conflict between a conversation partner and the robot during a conversation. Regarding recovery processing when such occurrence occurs.

A. Recovery Processing from Identification Failure FIG. 3 shows the control structure of a program that is executed by the robot 110 when the robot 110 fails in the identification processing of the dialogue partner, and realizes the recovery processing from the failure according to a predetermined procedure. . Identification of the conversation partner is executed by the facial image recognition PC 116 shown in FIG. 1 as described above. The face image recognition PC 116 notifies the integrated control PC 122 of the identified identifier of the conversation partner, its confidence level, and identifiers of some people who may be the conversation partner, although the confidence level is low.

Referring to FIG. 3, the program includes a step 360 of receiving the results of facial image recognition, and a step 362 of branching the flow of control depending on whether the received confidence is greater than a predetermined threshold. . The threshold value in this case cannot be determined unconditionally because it depends on the accuracy of face image recognition by the face image recognition PC 116. It is desirable to adjust this threshold based on the results of actual facial image recognition.

Further, in response to the affirmative determination in step 362, this program reads information regarding the person corresponding to the identifier identified as the conversation partner from the person information DB 92, and includes the information with a confident attitude (facial expression). Step 364: confirming the other party's name by speaking the name given to the other party; voice recognition of the other party's response to the utterance in step 364; determining whether or not the other party's response is affirmative; and controlling the flow of control according to the determination result. and branching step 366. If the determination in step 366 is affirmative, this means that the dialogue partner identification process has been successful, and that the dialogue partner is an acquaintance of the robot. Therefore, in step 368, the robot 110 classifies the conversation partner as an acquaintance, and starts a conversation with the conversation partner according to a scenario prepared in advance for a conversation with an acquaintance. At this time, it is preferable to control the robot 110 to greet the user with a smile.

This program further determines that when the determination in step 362 is negative (that is, the confidence level is below the threshold), or the determination in step 366 is negative (that is, the identification process is incorrect and the other party is not the person identified as the identification result). step 370, which is executed when the confidence level is higher than the threshold, and utters a question asking whether the conversation partner is meeting the robot 110 for the first time in a manner that makes the conversation partner appear less confident than when the confidence level is higher than the threshold; and the question in step 370. step 372 of voice-recognizing the dialogue partner's response to and branching the flow of control depending on whether the dialogue partner's response is affirmative or not. If the determination in step 372 is affirmative, it means that the conversation partner and robot 110 are meeting for the first time, and if negative, it means that the conversation partner and robot 110 must have interacted before.

In response to the negative determination in step 372, the program further utters an utterance to confirm with the dialogue partner whether the name of the person identified by the facial image recognition in step 360 matches the dialogue partner's name. step 374, and step 376, which performs speech recognition of the conversation partner's response to the utterance in step 374, and branches the flow of control depending on whether the response is affirmative or negative. If the determination in step 376 is affirmative, it means that the result of the first identification process was correct. Therefore, control proceeds to step 368, where the conversation partner is classified as an acquaintance and the robot is controlled to start a conversation with the acquaintance. In this case, in step 374, it is preferable to make the robot look like it lacks confidence. By doing this, it appears to the conversation partner that the robot is guessing who the conversation partner is. As a result, it has the effect of making the conversation partner feel the intelligence of the humanoid robot.

This program further includes a step 378 in which, in response to the negative determination in step 376, the robot 110 controls the robot 110 to make an utterance asking the dialogue partner for his or her name; step 380 of performing voice recognition on the response, determining whether or not the name of the other party included in the voice recognition result exists in the personal information DB 92, and branching the flow of control according to the result. If the determination in step 380 is affirmative, this person is an acquaintance of robot 110. Control therefore continues to step 368. If the determination in step 380 is negative, this person should be an acquaintance of the robot 110, but this information does not exist in the person information DB 92. Therefore, in this program, in step 382 and subsequent processes, the robot 110 is controlled to perform a process for gathering information about the other party with an apologetic expression towards the other party.

This program further includes a step 384 of controlling the robot 110 to make an utterance asking the other party for the name of the other party when the determination in step 372 is affirmative, and a step 384 of controlling the robot 110 to make an utterance to ask the other party's name; A step 386 in which it is determined whether the name matches the name of any of several persons listed as candidates in the face image recognition process in step 360, and the flow of control is branched according to the determination result. include.

If the determination result in step 386 is negative, the control proceeds to step 394, where the computer classifies the conversation partner as a person met for the first time (unknown person). Thereafter, the computer controls the robot 110 to perform a dialogue based on a script prepared in advance as a dialogue with a person the robot 110 is meeting for the first time.

Furthermore, in response to the affirmative determination result in step 386, the program makes an utterance to the effect that the robot 110 feels like it has met the conversation partner, with a slightly unsure expression. a step 388 in which the robot 110 is controlled to do so; a step 390 in which the control flow is branched depending on whether the dialogue partner's response to this utterance is affirmative; and a step 390 in which the robot 110 makes a happy expression when the determination in step 390 is affirmative. The method includes a step 392 of controlling the robot 110 so as to classify the conversation partner as an acquaintance of the robot 110. After step 392, similarly to step 368, the robot 110 is controlled to perform a dialogue with the acquaintance. If the determination at step 390 is negative, control proceeds to step 394, where the conversation partner is classified as a first-time partner.

In step 372, the other party acknowledges that this is the first time they are meeting the robot 110. Nevertheless, in steps 388 and 390, a process is performed to determine whether the other party has ever met the robot 110. This is because some people may avoid going out of their way to explain to a robot that they have met in the past when the other person does not recognize them, considering it troublesome. By including processes such as steps 388 and 390, it is expected that the dialogue partner will feel that the robot 110 has remembered him and will feel a sense of affinity towards the robot 110.

In this way, when a robot fails to identify a conversation partner, by having a conversation with the conversation partner that involves the expression of emotions, it is possible to extract information from the conversation partner for recovery from the failure. . Based on this information, the robot can recover from failure in a natural manner and start a dialogue according to the category of the dialogue partner.

B. Recovery Process from Utterance Conflict A communication failure that often occurs in dialogue is utterance conflict. Dialogue between people progresses as the speaker and listener take turns. The turn of the speaker is the speaking turn. Utterance turns usually change in a natural way. It is said that this is because some kind of turn-taking rule is established between the speaker and the listener. However, under some conditions, the turn exchange may fail and the two people may start speaking at almost the same time. This is a conflict of utterances.

It is thought that there is a tacit understanding between the two parties that the speaker has the right to speak until the turn is changed. This right is also called the right to speak, but in this specification it is referred to as the right to speak.

Conflicts in utterances during turn changes are considered to be primarily a failure on the part of the listener. In this embodiment, we will consider a case where the robot is responsible for a conflict in utterances and a case where the conversation partner is responsible. The former is illustrated in FIG. 4, and the latter is illustrated in FIG.

Referring to FIG. 4, a speech collision in which the robot is responsible will be described. In the conversation partner's utterance turn 400, after the conversation partner makes an utterance 410, the conversation partner attempts to make the next utterance 412 after a temporary interruption while maintaining the utterance turn. The robot misunderstands this temporary interruption as the end of its speaking turn and attempts to start speaking 414. As a result, the beginning portion of utterance 414 and the beginning portion of utterance 412 overlap in time, resulting in utterance collision 416.

Referring to FIG. 5, an utterance conflict in which the conversation partner is responsible is the opposite situation to that shown in FIG. Specifically, after the robot makes an utterance 440 during the utterance turn 430, the robot starts the next utterance 442 after a temporary interruption while maintaining the utterance turn. The dialogue partner misunderstands this temporary interruption as the end of the speaking turn and begins the next utterance 444. As a result, the beginning of utterance 442 and the beginning of utterance 444 overlap in time, resulting in utterance collision 446.

Note that just because the utterances of two speakers in a dialogue overlap in time, this does not immediately mean that there is a utterance conflict. Typically, while one speaker is speaking, the other speaker may chime in. Such exchanges should not be considered utterance conflicts. Furthermore, the other party may start speaking before one speaker has completely finished speaking. In this case as well, if it takes a short time to complete the speaking turn of the first speaker, it should not be considered an utterance conflict. These issues need to be taken into consideration when detecting speech collisions.

Using graphs showing exemplary programs shown in FIGS. 6 and 7, a situation where a speech collision is likely to occur and a target section (collision detection section) in which the occurrence of a speech collision is detected will be described. 6 and 7 are the same graphs. FIG. 6 shows an example of a situation in which a speech conflict in which the robot is responsible is likely to occur, and FIG. 7 shows an example in a situation in which a speech conflict in which the conversation partner is responsible is likely to occur.

Referring to FIG. 6, in this graph, two speech blocks and two question blocks are provided in this order following the leftmost start block. Next to the question block, a voice recognition block is provided which performs voice recognition of the other party's utterances. Following the speech recognition block, three paths are provided. One of these routes is selected depending on the dialogue partner's response recognized in the speech recognition block.

Each of these three paths includes two consecutive speech blocks. The last of these three paths joins the question block. Next to the question block, a voice recognition block is provided again. By voice recognition of the conversation partner's utterance in the voice recognition block, the conversation topic is selected from topics 1, 2, and topic 3, and the execution of this program is completed.

Referring to FIG. 6, an example of a situation in which a robot is likely to be responsible for a speech conflict is shown. When the robot operates according to the graph shown in FIG. 6, a utterance collision is likely to occur immediately after the conversation partner's utterance turn 460. Therefore, a collision detection section 464 for a speech collision surrounding the beginning of the speech block following the speech turn 460 of the dialogue partner is required. On the other hand, there is no robot speech block after the opponent's speech turn 462. Therefore, regarding the speech turn 462, there is no need for a collision detection section for speech collision.

Referring to FIG. 7, an example of a situation in which an utterance conflict in which the conversation partner is responsible is likely to occur is shown. When the robot speaks according to the graph shown in FIG. 7, three speech blocks immediately follow the start block of the graph. These three speech blocks constitute a speech turn 480 of the robot. In the robot's speech turn 480, there is a speech break after each speech block. In such a range, there is a possibility that the conversation partner may mistakenly think that the robot's speaking turn has ended and speak. As a result, utterance conflicts are likely to occur in which the conversation partner is responsible. Therefore, in the speech turn 480 shown in FIG. 7, the area including the boundaries of each block is collectively defined as a collision detection section 484.

After the robot's speaking turn 480 there is a speech recognition block. This part is the conversation partner's turn to speak. Furthermore, there is a robot speech turn 482 that includes three paths that are mutually parallel to the end of this speech recognition block. At the beginning of each utterance block within this utterance turn 482, a utterance conflict in which the conversation partner is responsible is likely to occur. Therefore, the portion including these is collectively defined as a collision detection section 486.

In this embodiment, duplication of utterances occurring outside of such a collision detection section is not considered as a utterance collision. Of course, such duplication of utterances may be treated as a utterance collision.

FIG. 8 shows the control structure of a program that realizes recovery from a speech collision in which the robot is responsible. Referring to FIG. 8, the program includes a step 600 in which the robot interrupts speech and a predefined facial expression (e.g., a surprised facial expression, a confused facial expression, and controlling the robot to make a facial expression (such as a facial expression). In step 602, the robot may be controlled so that it utters a voice such as "Ah!".

This program further includes, following step 602, a step 604 in which the robot executes a process for yielding the right to speak to the conversation partner, and a step 604 in which the robot determines whether or not a new collision has occurred and creates a new collision. If another collision occurs, control is returned to step 600; otherwise, this return processing is terminated in step 606.

Step 604, following step 602, determines whether or not there has been an interruption in the conversation partner's speech, and branches the flow of control according to the determination result; and in response to the affirmative determination in step 610, In response to a step 612 in which the speaking turn is adjusted by yielding the right to speak to the dialogue partner, and the determination in step 610 is affirmative and the execution of step 612 is completed, or the breakdown in step 610 is negative. , step 614 of waiting until the end of the conversation partner's speaking turn is detected, and in response to the detection of the end of the conversation partner's speaking turn, retransmitting the information that the robot was about to speak, and step 614. and step 616, which ends the process of step 604.

The reason for determining whether or not the dialogue partner's speech has been interrupted in step 610 is as follows. If the conversation partner does not interrupt the utterance, the robot can recover from the utterance conflict simply by interrupting the utterance. Therefore, in this case, it is sufficient for the robot to make a gesture indicating yielding the right to speak without making any particular utterance. In some cases, gestures may not be necessary. Therefore, the dialogue can be quickly repaired even if the robot does not speak.

On the other hand, if the conversation partner interrupts speaking, the robot needs to more politely yield the right to speak, since the conversation partner originally had the speaking turn. Therefore, in this case, the robot more politely makes an utterance clearly indicating that it is giving up the right to speak to the other party, and makes a gesture for that purpose. In this case, the robot's behavior may be to encourage the conversation partner to speak by, for example, uttering, "Please come first," and extending its hand toward the conversation partner. Of course, this is just one example, and various other utterances and gestures can be considered to repair the dialogue.

By performing such processing, it is possible to maintain the tempo of the dialogue between the robot and the dialogue partner, while realizing the robot's behavior that allows for natural dialogue.

FIG. 9 shows a control structure of a program for realizing dialogue recovery (return) from an utterance conflict in which the dialogue partner is responsible. Referring to FIG. 9, the program includes step 630 of interrupting the robot's speech in response to a detected speech conflict, and controlling the robot to make a facial expression indicating that the robot has noticed the speech conflict. In step 632, the control is performed in steps 636 and 638 according to whether or not the value of the robot's desire to speak, which is set as one parameter of the robot's emotion, is greater than a predetermined threshold in the utterance block being executed. and bifurcating the flow 634. Step 636 is a process for the robot to yield the right to speak to the other party, and step 638 is a process for the robot to maintain the speaking turn. In this embodiment, the value of the speech motivation of the robot 110 is set by the system designer, for example, when defining a corresponding speech block when creating a scenario. Of course, the value of the desire to speak may be set by some means when executing the scenario based on other conditions.

Step 636 includes a step 650 in which the flow of control is branched depending on whether or not there is an interruption in the conversation partner's utterance, and a step 652 in which the utterance turn is adjusted in response to an affirmative determination in step 650. including. The processing performed in step 652 is the same as the processing performed in step 612 of FIG. Step 636 further includes, in response to the affirmative determination of step 650 and the completion of step 652, or the negative determination of step 650, step 654 of waiting until the dialogue partner's speaking turn ends; In response to the completion of the other party's speaking turn, the robot retransmits the information that the robot was trying to convey through the utterance when the utterance collision occurred, and the step 636 is ended.

The program further determines whether further re-collisions have occurred in response to step 636 being completed, returns control to step 630 if the determination is positive, and returns control to step 630 if the determination is negative; The program includes step 640 of terminating execution of the program as having completed the repair.

Step 638 includes a step 660 in which a filler is expressed, a step 662 in which the robot adjusts the speech turn by informing the other party that the robot will maintain the speech turn following step 660, and a step 662 in which the robot adjusts the speech turn when a speech conflict occurs. and retransmitting the information that the program intended to convey through the utterance, and terminating the execution of the program. The filler in step 660 may be, for example, a meaningless voice such as "umm". The utterances made in step 662 are ones that make it clear that the robot will maintain the utterance turn, such as "Please let me say it first" or "Can I speak first?" That's fine.

FIG. 10 shows the control structure of a program for determining whether or not there is a speech conflict between the robot and the conversation partner. This program is started at each time in the robot's control loop, for example every 100 milliseconds.

Referring to FIG. 10, this program includes a step 700 in which the flow of control is branched according to a determination as to whether or not the robot is recovering from a collision, and when the determination in step 700 is negative, the robot's utterance and the dialogue partner are step 704, in which the flow of control is branched according to the determination as to whether or not the utterance of the robot overlaps with the utterance of and step 706 of branching the flow of control. Note that it is possible to determine whether or not the robot's utterance in step 706 is a mutual response by referring to a dictionary created by collecting utterance texts that are mutual responses in advance. The same applies when the speaker is the other party.

When the determination at step 700 is affirmative, when the determination at step 704 is negative, and when the determination at step 706 is affirmative, control proceeds to step 702. In step 702, it is determined that no utterance conflict has occurred, and the processing when no utterance conflict has occurred is executed.

The determination in step 700 results in that no speech collision is detected during recovery from the collision. Further, as determined in step 704, if the utterances of the robot and the utterance partner do not overlap in time, no utterance collision occurs. Further, as determined in step 706, if the robot's utterance is a mutual response, it will not be determined as a utterance collision even if it overlaps with the other party's utterance.

The program further includes a step 708 in which, in response to a negative determination in step 706, the flow of control is branched depending on whether the robot retains the right to speak after the robot's previous utterance; In response to the affirmative determination in step 728, the control flow is branched depending on whether the predicted speech length of the robot is equal to or greater than the threshold value T1 milliseconds. When the determination at step 728 is negative, it is concluded that no utterance collision occurs (step 736).

The predicted utterance length of the robot here refers to the maximum length of the robot's current utterance. Even if the robot makes all of its current utterances, if their length is small enough, there will actually be no utterance collision. This is the reason for the determination in step 728.

The program further includes a step 730 in which, in response to the affirmative determination in step 728, the flow of control is branched depending on whether the robot's current utterance position is a portion of a predetermined length at the end of the utterance or something else; In response to the determination in step 730 being negative, that is, in response to determining that the utterance is at the start position or the utterance body, the time period during which the robot's utterance and the conversation partner's utterance overlap is a threshold value T2. and step 732 of branching the flow of control depending on whether the time is greater than or equal to a millisecond.

Even if the predicted utterance length of the robot is long to some extent, if the current utterance position is at the end of the utterance, there is no need to consider it as a utterance collision. This is the reason for the determination in step 730. Further, as determined in step 708, the robot holds the right to speak. Therefore, if the robot's utterance position is not at the end of the utterance, and if the utterance overlap time is equal to or greater than the threshold, it can be concluded that an utterance conflict has occurred due to the conversation partner's responsibility. This is the reason for the determination in step 732. Therefore, this program further includes step 734 in which, if the determination in step 732 is affirmative, it is determined that an utterance conflict attributable to the dialogue partner has occurred.

The program further includes a step 736 in which it is determined that a speech conflict does not occur when the determination in step 728 is negative, the determination in step 730 is positive, or the determination in step 732 is negative.

This program further includes a step 710 in which, in response to the negative determination in step 708, the flow of control is branched depending on whether or not the dialogue partner's utterance is a mutual response, and in response to the negative determination in step 710. Step 712 branches the flow of control according to whether the predicted speech length of the robot is equal to or greater than the threshold value T1 milliseconds, and in response to an affirmative determination in step 712, the robot's current speech position is Step 714 branches the flow of control depending on whether the utterance is at the end, at the beginning, or at the main body; and step 718 of branching the flow of control depending on whether the utterance overlap time is equal to or greater than a threshold T2 milliseconds.

If the conversation partner's utterance is mutual, even if the robot's utterance and the conversation partner's utterance overlap, there is no need to consider that utterance overlap has occurred. The determination in step 710 is for this purpose.

When it is determined in step 714 that the robot's current utterance position is at the end of the utterance, control proceeds to step 702 and it is concluded that no utterance collision has occurred. When it is determined in step 714 that the robot's current utterance position is at the beginning of the utterance, control proceeds to step 716, where it is concluded that an utterance conflict for which the robot is responsible has occurred. When the determination in step 718 is affirmative, control proceeds to step 720, and it is concluded that an utterance conflict caused by the conversation partner's responsibility has occurred. If the determination in step 718 is negative, control proceeds to step 722 and it is determined that no speech conflict has occurred.

If the determination at step 708 is negative, the robot does not hold the right to speak. Therefore, if the robot's predicted utterance length is longer than the threshold in step 712 and the current utterance position is at the beginning of the utterance, it can be concluded that an utterance collision caused by the robot has occurred. Furthermore, if it is determined in step 714 that the robot's current utterance position is at the end of the utterance, it can be considered that a utterance collision has not substantially occurred. Furthermore, if the robot's current utterance position is the utterance body and duplicate utterances occur, it can be determined that this is when the conversation partner starts speaking. Therefore, if the overlapping time is equal to or greater than the threshold, it can be determined that an utterance conflict has occurred due to the dialog partner's responsibility; otherwise, it is considered that the dialog partner ended the utterance immediately, and a utterance conflict has actually occurred. It can be determined that this has not been done.

This program further includes a step 724 in which, when the determination in step 712 is negative, control is branched depending on whether or not the robot's utterance indicates the end of the conversation. If the determination in step 724 is affirmative, control proceeds to step 722 and it is concluded that no speech conflict has occurred. If the determination at step 724 is negative, control proceeds to step 726, where it is concluded that an utterance conflict for which the robot is responsible has occurred. Note that an example of the utterance that announces the end of the conversation in step 724 is a farewell greeting such as "goodbye." Further, the threshold value T1 may be, for example, 1300 milliseconds, and the threshold value T2 may be, for example, 1700 milliseconds.

FIG. 11 shows an example of the beginning 752, main body 754, and end 756 of one utterance 750 of the robot. In the example shown in FIG. 11, the beginning 752 is the range of 500 milliseconds from the start of the utterance. The tail 756 is a range of 500 milliseconds from the end of the predicted utterance length of the utterance. The main body 754 is the other part of the utterance 750. Of course, this is just one example, and the lengths of the beginning 752 and the end 756 are not limited to those shown in FIG. Further, the length of the beginning 752 and the length of the end 756 do not need to match.

2. Effects As described above, according to this embodiment, even when the robot fails to identify the conversation partner or when a speech conflict occurs during the conversation, the robot can repair the conversation according to a certain procedure and restore normal dialogue. You can return to At that time, the robot uses facial expressions and appropriate gestures to carry out restorative dialogue, so from the perspective of the dialogue partner, the effect is that communication can be restored in a natural manner, just like when speaking with a human. . Also, when a speech conflict occurs, unlike in the past, the robot does not always yield the right to speak to the other party. By setting the robot's desire to speak, if a speech conflict is the responsibility of the conversation partner, depending on the robot's desire to speak, the robot may maintain the right to speak and continue speaking, or it may yield the right to the conversation partner. Both operations are performed. This is a robot behavior that can be evaluated as being more human-like than conventional techniques, and has the effect of allowing the conversation partner to return to a normal conversation after an utterance conflict in a natural way.

2 Second Embodiment The robot according to the second embodiment has the control structure shown in FIG. 12 instead of the program executed by the robot according to the first embodiment shown in FIG. Run the program shown.

Referring to FIG. 12, the program executed by the robot according to the second embodiment is different from the program shown in FIG. The point is that the determination in step 770 is made in order to more reliably identify the person. In step 770, it is determined whether the attributes of the person described in the person information retrieved in the person information DB 92 in step 380 match the attributes of the other party. For example, if the record of the person information DB 92 includes the person's gender and date of birth, this information is compared with the gender and age estimated from the face image to be recognized in step 360. In this process, it is not possible to determine whether there is a complete match, but the probability (likelihood) that the person in the face image matches the gender recorded in the person information DB 92 and the age calculated from the date of birth. ) can be calculated by using a trained neural network.

If this likelihood is greater than or equal to a certain threshold, the determination at step 770 will be affirmative; otherwise, it will be negative. If the determination at step 770 is affirmative, control proceeds to step 368, where the recognized person is classified as an acquaintance. Otherwise, control proceeds to step 382, where the other party is classified as a person who is supposed to be an acquaintance but for whom there is no corresponding record in the personal information DB 92.

When identification of the other party by face image recognition fails, and even if the name the other party gave in response to the question in step 370 is in the person information DB 92, the person is not the same as the person recorded in the person information DB 92. It cannot be completely determined that they are the same person. By inserting the determination in step 770, it is possible to more accurately determine whether or not the conversation partner is the same person recorded in the personal information DB 92.

Third Third Embodiment The third embodiment is similar to the second embodiment in that a program whose control structure is shown in FIG. 13 is executed instead of the program shown in FIG. 3 of the first embodiment. different from.

The program shown in FIG. 13 is different from the program shown in FIG. 3 because, when the determination in step 380 is affirmative, the control is not immediately transferred to step 368, but as in the second embodiment, the dialogue partner is identified. The point is to provide processing to make it more reliable.

This program includes processing for identifying the conversation partner as accurately as possible when a plurality of records of a person having the name given by the conversation partner are found in the person information DB 92.

More specifically, in addition to each step shown in FIG. The method further includes step 800 of branching the flow of control depending on whether a plurality of results are found. If the determination in step 800 is negative, that is, if only one record is found, control proceeds to step 368, as in the first embodiment, and the conversation partner is classified as an acquaintance.

This program further includes, in response to the negative determination at step 800, that is, the number of records retrieved, a step for specifying a conversation partner using the retrieved record in the personal information DB 92. 802.

In step 802, in response to the affirmative determination in step 800, that is, in response to the fact that a plurality of records have been retrieved, the following step 822 is executed until a predetermined termination condition is satisfied for the plurality of records. Step 820 is included. The termination condition here is that either a record with information that matches the conversation partner is found among a plurality of records, or that no record with information that matches the conversation partner is found. Step 822 includes a step 840 in which information other than the name of the record to be processed is used to confirm whether or not the conversation partner is the person recorded in the record, and whether the determination in step 840 is affirmative or negative. and branching the flow of control according to step 842. For example, if the affiliation of each person is recorded in the person information DB 92, the robot asks the other person a question such as "Are you Mr. B from department A?" in step 840. If the other party answers this question in the affirmative, the determination at step 842 becomes affirmative. If the other party answers this question in the negative, the determination at step 842 will be negative.

If the determination in step 842 is affirmative, it is known that the conversation partner is the partner recorded in the record. Therefore, control exits step 822 and proceeds to step 368, where the conversation partner is classified as an acquaintance. If the determination at step 842 is negative, the robot performs similar processing using the next record. If the other party cannot be identified after performing steps 840 and 842 for all records, control proceeds to step 382. In step 382, the conversation partner is classified as a person who is supposed to be an acquaintance but is not recorded in the personal information DB 92, and a conversation is started.

As described above, according to the third embodiment, even if there are multiple records of a person corresponding to the name given by the conversation partner in step 378 in the person information DB 92, if there is a corresponding person among them, The person can be identified. The possibility of mistakenly identifying a person can be reduced. As a result, the effect of increasing the possibility that subsequent dialogue will proceed smoothly can be obtained.

Fourth Embodiment The fourth embodiment uses the program shown in FIG. The robot system is characterized by the fact that the robot system executes a program that shows the control structure to control the robot's movements.

Referring to FIG. 14, the program shown in FIG. 14 differs from the program shown in FIG. 9 in that between step 632 and step 634, the flow of control is branched depending on whether or not the utterance collision is the first collision. This point further includes 900. If the determination in step 900 is affirmative, control proceeds to step 636; if the determination in step 900 is negative, control proceeds to step 634. Step 634 and subsequent steps are the same as shown in FIG.

By providing this step 900, the following effects can be obtained. In the current dialogue between the dialogue partner and the robot, if an utterance conflict caused by the dialogue partner's responsibility occurs for the first time, the determination in step 900 will always be affirmative. Therefore, the robot always yields the right to speak to the person with whom it interacts. However, in the case of the second and subsequent utterances, the robot may maintain the right to speak or give the right to speak to the other party, depending on the robot's desire to speak. For example, when a speech conflict occurs in a conversation between people, the act of yielding the right to speak to the other person may be observed even if the person is not responsible. This behavior seems to be based on the expectation that the other person will do the same thing if the opposite situation occurs. These actions can be considered very human actions. In this embodiment, by having the robot perform such an action, the interaction with the robot becomes more natural and closer to the interaction with a human being from the perspective of the interaction partner.

Fifth Realization by Computer FIG. 15 is an external view of a computer system that operates as, for example, the integrated control PC 122 shown in FIG. 1. FIG. 16 is a hardware block diagram of the computer system shown in FIG. 15. The voice recognition PC 118, voice synthesis PC 120, face image recognition PC 116, and operation control PC 112 shown in FIG. 1 can also be realized by a computer system having almost the same configuration as the integrated control PC 122. Therefore, only the configuration of the integrated control PC 122 will be described here, and details of the configurations of the other PCs will not be repeated.

Referring to FIG. 15, this computer system 950 includes a computer 970 having a DVD (Digital Versatile Disc) drive 1002, a keyboard 974, a mouse 976, and a mouse 976, all connected to the computer 970, for interacting with a conversation partner. monitor 972. Of course, these are examples of configurations for when dialogue with a dialogue partner is required, and general hardware and software (e.g. touch panel, voice input, general pointing device) that can be used for dialogue with a dialogue partner to handle the system. If so, you can use anything. These are unnecessary if such dialogue with the dialogue partner is not expected.

Referring to FIG. 16, a computer 970 includes, in addition to a DVD drive 1002, a CPU (Central Processing Unit) 990, a GPU (Graphics Processing Unit) 992, and a bus 101 connected to the CPU 990, GPU 992, and DVD drive 1002. 0 and , a ROM (Read-Only Memory) 996 connected to the bus 1010 and storing a boot-up program for the computer 970 and the like.

The computer 970 further includes a RAM (Random Access Memory) 998 that is connected to the bus 1010 and stores instructions constituting the program, system programs, work data, etc., and an SSD (Solid Memory) that is a nonvolatile memory that is connected to the bus 1010 State Drive) 1000. The SSD 1000 is for storing programs executed by the CPU 990 and GPU 992, data used by the programs executed by the CPU 990 and GPU 992, and the like. The computer 970 further includes a network I/F (Interface) 1008 that provides connection to a network 986 (network 114 shown in FIG. 1) that enables communication with other terminals, and a USB (Universal Serial Bus) memory 984 that is removable. It includes a USB memory 984 and a USB port 1006 that provides communication with various parts within the computer 970.

The computer 970 is further connected to a bus 1010 and external devices such as a microphone 982, a speaker 980, a camera (not shown), and actuators of a robot (not shown) to perform input/output between internal parts such as the CPU 990 and external devices. It includes an input/output I/F 1004.

In the above embodiment, programs for realizing functions such as the operation control PC 112, the integrated control PC 122, the voice recognition PC 118, the voice synthesis PC 120, and the face image recognition PC 116 are all stored in the SSD 1000, RAM 998, DVD 978, etc. shown in FIG. It is stored in a USB memory 984 or a storage medium of an external device (not shown) connected via the network I/F 1008 and the network 986. Typically, these data and parameters are written into the SSD 1000 from the outside, for example, and loaded into the RAM 998 when the computer 970 is executed.

A computer program for operating this computer system so as to realize the functions of the operation control PC 112, integrated control PC 122, speech recognition PC 118, and speech synthesis PC 120 shown in FIG. It is stored on the DVD 978 that is installed, and transferred from the DVD drive 1002 to the SSD 1000. Alternatively, these programs are stored in the USB memory 984, the USB memory 984 is attached to the USB port 1006, and the programs are transferred to the SSD 1000. Alternatively, this program may be transmitted to computer 970 via network 986 and stored on SSD 1000.

The program is loaded into RAM 998 during execution. Of course, a source program may be input using the keyboard 974, monitor 972, and mouse 976, and the compiled object program may be stored in the SSD 1000. In the case of a script language as in the above embodiment, a script input using the keyboard 974 or the like may be stored in the SSD 1000. In the case of a program that operates on a virtual machine, it is necessary to install the program that functions as a virtual machine on the computer 970 in advance. Neural networks are used for facial image recognition, voice recognition, voice synthesis, etc. Neural networks are also used in programs that estimate the gender and age of a conversation partner from facial images. For these, a neural network trained by another system may be used, or the robot system 100 may perform the learning.

The CPU 990 reads the program from the RAM 998 according to the address indicated by an internal register called a program counter (not shown), interprets the instruction, and stores the data necessary for executing the instruction in the RAM 998 and the SSD 1000 according to the address specified by the instruction. Or read it from other devices and execute the process specified by the command. The CPU 990 stores the data of the execution result at an address specified by the program, such as the RAM 998, the SSD 1000, or a register within the CPU 990. Depending on the address, the computer outputs a command to the robot's actuator, a voice signal, etc. At this time, the value of the program counter is also updated by the program. Computer programs may be loaded directly into RAM 998 from DVD 978, from USB memory 984, or via network 986. Note that in the program executed by the CPU 990, some tasks (mainly numerical calculations) are dispatched to the GPU 992 according to instructions included in the program or according to an analysis result when the CPU 990 executes the instructions.

A program that causes the computer 970 to realize the functions of each part in each of the embodiments described above includes a plurality of instructions written and arranged to cause the computer 970 to operate to realize those functions. Some of the basic functions required to execute this instruction are provided by the operating system (OS) running on the computer 970, third party programs, modules of various toolkits installed on the computer 970, or the program execution environment. In some cases, it may be provided. Therefore, this program does not necessarily include all the functions necessary to implement the system and method of this embodiment. This program includes each of the above-mentioned devices and modules by statically linking or dynamically calling appropriate functions or modules in a controlled manner to obtain the desired results. It is sufficient to include only instructions for executing operations as its constituent elements. The manner in which computer 970 operates for this purpose is well known. Therefore, it will not be repeated here.

Note that the GPU 992 can perform parallel processing, and can execute a large amount of calculations associated with machine learning simultaneously in parallel or in a pipeline manner. For example, parallel computing elements found in a program when the program is compiled, or parallel computing elements discovered when the program is executed are dispatched from the CPU 990 to the GPU 992 and executed, and the results are sent directly or to the RAM 998. is returned to the CPU 990 via a predetermined address, and is substituted into a predetermined variable in the program.

Sixth Other Modifications In the embodiment described above, the robot 110 operates according to a specific scenario. However, the invention is not limited to such embodiments. Even in an embodiment in which the robot 110 selects its own behavior each time instead of a specific scenario, the method according to the above embodiment can be used when interacting with the other party. Further, in the above embodiment, the present invention is not only applicable to such cases as long as it relates to an error in identifying a dialogue partner at the beginning of a dialogue and a collision of utterances in the dialogue. When a dialogue breaks down because one party misunderstands the other party's utterances, or when the robot partner does not recognize the robot as a dialogue partner, the same way as above, by giving the robot an emotion and responding, it is possible to connect the robot to the robot. You can return to natural dialogue with others. Furthermore, in the above embodiment, one party to the dialogue is a robot, which has a physical entity. However, the invention is not limited to such embodiments. That is, the robot in this invention is not limited to a robot that has a physical entity. This invention can also be applied to images that imitate the human form, such as so-called avatars.

The embodiment disclosed this time is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim, with reference to the description of the detailed description of the invention, and all changes within the scope and meaning equivalent to the words described therein are defined. include.

60 Camera 62, 980 Speaker 66, 982 Microphone 92 Person information DB
100 Robot system 110 Robot 112 Operation control PC
114,986 Network 116 Facial image recognition PC
118 Voice recognition PC
120 Voice synthesis PC
122 Integrated control PC
150 Graph 360, 362, 364, 366, 368, 370, 372, 374, 376, 378, 400, 430, 460, 462 Utterance turn 412, 414, 440, 442, 444 Utterance 416, 446 Utterance collision 480, 482 Robot speech turn 950 computer system 970 computer

Claims (11)

a failure detection step in which the computer detects a failure in communication between the robot and the dialogue partner;
In response to the failure being detected in the failure detection step, the computer controls the robot to have a conversation with the conversation partner that involves the expression of emotion, according to a predetermined procedure. , the step of recovering from the failure using the information obtained in the dialogue,
The failure detection step includes:
A first step in which the computer confirms the result of the identification process with the conversation partner with a first attitude prepared in advance, depending on whether the reliability in the identification process of the conversation partner is higher than a predetermined threshold; a process of controlling the robot to make an utterance, and a second utterance for starting the dialogue partner identification procedure with a second attitude prepared in advance so as to appear less confident than the first attitude; selectively performing a process of controlling the robot to perform;
a third step for the computer to initiate the identification procedure with the second attitude in response to the interaction partner's response to the first utterance indicating an error in the result of the identification process; and controlling the robot to make the utterance. The step of performing the return includes:
a step in which the computer classifies the interaction partner as an acquaintance of the robot in response to the interaction partner's response to the first utterance indicating that the identification result is correct;
2. The method of claim 1, further comprising the step of: a computer controlling the robot to initiate a dialogue according to a previously prepared scenario for dialogue with an acquaintance. 3. The method of claim 1 or claim 2, wherein the second utterance and the third utterance are the same utterance. 4. The method according to claim 1, wherein the second utterance is an utterance in which the robot asks whether the conversation partner is meeting the robot for the first time. The step of performing the return further includes:
a step in which the computer determines whether or not the interaction partner's response to the second utterance affirms that the interaction partner is meeting the robot for the first time;
In response to the negative response of the dialogue partner in the determining step, the dialogue partner performs the identification process with a third attitude prepared in advance so as to appear even less confident than the second attitude. a computer controlling the robot to make a fourth utterance regarding whether or not the person is the person identified by the computer;
In response to the affirmative response of the dialogue partner to the fourth utterance, the computer classifies the dialogue partner as an acquaintance of the robot, and the computer classifies the dialogue partner as an acquaintance of the robot, and makes a fourth utterance prepared in advance so that the robot appears relieved. controlling the robot to initiate a dialogue with an attitude of
In response to the negative response of the dialogue partner to the fourth utterance, the computer displays a fifth attitude prepared in advance such as looking disappointed and performs an additional identification process. 5. A method according to any one of claims 2 to 4, comprising the step of controlling a robot. The additional identification process includes:
a computer controlling the robot to utter a question to the conversation partner asking for his or her name;
A computer generates a determination result by determining whether the name included in the response of the dialogue partner to the question asking for the name matches the name of the person registered in a person information database prepared in advance. the step of
In response to the affirmative determination result, the computer classifies the conversation partner as an acquaintance of the robot, and engages in a conversation with the acquaintance while displaying a fifth attitude prepared in advance so as to appear happy. controlling the robot to initiate a dialogue according to a scenario for;
In response to the determination result being negative, the computer classifies the conversation partner as a person unknown to the robot, and conducts a conversation with the conversation partner according to a scenario prepared in advance as a conversation with an unknown person. 6. The method of claim 5, wherein initiating includes controlling the robot. The additional identification process includes:
a computer controlling the robot to utter a question to the conversation partner asking for his or her name;
A computer generates a determination result by determining whether the name included in the response of the dialogue partner to the question asking for the name matches the name of the person registered in a person information database prepared in advance. the step of
In response to the affirmative determination result, the computer performs processing to confirm whether or not the conversation partner is the same person as the person registered in the person information database, and according to the confirmation result, a step of classifying a conversation partner into an acquaintance or an unknown person for the robot;
A dialogue according to a scenario for dialogue with an acquaintance, in which the computer displays a fifth attitude prepared in advance so as to appear happy in response to the robot's classification of the dialogue partner as an acquaintance of the robot. controlling the robot to initiate
In response to the determination result being negative or the conversation partner being classified as an unknown person to the robot, the computer selects the conversation partner according to a scenario prepared in advance as a conversation with an unknown person. 6. The method of claim 5, comprising: controlling the robot to initiate an interaction with the robot. The step of performing the return further includes:
a step in which the computer controls the robot to make a fifth utterance for identifying the conversation partner in response to the conversation partner's response to the fourth utterance being affirmative;
a step in which the computer generates a determination result regarding whether or not information identifying the conversation partner included in the conversation partner's response to the fifth utterance matches the result of the identification process;
In response to the affirmative determination result, the computer controls the robot to make a sixth utterance for confirming that the conversation partner corresponds to an acquaintance of the robot;
In response to the affirmative response of the conversation partner to the sixth utterance, the computer classifies the conversation partner as an acquaintance of the robot, and has a fifth attitude prepared in advance such that the robot appears happy. 6. The method of claim 5, comprising: controlling the robot to initiate a scenario-based interaction with an acquaintance while indicating the interaction with an acquaintance. In the step of performing the return, the computer classifies the conversation partner as a person unknown to the robot in response to the negative determination result, and creates a scenario prepared in advance as a conversation with an unknown person. 9. The method of claim 8, comprising controlling the robot to initiate a dialogue with the interaction partner according to the method. In the step of performing the return, the computer classifies the dialogue partner as an unknown person to the robot in response to the negative response of the dialogue partner to the sixth utterance, and classifies the dialogue partner as an unknown person. 10. The method according to claim 8, further comprising the step of controlling the robot to start a dialogue with the dialogue partner according to a scenario prepared in advance as a dialogue. A computer program for causing a computer to perform the method according to any one of claims 1 to 10.

Images