JP2022045276A - Interactive response device - Google Patents

Abstract

[Problem] To provide an interactive response device that can more smoothly communicate with a person when communicating with the person while displaying an image of a character. [Solution] In a system 1, an interactive response device 10 has a display unit 11 that displays an image of a character, a voice input/output unit 14 that outputs the voice of the character and inputs the voice of the surroundings, a storage unit 16 that stores a plurality of scenarios and a plurality of related information related to the plurality of scenarios, a detection unit 171 that detects a person, a selection unit 172 that selects related information corresponding to the voice input from the voice input/output unit from the plurality of related information and selects a scenario from the plurality of scenarios based on the selected related information, and a response unit 173 that responds by having the character converse with the detected person via the voice input/output unit according to the selected scenario. When the selection unit selects related information, it causes display information about the selected related information to be displayed on the display unit. [Selected Figure] FIG.

Description

The present invention relates to an interactive response device that interacts with a person.

Conventionally, guides have been assigned to transportation facilities such as airports and stations, commercial facilities, entertainment facilities, corporate offices, etc. to provide guidance or reception to persons who have visited the facilities. However, due to the problem of labor shortage of guides, a device for displaying an image of a predetermined character has been developed.

For example, Patent Document 1 discloses a device equipped with a video output device that installs a screen that matches the shape of the upper body of a person and projects an image of a receptionist according to the shape of the screen.

Japanese Unexamined Patent Publication No. 2011-150221

In an interactive response device that interacts with a person while displaying an image of a character, a natural reaction is not performed according to the other party's utterance, such as a dialogue between humans, and the person has an appropriate purpose of his or her utterance. It is difficult to know if it is recognized. Therefore, it is required that the user can smoothly interact with the person by recognizing the intention of the person's utterance.

An object of the present invention is to provide an interactive response device capable of more smoothly interacting with a person when interacting with the person while displaying an image of the character.

In order to solve such a problem, the present invention relates to a display unit for displaying an image of a character, an audio input / output unit for outputting a character's voice and inputting surrounding voice, a plurality of scenarios, and a plurality of scenarios. A storage unit that stores a plurality of related information, a detection unit that detects a person in a predetermined space, and a plurality of related information are selected from the related information corresponding to the voice input from the audio input / output unit. A selection unit that selects a scenario from multiple scenarios based on the selected related information, and a response unit that responds by interacting with a person detected by the character via the audio input / output unit according to the selected scenario. And, when the related information is selected, the selection unit provides an interactive response device for displaying the display information about the selected related information on the display unit.

In this interactive response device, it is preferable that the selection unit selects related information including terms included in the voice input from the voice input / output unit from a plurality of related information.

In this interactive response device, the selection unit displays the character information in which the voice input from the voice input / output unit is converted into text on the display unit, and the display information can be distinguished from other character information in the character information. It is preferable to display it.

In this interactive response device, it is preferable that the selection unit displays the display information on the display unit in association with the head region of the character.

In this interactive response device, the display information is character information indicating the purpose of the voice input from the voice input / output unit, and the selection unit displays the character information obtained by converting the voice input from the voice input / output unit into text. It is preferable that the display information is displayed on the display unit and the character information other than the display information is not displayed on the display unit.

The interactive response device according to the present invention can interact with a person more smoothly when interacting with the person while displaying an image of the character.

It is a figure which shows the whole system configuration of a guidance system 1. It is a schematic diagram which shows the appearance of the response device 10. It is a schematic diagram which shows an example of the data structure of a scenario table 161. It is a flowchart which shows the example of the operation of a display process. (A) and (b) are schematic diagrams showing an example of a character display mode. (A) is a schematic diagram showing an example of a display mode of a character, and (b) is a schematic diagram showing an example of a display mode of display information. (A) and (b) are schematic diagrams showing an example of a display mode of display information.

Hereinafter, the guidance system according to the embodiment will be described with reference to the drawings.

FIG. 1 is a diagram showing an overall system configuration of the guidance system 1 according to the embodiment.
The guidance system 1 is an interactive guidance system. The guidance system 1 guides users by interacting with users (persons) at target facilities such as airports, stations, commercial facilities (department stores, supermarkets, etc.), entertainment venues, and corporate offices. Or, the receptionist will respond. In the present embodiment, as an example of the guidance system, a monitoring system that monitors and guards the target facility will be described, but the guidance system may be a system that simply guides or accepts the user.
As shown in FIG. 1, the guidance system 1 has a plurality of response devices 10 and a management device 20 and the like. Each response device 10 is a monitoring display device that is installed in a different target space, responds to users in each target space, and monitors and guards each target space. The target space is, for example, the entrance of the target facility or each floor. The management device 20 is a device installed at a monitoring table or the like of a security center operated by a security company or a disaster prevention center in a target facility, and collects and manages monitoring results (photographed images and the like) by each response device 10. The response device 10 and the management device 20 are connected to each other via a communication network such as an intranet, the Internet, or a mobile phone network.
The guidance system 1 may have a plurality of management devices 20 for managing a plurality of different target facilities. In that case, the guidance system 1 further has a server device arranged in a monitoring center installed at a position away from each target facility, and the server device collects and manages information managed by each management device 20. You may.

The response device 10 is a display device such as a mirror signage. The response device 10 includes a display unit 11, a sensor 12, an image pickup unit 13, an audio input / output unit 14, a communication unit 15, a storage unit 16, a control unit 17, and the like.

The display unit 11 is a mirror type display or the like, and displays various information such as images and texts according to instructions from the control unit 17. In particular, the display unit 11 displays an image of a character imitating a standing guard according to an instruction from the control unit 17. The display unit 11 may be a non-mirror type display such as a liquid crystal display or an organic EL display. Further, the character displayed on the display unit 11 is not limited to a character imitating a security guard, but may be a character imitating a store clerk or a receptionist. The details of the display unit 11 will be described later.

The sensor 12 is a human body detection sensor that periodically detects a person in the target space and obtains the position of the detected person. As the sensor 12, for example, a laser ranging sensor is used. The laser ranging sensor irradiates a near-infrared pulsed laser with a predetermined wavelength (eg, about 870 nm) in a predetermined angle step (eg, in 0.25 ° increments) horizontally over a preset scanning range (eg, 180 °). Then, the reflected light of the laser is detected. The laser ranging sensor measures the distance to an object that reflects the laser, for example, by the Time-of-Flight method. The sensor 12 scans the entire scanning range at a fixed cycle (for example, 200 msec), measures the distance to the point where the laser is reflected in each direction within the scanning range, and uses the measured distance as the pulse laser. The ranging data associated with the projected direction is output to the control unit 17. The sensor 12 may determine the position of a person by using another known distance measuring method such as a phase difference method or a triangulation method. Further, a plurality of sensors 12 may be provided.

The image pickup unit 13 includes a photoelectric conversion element having sensitivity to visible light such as a CCD element or a C-MOS element, an imaging optical system for forming an image on the photoelectric conversion element, and electricity output from the photoelectric conversion element. It has an A / D converter that amplifies a signal and converts it to analog / digital (A / D). The image pickup unit 13 converts the captured image of each color of RGB into a digital input image in which each pixel has a luminance value in the range of 0 to 255, and outputs the image to the control unit 17. In addition, a plurality of image pickup units 13 may be provided.

The audio input / output unit 14 includes a microphone and an A / D converter that amplifies an electric signal output from a photoelectric conversion element and converts it into analog / digital (A / D). The voice input / output unit 14 inputs surrounding voice, converts the input voice into digital input voice, and outputs the input voice to the control unit 17. The voice input / output unit 14 further has a speaker or the like, and outputs the character's voice according to an instruction from the control unit 17.

The communication unit 15 has, for example, a communication interface circuit compliant with TCP / IP or the like, and connects to a communication network such as an intranet or the Internet. Alternatively, the communication unit 15 has a communication interface circuit conforming to, for example, a W-CDMA system or an LTE system, and connects to a communication network such as a mobile communication network via a base station. The communication unit 15 outputs the data received from the communication network to the control unit 17, and transmits the data input from the control unit 17 to the communication network.

The storage unit 16 includes a semiconductor memory such as a ROM or RAM, an optical disk drive such as a magnetic disk or a CD-ROM or a DVD-ROM, and a recording medium thereof. The storage unit 16 stores a computer program for controlling the response device 10 and various data, and inputs and outputs these information to and from the control unit 17. The computer program may be installed in the storage unit 16 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like.
Further, the storage unit 16 stores as data the installation position of the response device 10 in the target space and the position of a specific area such as an entrance gate or an exclusion zone. Further, the storage unit 16 stores the arrangement position of the sensor 12 in the target space and the irradiation direction of the pulse laser as data. Further, the storage unit 16 stores as data map information indicating the correspondence relationship between each pixel in the image captured by the image pickup unit 13 and each position in the target space. Further, the storage unit 16 stores as data a scenario table 161 that manages a plurality of scenarios for causing the character to interact with a person in the target space. The details of the scenario table 161 will be described later.

The control unit 17 has a processor such as a CPU and an MPU, a memory such as a ROM and a RAM, and peripheral circuits thereof, and executes various signal processing of the response device 10. The control unit 17 includes a detection unit 171, a selection unit 172, a response unit 173, and the like, which are implemented as functional modules of a program that operates on the processor. The selection unit 172 is an example of a recognition unit. The control unit 17 displays and controls the operation of the character to be displayed according to the recognized situation of the target space and the like. Further, when the control unit 17 detects a person who has stopped in the vicinity of the response device 10 (for example, within 1 m), or when it detects an utterance from the person, the control unit 17 outputs voice so as to interact with the person. To control. A DSP, LSI, ASIC, FPGA, or the like may be used as the control unit 17.
The control unit 17 transmits the input image captured by the image pickup unit 13 or the input voice collected by the audio input / output unit 14 to the management device 20 via the communication unit 15. Further, the control unit 17 displays the image of the character on the display unit 11, outputs the voice of the character from the voice input / output unit 14, and voices the voice of the observer received from the management device 20 via the communication unit 15. Output from the input / output unit 14.

The management device 20 is, for example, a personal computer or the like. The management device 20 includes a second display unit 21, a second voice input unit 22, a second voice output unit 23, a second communication unit 23, a second storage unit 24, a second control unit 25, and the like.

The second display unit 21 is a liquid crystal display, an organic EL display, or the like, and displays various information such as images and texts according to instructions from the second control unit 25.
The second audio input / output unit 22 has the same configuration as the audio input / output unit 14, converts the input audio into digital audio data and outputs it to the second control unit 25, and the second control unit 25. Outputs audio according to the instructions from.

The second communication unit 23 has, for example, a communication interface circuit compliant with TCP / IP or the like, and connects to a communication network such as an intranet or the Internet. The second communication unit 23 outputs the data received from the communication network to the second control unit 25, and transmits the data input from the second control unit 25 to the communication network.

The second storage unit 24 has a semiconductor memory, a magnetic disk or an optical disk drive similar to the storage unit 16, and a recording medium thereof. The second storage unit 24 stores a computer program for controlling the management device 20 and various data, and inputs and outputs these information to and from the second control unit 25. The computer program may be installed in the second storage unit 24 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like.

The second control unit 25 has the same processor, memory, and peripheral circuits as the control unit 17, and executes various signal processing of the management device 20. A DSP, LSI, ASIC, FPGA, or the like may be used as the second control unit 25.
The second control unit 25 displays the input image received from the response device 10 via the second communication unit 23 on the second display unit 21, and the input voice received from the response device 10 via the second communication unit 23. Is output from the second audio output unit 23. Further, the second control unit 25 transmits the voice input by the observer from the second voice input unit 22 to the response device 10 via the second communication unit 23.

FIG. 2 is a schematic view showing the appearance of the response device 10. As shown in FIG. 2, the display unit 11 of the response device 10 is supported by the pedestal 18. The response device 10 is arranged so that the display unit 11 faces the target space A.
The display unit 11 is formed by attaching a mirror panel 111 to the display surface of a monitor such as a liquid crystal display or an organic EL display, for example. The mirror panel 111 transmits light (image) from the inside (monitor side) while reflecting light from the outside (target space side). As a result, the display unit 11 reflects and displays the image of the person H coming and going in the target space A, and displays the image of the character C output from the control unit 17.
A sensor 12 is arranged on the outer frame 112 of the mirror panel 111 so as to be able to project a near-infrared pulse laser toward the target space A. Similarly, in the outer frame 112, the imaging unit 13 is arranged so that the target space A can be imaged, and the audio input / output unit 14 can collect the sound of the target space A and output the sound toward the target space A. Is placed in. The sensor 12, the image pickup unit 13, and the audio input / output unit 14 are not limited to being arranged in the outer frame 112, and may be arranged through the mirror panel 111.

FIG. 3 is a schematic diagram showing an example of the data structure of the scenario table 161.
As shown in FIG. 3, in the scenario table 161, the identification number (scenario ID) of each scenario, the scenario, the related information, the response information, and the like are stored in association with each of the plurality of scenarios.

The scenario is information showing the flow of dialogue for the character to interact with the person in the target space, and is used when the control unit 17 causes the character to interact with the person in the target space. As a scenario, for example, a scenario of dialogue regarding facility guidance (location inquiry), agency request, report, daily conversation, etc. is set in advance.

The related information is information related to each scenario, and is used by the control unit 17 to select a scenario. As related information, keywords that are likely to be included in the utterance of a person in the target space, or identification information that can identify each scenario according to the meaning or purpose of the utterance, etc. are set in advance. One or more keywords or identification information is set as the related information of each scenario.
For example, for a dialogue scenario related to facility guidance (location inquiry), terms (toilet, conference room, etc.) that can be the subject of an inquiry are set as keywords. Similarly, terms such as appointments are set as keywords for the scenario of the dialogue regarding the agency request. In addition, terms for asking for help and terms such as suspicious persons are set as keywords for the scenario of dialogue regarding reporting.
On the other hand, as the identification information, a discriminator pre-learned using a plurality of sample voices including various types of utterances is set by a learning device using a machine learning technique such as a neural network. When each sample voice is input, the classifier has the certainty (probability) that the sample voice corresponds to each scenario, the keyword (toilet, conference room), the meaning or purpose (toilet, conference room) indicated by the sample voice (toilet, conference room). Pre-learned to output the location of the toilet, visit by appointment).

The response information is information indicating the voice to be output to the character in each scenario. It should be noted that a plurality of response information subdivided according to the utterance content of the person may be set for one scenario. In that case, each response information is set in association with the input voice information.

FIG. 4 is a flowchart showing an example of the operation of the display processing by the response device 10. This flowchart is executed mainly by the control unit 17 in cooperation with each element of the response device 10 based on the program stored in the storage unit 16 in advance. While the display process is being executed, the control unit 17 continues to display the image of the character on the display unit 11. In particular, when the control unit 17 moves the line-of-sight direction and the body direction of the character C within a predetermined range (for example, 45 degrees each with respect to the front) when the user is not detected, the control unit 17 detects the user. The image display of the character C is controlled so as to execute a normal operation toward the user who has detected the line of sight of the character C. As a result, it is possible to display an image that seems to be a security guard who performs real guards, and the presence as a security guard is demonstrated.

FIG. 5A is a schematic diagram showing an example of a display mode of a character performing a normal operation.
In the example shown in FIG. 5A, the selection unit 172 changes the operation of the character C so that the line of sight of the character C is slightly tilted toward the person H in the target space. The response device 10 divides an image of one character C into predetermined parts such as eyes, nose, mouth, facial contour, neck, body, arms, and legs, and assigns polygons to each part. .. Further, the response device 10 sets the position and orientation of the polygon of each part corresponding to each line-of-sight direction in the storage unit 16 in advance for each of the plurality of line-of-sight directions. The selection unit 172 updates the image display of the character C by moving the polygon so that the character C faces a predetermined line-of-sight direction and moving each part according to the polygon. In the normal operation, the selection unit 172 does not change the line of sight of the character C, but changes the operation of the character C by moving other arbitrary parts such as causing the character C to strike an aizuchi at regular intervals. May be good. Further, the selection unit 172 may stop the character C without changing the operation of the character C in the normal operation.

Further, when the control unit 17 detects the user or the utterance from the user in the vicinity of the response device 10 (for example, within 1 m), the control unit 17 shifts from the normal operation to the response operation, and the character C shifts to the use. By controlling the voice output so as to interact with the person, the response operation such as guidance and reception to the user of the facility is executed.

First, the detection unit 171 causes the image pickup unit 13 to image the target space, and acquires an input image obtained by capturing the target space from the image pickup unit 13 (step S101).

Next, the detection unit 171 executes a detection process for detecting a person in the target space (step S102). The detection unit 171 detects a person who has stopped, for example, in the vicinity of the response device 10 (for example, within 1 m).
The detection unit 171 detects a person from the input image. The detection unit 171 uses, for example, inter-frame subtraction processing or background subtraction processing to extract a change region having a temporal change in the luminance value in a plurality of input images sequentially captured. The detection unit 171 extracts a change area considered to be a person as a person area based on feature quantities such as the size and shape of each extracted change area. The detection unit 171 extracts edge pixels for the extracted person area by using a Sobel filter or the like so that the inclination direction of the luminance change can be known. The detection unit 171 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels by using, for example, a generalized Hough transform, and is surrounded by the edge distribution. The area is extracted as a face area. The detection unit 171 may extract the face region by using another known method such as an Adaboost classifier. When the face area is extracted from the input image, the detection unit 171 detects a person included in the person area corresponding to the face area as a person in the target space. Then, based on the map information, the detection unit 171 identifies a position in the target space corresponding to the pixel at the lowermost end of the person area in which the feet of the detected person are captured as the person position. When a plurality of face areas are extracted, the detection unit 171 detects a person for each of the extracted face areas and identifies the position of the person.

The detection unit 171 may acquire distance measurement data from the sensor 12 and detect a person from the acquired distance measurement data. The detection unit 171 compares the acquired distance measurement data with a preset reference measurement data, extracts a direction (scanning direction) in which a distance change of a predetermined value or more occurs, and has a continuity of a predetermined value or more (for example, general). A distance change direction group having a person's size) is detected as a person in the target space. The detection unit 171 is a target based on the arrangement position of the sensor 12 in the target space, the irradiation direction of the pulse laser, and the representative distance measurement data (for example, the scanning direction and the measurement distance at the center of the distance change direction group) related to the detected person. Identify the position of a person in space. When a plurality of distance change direction groups are extracted, the detection unit 171 determines whether or not the person is a person for each distance change direction group, and specifies the position of the person.
Further, the detection unit 171 may acquire an input voice from the voice input / output unit 14 and detect a person from the acquired input voice. The detection unit 171 determines that a person exists in the target space when the sound pressure of the input voice is equal to or higher than a preset threshold value. The detection unit 171 may determine that a person exists in the target space when the frequency of the input voice is within a preset range.

Next, the detection unit 171 determines whether or not a person has been detected in the vicinity of the response device 10 in the detection process (step S103). When a person is not detected in the vicinity of the response device 10 in the detection process, the detection unit 171 returns the process to step S101, and the character C continues the normal operation.

On the other hand, when a person is detected in the vicinity of the response device 10 in the detection process, the selection unit 172 executes a recognition process for recognizing the input voice input from the voice input / output unit 14, and the utterance content by the person in the recognition process. It is determined whether or not the voice related to the above is recognized (step S104). The selection unit 172 applies known voice recognition technology and natural language processing technology to the input voice, extracts the character information corresponding to the input voice, that is, the character information obtained by converting the input voice into text, and detects the utterance content by the person. do. If the voice related to the utterance content is not recognized, the selection unit 172 returns the process to step S101.

On the other hand, when the voice related to the utterance content is recognized, the selection unit 172 controls the image display of the character so as to execute the response operation (step S105). That is, the selection unit 172 controls the image display of the character so as to execute the response operation at the timing when the voice related to the utterance content is recognized. However, when the selection unit 172 controls the image display of the character to indicate that the detected person has already obtained the specific recognition information in step S107 described later, the process of step S105 is performed. Is omitted.

Further, the selection unit 172 receives character information corresponding to the input voice when the voice related to the utterance content is recognized, that is, character information corresponding to all utterances by a person in the target space (character information obtained by converting all utterances into text). ) May be displayed on the display unit 11. Further, the selection unit 172 may control the image display of the character so as to execute the response operation when the person is detected, not when the voice related to the utterance content is recognized.

Next, the selection unit 172 executes the first selection process of selecting (identifying) the related information corresponding to the input voice from the plurality of related information stored in the scenario table, and the related information in the first selection process. Is determined (step S106).
In the first selection process, the selection unit 172 determines whether or not each keyword set as related information is included in the character information extracted from the input voice. When the character information includes a keyword, the selection unit 172 selects the keyword as the related information corresponding to the input voice, and displays the keyword as the related information (an image diagram of the character information indicating the related information, an icon, etc.). ). That is, the selection unit 172 selects related information including terms included in the input voice as the related information corresponding to the input voice. As a result, the selection unit 172 can accurately select a scenario suitable for the input voice. On the other hand, when the character information extracted from the input voice does not include the keyword, the selection unit 172 does not select the related information corresponding to the input voice. In determining whether or not a keyword is included, morphological analysis of the recognized voice may be performed and matching of the part of speech with the keyword may be considered.
Further, the selection unit 172 inputs the input voice to each classifier set as the related information, and acquires the accuracy and the keyword, the meaning or the purpose output from each classifier. When the accuracy acquired from any of the classifiers is equal to or higher than a preset threshold value, the selection unit 172 selects the classifier with the highest accuracy as the related information corresponding to the input voice and acquires it from the classifier. Select a keyword, meaning or purpose as display information for the related information. As a result, the selection unit 172 can flexibly select a scenario suitable for the input voice. On the other hand, when the accuracy acquired from any of the classifiers is less than the threshold value, the selection unit 172 does not select the related information corresponding to the input voice.
If the related information is not selected in the first selection process, the selection unit 172 returns the process to step S101.

On the other hand, when the related information is selected in the first selection process, the selection unit 172 acquires the specific recognition information by selecting (identifying) the related information from the input voice recognition result. That is, the selection unit 172 acquires the related information as specific recognition information. In that case, the selection unit 172 controls the image display of the character so as to indicate to the person to be interacted with that the specific recognition information has been obtained (step S107). That is, the selection unit 172 controls the image display of the character so as to indicate to the dialogue target person that the specific recognition information has been obtained at the timing when the specific recognition information is obtained from the recognition result of the input voice. ..

FIG. 5B is a schematic diagram showing an example of a character display mode when specific recognition information is acquired.
As shown in FIG. 5B, the selection unit 172 controls the image display of the character C so as to indicate to the person to be interacted with that the specific recognition information has been acquired by changing the operation of the character C. .. The selection unit 172 normally controls the image display of the character C so as to execute a first reaction operation (for example, nodding) that reacts to the user's utterance at a predetermined timing when the user's utterance is recognized (FIG. 5 (FIG. 5). a) See). On the other hand, the selection unit 172 controls the image display of the character C so as to execute the second reaction operation (for example, nodding greatly) having a larger operation range than the first reaction operation when the specific recognition information is acquired. That is, the character C so that the selection unit 172 executes the first reaction operation that reacts to the voice at the timing of recognizing the voice related to the utterance content, and executes the second reaction operation at the timing when the specific recognition information is obtained. Control the image display of.

FIG. 6A is a schematic diagram showing another example of the display mode of the character.
As shown in FIG. 6A, the selection unit 172 controls the image display of the character C so as to indicate to the dialogue target person that the specific recognition information has been acquired by changing the facial expression of the character C. .. In the example shown in FIG. 6A, the selection unit 172 changes the facial expression of the character C so that the character C smiles. The response device 10 stores, for example, a face image of the character C in the storage unit 16 in advance for each of a plurality of facial expressions. The selection unit 172 changes the facial expression of the character C by replacing the face image of the character C displayed on the display unit 11 with another face image stored in the storage unit 16. The selection unit 172 may change the facial expression of the character C to another facial expression such as a serious facial expression instead of changing the facial expression of the character C so that the character C smiles.

As a result, the person in the target space can grasp that the response device 10 has some recognition for his / her utterance and can obtain a sense of security, so that the response device 10 improves the user experience. be able to.

Further, the selection unit 172 causes the display unit 11 to display the display information about the selected related information (step S108). The selection unit 172 causes the display unit 11 to display the display information separately from the character C.

FIG. 6B is a schematic diagram showing an example of a display mode of display information.
As shown in FIG. 6B, the selection unit 172 causes the display unit 11 to display the character information T1 in which the input voice is converted into text. The character information T1 is character information obtained by converting all utterances by a person in the target space into text. Then, the selection unit 172 highlights the selected display information T2 in the character information T1 and displays the selected display information T2 in the character information T1 so as to be distinguishable from other character information. In the example shown in FIG. 6B, the selection unit 172 underlines the display information T2 so that the display information T2 can be distinguished from other character information. The selection unit 172 displays the display information T2 by highlighting the display information T2 or by making the color, thickness, font, etc. of the display information T2 different from the color, thickness, font, etc. of other character information. Information T2 may be displayed so as to be distinguishable from other character information. As a result, the person in the target space can grasp which term in the utterance he / she has uttered as a keyword, and whether the response device 10 correctly recognizes the intention of his / her utterance. You can judge whether or not. Therefore, the person in the target space can get a sense of security, and the response device 10 can improve the user experience. Further, if the person in the target space recognizes a term that he / she does not intend as a keyword, the person in the target space can immediately correct the term, and as a result, the response device 10 takes the time required for the response. It can be shortened.

FIG. 7A is a schematic diagram showing another example of the display mode of the display information.
As shown in FIG. 7A, the selection unit 172 causes the display unit 11 to display the display information T3 in association with the head region of the character C. In the example shown in FIG. 7A, the selection unit 172 displays the display information T3 in association with the head region of the character C by displaying the display information T3 from the head region of the character C by a balloon. The selection unit 172 may display the display information T3 in association with the character C by displaying the display information T3 in the vicinity of the head region of the character C without using a balloon. Further, in the example shown in FIG. 7A, the character information T4 in which the input voice is converted into text is also displayed on the display unit 11. However, when the display information T3 is displayed in association with the character C, the character information T4 obtained by converting the input voice into text may not be displayed. As a result, the person in the target space can easily visually grasp the purpose of the utterance recognized by the response device 10 (character), so that the response device 10 can improve the user experience. Further, if the person in the target space recognizes the purpose of the utterance in the direction not intended by him / her, he / she can correct it immediately. As a result, the person in the response device 10 needs to respond. The time can be shortened.

FIG. 7B is a schematic diagram showing still another example of the display mode of the display information.
As shown in FIG. 7B, the display information T5 is character information indicating the purpose of the input voice. The selection unit 172 displays the display information T5 on the display unit 11 and does not display the character information other than the display information T5 on the display unit 11 when displaying the character information obtained by converting the input voice into text on the display unit 11. As a result, the person in the target space can pay attention to the purpose of the utterance recognized by the response device 10, so that the response device 10 can improve the user experience. Further, if the person in the target space recognizes the purpose of the utterance in the direction not intended by him / her, he / she can correct it immediately. As a result, the person in the response device 10 needs to respond. The time can be shortened.

Further, as shown in FIGS. 6 (b), 7 (a), and (b), the selection unit 172 displays an image of the character C so as to indicate to the person to be interacted with that the specific recognition information has been obtained. While controlling, the selected (specified) display information is displayed on the display unit 11. As a result, the person in the target space can grasp the purpose of the utterance recognized by the response device 10 while grasping that the response device 10 recognizes his / her own utterance. Therefore, the response device 10 can be used by the user. You can improve the experience.
The process of step S107 is omitted, and the selection unit 172 displays the display information on the display unit 11 without controlling the image display of the character C so as to indicate to the person to be interacted with that the specific recognition information has been obtained. It may be displayed. Further, the process of step S108 is omitted, and the selection unit 172 displays the image of the character C so as to show the person to be interacted with that the specific recognition information has been obtained without displaying the display information on the display unit 11. You may control it.

Next, has the selection unit 172 executed a second selection process of selecting a scenario from a plurality of scenarios stored in the scenario table based on the selected related information, and selected the scenario in the second selection process? It is determined whether or not (step S109).
In the second selection process, the selection unit 172 counts the number of keywords selected in the first selection process for each scenario stored in the scenario table. When the number counted for any scenario is a predetermined number (for example, 2) or more, the selection unit 172 selects the scenario with the largest number counted. On the other hand, the selection unit 172 does not select a scenario when the number counted for any scenario is less than a predetermined number.
You may select a scenario by setting conditions (AND condition, NOT condition, etc.) related to the combination of keywords for each scenario. Taking the scenario ID S01 as an example, the selection unit 172 recognizes "information asking for a place (" place "," where "," want to go ", etc.)" (AND condition) in addition to the "toilet". And scenario S01 are selected, while the selection unit 172 does not select scenario S01 when "clogged" (NOT condition) is recognized in addition to "toilet". In this way, by considering the combination with the keyword "toilet", it is possible to appropriately select a scenario according to the content of the user's utterance.
Further, the selection unit 172 calculates the total value or the average value of the accuracy acquired from the classifier selected in the first selection process for each scenario stored in the scenario table. When the total value or the average value calculated for any of the scenarios is equal to or larger than a predetermined value, the selection unit 172 selects the scenario having the largest calculated total value or the average value. On the other hand, the selection unit 172 does not select a scenario when the total value or the average value calculated for any scenario is less than a predetermined value.
If the scenario is not selected in the second selection process, the selection unit 172 returns the process to step S101.

On the other hand, when a scenario is selected in the second selection process, the response unit 173 responds to the person detected by the detection unit 171 according to the selected scenario (step S110), and returns the process to step S101. The response unit 173 responds to the person by having the character interact with the detected person via the voice input / output unit 14. The response unit 173 causes the character to interact with a person in the target space by outputting the voice of the character from the voice input / output unit 14 according to the response information stored in association with the selected scenario in the scenario table.

As described above, when the related information used for selecting the dialogue scenario is selected, the response device 10 according to the present invention displays the display information about the selected related information. As a result, the person in the target space can grasp the purpose of the utterance recognized by the response device 10, recognize that his / her intention is appropriately transmitted, and can obtain a sense of security. Therefore, the response device 10 can interact with the person more smoothly when interacting with the person while displaying the image of the character.

Further, the response device 10 according to the present invention displays a character image so as to indicate to the person to be interacted with that the specific recognition information has been obtained at the timing when the specific recognition information is obtained for the input voice. Control. As a result, the person in the target space can grasp that the response device 10 has some recognition for his / her utterance, and can obtain a sense of security. Therefore, the response device 10 can interact with the person more smoothly when interacting with the person while displaying the image of the character.

Conventionally, the interactive information processing device has only communicated with the user at regular intervals to the extent that the information processing device strikes an aizuchi. Therefore, the user may not be able to grasp whether the information processing device correctly recognizes his / her utterance in the dialogue with the information processing device, and may feel anxious whether he / she can obtain an appropriate answer to what he / she is looking for. there were. On the other hand, the response device 10 conveys the recognition and / or the recognized content to the user at the timing when the specific recognition information is obtained, so that the user can be relieved and the user experience can be improved. Is possible.

In the above, an example of displaying the character information T4 corresponding to the input voice on the display unit 11 is shown, but it may not be preferable to display the character information T4 depending on the utterance content of the user. For example, violent content, discriminatory content, privacy-related content, etc. Therefore, the utterance content (content unsuitable for display) that is not preferable to be displayed is stored in the storage unit in advance, and when the character information corresponding to the input voice is unsuitable for display, the selection unit 172 hides the relevant part. Make it invisible by making it equal to. For example, if the user recognizes that "I'm Sato who came with a promise of a meeting", and if "personal name" is registered as information unsuitable for display as privacy information, "I came with a promise of a meeting **". Is displayed. By this non-display processing of unsuitable content, it is possible to prevent not only the user's utterance content from being unfavorable for display but also the display of inappropriate content due to an error in voice recognition.

In the above, the recognition result is displayed in the dialogue with the user to indicate that the user recognizes the intention of the utterance, but the voice is heard at the timing when the user approaches the response device 10. You may indicate to the user that the input is being accepted. For example, when the normal operation is switched to the response operation, the selection unit 172 displays on the display unit 11 an icon indicating the surrounding environmental sound, a voice sound collection state, a schematic waveform indicating the sound collection level, and the like. Alternatively, the icon related to voice recognition may be always displayed, and the display of the icon may be changed so as to indicate that the voice input is being accepted when the response operation is switched. As a result, the user can grasp that he / she has been recognized, and can also grasp that the character C is accepting voice input, so that he / she can speak naturally.

10 Response device, 11 Display unit, 14 Audio input / output unit, 16 Storage unit, 171 detection unit, 172 selection unit, 173 Response unit

Claims (5)

A display unit that displays a character image and
An audio input / output unit that outputs the character's voice and inputs surrounding voice, and
A storage unit that stores a plurality of scenarios and a plurality of related information related to the plurality of scenarios.
A detector that detects a person in a predetermined space,
A selection unit that selects related information corresponding to the voice input from the voice input / output unit from the plurality of related information, and selects a scenario from the plurality of scenarios based on the selected related information. ,
It has a response unit that responds by having the character interact with the detected person via the audio input / output unit according to the selected scenario.
When the related information is selected, the selection unit causes the display unit to display display information about the selected related information.
An interactive response device characterized by this. The interactive response device according to claim 1, wherein the selection unit selects related information including terms included in the voice input from the voice input / output unit from the plurality of related information. The selection unit displays character information obtained by converting the voice input from the voice input / output unit into text on the display unit, and displays the display information in the character information so as to be distinguishable from other character information. , The interactive response device according to claim 1 or 2. The interactive response device according to claim 1 or 2, wherein the selection unit associates the display information with the head region of the character and displays the display information on the display unit. The display information is character information indicating the purpose of the voice input from the voice input / output unit.
The selection unit displays the display information on the display unit and displays character information other than the display information on the display unit when displaying the character information obtained by converting the voice input from the audio input / output unit into text on the display unit. The interactive response device according to claim 1 or 2, which is not displayed on the display unit.

Images