JP2015173857A - Electronic apparatus and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic apparatus capable of achieving pseudo communication with persons existed in the past and animals.SOLUTION: The electronic apparatus includes storage means, speech recognition means, and speech synthesis means. The storage means stores information related to a person existed in the past or an animal. The speech recognition means generates a sentence for responding to the content of an utterance of a user. The speech synthesis means synthesizes a speech of the person or the animal on the basis of the information stored in the storage means and the generated sentence.

Description

  Embodiments described herein relate generally to an electronic device that processes sound and images, and an information processing method applied to the device.

  Festival utensils such as Buddhist altars are purchased, for example, when a family dies, and are used to provide for the family where the remaining family has died. In recent years, Buddhist altars for dead pets have become popular. Such Buddhist altars have also been proposed that can display deceased information such as ritual names and relics.

Japanese Patent No. 4682265

  However, in the Buddhist altar, even if the remaining family was able to make a visit to feed the family who died towards the Buddhist altar or the dead pet, the remaining family (especially the elderly living alone) It is difficult to reduce loneliness and sadness. Therefore, in order to reduce the loneliness and sadness of the remaining family, it may be expected to realize pseudo communication with people and animals (pets) that existed in the past.

  An object of this invention is to provide the electronic device and information processing method which can implement | achieve pseudo | simulation communication with the person and animal which existed in the past.

  According to the embodiment, the electronic device includes a storage unit, a voice recognition unit, and a voice synthesis unit. The storage means stores information on a person or animal that has existed in the past. The voice recognition means generates a sentence for responding to the user's utterance content. The voice synthesizing unit synthesizes the human or animal voice based on the information stored in the storage unit and the generated sentence.

The figure which shows the example of the dialogue system provided with the electronic device which concerns on embodiment. The figure which shows another example of the dialogue system provided with the electronic device of the embodiment. The figure for demonstrating the example by which a pseudo dialogue is provided to a user using the electronic device of the embodiment. 2 is an exemplary block diagram showing an example of the system configuration of the electronic apparatus of the embodiment. FIG. 2 is an exemplary block diagram showing an example of the functional configuration of a dialog application program executed by the electronic apparatus of the embodiment. FIG. 6 is an exemplary flowchart illustrating an example of the procedure of a dialogue control process which is executed by the electronic apparatus of the embodiment. The perspective view which shows the example of the external appearance of the electronic device of the embodiment.

  Hereinafter, embodiments will be described with reference to the drawings.

  First, with reference to FIG. 1, the outline | summary of the dialogue system with the person (deceased) who existed in the past provided with the electronic device which concerns on one Embodiment is demonstrated. This electronic device can be realized as, for example, a tablet computer, a notebook personal computer, a smartphone, a PDA, a digital television receiver, or an embedded system incorporated in various electronic devices. Below, the case where this electronic device is implement | achieved as the tablet computer 2 is assumed.

  The tablet computer 2 is incorporated in a festival tool for feeding a deceased person, such as the Buddhist altar 1 or a tombstone. The tablet computer 2 may be fixed to the Buddhist altar 1 or may be detachably installed on the Buddhist altar 1. That is, the tablet computer 2 may be provided (sold) in a state of being incorporated in the Buddhist altar 1, or may be installed as an option on the conventional Buddhist altar 1 (for example, an altar already in the home).

  The tablet computer 2 is supplied with power from an AC external power supply or a battery. A cradle (charger) for supplying power from the AC external power source to the tablet computer 2 and charging the battery may be provided at the position where the tablet computer 2 is installed in the Buddhist altar 1.

  The tablet computer 2 includes a camera 16, a touch screen display 17, speakers 18A and 18B, and microphones 19A and 19B. The camera 16 acquires an image (face image) of a person visiting the Buddhist altar 1, that is, an image of a person in front of the camera 16. The microphones 19 </ b> A and 19 </ b> B acquire sound generated by the person's speech. The tablet computer 2 synthesized the face image of the deceased person for the artificial (pseudo) dialogue with the deceased person based on the acquired image and sound of the visiting person. Synthesize human speech. Then, the tablet computer 2 displays the face image on the screen of the touch screen display 17 using the synthesized face image and the synthesized voice, and outputs the voice from the speakers 18A and 18B. The screen of the touch screen display 17 may display not only an image of a deceased person but also an image of Buddhist tools such as a pot, a flower stand, a fire stand, an incense burner, an offering, a flower Buddha water device, and a phosphorus.

  As a result, the user can achieve pseudo communication with the deceased person by the Buddhist altar 1 in which the dialogue techniques are integrated, so that sadness and loneliness can be reduced.

  Note that the process of synthesizing the face image of the deceased person and the process of synthesizing the sound of the deceased person based on the image and sound of the visiting person are executed by the dialogue engine provided in the cloud server 3. Also good. In this case, the tablet computer 2 transmits the acquired image and audio data of the visiting person to the cloud server 3 via the network. The tablet computer 2 receives the face image and audio data of the deceased person synthesized by the cloud server 3 and displays the face image of the deceased person on the screen of the touch screen display 17 using the data. Then, the voice of the person who died is output from the speakers 18A and 18B.

  In addition, as shown in FIG. 2, the altar 1 may be an altar for feeding dead animals. The tablet computer 2 uses the acquired image and sound of the visiting person to generate an image (face image or whole body image) of the animal for artificial (pseudo) communication with the dead animal. Synthesize and synthesize the animal's voice. Then, the tablet computer 2 displays the image on the screen of the touch screen display 17 using the synthesized image and the synthesized voice, and outputs the voice from the speakers 18A and 18B.

  Next, an example of a trigger for starting a pseudo dialogue (communication) between a user (person who visits) and a person or animal that has existed in the past will be described with reference to FIG.

  For example, the tablet computer 2 starts a dialogue when a face image of the user is detected from an image acquired by the camera module 16. Moreover, the tablet computer 2 starts a dialogue when, for example, an utterance by the user is detected from voices acquired by the microphones 19A and 19B. Note that the detected utterance may be an utterance including a predetermined word such as a call to the name of a dead person or a dead animal.

  Furthermore, when the sound of “Rin” 5 arranged in the vicinity of the Buddhist altar 1 is detected from the sound acquired by the microphones 19A and 19B, the tablet computer 2 is connected with a dead person or a dead animal. A dialogue may be started. Further, a communication connection (wired or wireless) such as Bluetooth (registered trademark) is established between “Rin” 5 having a communication function and the tablet computer 2, and the user responds to an operation using “Rin” 5. Thus, a signal requesting the start of the dialogue may be transmitted to the tablet computer 2.

FIG. 4 is a diagram showing a system configuration of the tablet computer 2.
As shown in FIG. 4, the tablet computer 2 includes a CPU 100, a system controller 101, a touch controller 102, a memory 103, a display controller 104, a BIOS-ROM 105, an embedded multimedia card (eMMC) 106, and a power management IC (PMIC) 108. , Audio Codec 110, 3G / LTE module 111, LAN module 112, BT module 113, card controller 114, USB connector 115, camera module 16, and the like.

  The CPU 100 is a processor that controls operations of various components in the tablet computer 2. The CPU 100 executes various software loaded into the memory 103 from the eMMC 106 that is a storage device. The eMMC 106 is an external storage device for an embedded device using a flash memory, and is connected to the outside through the same interface as the multimedia card (MMC). The software includes an operating system (OS) 201 and various application programs. The application program includes a dialog application program 202. For example, the interactive application program 202 has a function of providing a user who uses the tablet computer 2 (for example, a person who visits the Buddhist altar 1) with a pseudo dialog with a person or animal that has existed in the past.

  The CPU 100 also executes a basic input / output system (BIOS) stored in the BIOS-ROM 105. The BIOS is a program for hardware control.

  The system controller 101 is a device that connects between the local bus of the CPU 100 and various components. The system controller 101 also includes a memory controller that controls access to the memory 103. The system controller 101 also has a function of executing communication with the display controller 104 via a PCI EXPRESS serial bus or the like.

  The display controller 104 is a display controller that controls the LCD 17 </ b> A used as a display monitor of the tablet computer 2. A display signal generated by the display controller 104 is sent to the LCD 17A. The LCD 17A displays a screen image based on the display signal. A touch panel 17B is disposed on the LCD 17A. The touch controller 102 outputs an operation signal indicating an operation position on the touch panel 17B to the system controller 101 in response to an operation using a finger or the like on the touch panel 17B.

  Furthermore, the system controller 101 also has a function of executing communication with the Audio Codec 110. The Audio Codec 110 generates audio data in a predetermined format by encoding audio detected by the microphones 19A and 19B (for example, audio generated by the user). Also, the Audio Codec 110 decodes audio data to be reproduced and outputs the decoded sound from the speakers 18A and 18B. The system controller 101 also incorporates a USB controller for executing communication with an external device via a USB 2.0 standard cable connected to the USB connector 115.

  The PMIC 108 is a one-chip microcomputer for managing power supplied from the AC or the battery 108A. The PMIC 108 has a function of powering on or powering off the tablet computer 2 in accordance with the operation of the power button by the user.

  The camera module 16 captures an image and outputs data of the captured image to the system controller 101. The camera module 16 can also capture a plurality of continuous images such as a moving image (video).

  The 3G / LTE module 111 is a device configured to execute 3G mobile communication or LTE communication via the antenna 111A. The LAN module 112 is a communication device that performs, for example, wireless communication of the IEEE 802.11g standard or wired communication of the IEEE 802.3 standard via the antenna 112A. The Bluetooth module 110 is a communication device that performs Bluetooth communication with an external device via the antenna 113A.

  The card controller 114 writes and reads data to and from a memory card such as an SD card inserted into a card slot provided in the computer main body 11. The human sensor 116 is, for example, a sensor that detects whether a person is present in front of the screen of the tablet computer 2.

  Next, the functional configuration of the interactive application program 202 executed by the tablet computer 2 will be described with reference to FIG. As described above, the dialogue application program 202 has a function of providing, for example, a pseudo dialogue with a person or animal that has existed in the past to a user who uses the tablet computer 2 (for example, a person visiting the Buddhist altar 1), Etc. The dialogue application program 202 includes a voice recognition unit 302, an intention understanding unit 303, a face / person recognition unit 305, a facial expression recognition unit 306, a dialogue control unit 307, a voice synthesis unit 308, a face generation unit 310, a facial expression synthesis unit 311, and the like. Prepare.

  First, the voice recognition unit 302 and the face / person recognition unit 305 determine whether a dialogue with a person or animal that has existed in the past has been requested. For example, when a predetermined voice is detected from the voices generated by the microphones 19A and 19B, the voice recognition unit 302 determines that a dialogue with a person who has existed in the past has been requested. For example, when a person is detected from an image captured by the camera module 16, the face / person recognition unit 305 determines that an interaction with a person who has existed in the past is requested. When a person is detected by the human sensor 116, it may be determined that an interaction with a person who has existed in the past is requested.

  When dialogue with a person who has existed in the past is requested, voice (voice data) from the user's utterance is inputted by voice input 301 using the microphones 19A and 19B, and by image input 304 using the camera module 16. The user's image (image data) is input. The voice recognition unit 302, the intention understanding unit 303, and the dialogue control unit 307 generate a sentence for responding to the user's utterance content. The speech synthesizer 308 uses the information about the person or animal that died (that is, the person or animal that existed in the past) stored in the databases 404 to 414 and the sentence or the person who died based on the generated sentence. To synthesize the voice. Further, the face generation unit 310 and the facial expression synthesis unit 311 are based on information about the person or animal that has died stored in the databases 404 to 414 and the generated sentence (face of the person or animal that has died). Image).

  The voice recognition unit 302 generates a text corresponding to the voice by performing voice recognition processing on the voice generated by the user using the voice recognition dictionary 401. The speech recognition dictionary 401 includes, for example, information indicating the correspondence between phonemes, which are the smallest units of words, and feature quantities, and information indicating the correspondence between phoneme strings and words.

  More specifically, first, the speech recognition unit 302 removes noise from speech and calculates a feature amount. The speech recognition unit 302 uses the speech recognition dictionary 401 to generate a phoneme sequence corresponding to speech based on the calculated feature amount. Then, the voice recognition unit 302 uses the voice recognition dictionary 401 to determine a word corresponding to the converted phoneme string. Note that the speech recognition unit 302 determines a plurality of word candidates corresponding to the phoneme string, and determines a word string (text) corresponding to the phoneme string based on the likelihood of the connection of consecutive words. Also good.

  Next, the intention understanding unit 303 uses the knowledge database 402 and the intention understanding database 403 to detect the intention of the generated text (word string). The knowledge database 402 includes information indicating knowledge such as contents and features corresponding to words. The intention understanding database 403 includes information for determining the contents (intentions) of text (word strings) from information indicating knowledge corresponding to words. For example, the intent understanding unit 303 detects that the generated text is a greeting word or a call to the names of people or animals that existed in the past.

  In addition, the face / person recognition unit 305 uses the face / expression database 404 to perform face recognition processing on the input user image, thereby detecting a face image region from the image and corresponding to the face image. Determine the person. Then, the facial expression recognition unit 306 determines the facial expression corresponding to the facial image by performing facial expression recognition processing on the facial image of the user using the facial / expression database 404. In the face / expression database 404, for example, information indicating a face image corresponding to one or more persons using the tablet computer 2 and information for identifying an expression are stored. These pieces of information are stored not as the image itself but as, for example, feature amounts.

  More specifically, the face / person recognition unit 302 calculates a feature amount for recognizing a person using the face image of the user. Then, the face / person recognition unit 302 calculates the degree of similarity between the calculated feature quantity and the feature quantity corresponding to the face image of the person stored in the face / expression database 404, and the face / expression database 404 stores the face The person of the face image having the highest similarity is determined from the persons storing the images.

  The expression recognition unit 306 identifies an expression corresponding to the user's face image. The identified facial expressions include a plurality of facial expressions such as joy, surprise, fear, disgust, anger, sadness, and no expression. The facial expression recognition unit 306 calculates a feature amount for recognizing a facial expression using the user's face image. The facial expression recognition unit 306 calculates the similarity between the feature amount corresponding to the face image of the user and the feature amount corresponding to each facial expression, and determines the facial expression having the highest similarity from the plurality of facial expressions.

  Next, the dialogue control unit 307 generates a sentence for responding to the user's utterance content. The dialogue control unit 307 is a sentence for responding to the content of the user's utterance based on the text corresponding to the voice of the user's utterance and the information on the person or animal that existed in the past stored in the databases 405 to 410. Is generated. The speech synthesizer 308 synthesizes the speech of a person or animal that has existed in the past based on the information about the person or animal that has existed in the past stored in the databases 411 to 413 and the generated sentence. That is, the dialogue control unit 307 and the speech synthesis unit 308 synthesize the speech of a person or animal that has existed in the past based on the text corresponding to the speech by the user's utterance and information on the person or animal that has existed in the past. .

  The dialogue control unit 307 is based on the text corresponding to the voice of the user's utterance, the person corresponding to the user's face image, the facial expression corresponding to the face image, and information on the person or animal that existed in the past. Thus, a sentence for responding to the user's utterance content may be generated. In other words, the dialogue control unit 307 and the voice synthesis unit 308 are configured such that the text corresponding to the voice of the user's utterance, the person corresponding to the user's face image, the facial expression corresponding to the face image, and the person or animal that existed in the past Based on the information regarding the voice of a person or animal that existed in the past may be synthesized.

  In addition, the face generation unit 310 and the facial expression synthesis unit 311 are based on the information about the person or animal that existed in the past stored in the database 414 and the sentence generated by the dialogue control unit 307. Or synthesize animal images. In other words, the dialogue control unit 307, the face generation unit 310, and the facial expression synthesis unit 311 are configured so that the person or animal that has existed in the past is based on the text corresponding to the speech by the user's utterance and information on the person or animal that has existed in the past. Synthesize the image. Note that the dialogue control unit 307, the face generation unit 310, and the facial expression synthesis unit 311 include text corresponding to the voice generated by the user's speech, a person corresponding to the user's face image, an expression corresponding to the user's face image, An image of a person or animal that existed in the past may be synthesized based on information about the person or animal that existed in the past.

  More specifically, when the user wants to talk with a person who has existed in the past (a person who died), the dialogue control unit 307 includes a family / person database 405, a deceased person preference database 406, a deceased person experience database 407, and a chat database. 408, the conversation history database 409, and the conversation person database 410, and the user based on the intention of the text detected by the intention understanding unit 303 and the person and expression determined from the user's face image. A sentence (sentence) for responding to the utterance content is generated.

  The family / person database 405 includes information indicating a family tree showing a family structure of a person who has existed in the past and a friend relationship, and information indicating a relationship between an animal (pet) and a family existing in the past. The deceased person preference database 406 includes information indicating hobbies, likes, dislikes, etc. of people who existed in the past, and information indicating likes, dislikes, etc. of animals that existed in the past. The deceased experience database 407 includes information indicating events experienced by people who existed in the past and memories with animals that existed in the past. The chat database 408 includes information indicating conversations and words that can be used for chat. The conversation history database 409 includes information indicating a history of conversation with a person who has existed in the past and information indicating a history of words spoken to animals that have existed in the past. The conversation person database 410 includes information (eg, age, sex, occupation, hobbies, etc.) regarding the user (ie, a person who has interacted with a person or animal that has existed in the past).

  The dialog control unit 307 responds to the user's utterance content by further using new information such as news and weather and the current date (for example, the season corresponding to the date and the event corresponding to the date). The sentence may be generated.

  The speech synthesizer 308 then synthesizes the speech of a person who has existed in the past to output the sentence generated by the dialog controller 307 using the synthesis dictionary 411, the voice database 412, and the speech database 413. To do. The synthesis dictionary 411 includes information indicating correspondence between words and phoneme strings, and information indicating correspondence between phonemes and speech signals (speech data). The similar voice database 412 includes information indicating the voice (voice quality) of a person who has existed in the past and information indicating the voice quality of an animal call that has existed in the past. The speech database 413 includes information indicating the intonation and rhythm of a person's speech that existed in the past, and information indicating the rhythm of how an animal has existed in the past.

  In addition, the face generation unit 310 and the expression synthesis unit 311 use the deceased face / expression database 414 to synthesize a face image of a person existing in the past based on the generated sentence and the determined person and expression. To do. The deceased person's face / expression database 414 includes information indicating face images of people in the past and various expressions (eg, joy, surprise, fear, disgust, anger, sadness, expressionlessness, etc.) It includes information indicating images (face images) and various facial expressions and types.

  The face generation unit 310 and the facial expression synthesis unit 311 use the deceased face / expression database 414 to change the past according to the synthesized speech based on the generated sentence and the determined person and facial expression. A moving image of a person (a moving image of a face) may be synthesized. The face generation unit 310 and the facial expression synthesis unit 311 synthesize a moving image whose facial expression changes in accordance with, for example, the inflection and rhythm of the synthesized voice and the content spoken by the voice.

  Next, the voice of the deceased synthesized from the speakers 18A, 18B is output by the audio output 309 using the speakers 18A, 18B, and the deceased synthesized on the screen of the LCD 17A by the face output (image output) 312 using the LCD 17A. The face image is displayed.

  The above-mentioned family / person database 405, deceased person preference database 406, deceased person experience database 407, conversation history database 409, conversation person database 410, voice database 412, speech database 413, deceased face / expression database 414 are, for example, It is created using life logs including images, sounds (speech of conversation), texts, etc. collected in advance before death or before the animal dies, questionnaire results answered by the person, and the like. In the life log, for example, a date, an event, an image, a person who was together, a place, and the like may be recorded in association with each other. Databases 405 to 407, 409, 410, and 412 that store information on memories and experiences of the deceased person, information that expresses memories and characteristics with animals that existed in the past, using such information. 414 can be constructed.

These databases 405 to 407, 409, 410, 412 to 414 are sites of social networking services (SNS) related to the person or animal (or the owner of the animal) after the death of the person or the animal. (For example, Facebook (registered trademark), Twitter (registered trademark), various blogs, etc.), created using information such as images, video, audio, text, etc. stored locally or on a network storage May be. Even in a site of a social networking service, for example, a date, an event, an image, a person who was together, a place, and the like may be recorded in association with each other, and using such information, the databases 405 to 407, 409, 410, 412 to 414 can be constructed.
By using such a database, a more realistic dialogue (communication) with people and animals that existed in the past can be realized.

  When the user tries to talk with an animal that has existed in the past, the dialogue control unit 307 includes a family / person database 405, a deceased preference database 406, a deceased experience database 407, a chat database 408, a conversation history database 409, and a conversation. Using at least one of the person database 410, a cry for responding to the user based on the intention of the text detected by the intention understanding unit 303 and the person and facial expression determined from the face image of the user ) And determine the type. Then, the speech synthesis unit 308 synthesizes animal calls that existed in the past for outputting the calls determined by the dialogue control unit 307 using the similar database 412 and the speech database 413.

  In addition, the face generation unit 310 and the expression synthesis unit 311 use the deceased face / expression database 414 to exist in the past according to the determined call and type, and the person and expression determined from the user's face image. Synthesize the animal images. The face generation unit 310 and the facial expression synthesis unit 311 use the deceased human face / expression database 414 based on the synthesized speech and the synthesized voice based on the person and facial expression determined from the user's facial image. A moving image of a changing animal may be synthesized. For example, the face generation unit 310 and the facial expression synthesis unit 311 synthesize a moving image whose facial expression and type change according to the inflection and rhythm of the synthesized voice and the content of the voice.

  Then, the audio of the animal synthesized from the speakers 18A and 18B is output by the audio output 309 using the speakers 18A and 18B, and synthesized on the screen of the LCD 17A by the face output (image output) 312 using the LCD 17A. An animal image is displayed.

  In addition, the case where the conversation with each of the several person and animal which existed in the past is performed by one Buddhist altar 1 is also assumed. In that case, the dialogue control unit 307 may determine a person or an animal that existed in the past as a partner of the dialogue according to the user who is going to dialogue. The dialogue control unit 307, the speech synthesis unit 308, the face generation unit 310, and the facial expression synthesis unit 311 output synthesized speech corresponding to the determined person or animal existing in the past from the speakers 18A and 18B and are synthesized. The displayed image is displayed on the screen of the LCD 17A.

  Next, an example of the procedure of the dialogue control process executed by the tablet computer 2 will be described with reference to the flowchart of FIG.

  First, the voice recognition unit 302 and the face / person recognition unit 305 determine whether or not a dialogue with a deceased person has been requested (block B101). The voice recognition unit 302 and the face / person recognition unit 305, for example, when a person is detected from an image captured by the camera module 16, or when a predetermined voice is detected from the voices generated by the microphones 19A and 19B, If a person is detected by the human sensor 116, etc., it is determined that a dialogue with the dead person has been requested.

  When the dialogue is not requested (NO in block B101), the process returns to block B101, and it is determined again whether the dialogue is requested.

  On the other hand, when a dialogue with a deceased person is requested (YES in block B101), voice input by the user is input by voice input 301 using the microphones 19A and 19B, and image input using the camera module 16 is performed. In 304, the image of the user is input (block B102).

  The speech recognition unit 302 uses the speech recognition dictionary 401 to generate text corresponding to the input speech (block B103). Then, the intention understanding unit 303 detects the intention of the generated text using the knowledge database 402 and the intention understanding database 403 (block B104).

  Further, the face / person recognition unit 305 detects a face image from the input image using the face / expression database 404 and determines a person corresponding to the face image (block B105). Then, the facial expression recognition unit 306 uses the face / expression database 404 to determine a facial expression corresponding to the face image detected by the face / person recognition unit 305 (block B106).

  Next, the dialogue control unit 307 uses the family / person database 405, the deceased person preference database 406, the deceased person experience database 407, the chat database 408, the conversation history database 409, and the conversation person database 410 to detect the intention of the detected text. A sentence (sentence) for responding to the user according to the determined person and facial expression is generated (block B107). The speech synthesizer 308 synthesizes the speech of the deceased for outputting the generated sentence by speech using the synthesis dictionary 411, the similar voice database 412, and the speech database 413 (block B108).

  Further, the face generation unit 310 and the expression synthesis unit 311 use the deceased face / expression database 414 to synthesize the deceased's face image according to the generated sentence and the determined person and expression (block B109). ). The voice of the deceased person synthesized from the speakers 18A and 18B is output by the audio output 309 using the speakers 18A and 18B, and the deceased person synthesized on the screen of the LCD 17A by the face output (video output) 312 using the LCD 17A. Are displayed (block B110).

  The dialogue control process described above may be executed not by the tablet computer 2 but by the cloud server 3 connected to the tablet computer 2 via a network. In that case, in the tablet computer 2, the synthesized voice of the deceased is output from the speakers 18A and 18B based on the processing result by the cloud server 3, and the synthesized image of the deceased is displayed on the screen of the LCD 17A.

  In the above description, an example of a dialogue process with a person who has existed in the past has been described, but a dialogue (communication) with an animal that has existed in the past can also be realized by a similar process. In the case of a dialogue with an animal that has existed in the past, the dialogue control unit 307 determines, for example, a cry or type that responds to the user, instead of a sentence (sentence) that responds to the user.

  Further, as shown in FIG. 7, the electronic device of the present embodiment may be realized as a television receiver 50. The television receiver 50 has a dialog function for realizing the above-described dialog with people and animals that existed in the past. On the screen of the LCD 58 provided in the television receiver 50, not only images of people and animals that existed in the past, but also images of Buddhist altars, ranks, flower stands, fire stands, incense burners, offerings, flower Buddha water devices, phosphorus, An image of a Buddhist ware such as may be displayed.

  The digital television receiver 50 has a thin box-shaped housing. An LCD 58 is provided on the front surface of the digital television receiver 50 so as to be superimposed on the box-shaped housing. A light receiving unit 51, an LED 52, a human sensor 54, a microphone 55, and the like are arranged at the lower front portion of the digital television receiver 50. The light receiving unit 51 receives a signal transmitted by the remote controller. The LED 52 indicates power on / off, standby state, and the like. The human sensor 54 detects that a person is present in front of the television receiver 50. The microphone 55 acquires sound produced by the voice of a person who uses the television receiver 50 (for example, a person who visits the Buddhist altar displayed on the screen of the television receiver 50). In addition, a camera module 53 that captures an image (face image) of a person using the television receiver 50 is provided on the front upper portion of the digital television receiver 50.

  Further, an operation unit 56 and a speaker 57 are provided on the side surface of the digital television receiver 50. Various buttons for operating the television receiver 50 are provided on the operation unit 56. The operation unit 56 includes, for example, a main power button for powering on / off the digital television receiver 50, a channel button for switching channels, and the like. The speaker 56 outputs sound.

When the above interactive function is used in the digital television receiver 50, the interactive application program 202 described with reference to FIG. 5 is executed by, for example, a processor provided in the digital television receiver 50. Thus, based on the voice of the user's utterance acquired by the microphone 55, the user's face image acquired by the camera module 53, and information on the person or animal that existed in the past, the speaker 57 has existed in the past. A synthesized (artificial) voice of a person or animal is output, and a synthesized image of a person or animal that has existed in the past is displayed on the screen of the LCD 58.
Therefore, the user can use the television receiver 50 to visit a dead person or a dead animal, or perform a pseudo dialogue with a dead person or a dead animal.

  As described above, according to the present embodiment, it is possible to realize pseudo communication with a person who has existed in the past or a dead animal. The databases 405 to 414 store information related to people or animals that existed in the past. The voice recognition unit 302, the intention understanding unit 303, and the dialogue control unit 307 generate a sentence for responding to the user's utterance content. The speech synthesizer 308 synthesizes the speech of a person or animal that existed in the past based on the information stored in the databases 405 to 414 and the generated sentence.

  Users (people who visit the Buddhist altar 1) can perform simulated conversations with people and animals that existed in the past by means of synthesized speech, thus reducing the sadness and loneliness of losing their family and pets. be able to. In addition, such an interactive function increases the chances of providing the deceased and ancestors, and allows the user to remember events with people and animals that existed in the past.

  Furthermore, this interactive function can contribute to expanding business opportunities in the industry for manufacturing and selling festival tools as new functions of the Buddhist altar 1 and the tombstone. In addition, as a dialog application program 202 executed on the tablet computer 2 or the smartphone, a new opportunity for feeding a dead person or a dead animal can be provided to the user.

  Note that all the procedures of the dialog control process of the present embodiment can be executed by software. For this reason, it is possible to easily realize the same effect as that of the present embodiment only by installing and executing this program on a normal computer through a computer-readable storage medium storing the program for executing the procedure of the interactive control process. Can do.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  202 ... Dialog application program, 301 ... Voice input, 302 ... Voice recognition unit, 303 ... Intent understanding unit, 304 ... Image input, 305 ... Face / person recognition unit, 306 ... Expression recognition unit, 307 ... Dialog control unit, 308 ... Speech synthesis unit, 309 ... voice output, 310 ... face generation unit, 311 ... facial expression synthesis unit, 312 ... face output, 401 ... speech recognition dictionary, 402 ... knowledge database, 403 ... intention understanding database, 404 ... face / expression database, 405 ... Family / person database, 406 ... Deceased person preference database, 407 ... Deceased person experience database, 408 ... Chatting database, 409 ... Conversation history database, 410 ... Conversation person database, 411 ... Synthetic dictionary, 412 ... Voice database, 413 ... Talk database, 414 ... Deceased face / expression database

Claims (11)

Storage means for storing information about a person or animal that existed in the past;
Speech recognition means for generating a sentence for responding to the user's utterance content;
An electronic device comprising: information stored in the storage means; and voice synthesis means for synthesizing the human or animal voice based on the generated sentence.   The electronic device according to claim 1, wherein the voice recognition unit generates a text corresponding to a voice uttered by the user, and generates the sentence based on the text and information stored in the storage unit. Further comprising face recognition means for determining a person corresponding to the face image and a facial expression corresponding to the face image using the user's face image;
The speech recognition unit generates the sentence based on the text, a person corresponding to the face image, a facial expression corresponding to the face image, and information stored in the storage unit. Electronics.   The image processing device further comprises image combining means for combining the human or animal image based on the sentence, the person corresponding to the face image, the facial expression corresponding to the face image, and the information stored in the storage means. The electronic device according to claim 3.   The image synthesizing unit changes according to the synthesized voice based on the sentence, the person corresponding to the face image, the facial expression corresponding to the face image, and the information stored in the storage unit. The electronic apparatus according to claim 4, wherein the moving image of the person or animal is synthesized.   The electronic apparatus according to claim 1, further comprising an image synthesis unit that synthesizes an image of the person or animal based on the information stored in the storage unit and the sentence.   The information on the person or animal that existed in the past is information indicating at least one of the face, expression, voice, and way of speaking of the person that existed in the past, or the face, expression, call, and type of animal that existed in the past. The electronic device according to claim 1, comprising information indicating at least one.   The electronic device according to claim 1, wherein the information related to the person or animal existing in the past includes information acquired from a life log.   The electronic device according to claim 1, wherein the information related to the person or animal existing in the past includes information acquired from a site of a social networking service (SNS). Store information about people or animals that existed in the past in the storage device,
Generate a sentence to respond to the user's utterance,
An information processing method for synthesizing the voice of the person or animal based on the information stored in the storage device and the generated sentence. A program executed by a computer, wherein the program is
A procedure for storing information about people or animals that existed in the past;
A procedure for generating a sentence to respond to the user's utterance,
A program that causes the computer to execute the stored information and a procedure for synthesizing the voice of the person or animal based on the generated sentence.

Images