JP2019128384A - System, method, and program for processing information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information so as not to give a feeling of strangeness to a user. SOLUTION: One aspect of the present invention is a response unit for outputting a response content to a voice emitted by a user, specific information different from the response content to an output unit, and an output mode of the specific information. Output of the specific information when the user's instruction is received after changing to the first output mode, which is harder for the user to hear than the third output mode, which is the output mode of the response content, and outputting to the output unit. It is an information processing system including a control unit that changes the mode to a second output mode that is easier for the user to hear than the first output mode and outputs the specific information to the output unit. [Selection diagram] Fig. 1

Description

  The present invention relates to an information processing system, an information processing method, and a program.

  Conventionally, during a navigation process of searching for a route to a destination and guiding a guided route according to a search result, a voice advertisement or a questionnaire related to a voice advertisement is output by voice output means based on dialogue with the user. An information processing apparatus including an output control unit is disclosed (for example, Patent Document 1).

JP, 2017-58315, A

  However, in the conventional technology, the output of the sound sometimes gives the user a sense of incongruity.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an information processing system, an information processing method, and a program capable of providing information so as not to give a user a sense of incongruity. One of them.

  One aspect of the present invention provides a response unit that outputs a response content to a voice uttered by a user and specific information different from the response content to an output unit, an output mode of the specific information, When the user's instruction is accepted after changing to the first output mode that is more difficult for the user to hear than the third output mode, which is the output mode, and outputting to the output unit, the output mode of the specific information is It is an information processing system provided with the control part which changes to the 2nd output mode which the user can easily hear rather than the 1st output mode, and makes the output part output the above-mentioned specific information.

  According to one embodiment of the present invention, information can be provided so as not to give the user a sense of incongruity.

FIG. 1 is a diagram showing the configuration of an information processing system 1; 5 is a flowchart showing an example of the flow of processing executed by the information processing system 1; It is a figure which shows an example of the content of the environment pattern information. It is a figure which shows an example of the content of the advertisement information. It is a figure which shows an example of the content of the output level information. It is a figure which shows an example of the conversation between a user and the automatic response apparatus. 4 is a flowchart illustrating an example of a flow of processing executed by a terminal device 10 and an automatic response device 40. It is a figure which shows an example of the content of the usage information. It is a figure which shows an example of a function structure of 40 A of automatic response apparatuses contained in the information processing system 1A of 2nd Embodiment. It is a flowchart which shows an example of the flow of the process performed by the terminal device 10 and the automatic response apparatus 40A of 2nd Embodiment. It is a figure which shows an example of the content of the instruction | indication correspondence information. It is a figure which shows an example of the conversation with the user of 2nd Embodiment, and the automatic response apparatus. It is a figure which shows the change of the sound volume at the time of the information of an advertisement being output. It is a figure which shows an example of a function structure of the information processing system 1B of 3rd Embodiment. It is a flowchart which shows an example of the flow of the process performed by the automatic response apparatus 40B. It is the figure which shows an example of the image displayed on the conversation and the display part 15 of 3rd Embodiment (the 1). It is the figure which shows an example of the image displayed on the conversation and the display part 15 of 3rd Embodiment (the 2). It is a figure (the 3) which shows an example of the image displayed on the conversation and the display part 15 of 3rd Embodiment. It is a figure (the 4) which shows an example of the image displayed on the conversation and the display part 15 of 3rd Embodiment.

  Hereinafter, embodiments of an information processing system, an information processing method, and a program according to the present invention will be described with reference to the drawings.

<Overview (common items)>
An information processing system is implemented by one or more processors. The information processing system causes the output unit to output the response content to the voice uttered by the user and the specific information different from the response content. The “response content” is, for example, information determined by an automatic response device operating by a machine-learned model such as artificial intelligence (AI) or deep learning. The “specific information” is information that does not correspond to a response to a voice uttered by the user, such as an advertisement, a greeting, an utterance that triggers a conversation, or an announcement (for example, a request for a recommendation or a password change).

[Overview (Part 1)]
The information processing system controls the output mode of the specific information in accordance with the usage degree of the user device (for example, a microphone or a speaker) to which audio is input or output. The “degree of use” is, for example, a value based on the number or frequency of voice input to the user device, or a value based on the number or frequency of voice output to the user device. For example, the output amount of the specific information is increased as the usage degree of the user device is higher. That is, a large number of talks and voice advertisements from the automatic response device are output to a user who frequently uses voice input or output. In addition, the higher the degree of use of the user device, the more control is performed so that the user can easily hear the output mode of the specific information. The “output mode” is, for example, the volume of the sound, the pitch of the sound, and the tempo at which information is output. The outline (part 1) will be described focusing on the first embodiment described later.

[Overview (Part 2)]
When the information processing system receives an instruction from the user after changing the output mode of the specific information to the first output mode in which the user is less likely to hear than the third output mode of the response content and causing the output unit to output The output mode of the specific information is changed to the second output mode, and the specific information is output to the output unit. The “second output mode” is an output mode that is easier for the user to hear than the first output mode. That is, the volume of only the specific information (for example, voice advertisement) is reduced in the dialogue with the automatic response device, and the volume is increased according to the request or operation from the user. The outline (No. 2) will be described focusing on a second embodiment described later.

[Overview (Part 3)]
The information processing system causes the output unit to output the contents of the response in accordance with the output mode corresponding to the first character, and causes the output unit to output the specific information in accordance with the output mode corresponding to the second character. Furthermore, the information processing system causes the output unit to output a conversation between the first character and the second character. The “first character” is, for example, a character that interacts with the user or responds to the user's speech in everyday life. The “second character” is, for example, a character that is different from the first character and is associated with specific information (for example, an advertisement). In this way, the first character interacting with the user and the second character corresponding to the voice advertisement arouse the user's interest in the advertisement. The outline (No. 3) will be described focusing on a third embodiment described later.

First Embodiment
[overall structure]
FIG. 1 is a diagram showing the configuration of the information processing system 1. The information processing system 1 includes, for example, a terminal device 10, an automatic response device 40, and an advertisement providing device 80. These devices communicate with each other via a network NW. The network NW includes, for example, a wide area network (WAN), a local area network (LAN), the Internet, a dedicated line, a wireless base station, a provider, and the like. In the present embodiment, the automatic response device 40 is an example of the “information processing system”. Further, the “information processing system” may include the terminal device 10 and / or the advertisement providing device 80.

[Functional configuration of terminal device]
The terminal device 10 is, for example, a smart speaker (Artificial intelligence speaker), a smartphone, a tablet terminal, a personal computer, or the like. In the first embodiment, the terminal device 10 will be described as a smart speaker.

  The terminal device 10 includes, for example, a microphone 12, a speaker 14, a voice recognition unit 16, a voice generation unit 18, a terminal control unit 20, a terminal device side communication unit 22, and a storage unit 30. The voice recognition unit 16, the voice generation unit 18, and the terminal control unit 20 are configured such that, for example, a hardware processor such as a CPU (Central Processing Unit) executes an application program (application 32) stored in a storage unit 30 such as a flash memory. It is realized by executing. The application 32 may be downloaded from, for example, a server device or the like via a network, or may be preinstalled on the terminal device 10 in advance. Note that, instead of the application program, a browser having the same function as that described below may be used as a UA (User Agent). Note that some or all of the functions included in the terminal device 10 may be included in the automatic response device 40.

  The microphone 12 acquires the sound emitted by the user or the environmental sound of the environment where the terminal device 10 exists. The speaker outputs sound corresponding to the information generated by the sound generation unit 18.

  The voice recognition unit 16 converts the voice acquired by the microphone 12 into digital data (voice data). The sound generation unit 18 generates information according to the sound to be output to the speaker 14 based on the information transmitted by the automatic response device 40.

  The terminal control unit 20 transmits the digital data converted by the voice recognition unit 16 to the automatic response device 40 using the terminal device side communication unit 22. The terminal control unit 20 acquires the information transmitted by the automatic response device 40 via the terminal device side communication unit 22.

  The terminal device side communication unit 22 is, for example, a wireless communication interface. The terminal device side communication unit 22 acquires information transmitted by the automatic response device 40 or transmits a processing result processed in the terminal device 10 to the automatic response device 40.

[Functional configuration of automatic answering device]
The automatic response apparatus 40 includes, for example, a user identification unit 42, an environment analysis unit 43, a pattern identification unit 44, an interpretation unit 46, a response unit 48, a provision control unit 50, a learning unit 52, and a response device. A side communication unit 54, a first storage unit 60, and a second storage unit 70 are provided. For the user identification unit 42, the environment analysis unit 43, the pattern identification unit 44, the interpretation unit 46, the response unit 48, the provision control unit 50, and the learning unit 52, for example, a hardware processor such as a CPU is a storage device (e.g. This is realized by executing a program stored in one storage unit 60). In addition, these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), It may be realized by cooperation of software and hardware. In addition, the above program may be stored in advance in a storage device, or stored in a removable storage medium such as a DVD or a CD-ROM, and the storage medium is attached to the drive device of the automatic response device 40 May be installed in the storage device. The first storage unit 60 and the second storage unit 70 are realized by, for example, a ROM (Read Only Memory), a flash memory, an SD card, a RAM (Random Access Memory), a register, and the like.

  For example, user specifying information 62, environment specifying information 63, environment pattern information 64, regular expression information 66, and scenario information 68, which will be described later, are stored in the first storage unit 60. In the second storage unit 70, for example, output degree information 72 and usage information 74 described later are stored. The first storage unit 60 and the second storage unit 70 do not necessarily have to be realized by separate storage devices, and may be different storage areas in an integrated storage device.

  For example, the user identification unit 42 extracts a component of voice data (hereinafter referred to as an utterance component) estimated to represent a human voice from the voice data transmitted by the terminal device 10. The user identification unit 42 collates the extracted utterance component with the information included in the user identification information 62, and identifies the person who uttered the voice represented by the extracted utterance component. The user identification information 62 is information in which the identification information of the user is associated with information indicating the characteristics of the voice of the user (for example, a voice print pattern or a frequency pattern).

  The user identification unit 42 may also refer to the user identification information 62 to identify the type of person present in the vicinity of the user who issued the voice. In this case, the user identification information 62 includes information indicating the features of the voice of the user's family, friends, etc. in advance. In addition, the user specifying unit 42 acquires information indicating a connection state between the terminal device owned by the family and the Wi-Fi router via the terminal device side communication unit 22, and based on the acquired information, the terminal It may be determined whether the owner of the device exists near the position where the Wi-Fi router is installed.

  For example, the environment analysis unit 43 extracts a component of sound data (hereinafter referred to as an environment sound component) estimated to represent an environment sound other than a human voice from the sound data transmitted by the terminal device 10. The environment analysis unit 43 collates the extracted environmental sound component with the information included in the environment specifying information 63, and specifies the magnitude of the environmental sound represented by the extracted environmental sound component and the cause of the environmental sound. To do. The environment specifying information 63 is information in which identification information of an occurrence factor of environmental sound and a feature of sound for each occurrence factor of environmental sound are associated with each other.

  The pattern specifying unit 44 specifies an environment pattern based on, for example, the environment pattern information 64, the processing result of the user specifying unit 42, and the processing result of the environment analysis unit 43. An environmental pattern is a pattern classified according to a predetermined standard about the environment where a user exists. Details will be described later.

  For example, the interpretation unit 46 converts voice data corresponding to a human voice into text information, and further collates the text information with the regular expression information 66 to interpret the meaning of the user's utterance. For example, it is assumed that the user uttered “Tell me how to get from Shinjuku to Shibuya”. The interpretation unit 46 performs morphological analysis on the utterance and divides the utterance into parts of speech. Then, the interpretation unit 46 generates a search key in which Shinjuku and Shibuya corresponding to proper nouns and place names are converted into codes, and searches the regular expression information 68. In the regular expression information 68, information (regular expression) obtained by converting a proper noun into an abstract code is registered. For example, information to which a text such as “Tell me how to go from OO to XX” or “Tell me how to go from OO to XX” is registered as a regular expression.

  The response unit 48 acquires, for example, text information corresponding to “Teach me how to get from (proper noun, place) to (proper noun, place)” included in the regular expression information 68, from 〇 to ×× Recognize that you should provide directions.

  Then, the response unit 48 embeds “Shinjuku” and “Shibuya”, which are the encoded original information, in the (proprietary noun, place) part, and uses “I want to know how to get from Shinjuku to Shibuya” Recognize the will of the The response unit 48 performs a network search and obtains directions from Shinjuku to Shibuya. The response unit 48 refers to the scenario information 68, for example, and generates voice source information to be output from the terminal device 10, which indicates the way from Shinjuku to Shibuya. As the scenario information 68, for example, the contents to be responded to the user's speech are held in advance. That is, the content of the response to the utterance in which the user has the intention "I want to know the way from 〇 to ×" is held. The scenario information 68 is prepared for each user, for example, so that the response content matches the preference or the like of the user.

  Note that some or all of the functions included in the automatic response device 40 such as the response unit 48 may be provided in the terminal device 10. Information such as regular expression information 66 and scenario information 68 may also be stored in the storage device of the terminal device 10.

  The provision control unit 50 transmits the voice source information to the terminal device 10 using the response device side communication unit 54 in order to cause the terminal device 10 to output the voice source information generated by the response unit 48. Further, the provision control unit 50 transmits the voice source information to the terminal device 10 using the response device side communication unit 54 in order to cause the terminal device 10 to output the voice source information transmitted by the advertisement providing device 80. .

  Further, the provision control unit 50 designates an output mode of the response content or the specific information, and outputs the response content or the specific information in the specified output mode to the speaker 14 of the terminal device 10, the designated output mode and the response content Or the information which matched specific information is transmitted to the terminal device 10 using the response apparatus side communication part 54. FIG. The function of the provision control unit 50 may be included in the terminal device 10.

  The learning unit 52 learns the response content or specific information content output to the speaker 14 of the terminal device 10, the output mode of the output information, the user's reaction, and the environment pattern. The learning is, for example, learning using artificial intelligence or machine learning such as deep learning.

  The response device side communication unit 54 includes a communication interface such as a network interface card. The response device side communication unit 54 acquires information transmitted by the terminal device 10 or the advertisement providing device 80, or transmits a processing result processed in the automatic response device 40 to the terminal device 10 or the advertisement providing device 80. .

[Advertising equipment]
The advertisement providing apparatus 80 includes, for example, an information providing unit 82, an advertisement providing apparatus side communication unit 84, and an advertisement providing apparatus side storage unit 90. The information providing unit 82 extracts an advertisement to be provided to the user based on the information input by the user's utterance or the response content of the automatic response apparatus 40, and information on the extracted advertisement (for example, voice source information and voice) To the automatic response device 40.

  The advertisement providing device side communication unit 84 includes a communication interface such as a network interface card. The advertisement providing device side communication unit 84 acquires information transmitted by the automatic response device 40 or transmits a processing result processed in the advertisement providing device 80 to the automatic response device 40. The advertisement providing device side storage unit 90 stores advertisement information 92 to be described later. The advertisement providing device 80 and the automatic response device 40 may be provided as an integrated device.

[Flowchart (processing to determine output degree)]
FIG. 2 is a flowchart showing an example of the flow of processing executed by the information processing system 1. The present process is a process of controlling the amount of audio output from an artifact according to the amount of use of the audio UI (user interface / user device) by the user. This voice UI is voice recognition.

  First, the terminal device 10 determines whether or not a voice is input by the user (S10). When voice is input by the user (when conversation between the user and the automatic response device 40 is started), the input voice data (speech component and environmental sound component) is transmitted to the automatic response device 40 .

  The automatic response apparatus 40 acquires the utterance component, and identifies the user based on the acquired utterance component and the user identification information 62 (S20). The automatic response apparatus 40 acquires an environmental sound component, and specifies an environmental pattern based on the acquired environmental sound component and the environmental pattern information 64 (S22).

  FIG. 3 is a diagram showing an example of the contents of the environmental pattern information 64. As shown in FIG. The environment pattern information 64 is information in which a plurality of environment patterns are associated with classification criteria. The classification criteria of the environmental pattern include, for example, the day of the week, the time, the number of persons existing around the user, the type of person, the size of the environmental sound in which the user exists, and the user Classification based on at least one or more of the user's environment (home, office, city), location where the user exists, and user schedule (currently registered in advance) It is a pattern.

  The environment where the user exists, the position where the user exists, or the schedule of the user are, for example, information set in advance by the user. Further, the environment where the user exists or the position where the user exists may be specified based on information measured by a positioning device using a GPS (Global Positioning System) (not shown). . The user's schedule may be information acquired by the terminal device 10 from another device via the network NW.

  Next, the automatic response apparatus 40 requests the advertisement providing apparatus 80 to determine the content of the advertisement to be provided to the specified user (S24). At this time, the automatic response apparatus 40 transmits, to the advertisement providing apparatus 80, text information obtained by converting the information included in the voice input to the terminal apparatus 10 into text information.

  In response to the request from the automatic response device 40, the advertisement providing device 80 refers to the advertisement information 92 and determines the content of the advertisement to be provided to the user corresponding to the text information (S30). If there is no advertisement to be provided to the user, the advertisement providing apparatus 80 transmits a message to that effect to the automatic response apparatus 40.

  FIG. 4 is a diagram showing an example of the contents of the advertisement information 92. As shown in FIG. The advertisement information 92 is information in which a character, a product (or service), a scenario, and a keyword are associated with an advertisement ID. The "character" is a person having a predetermined feature, an animal, a plant, a creation, an artifact, etc. The character may be provided for each product, or may be provided for each of a plurality of products or for each campaign.

  A “scenario” defines the content and order of words (or actions) issued by a character. A scenario is provided, for example, for each character. Further, in the advertisement information 92, in addition to the scenario, the characteristics of the character such as voice tone and tempo are associated with the character. Characters for each product and campaign act autonomously based on scenarios (action rules).

  “Keyword” is a word associated with an advertisement. [Keyword] is a word having the same meaning as the meaning (semantic information) of a word indicating a product, or a word related to the meaning of a word indicating a product. Related words are words that are generally recalled from words that describe products. For example, the advertisement providing device 80 is associated with a matching keyword when the word input by the user or the word included in the voice emitted by the automatic response device 40 matches the keyword of the advertisement information 92. Information corresponding to the advertisement ID (speech original information etc. spoken by the character) is transmitted to the automatic response device 40. Note that the advertisement providing device 80 may determine information to be provided to the user by a machine-learned model such as artificial intelligence or deep learning.

  Next, the automatic response device 40 refers to the output degree information 72 described later, determines the output degree of the advertisement according to the environment pattern, and instructs the terminal device 10 to output the advertisement with the determined output degree. (S26). Next, the terminal device 10 outputs an advertisement based on the instruction of the automatic response device 40 (S12). Thereby, the process of one routine of this flowchart is completed.

  FIG. 5 is a diagram showing an example of the content of the output degree information 72. As shown in FIG. The output degree information 72 is prepared, for example, for each environment pattern. Further, the output degree information 72 is information in which the past usage degree of the environment pattern and the output degree of outputting the advertisement are associated with the user ID.

  “Past usage” refers to the degree to which a user has been provided with voice information (for example, an advertisement) from the speaker 14 in the past, or the degree to which the user has previously input information into the microphone 12 using voice. is there. The “output degree” is the volume of sound that is output when information is output to the user using the speaker 14. The “output degree” is an example of the “output mode”. The output degree is set, for example, so that the magnitude of the sound to be output becomes larger as the past use degree increases. Note that the degree of output of music may be excluded from the “degree of provision of information by voice from the speaker 14”.

  Further, in the output level information 72, instead of the output level, other output modes may be associated. The aspect related to output includes, for example, the volume of the sound, the pitch of the sound, the tempo at which the content of the advertisement is output, and the like. For example, the aspect related to the output is set so that the user can easily hear as the past usage degree increases.

  Further, when the environmental sound of the environment where the user exists is greater than or equal to a predetermined level, the degree of change in the output mode of the specific information may be smaller than when the environmental sound is less than the predetermined level. That is, in an environment where the environmental sound is originally large, the output of the specific information may not be increased.

  As described above, by referring to the output degree information 72, the automatic response apparatus 40 can provide information so as not to give a sense of discomfort to the user.

  In the above description, as an example, an example in which the output mode is controlled based on the degree of use when the user inputs voice is described. However, the automatic response device 40 simply utters or outputs information. In such a case, the output mode may be controlled based on the degree of use.

[Specific example (1)]
FIG. 6 is a view showing an example of the conversation between the user and the automatic response device 40. As shown in FIG. For example, as shown in FIG. 6A, (1) the user inputs to the microphone 12 "I want a new car".
(2) The automatic response device 40 responds, "What kind of car do you want?" In the output mode of the first character.

  Next, as shown in FIG. 6 (B), (3) the user inputs to the microphone 12 that “a car with good fuel efficiency is good”. (4) The automatic response device 40 responds “It is good because it can be saved” in the output mode of the first character. Then, (5) the automatic response apparatus 40 utters "The car A has good fuel consumption." In the output mode of the second character. This output mode of the second character is an output mode corresponding to the usage degree of the user device.

  Next, as shown in FIG. 6C, the user inputs (6) “Tell me in detail” to the microphone 12. (7) The automatic response device 40 responds with the output mode of the second character, “Car A is an electric vehicle.

  Thus, when a keyword appears in the conversation between the first character and the user, the automatic response device 40 outputs an advertisement based on the keyword in the output mode according to the usage degree of the user device. Provide to users. As a result, information can be provided so as not to give the user a sense of incongruity.

  Note that if the car A does not show interest in (6) above, the second character may not speak after that. In addition, when the car A does not show an interest, another car may be introduced in the output mode of the character corresponding to the other car.

  In addition, when it is desired to provide a car advertisement, the automatic response apparatus 40 may make an utterance or a response to talk to the first character on a topic of the car. In this case, for example, the automatic response apparatus 40 utters the keyword and the keyword described above. For example, a character conversation is selected based on specific information to be output.

  In the above-described example, the output level of the second character's utterance is changed. However, the output level of the first character's utterance may be changed. The output level may be changed based on the degree of conversation between the user and the character. For example, when the degree of conversation between the first character and the user is higher than the degree of conversation between the Nth character (N is an arbitrary natural number) and the user, the degree to which the first character talks to the user is Make it more than the N character talks to the user.

[Flowchart (learning process)]
FIG. 7 is a flowchart showing an example of the flow of processing executed by the terminal device 10 and the automatic response device 40. The processes of S40, S50, and S52 in the flowchart of FIG. 6 are the same as the processes of S10, S20, and S22 in the flowchart of FIG.

  After the processing of S52, the automatic response device 40 determines whether or not the own device has provided information to the user (S54). When the information is provided to the user, the automatic response device 40 acquires the content of the provided information and the user's reaction after the information is provided, and stores the acquired response as the usage information 74 in the second storage unit 70. (S56).

  FIG. 8 is a diagram showing an example of the contents of the usage information 74. As shown in FIG. The usage information 74 includes, for each user, information input by the user in the past, information output to the user in the past, input information, output mode of the output information, environment, This is information in which a pattern and a user's reaction (for example, an instruction) to the output information are associated with each other.

  Next, the automatic response device 40 determines whether or not a predetermined timing has been reached (S58). If the predetermined timing has not been reached, the process of one routine of this flowchart is terminated. When the predetermined timing is reached, the automatic response device 40 learns the usage information 74 as learning data (S60). Thereby, the process of one routine of this flowchart is completed.

  As described above, the user's preference can be grasped by learning the reaction of the user when providing information to the user, the environmental pattern, the output mode of the information, and the content of the information. Then, the learning unit 52 can generate or update the output degree information 72 by reflecting the preference of the user.

  For example, if a parent is present around the user on a Saturday, 7 o'clock to 8 o'clock, or if the user is at home, or if the private schedule is scheduled, It is assumed that the usage information 74 includes information indicating that the specific information is instructed to be output so as to be suppressed more than in the situation. In this case, the learning unit 52 generates the output degree information 72 so as to suppress the output of the specific information in the environment pattern corresponding to the above-described situation.

  As described above, an artificial object such as a smart speaker that performs voice interaction with a user may feel complicated if the user performs excessive voice advertisement or talking, but in this embodiment, the user's voice interaction Since the voice advertisement and the conversation are adjusted according to the degree of use and the situation in which the interaction is performed, information can be provided so as not to give the user a sense of incongruity.

  In the example described above, the provision control unit 50 has been described as controlling the output mode of the specific information in accordance with the usage degree of the user device for which the voice is input or output, but instead of this, (Or in addition), it may be changed as follows. That is, the providing control unit 50 controls the output mode of the second response content according to the usage degree of the user device. This “second response content” is a response content to the voice uttered by the user and includes an advertisement. For example, in this case, the automatic response device 40 determines the response content including the advertisement, and causes the terminal device 10 to output the determined response content in an output mode corresponding to the usage degree of the user device. Thus, since the response content itself becomes an advertisement and the control mode of the response content is controlled, information can be provided so as not to give the user a sense of incongruity.

  According to the first embodiment described above, the provision control unit 50 controls the output mode of the specific information according to the usage degree of the user device to which the voice is input or output, thereby allowing the user to Information can be provided so as not to give a sense of incongruity.

Second Embodiment
The second embodiment will be described below. The provision control unit 50 changes the output mode of the specific information to the first output mode in which the user is less likely to hear than the third output mode of the response content and causes the output unit to output it, and then receives the user's instruction. In this case, the output mode of the specific information is changed to the second output mode that is easier for the user to hear than the first output mode, and the specific information is output to the output unit. The differences from the first embodiment will be mainly described.

  FIG. 9 is a diagram illustrating an example of a functional configuration of an automatic response device 40A included in the information processing system 1A according to the second embodiment. The automatic response device 40A includes a second storage unit 70A in place of the second storage unit 70. For example, in addition to the output degree information 72 and the use information 74, the second storage unit 70A further includes instruction correspondence information 76 (details will be described later).

  When outputting the specific information to the terminal device 10, the response unit 48 of the second embodiment changes the output mode of the specific information to the first mode in which the user is less likely to hear the user than the third output mode of the response content, The specific information is output to the terminal device 10.

  After the specific information is output to the terminal device 10 as described above, when the user's instruction is received, the automatic response device 40A can easily hear the application mode of the specific information than the first output mode. It changes to a 2nd output mode, and makes specific information be output to the terminal device 10. FIG. The second output mode is, for example, a mode in which the volume is larger than the first output mode, the sound frequency band is easy for the user to hear, and the tempo at which information is output is appropriate.

  In the process of changing to the first mode in which the user hardly hears, if the environmental sound of the environment in which the user is present is greater than or equal to the predetermined size, the specific information is more specific than if the environmental sound is less than the predetermined size. The output mode may not be changed, or the change degree of the output mode may be reduced. This is because even if the output mode is changed in an environment where the environmental sound is large originally, the influence on the user is small.

[flowchart]
FIG. 10 is a flow chart showing an example of the flow of processing executed by the terminal device 10 and the automatic response device 40A of the second embodiment. This process is a process executed after the specific information is output in the first output mode. The processes of S60, S70 and S72 of the flowchart of FIG. 10 are the same as the processes of S10, S20 and S22 of the flowchart of FIG.

  Next, the automatic response device 40A refers to the instruction correspondence information 76, and matches the combination of the specified user, the specified environment pattern, and information (instruction contents) included in the input voice. The output mode of the advertisement information is determined (S74). The content of the instruction is information on the instruction that the user has requested regarding the output of information. The content of the instruction is, for example, raising the volume, outputting information slowly, outputting information with a high sound, outputting information output a few seconds ago, or a combination thereof. .

  FIG. 11 is a diagram showing an example of the content of the instruction correspondence information 76. As shown in FIG. The instruction correspondence information 76 is information used to determine in which output mode the information is to be output in response to an instruction issued by the user. The instruction correspondence information 76 is, for example, information in which the user ID, the instruction content, and the output mode are associated with each other for each environment pattern.

  Next, the automatic response device 40A instructs the terminal device 10 to output the advertisement information in the determined output mode (S76). Next, the terminal device 10 causes the speaker 14 to output the advertisement information in the determined output mode based on the instruction of the automatic response device 40A (S62). Thereby, the process of one routine of this flowchart is completed.

  As described above, since the automatic response device 40 changes the output mode according to the user's request, information can be provided so as not to give the user a sense of incongruity.

[Specific example (part 2-1)]
FIG. 12 is a diagram illustrating an example of a conversation between the user of the second embodiment and the automatic response device 40A. For example, as shown in FIG. 12A, (1) the user inputs to the microphone 12 "I want a new car". (2) The automatic response device 40A responds, "What kind of car do you want?" In the output mode of the first character.

  Next, as shown in FIG. 12 (B), (3) the user inputs to the microphone 12 that “a car with good fuel efficiency is good”. (4) The automatic response device 40 </ b> A responds “It is good because it can be saved” in the output mode of the first character.

  Next, for example, when the user does not make a speech for several seconds, as shown in FIG. 12C, (5) the automatic response device 40A is an output mode of the second character and in the first output mode. Say, "We recommend car A ....".

  (6) The user is interested in the information output in (5) but can not hear it because the volume is small, so he utters "I can not hear it." Then, (7) the automatic response device 40A is the output mode of the second character, increases the volume, and causes the terminal device 10 to output the information output in (5) above. That is, the second character speaks again, “We recommend car A.”

  In this way, the output mode when the second character outputs information is made more difficult for the user to hear than the output mode when the first character outputs information. Can be suppressed. Further, convenience for the user can be improved by making it easy for the user to hear the output mode when the second character outputs information in response to the user's request.

  In the above description, the example in which the specific information is output when the user inputs a voice has been described as an example. However, when the automatic response device 40A simply outputs the specific information, as described above. The output mode may be controlled. Also, for example, the conversation between the first character and the second character may be selected based on the specific information to be output.

[Specific example (2-2)]
FIG. 13 is a diagram showing a change in volume when advertisement information is output. The vertical axis in FIG. 13 indicates the magnitude of sound, and the horizontal axis in FIG. 13 indicates time. The length (time) of each advertisement of advertisements A to C described below is, for example, a predetermined second (for example, about 15 seconds). It is assumed that advertisement information is scheduled to be output in the order of advertisements A to C. In this case, for example, the advertisement A is output, the advertisement B is output, and the user instructs to increase the volume while the content of the advertisement B is being output (time T in FIG. 13). At time T, the automatic response device 40A causes the terminal device 10 to output the contents of the advertisement B from the beginning. That is, the advertisement B is re-outputted from the portion going back for a predetermined time or the portion with reduced volume. Further, after that, after the contents of the advertisement B are output as illustrated, the automatic response apparatus 40A may lower the volume to the level before raising the volume, or may keep the volume increased.

  As described above, when the automatic response device 40A instructs the user, the user can obtain desired information because the advertisement output at the time of the instruction is output from the beginning .

  In the example described above, the content B is output from the beginning based on the user's instruction, but may be output from the beginning of the advertisement A, or a predetermined time from when the user's instruction is given You may output from the information output previously. Information to be re-outputted may be determined based on the content of the user's utterance (for example, the degree of urgency). Further, the automatic response device 40A determines how long before the advertisement is reproduced again from the time when the user's instruction is made based on the tendency of the user's instruction in the past or the preset condition. Also good.

[Others]
The provision control unit 50 may change the output mode of the specific information to the first output mode based on the attribute of the specific information and cause the output unit to output the specific information. The attribute of the specific information includes at least one of information related to advertisement, information related to device operation, music, and information related to a time limit related to the user (information such as a password change time limit). For example, the provision control unit 50 may change the output mode of the information related to the advertisement to the first output mode, and the specific information of the other attributes may not change the output mode.

  The provision control unit 50 may change the output mode of the specific information to the first output mode based on the type of the advertisement and cause the output unit to output the specific information. The type of advertisement is, for example, the type of product corresponding to the advertisement. For example, the provision control unit 50 changes the output mode of the car advertisement to the first output mode, but does not change the output mode of the real estate advertisement to the output unit without changing to the first output mode. May be.

  Further, the provision control unit 50 changes the output mode of the specific information to the first output mode based on the type of advertisement and the result of the user's instruction made in the past, and the specific information is output to the output unit. It may be output. For example, the learning unit 52 learns the types of advertisements and the results of user instructions made in the past. For example, when the advertisement of a car is output, the learning unit 52 instructs the user to increase the volume, but when the advertisement of real estate is output, the learning unit 52 learns that the user instructs to decrease the volume. In this case, for example, the provision control unit 50 changes the output mode of the car advertisement to the first output mode, but does not change the output mode of the real estate advertisement to the first output mode. May be output.

  Further, the provision control unit 50 may adopt the above-described concept and change the output mode of the specific information to the first output mode based on the environment pattern corresponding to the user. For example, in a certain environment, the learning unit 52 learns that it is preferable for the user that the specific information is output in the first output mode. The providing control unit 50 outputs the specific information in the first output mode based on the learning result.

  Further, the output mode of information specified by the user (for example, information of a predetermined attribute) is changed to the first output mode, and the output mode of information not specified is not changed to the first output mode. Also good.

  The instruction correspondence information 76 is generated by the learning unit 52. For example, after the specific information is output to the output unit in the first output mode, the learning unit 52 changes the content of the instruction received by the user and the output mode of the specific information changed based on the instruction for each environment pattern. learn. Then, the learning unit 52 learns how to change the output mode of the specific information in the predetermined environment pattern, and generates the instruction correspondence information 76 that matches the user's preference.

  For example, if a parent is present around the user on a Saturday, 7 o'clock to 8 o'clock, the learning unit 52, if the user is at home, or if a private schedule is scheduled In the band, the output mode of the changed specific information is learned based on the content of the instruction received by the user and the instruction, and the instruction correspondence information 76 is generated based on the learning result. For example, when the user tends to output specific information with volume "10" in a predetermined environment pattern, the second output mode when volume change instruction is given in the instruction correspondence information 76 is volume "10". Set to

  According to the second embodiment described above, the provision control unit 50 changes the output mode of the specific information to the first output mode that is more difficult for the user to hear than the output mode of the response content, and causes the output unit to output it. After that, when the user's instruction is accepted, the output mode of the specific information is changed to a second output mode in which the user can easily hear more than the first output mode, and the specific information is output to the output unit. Information can be provided so as not to give the user a sense of discomfort.

  For example, if the voice advertisement is output as it is for the extension of the dialogue between the automatic response device and the user, it may seem annoying or regarded as stealth marketing (stemmer), but as in this embodiment, After the specific information is changed to the first output mode that is difficult for the user to hear and output to the output unit, the specific information is changed to the second output mode according to the user's instruction and output to the output unit. Suppress what you feel, or what is considered stealth marketing.

Third Embodiment
The third embodiment will be described below. The providing control unit 50 causes the output unit to output a conversation between the first character that outputs the response content and the second character that outputs the specific information. The differences from the first embodiment will be mainly described.

  FIG. 14 is a diagram illustrating an example of a functional configuration of the information processing system 1B according to the third embodiment. The information processing system 1B includes, for example, a terminal device 10B, an automatic response device 40B, and an advertisement providing device 80B.

  The terminal device 10B includes a display unit 15 and an image generation unit 19 in addition to the functional configuration of the terminal device 10 of the first embodiment. The display unit 15 displays an image based on the control of the image generation unit 19. The image generation unit 19 displays an image on the display unit 15 based on the information transmitted by the automatic response device 40B. For example, based on the information transmitted by the automatic response device 40B, the sound generation unit 18 and the image generation unit 19 may use the contents of the image displayed on the display unit 15 and the contents of the sound output on the speaker 14. The speaker 14 and the display unit 15 are controlled in a coordinated manner so that the intended timing is reached. Hereinafter, a combination of the sound generation unit 18 and the image generation unit 19 is referred to as a “generation unit 17”.

  In addition to the functional configuration of the automatic response device 40 of the first embodiment, the automatic response device 40B further includes an image providing unit 49, and includes a first storage unit 60B instead of the first storage unit 60 of the first embodiment. . For example, in addition to the information stored in the first storage unit 60 of the first embodiment, the first storage unit 60B further stores motion information 69. The motion information 69 is information in which the motion of the character talking with the user is defined. The image providing unit 49 provides the terminal device 10 with information for generating an image to be displayed on the terminal device 10B based on the information included in the motion information 69 or the information provided by the advertisement providing device 80B. The information for generating the image is associated with the utterance output to the speaker 14 with the timing for changing the image. Hereinafter, the combination of the response unit 48 and the image providing unit 49 is referred to as a “response providing unit 47”.

  The advertisement providing device 80B includes an advertisement providing device side storage unit 90B in place of the advertisement providing device side storage unit 90 of the first embodiment. The advertisement provision device side storage unit 90 includes, for example, advertisement information 92B. The advertisement information 92B further includes advertisement motion information 93 in addition to the information of the advertisement information 92 of the first embodiment. The advertisement motion information 93 is information that defines the movement of the character associated with the advertisement ID.

[flowchart]
FIG. 15 is a flowchart showing an example of the flow of processing executed by the automatic response device 40B. First, the response providing unit 47 causes a conversation between the first character and the second character (S80). Next, the advertisement providing unit 47 causes the second character to output advertisement information (S82).

  Next, the automatic response device 40B determines whether or not the user has input voice according to the output advertisement information (first specific information) (S84). Note that, instead of the voice, it may be determined whether a predetermined operation has been performed. When the user has not input a voice, the process of one routine of this flowchart ends.

  When the user inputs a voice, the automatic response apparatus 40B determines whether the user feels it bothersome to output the advertisement information (S86). “I feel bothersome” means, for example, that the information contained in the input voice has a negative meaning with respect to the output of the information of the advertisement. More specifically, for example, when an utterance having a meaning such as "make quiet", "stop", "drop the sound" is made, it is determined that the user feels troublesome. If the user does not feel bothersome, the processing of one routine of this flowchart ends. If the user does not feel troublesome at S86, the automatic response apparatus 40B causes the output unit to output second identification information that is more detailed than the first identification information. The detailed information is, for example, the explanatory content when the first specific information is a product name or a product attribute.

  When the user feels troublesome, the response providing unit 47 stops outputting the information of the advertisement (S88). In addition, it may replace with a stop and may change an output mode based on a user's reaction. For example, when the user inputs “turn down the sound”, the sound at which the advertisement information is output is controlled to be small. Thus, the processing of one routine of this flowchart ends.

  As described above, it is possible for the user to have more interest in information by causing the characters to have a conversation and output the information of the advertisement. In addition, since the output of information is suppressed according to the reaction of the user, the convenience for the user is improved.

[Specific example (3-1)]
FIG. 16 is a diagram (part 1) illustrating an example of an image displayed on the conversation and display unit 15 of the third embodiment. A provision control part makes a 1st character and a 2nd character have conversation based on the information provided to the user. For example, as shown in FIG. 16, (1) the second character CR2 utters "How is the weather today?" (2) The first character CR1 responds, "It is clear in the forecast."

  Next, (3) the second character CR2 utters, "It's a driving weather." Next, (4) the first character CR1 responds with "Yes." Next, (5) the second character CR2 utters, "If it says so, a car best suited for driving has been released."

  In this way, by letting the characters talk and introduce the product, the user can naturally have an interest in the product.

[Specific example (3-2)]
For example, based on the conversation between the first character and the user, the automatic response device 40B acquires preference information such as the user's preference, preference, and action schedule. The preference information is, for example, information such as a user's taste, a facility or place frequently used, a product frequently purchased, a product or service desired to be purchased, or the like.

  FIG. 17 is a diagram (part 2) illustrating an example of an image displayed on the conversation and display unit 15 according to the third embodiment. The provision control unit 50 inquires of the user whether, for example, the conversation information included in the conversation between the user and the first character is to be reflected in the content of the specific information output by the second character, and grants the user permission If it is obtained, the conversation information is reflected in the content of the specific information.

  For example, as shown in FIG. 17, (1) the first character CR1 utters "User A. Can you tell other people that User A is thinking of purchasing a car?" Assume that user A answers "Yes" to this utterance. (2) The first character CR1 responds, "It's good. I will tell other people. I will surely find a good car!" As described above, the first character provides information such as the user's interest and tendency to the second character, thereby optimizing the information output by the second character.

  FIG. 18 is a diagram (part 3) illustrating an example of an image displayed on the conversation and display unit 15 according to the third embodiment. After the response in (2) of FIG. 17 described above, the following conversation is performed at a predetermined timing. (1) While the second character CR2 is not displayed on the display unit 15, for example, it utters "I'm sorry." Next, (2) the first character CR1 responds with "Who is it?" Next, (3) the second character CR2 utters, "Are you going for a while?" Next, (4) the first character CR1 utters, "User A, I asked who I am. Can I insert it?" In response to this utterance, it is assumed that the user A responds as "I can insert it." Next, (5) the first character CR1 speaks "Please enter" according to the speech of the user A. Thereafter, the image shown in FIG. 19 is displayed on the display unit 15.

  FIG. 19 is a diagram (part 4) illustrating an example of an image displayed on the conversation and display unit 15 according to the third embodiment. (1) While the second character CR2 is displayed on the display unit 15, for example, he / she utters, "I asked for an introduction because I asked that I was looking for a car." Next, (2) the first character CR1 responds, "User A, do you want to hear a story?" When the user makes a positive utterance in response to this response, for example, the second character CR2 introduces a product. When the user makes a negative utterance in response to this response, for example, the second character CR2 disappears without introducing a product.

  As described above, when permission is obtained for the handling of preference information, advertisement information corresponding to the preference information of the user is output, so that it is possible to prevent the user from feeling annoying. It is possible to improve the convenience for the disabled.

  In the example described above, an example in which the first character CR1 and the second character CR2 talk is described, but instead (or in addition) the second character CR2 and the third character talk Good. The third character is, for example, a character promoting a product (or service) that competes (or is associated with) a product (or service) recommended by the second character CR2.

  According to the third embodiment described above, the provision control unit 50 causes the output unit to output the response content in the output mode according to the first character, and outputs the specific information in the output mode according to the second character. By causing the unit to output and causing the output unit to output the conversation between the first character and the second character, it is possible to make the user more interested in the information.

  In addition, in the information processing system 1 of each embodiment mentioned above, although the terminal device 10 demonstrated as what was one, two or more terminal devices 10 may be provided. In this case, the automatic response apparatus 40 acquires, for example, voice data input to the terminal apparatus 10 together with identification information of the apparatus from the first terminal apparatus 10 or the second terminal apparatus 10. Then, the automatic response apparatus 40 causes the first terminal device 10 to output the response content in the output mode of the first character with reference to the acquired identification information, and the second terminal device 10 outputs the output mode of the second character. To output specific information.

  According to the embodiment described above, the response unit for causing the output unit to output the response content for the voice emitted by the user and the specific information different from the response content, the output mode of the specific information, the response The output mode of the specific information is received when the user's instruction is received after changing to a first output mode in which the user is less likely to hear than the third output mode, which is the content mode, and causing the output unit to output Changing the information to a second output mode that is easier for the user to hear than the first output mode, and providing a control unit that causes the output unit to output the specific information so that the user does not feel discomfort Can be provided.

  As mentioned above, although the form for carrying out the present invention was explained using an embodiment, the present invention is not limited at all by such an embodiment, and various modification and substitution within the range which does not deviate from the gist of the present invention Can be added.

1, 1A, 1B: information processing system, 10: terminal device, 12: microphone, 14: speaker, 15: display unit, 16: voice recognition unit, 18: voice generation unit, 19: image generation unit, 40, 40A, DESCRIPTION OF SYMBOLS 40B ... Automatic response apparatus, 42 ... User identification part, 43 ... Environment analysis part, 46 ... Interpretation part, 48 ... Response part, 49 ... Image provision part, 50 ... Provision control part, 52 ... Learning part, 80 ... Advertisement offer Device, 82 ... information providing unit

Claims (19)

A response unit that outputs a response content to a voice uttered by the user and specific information different from the response content to the output unit;
The user's instruction was accepted after changing the output mode of the specific information to a first output mode in which the user is less likely to hear than the third output mode, which is the output mode of the response content, and causing the output unit to output In this case, the control unit causes the output unit to output the specific information by changing the output mode of the specific information to a second output mode in which the user can easily hear the user than the first output mode;
An information processing system comprising: When the control unit receives an instruction from the user, the control unit changes the output mode of the specific information based on the mode of the received instruction.
The information processing system according to claim 1. The control unit is configured to output the specific information output by the output unit in the first output mode when the user receives an instruction when the output unit outputs the specific information in the first output mode. Change the mode to the second output mode, and output again to the output unit,
The information processing system according to claim 1. The first output mode is an output mode in which the sound volume is smaller than in the third output mode, the frequency band of sound is difficult for the user to hear, or the tempo at which information is output is early.
The information processing system according to any one of claims 1 to 3. The control unit changes the output mode of the specific information to the first output mode based on the attribute of the specific information, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 4. The attribute of the specific information includes at least one of information on an advertisement, information on operation of a device, music, and information on a deadline related to a user,
The information processing system according to any one of claims 1 to 5. The specific information includes information on an advertisement,
The control unit changes the output mode of the specific information to the first output mode based on the type of the advertisement, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 6. The specific information includes information on an advertisement,
The control unit changes the output mode of the specific information to the first output mode based on the type of the advertisement and the result of the user's instruction given in the past, and outputs the specific information. To output to
The information processing system according to any one of claims 1 to 7. The control unit changes the output mode of the specific information to the first output mode based on the attribute of the specific information and the attribute of the specific information specified by the user in advance. Output to the output unit,
The information processing system according to any one of claims 1 to 8. The control unit changes the output mode of the specific information to the first output mode based on a time zone, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 9. The control unit changes the output mode of the specific information to the first output mode based on an environment in which the user exists, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 10. The control unit changes the output mode of the specific information to the first output mode based on the position where the user exists, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 11. The control unit changes the output mode of the specific information to the first output mode based on a person present around the user and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 12. The control unit changes the output mode of the specific information to the first output mode based on the user's schedule information, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 13. The control unit changes the output mode of the specific information to the first output mode based on the environmental sound of the environment in which the user is present, and causes the output unit to output the specific information.
The information processing system according to any one of claims 1 to 14. The user's instruction is performed by voice input to a user device for which voice is input, or by a predetermined operation.
The information processing system according to any one of claims 1 to 15. The control unit causes the output unit to output the response content in an output mode according to the first character, and causes the output unit to output the specific information in an output mode according to a second character.
The information processing system according to any one of claims 1 to 16. One or more computers
The response content for the voice uttered by the user and the specific information different from the response content are output to the output unit,
The user's instruction was accepted after changing the output mode of the specific information to a first output mode in which the user is less likely to hear than the third output mode, which is the output mode of the response content, and causing the output unit to output In this case, the output mode of the specific information is changed to a second output mode that is easier for the user to hear than the first output mode, and the specific information is output to the output unit.
Information processing method. On one or more computers,
The response content for the voice uttered by the user and the specific information different from the response content are output to the output unit,
The user's instruction was accepted after changing the output mode of the specific information to a first output mode in which the user is less likely to hear than the third output mode, which is the output mode of the response content, and causing the output unit to output In this case, the output mode of the specific information is changed to a second output mode that is easier for the user to hear than the first output mode, and the specific information is output to the output unit.
program.

Images