JP6774438B2 - Information processing systems, information processing methods, and programs - Google Patents

Description

The present invention relates to information processing systems, information processing methods, and programs.

Conventionally, during a navigation process in which a route search to a destination is performed and a guidance route is guided according to the search result, a voice advertisement or a questionnaire related to the voice advertisement is output by voice output means based on a dialogue with a user. An information processing device including an output control means for making the device is disclosed (for example, Patent Document 1).

JP-A-2017-58315

However, in the conventional technology, the audio output may give the user a sense of discomfort.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an information processing system, an information processing method, and a program capable of providing information so as not to give a sense of discomfort to the user. It is one of.

One aspect of the present invention is a response unit that outputs a response content to a voice emitted by a user and specific information different from the response content to an output unit, and a user device to which the voice is input or output. This is an information processing system including a control unit that controls an output mode of the specific information according to the degree of utilization of the information.

According to one aspect of the present invention, information can be provided so as not to give the user a sense of discomfort.

It is a figure which shows the structure of the information processing system 1. It is a flowchart which shows an example of the flow of processing executed by the information processing system 1. It is a figure which shows an example of the content of the environment pattern information 64. It is a figure which shows an example of the content of advertisement information 92. It is a figure which shows an example of the content of output degree information 72. It is a figure which shows an example of the conversation between a user and an automatic response device 40. It is a flowchart which shows an example of the flow of processing executed by the terminal apparatus 10 and the automatic response apparatus 40. It is a figure which shows an example of the content of the usage information 74. It is a figure which shows an example of the functional structure of the automatic response device 40A included in the information processing system 1A of the 2nd Embodiment. It is a flowchart which shows an example of the flow of processing executed by the terminal apparatus 10 and the automatic response apparatus 40A of 2nd Embodiment. It is a figure which shows an example of the content of instruction correspondence information 76. It is a figure which shows an example of the conversation between the user of 2nd Embodiment and the automatic response device 40. It is a figure which shows the change of the volume when the advertisement information is output. It is a figure which shows an example of the functional structure of the information processing system 1B of 3rd Embodiment. It is a flowchart which shows an example of the flow of processing executed by the automatic response device 40B. It is a figure (the 1) which shows an example of the conversation and the image displayed on the display part 15 of the 3rd Embodiment. FIG. 2 is a diagram (No. 2) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. FIG. 3 is a diagram (No. 3) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. FIG. 4 is a diagram (No. 4) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment.

Hereinafter, the information processing system, the information processing method, and the embodiment of the program of the present invention will be described with reference to the drawings.

<Overview (common matters)>
The information processing system is realized by one or more processors. The information processing system outputs the response content to the voice uttered by the user and specific information different from the response content to the output unit. The "response content" is information determined by an automatic response device that operates by a machine-learned model such as AI (Artificial Intelligence) or deep learning. The "specific information" is information that does not correspond to a response to a voice uttered by a user, such as an advertisement, a greeting, an utterance that triggers a conversation, or a notification (for example, a recommendation or a request for changing a password).

[Overview (1)]
The information processing system controls the output mode of specific information according to the degree of utilization of a user device (for example, a microphone or a speaker) whose voice is input or output. The "utilization degree" is, for example, a value based on the number or frequency of inputting voice to the user device, or a value based on the number of times or frequency of outputting voice to the user device. For example, the higher the usage of the user device, the larger the output amount of specific information. That is, a lot of talks and voice advertisements from the automatic response device are output to the user who usually uses voice input or output a lot. Further, the higher the degree of use of the user device, the easier it is to control the output mode of the specific information so that the user can easily hear it. The "output mode" is, for example, the loudness of a sound, the pitch of a sound, and the tempo at which information is output. The outline (No. 1) will be described focusing on the first embodiment described later.

[Overview (Part 2)]
When the information processing system receives the user's instruction after changing the output mode of the specific information to the first output mode that is harder for the user to hear than the third output mode of the response content and outputting it to the output unit. The output mode of the specific information is changed to the second output mode, and the specific information is output to the output unit. The "second output mode" is an output mode that is easier for the user to hear than the first output mode. That is, in the dialogue with the automatic response device, the volume of only specific information (for example, voice advertisement) is reduced, and the volume is increased in response to a request or operation from the user. The outline (No. 2) will be described focusing on the second embodiment described later.

[Summary (3)]
The information processing system outputs the response content to the output unit according to the output mode according to the first character, and outputs the specific information to the output unit according to the output mode according to the second character. Further, the information processing system causes the output unit to output the conversation between the first character and the second character. The "first character" is, for example, a character that interacts with the user and responds to the user's utterance in daily life. The "second character" is, for example, a character different from the first character and associated with specific information (for example, an advertisement). In this way, the first character interacting with the user and the second character corresponding to the voice advertisement arouse the user's interest in the advertisement by having a conversation. The outline (No. 3) will be described focusing on the third embodiment described later.

<First Embodiment>
[overall structure]
FIG. 1 is a diagram showing a configuration of an information processing system 1. The information processing system 1 includes, for example, a terminal device 10, an automatic response device 40, and an advertisement providing device 80. These devices communicate with each other via the network NW. The network NW includes, for example, WAN (Wide Area Network), LAN (Local Area Network), the Internet, a dedicated line, a wireless base station, a provider, and the like. In this embodiment, the automatic response device 40 is an example of an "information processing system". Further, the "information processing system" may include the terminal device 10 and / or the advertisement providing device 80.

[Functional configuration of terminal device]
The terminal device 10 is, for example, a smart speaker (Artificial intelligence speaker), a smartphone, a tablet terminal, a personal computer, or the like. In the first embodiment, the terminal device 10 will be described as being a smart speaker.

The terminal device 10 includes, for example, a microphone 12, a speaker 14, a voice recognition unit 16, a voice generation unit 18, a terminal control unit 20, a terminal device side communication unit 22, and a storage unit 30. In the voice recognition unit 16, the voice generation unit 18, and the terminal control unit 20, for example, a hardware processor such as a CPU (Central Processing Unit) stores an application program (application 32) stored in a storage unit 30 such as a flash memory. It is realized by executing. The application 32 may be downloaded from a server device or the like via a network, or may be pre-installed in the terminal device 10 in advance. Instead of the application program, a browser having the same functions as those described below may be used as the UA (User Agent). Note that some or all of the functions included in the terminal device 10 may be included in the automatic response device 40.

The microphone 12 acquires the voice emitted by the user or the environmental sound of the environment in which the terminal device 10 exists. The speaker outputs voice according to the information generated by the voice generation unit 18.

The voice recognition unit 16 converts the voice acquired by the microphone 12 into digital data (voice data). The voice generation unit 18 generates information according to the voice to be output to the speaker 14 based on the information transmitted by the automatic response device 40.

The terminal control unit 20 transmits the digital data converted by the voice recognition unit 16 to the automatic response device 40 by using the communication unit 22 on the terminal device side. The terminal control unit 20 acquires the information transmitted by the automatic response device 40 via the terminal device side communication unit 22.

The terminal device side communication unit 22 is, for example, a wireless communication interface. The terminal device side communication unit 22 acquires the information transmitted by the automatic response device 40, and transmits the processing result processed by the terminal device 10 to the automatic response device 40.

[Functional configuration of interactive voice response]
The automatic response device 40 includes, for example, a user identification unit 42, an environment analysis unit 43, a pattern identification unit 44, an interpretation unit 46, a response unit 48, a provision control unit 50, a learning unit 52, and a response device. It includes a side communication unit 54, a first storage unit 60, and a second storage unit 70. In the user identification unit 42, the environment analysis unit 43, the pattern identification unit 44, the interpretation unit 46, the response unit 48, the provision control unit 50, and the learning unit 52, for example, a hardware processor such as a CPU is stored in a storage device (for example, a first unit). It is realized by executing the program stored in 1 storage unit 60). Further, these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by the collaboration of software and hardware. Further, the above program may be stored in a storage device in advance, or is stored in a removable storage medium such as a DVD or a CD-ROM, and the storage medium is attached to the drive device of the automatic response device 40. It may be installed in the storage device. The first storage unit 60 and the second storage unit 70 are realized by, for example, a ROM (Read Only Memory), a flash memory, an SD card, a RAM (Random Access Memory), a register, or the like.

The first storage unit 60 stores, for example, user-specific information 62, environment-specific information 63, environment pattern information 64, regular expression information 66, and scenario information 68, which will be described later. For example, the output degree information 72 and the usage information 74, which will be described later, are stored in the second storage unit 70. The first storage unit 60 and the second storage unit 70 do not necessarily have to be realized by separate storage devices, and may be different storage areas in the integrated storage device.

For example, the user identification unit 42 extracts a component of voice data (hereinafter, utterance component) presumed to represent a human voice from the voice data transmitted by the terminal device 10. The user identification unit 42 collates the extracted utterance component with the information contained in the user identification information 62, and identifies the person who has emitted the voice represented by the extracted utterance component. The user identification information 62 is information in which user identification information is associated with information indicating the characteristics of the user's voice (for example, a voiceprint pattern or a frequency pattern).

Further, the user identification unit 42 may refer to the user identification information 62 and specify the type of a person existing in the vicinity of the user who has emitted the voice. In this case, the user identification information 62 includes information indicating the characteristics of the voice of the user's family and friends in advance. In addition, the user identification unit 42 acquires information indicating the connection status between the terminal device owned by the family and the Wi-Fi router via the communication unit 22 on the terminal device side, and based on the acquired information, the terminal The owner of the device may determine if it is near the location where the Wi-Fi router is installed.

For example, the environmental analysis unit 43 extracts a component of voice data (hereinafter, environmental sound component) that is presumed to represent an environmental sound other than a human voice from the voice data transmitted by the terminal device 10. The environmental analysis unit 43 collates the extracted environmental sound component with the information contained in the environmental identification information 63, and identifies the loudness of the environmental sound represented by the extracted environmental sound component and the cause of the environmental sound. To do. The environment-specific information 63 is information in which the identification information of the environmental sound generation factor and the sound characteristics of each environmental sound generation factor are associated with each other.

The pattern identification unit 44 specifies an environment pattern based on, for example, the environment pattern information 64, the processing result of the user identification unit 42, and the processing result of the environment analysis unit 43. The environment pattern is a pattern in which the environment in which the user exists is classified according to a predetermined standard. Details will be described later.

For example, the interpretation unit 46 converts the voice data corresponding to the human voice into text information, and further collates the text information with the regular expression information 66 to interpret the meaning of the user's utterance. For example, assume that the user utters "Tell me how to get from Shinjuku to Shibuya." The interpretation unit 46 morphologically analyzes the above utterance and divides the utterance into part of speech. Then, the interpretation unit 46 generates a search key in which Shinjuku and Shibuya corresponding to proper nouns and place names are converted into codes, and searches for regular expression information 68. Information (regular expression) obtained by converting a proper noun into an abstract code is registered in the regular expression information 68. For example, information with texts such as "Tell me how to get from XX to XX" and "Tell me how to get from XX to XX" is registered as a regular expression.

The response unit 48 acquires, for example, the text information corresponding to "Tell me how to get from (proper noun, place) to (proper noun, place)" included in the regular expression information 68, and from XX to XX. Recognize that you should provide directions for.

Then, the response unit 48 uses "I want to know how to get from Shinjuku to Shibuya" by embedding the encoded original information "Shinjuku" and "Shibuya" in the (proper noun, place) part. Recognize the intention of the person. The response unit 48 performs a network search and obtains directions from Shinjuku to Shibuya. The response unit 48 refers to, for example, the scenario information 68, and generates voice source information for output by the terminal device 10, which indicates the direction from Shinjuku to Shibuya. The scenario information 68 holds, for example, the content to be responded to the utterance of the user in advance. That is, the content of the response to the utterance in which the user has the intention of "knowing the way from XX to XX" is retained. The scenario information 68 is prepared for each user, for example, so that the response content matches the user's preference and the like.

Note that some or all of the functions included in the automatic response device 40 such as the response unit 48 may be provided in the terminal device 10. In addition, information such as regular expression information 66 and scenario information 68 may also be stored in the storage device of the terminal device 10.

The providing control unit 50 transmits the voice source information to the terminal device 10 by using the response device side communication unit 54 in order to output the voice source information generated by the response unit 48 to the terminal device 10. Further, the provision control unit 50 transmits the voice source information to the terminal device 10 by using the response device side communication unit 54 in order to output the voice source information transmitted by the advertisement providing device 80 to the terminal device 10. ..

Further, the provision control unit 50 specifies the output mode of the response content or the specific information, and in order to output the response content or the specific information to the speaker 14 of the terminal device 10 in the designated output mode, the designated output mode and the response content. Alternatively, the information associated with the specific information is transmitted to the terminal device 10 by using the communication unit 54 on the response device side. The function of the provided control unit 50 may be provided in the terminal device 10.

The learning unit 52 learns the content of the response or specific information output to the speaker 14 of the terminal device 10, the output mode of the output information, the reaction of the user, and the environmental pattern. Learning is, for example, learning using artificial intelligence, machine learning such as deep learning, and the like.

The communication unit 54 on the response device side includes a communication interface such as a network interface card. The response device side communication unit 54 acquires the information transmitted by the terminal device 10 or the advertisement providing device 80, and transmits the processing result processed by the automatic response device 40 to the terminal device 10 or the advertisement providing device 80. ..

[Advertising provider]
The advertisement providing device 80 includes, for example, an information providing unit 82, an advertisement providing device side communication unit 84, and an advertisement providing device side storage unit 90. The information providing unit 82 extracts an advertisement to be provided to the user based on the information input by the user's utterance or the response content of the automatic response device 40, and information about the extracted advertisement (for example, voice source information and voice). The output mode) is provided to the automatic response device 40.

The communication unit 84 on the advertisement providing device side includes a communication interface such as a network interface card. The advertisement providing device side communication unit 84 acquires the information transmitted by the automatic response device 40, and transmits the processing result processed by the advertisement providing device 80 to the automatic response device 40. The advertisement information 92, which will be described later, is stored in the advertisement providing device side storage unit 90. The advertisement providing device 80 and the automatic response device 40 may be provided as an integrated device.

[Flowchart (process to determine the degree of output)]
FIG. 2 is a flowchart showing an example of the flow of processing executed by the information processing system 1. This process is a process of controlling the amount of voice output from an artificial object according to the amount of voice UI (user interface / user device) used by the user. This voice UI is voice recognition.

First, the terminal device 10 determines whether or not the voice has been input by the user (S10). When voice is input by the user (when a conversation between the user and the automatic response device 40 is started), the input voice data (speech component and environmental sound component) is transmitted to the automatic response device 40. ..

The automatic response device 40 acquires the utterance component and identifies the user based on the acquired utterance component and the user identification information 62 (S20). The automatic response device 40 acquires an environmental sound component and identifies the environmental pattern based on the acquired environmental sound component and the environmental pattern information 64 (S22).

FIG. 3 is a diagram showing an example of the contents of the environment pattern information 64. The environment pattern information 64 is information in which a plurality of environment patterns and classification criteria are associated with each other. The classification criteria for environmental patterns are, for example, the day and time, the number of people around the user, the type of person, the loudness of the environmental sound in which the user exists, and the presence of the user. It is classified based on at least one item among the environment (home, office, city), the location where the user exists, the user's schedule (pre-registered current schedule), etc. It is a pattern.

The environment in which the user exists, the location in which the user exists, or the schedule of the user is, for example, information set in advance by the user. Further, the environment in which the user exists or the position where the user exists may be specified based on the information positioned by the positioning device using GPS (Global Positioning System) (not shown). .. Further, the user's schedule may be information acquired by the terminal device 10 from another device via the network NW.

Next, the automatic response device 40 requests the advertisement providing device 80 to determine the content of the advertisement to be provided to the specified user (S24). At this time, the automatic response device 40 transmits the text information obtained by converting the information contained in the voice input to the terminal device 10 into the text information to the advertisement providing device 80.

In response to the request of the automatic response device 40, the advertisement providing device 80 refers to the advertisement information 92 and determines the content of the advertisement to be provided to the user corresponding to the text information (S30). If the advertisement to be provided to the user does not exist, the advertisement providing device 80 transmits to that effect to the automatic response device 40.

FIG. 4 is a diagram showing an example of the content of the advertisement information 92. The advertisement information 92 is information in which a character, a product (or service), a scenario, and a keyword are associated with the advertisement ID. A "character" is a person having a predetermined characteristic, an animal, a plant, a creative work, an artificial object, or the like that looks like a person. The character may be provided for each product, for each of a plurality of products, or for each campaign.

The "scenario" defines the content and order of words (or words and actions) uttered by the character. The scenario is provided for each character, for example. Further, in the advertisement information 92, in addition to the scenario, character characteristics such as voice tone and tempo are associated with the character. Characters for each product or campaign act autonomously based on scenarios (action rules).

A "keyword" is a word associated with an advertisement. A [keyword] is a word having the same meaning as the meaning (semantic information) of a word indicating a product, or a word related to the meaning of a word indicating a product. Related words are words that are commonly recalled from words that indicate a product. For example, the advertisement providing device 80 is associated with a matching keyword when the word input by the user or the word included in the voice uttered by the automatic response device 40 matches the keyword of the advertisement information 92. Information corresponding to the advertisement ID (voice source information uttered by the character, etc.) is transmitted to the automatic response device 40. The advertisement providing device 80 may determine the information to be provided to the user by a machine-learned model such as artificial intelligence or deep learning.

Next, the automatic response device 40 determines the output degree of the advertisement according to the environment pattern with reference to the output degree information 72 described later, and instructs the terminal device 10 to output the advertisement at the determined output degree. (S26). Next, the terminal device 10 outputs an advertisement based on the instruction of the automatic response device 40 (S12). As a result, the processing of one routine of this flowchart is completed.

FIG. 5 is a diagram showing an example of the contents of the output degree information 72. The output degree information 72 is prepared for each environment pattern, for example. Further, the output degree information 72 is information in which the past usage degree in the environment pattern and the output degree for outputting the advertisement are associated with the user ID.

The "past usage degree" is the degree to which the user has received voice information (for example, advertisement) from the speaker 14 in the past, or the degree to which the user has input information to the microphone 12 by voice in the past. is there. The "output degree" is the loudness of the output sound when the information is output to the user by using the speaker 14. "Output degree" is an example of "output mode". The degree of output is set so that, for example, the greater the degree of use in the past, the louder the output sound. In addition, in "the degree of receiving information by voice from the speaker 14," the degree of outputting music may be excluded.

Further, in the output degree information 72, other modes related to output may be associated with the output degree information 72 instead of the output degree. The output aspect includes, for example, the loudness of the sound, the pitch of the sound, the tempo at which the content of the advertisement is output, and the like. The output aspect is set so that the user can easily hear the output, for example, the greater the degree of use in the past.

Further, when the environmental sound of the environment in which the user exists is equal to or larger than a predetermined loudness, the degree of change in the output mode of the specific information may be smaller than that when the environmental sound is less than the predetermined loudness. That is, in an environment where the environmental sound is originally loud, it is not necessary to increase the output of specific information.

As described above, the automatic response device 40 can provide information by referring to the output degree information 72 so as not to give a sense of discomfort to the user.

In the above description, as an example, when the user inputs voice, the output mode is controlled based on the degree of use, but the automatic response device 40 simply speaks or outputs information. In such a case, the output mode may be controlled based on the degree of utilization.

[Specific example (1)]
FIG. 6 is a diagram showing an example of a conversation between the user and the automatic response device 40. For example, as shown in FIG. 6A, (1) the user inputs "I want a new car" to the microphone 12.
(2) The automatic response device 40 responds with "what kind of car do you want?" In the output mode of the first character.

Next, as shown in FIG. 6 (B), (3) the user inputs to the microphone 12 "I like a car with good fuel economy." (4) The automatic response device 40 responds with the output mode of the first character, "It's good because it saves money." Then, (5) the automatic response device 40 utters "Car A has good fuel economy" in the output mode of the second character. The output mode of the second character is an output mode according to the degree of use of the user device.

Next, as shown in FIG. 6 (C), (6) the user inputs "Tell me in detail." To the microphone 12. (7) The automatic response device 40 responds in the output mode of the second character, "Car A is an electric vehicle. It can travel 000 km on a full charge."

In this way, when a keyword appears in the conversation between the first character and the user, the automatic response device 40 outputs an advertisement based on the keyword in an output mode according to the degree of use of the user device, and outputs an advertisement based on the keyword. And provide it to the user. As a result, it is possible to provide information so as not to give the user a sense of discomfort.

If the car A is not interested in the above (6), the second character does not have to speak after that. Further, if the car A is not interested, the other car may be introduced in the output mode of the character corresponding to the other car.

Further, when it is desired to provide an advertisement for a car, the automatic response device 40 may make an utterance or a response to the first character as if talking about the topic of the car. In this case, for example, the automatic response device 40 makes the above-mentioned keywords and utterances that induce the keywords. For example, a character conversation is selected based on the specific information to be output.

Further, in the above-described example, the output degree of the utterance of the second character is changed, but the output degree of the utterance of the first character may be changed. Further, the output degree may be changed based on the degree of conversation between the user and the character. For example, if the degree of conversation between the first character and the user is higher than the degree of conversation between the Nth character (N is an arbitrary natural number) and the user, the degree of conversation between the first character and the user is determined to be the first. Make it more than the N character talks to the user.

[Flowchart (learning process)]
FIG. 7 is a flowchart showing an example of the flow of processing executed by the terminal device 10 and the automatic response device 40. The processing of S40, S50, and S52 in the flowchart of FIG. 6 is the same as the processing of S10, S20, and S22 of the flowchart of FIG. 2, and thus the description thereof will be omitted.

After the processing of S52, the automatic response device 40 determines whether or not the own device has provided the information to the user (S54). When the information is provided to the user, the automatic response device 40 acquires the content of the provided information and the user's reaction after the information is provided, and stores the acquired reaction as the usage information 74 in the second storage unit 70. (S56).

FIG. 8 is a diagram showing an example of the contents of the usage information 74. The usage information 74 includes information input by the user in the past or information output to the user in the past, information input or output mode of the output information, and an environment for each user. The pattern and the user's reaction (for example, instruction) to the output information are information associated with each other.

Next, the automatic response device 40 determines whether or not a predetermined timing has been reached (S58). If the predetermined timing has not been reached, the processing of one routine of this flowchart ends. When the predetermined timing is reached, the automatic response device 40 learns the usage information 74 as learning data (S60). As a result, the processing of one routine of this flowchart is completed.

As described above, the user's preference can be grasped by learning the user's reaction when the information is provided to the user, the environmental pattern, the information output mode, and the content of the information. Then, the learning unit 52 can generate or update the output degree information 72 by reflecting the user's preference.

For example, on Saturdays, from 7:00 to 8:00, when there are parents around the user, when the user is at home, or when a private schedule is scheduled, other times. It is assumed that the usage information 74 includes information indicating that the specific information has been instructed to be output so as to suppress the situation more than in the case of the situation. In this case, the learning unit 52 generates the output degree information 72 so as to suppress the output of the specific information in the environment pattern corresponding to the above-mentioned situation.

As described above, the user may feel complicated when an artificial object such as a smart speaker that interacts with the user excessively voice advertisements or talks, but in the present embodiment, the voice interaction of the user is performed. Since the voice advertisement and the conversation are adjusted according to the degree of use and the situation in which the interaction is performed, it is possible to provide information so as not to give the user a sense of discomfort.

In the above-described example, the provision control unit 50 has been described as controlling the output mode of specific information according to the degree of use of the user device to which the voice is input or output. (Or in addition), it may be changed as follows. That is, the provision control unit 50 controls the output mode of the second response content according to the degree of use of the user device. This "second response content" is the content of the response to the voice uttered by the user and includes the advertisement. For example, in this case, the automatic response device 40 determines the response content including the advertisement, and causes the terminal device 10 to output the determined response content in an output mode according to the degree of use of the user device. In this way, since the response content itself becomes an advertisement and the control mode of the response content is controlled, it is possible to provide information so as not to give a sense of discomfort to the user.

According to the first embodiment described above, the provision control unit 50 controls the output mode of the specific information according to the degree of use of the user device to which the voice is input or output, thereby causing the user. Information can be provided so as not to give a sense of discomfort.

<Second Embodiment>
Hereinafter, the second embodiment will be described. The provision control unit 50 changes the output mode of the specific information to the first output mode, which is harder for the user to hear than the third output mode of the response content, outputs the output to the output unit, and then accepts the user's instruction. In this case, the output mode of the specific information is changed to the second output mode that is easier for the user to hear than the first output mode, and the specific information is output to the output unit. The differences from the first embodiment will be mainly described.

FIG. 9 is a diagram showing an example of the functional configuration of the automatic response device 40A included in the information processing system 1A of the second embodiment. The automatic response device 40A includes a second storage unit 70A instead of the second storage unit 70. The second storage unit 70A includes, for example, output degree information 72 and usage information 74, as well as instruction correspondence information 76 (details will be described later).

When the response unit 48 of the second embodiment outputs the specific information to the terminal device 10, the output mode of the specific information is changed to the first mode in which the user is harder to hear than the third output mode of the response content. The specific information is output to the terminal device 10.

After outputting the specific information to the terminal device 10 as described above, the automatic response device 40A makes it easier for the user to hear the application mode of the specific information than the first output mode when the user's instruction is received. The terminal device 10 is made to output the specific information by changing to the second output mode. The second output mode is, for example, a mode in which the volume is louder than the first output mode, the frequency band of the sound is easy for the user to hear, and the tempo at which the information is output is appropriate.

In the process of changing to the first mode, which is difficult for the user to hear, when the environmental sound of the environment in which the user exists is equal to or larger than the predetermined loudness, the specific information is more important than when the environmental sound is less than the predetermined loudness. The output mode may not be changed, or the degree of change in the output mode may be reduced. This is because even if the output mode is changed in an environment where the environmental sound is originally loud, the influence on the user is small.

[flowchart]
FIG. 10 is a flowchart showing an example of a processing flow executed by the terminal device 10 and the automatic response device 40A of the second embodiment. This process is a process executed after the specific information is output in the first output mode. The processing of S60, S70, and S72 in the flowchart of FIG. 10 is the same as the processing of S10, S20, and S22 of the flowchart of FIG. 2, and thus the description thereof will be omitted.

Next, the automatic response device 40A refers to the instruction correspondence information 76, and matches the combination of the specified user, the specified environment pattern, and the information (content of the instruction) included in the input voice. The output mode of the advertisement information is determined (S74). The content of the instruction is the information of the instruction requested by the user regarding the output of the information. The content of the instruction is, for example, raising the volume, slowly outputting information, outputting information with a high sound, outputting information output a few seconds ago, or a combination thereof. ..

FIG. 11 is a diagram showing an example of the contents of the instruction correspondence information 76. The instruction correspondence information 76 is information used for determining in what output mode the information is output in response to the instruction given by the user. The instruction correspondence information 76 is, for example, information in which a user ID, an instruction content, and an output mode are associated with each other for each environment pattern.

Next, the automatic response device 40A instructs the terminal device 10 to output the advertisement information in the determined output mode (S76). Next, the terminal device 10 causes the speaker 14 to output the advertisement information in the determined output mode based on the instruction of the automatic response device 40A (S62). As a result, the processing of one routine of this flowchart is completed.

As described above, since the automatic response device 40 changes the output mode according to the request of the user, it is possible to provide information so as not to give the user a sense of discomfort.

[Specific example (No. 2-1)]
FIG. 12 is a diagram showing an example of a conversation between the user of the second embodiment and the automatic response device 40A. For example, as shown in FIG. 12 (A), (1) the user inputs "I want a new car" to the microphone 12. (2) The automatic response device 40A responds with "what kind of car do you want?" In the output mode of the first character.

Next, as shown in FIG. 12 (B), (3) the user inputs to the microphone 12 "I like a car with good fuel economy." (4) The automatic response device 40A responds with "It's good because it saves money" in the output mode of the first character.

Next, for example, when the user does not speak for about several seconds, (5) the automatic response device 40A is the output mode of the second character and in the first output mode, as shown in FIG. 12 (C). , "I recommend car A ...."

(6) The user was interested in the information output in (5) above, but could not hear it because the volume was low, so he said, "I can't hear it." Then, (7) the automatic response device 40A is the output mode of the second character, and the volume is raised to output the information output in (5) above to the terminal device 10. That is, the second character speaks again, "I recommend car A ....".

In this way, the output mode when the second character outputs the information is harder for the user to hear than the output mode when the first character outputs the information, which makes the user feel annoyed. Can be suppressed. In addition, the convenience for the user can be improved by making the output mode when the second character outputs the information in response to the user's request easy for the user to hear.

In the above description, as an example, when the user inputs voice, the specific information is output. However, when the automatic response device 40A simply outputs the specific information, as described above. The output mode may be controlled. Further, for example, a conversation between the first character and the second character may be selected based on the specific information to be output.

[Specific example (2-2)]
FIG. 13 is a diagram showing a change in volume when advertising information is output. The vertical axis of FIG. 13 indicates loudness, and the horizontal axis of FIG. 13 indicates time. The length (time) of each of the advertisements A to C described below is, for example, a predetermined second (for example, about 15 seconds). It is assumed that the advertisement information is to be output in the order of advertisements A to C. In this case, for example, the user instructed to raise the volume while the advertisement A was output, the advertisement B was output, and the content of the advertisement B was being output (time T in FIG. 13). At time T, the automatic response device 40A causes the terminal device 10 to output the content of the advertisement B from the beginning. That is, the advertisement B is re-output from the portion that goes back a predetermined time or the portion where the volume is reduced. Further, after that, the automatic response device 40A may lower the volume before raising the volume after the content of the advertisement B is output as shown in the figure, or may keep the volume raised.

As described above, when the automatic response device 40A is instructed by the user, the advertisement output at the time of the instruction is output from the beginning, so that the user can acquire desired information. ..

In the above example, the content B is output from the beginning based on the user's instruction, but the content B may be output from the beginning of the advertisement A, or a predetermined time from the time when the user's instruction is given. You may output from the information that was output before. In addition, the information to be re-output may be determined based on the content of the user's utterance (for example, the degree of urgency). In addition, the automatic response device 40A determines how long before the advertisement is replayed from the time when the user's instruction is given, based on the tendency of the user's instruction in the past or the preset conditions. May be good.

[Other]
The provision control unit 50 may change the output mode of the specific information to the first output mode and output the specific information to the output unit based on the attribute of the specific information. The attribute of the specific information includes at least one of information about advertisement, information about operation of a device, music, and information about a deadline related to a user (information such as a password change deadline). For example, the provision control unit 50 does not have to change the output mode of the information related to the advertisement to the first output mode, and the specific information of other attributes does not change the output mode.

The provision control unit 50 may change the output mode of the specific information to the first output mode based on the type of the advertisement and output the specific information to the output unit. The type of advertisement is, for example, the type of product corresponding to the advertisement. For example, the provision control unit 50 changes the output mode of the car advertisement to the first output mode, but outputs the output mode of the real estate advertisement to the output unit without changing to the first output mode. You may.

Further, the provision control unit 50 changes the output mode of the specific information to the first output mode based on the type of advertisement and the result of the instruction given by the user in the past, and sends the specific information to the output unit. It may be output. For example, the learning unit 52 learns the type of advertisement and the result of the user's instruction given in the past. For example, the learning unit 52 learns that when a car advertisement is output, the user has instructed to increase the volume, but when a real estate advertisement is output, the user has instructed to decrease the volume. In this case, for example, the provision control unit 50 changes the output mode of the car advertisement to the first output mode, but does not change the output mode of the real estate advertisement to the first output mode. May be output to.

Further, the provision control unit 50 may adopt the above-mentioned concept and change the output mode of the specific information to the first output mode based on the environment pattern corresponding to the user. For example, in a certain environment, the learning unit 52 learns that it is preferable for the user to output specific information in the first output mode. The provision control unit 50 outputs specific information in the first output mode based on the learning result.

Further, the output mode of the information specified by the user (for example, the information of a predetermined attribute) is changed to the first output mode, and the output mode of the information not specified is not changed to the first output mode. May be good.

Further, the instruction correspondence information 76 is generated by the learning unit 52. For example, after the specific information is output to the output unit in the first output mode, the learning unit 52 outputs the specific information output mode changed based on the content of the instruction received by the user and the instruction for each environment pattern. learn. Then, the learning unit 52 learns how the output mode of the specific information is changed in the predetermined environment pattern, and generates the instruction correspondence information 76 that matches the preference of the user.

For example, the learning unit 52 may be on Saturday, from 7:00 to 8:00, when there are parents around the user, when the user is at home, or when a private schedule is scheduled. In the band, the content of the instruction received by the user and the output mode of the specific information changed based on the instruction are learned, and the instruction correspondence information 76 is generated based on the learning result. For example, when the user tends to output specific information on the volume "10" in a predetermined environment pattern, the second output mode when the volume change instruction is given in the instruction correspondence information 76 is the volume "10". Is set to.

According to the second embodiment described above, the provision control unit 50 changes the output mode of the specific information to the first output mode that is harder for the user to hear than the output mode of the response content, and causes the output unit to output the information. Later, when the user's instruction is received, the output mode of the specific information is changed to the second output mode that is easier for the user to hear than the first output mode, and the specific information is output to the output unit. Information can be provided so as not to give the user a sense of discomfort.

For example, if a voice advertisement is output as it is as an extension of the dialogue between the automatic response device and the user, it may seem annoying or may be regarded as stealth marketing (stemmer), but as in the present embodiment, It is troublesome to change the specific information to the first output mode, which is difficult for the user to hear, and output it to the output unit, and then change the specific information to the second output mode and output it to the output unit according to the user's instruction. Suppress what makes you feel or what is considered stealth marketing.

<Third Embodiment>
Hereinafter, the third embodiment will be described. The provision control unit 50 causes the output unit to output a conversation between the first character that outputs the response content and the second character that outputs the specific information. The differences from the first embodiment will be mainly described.

FIG. 14 is a diagram showing an example of the functional configuration of the information processing system 1B of the third embodiment. The information processing system 1B includes, for example, a terminal device 10B, an automatic response device 40B, and an advertisement providing device 80B.

The terminal device 10B further includes a display unit 15 and an image generation unit 19 in addition to the functional configuration of the terminal device 10 of the first embodiment. The display unit 15 displays an image under the control of the image generation unit 19. The image generation unit 19 causes the display unit 15 to display an image based on the information transmitted by the automatic response device 40B. For example, the voice generation unit 18 and the image generation unit 19 have the content of the image displayed on the display unit 15 and the content of the voice output to the speaker 14 based on the information transmitted by the automatic response device 40B. The speaker 14 and the display unit 15 are controlled in cooperation with each other so as to have an intended timing. Hereinafter, the combination of the voice generation unit 18 and the image generation unit 19 will be referred to as a “generation unit 17”.

The automatic response device 40B includes an image providing unit 49 in addition to the functional configuration of the automatic response device 40 of the first embodiment, and includes a first storage unit 60B in place of the first storage unit 60 of the first embodiment. .. In the first storage unit 60B, for example, in addition to the information stored in the first storage unit 60 of the first embodiment, motion information 69 is further stored. The motion information 69 is information in which the movement of the character talking with the user is defined. The image providing unit 49 provides the terminal device 10 with information for generating an image displayed on the terminal device 10B based on the information included in the motion information 69 or the information provided by the advertisement providing device 80B. The information for generating the image is associated with the timing of changing the image with respect to the utterance output to the speaker 14. Hereinafter, the combination of the response unit 48 and the image providing unit 49 will be referred to as a “response providing unit 47”.

The advertisement providing device 80B includes an advertisement providing device side storage unit 90B instead of the advertisement providing device side storage unit 90 of the first embodiment. The advertisement providing device side storage unit 90 includes, for example, advertisement information 92B. The advertisement information 92B further includes the advertisement motion information 93 in addition to the information of the advertisement information 92 of the first embodiment. The advertisement motion information 93 is information that defines the movement of the character associated with the advertisement ID.

[flowchart]
FIG. 15 is a flowchart showing an example of the flow of processing executed by the automatic response device 40B. First, the response providing unit 47 causes the first character and the second character to have a conversation (S80). Next, the advertisement providing unit 47 causes the second character to output the information of the advertisement (S82).

Next, the automatic response device 40B determines whether or not the user has input the voice according to the output advertisement information (first specific information) (S84). In addition, instead of voice, it may be determined whether or not a predetermined operation has been performed. If the user has not input voice, the processing of one routine of this flowchart ends.

When the user inputs a voice, the automatic response device 40B determines whether or not the user feels annoyed with the output of the advertisement information (S86). "I feel annoyed" means that, for example, the information contained in the input voice has a negative meaning with respect to the output of the information of the advertisement. More specifically, for example, when an utterance having a meaning such as "quiet", "stop", or "lower the sound" is made, it is determined that the user feels annoyed. If the user does not find it bothersome, the processing of one routine in this flowchart ends. If the S86 does not feel annoying, the automatic response device 40B causes the output unit to output the second specific information, which is more detailed information than the first specific information. The detailed information is, for example, descriptive content when the first specific information is a product name or an attribute of a product.

When the user feels annoyed, the response providing unit 47 stops outputting the advertisement information (S88). Instead of stopping, the output mode may be changed based on the reaction of the user. For example, when the user inputs "lower the sound", the sound at which the advertisement information is output is controlled to be small. As a result, the processing of one routine of this flowchart is completed.

As described above, by having the characters talk with each other and output the information of the advertisement, it is possible to make the user more interested in the information. In addition, since the output of information is suppressed according to the reaction of the user, the convenience for the user is improved.

[Specific example (No. 3-1)]
FIG. 16 is a diagram (No. 1) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. The provision control unit causes the first character and the second character to have a conversation based on the information provided to the user. For example, as shown in FIG. 16, (1) the second character CR2 utters "How is the weather today?" (2) The first character CR1 responds, "It's fine in the forecast."

Next, (3) the second character CR2 utters, "It's a good day to drive." Next, (4) the first character CR1 responds with "That's right." Next, (5) the second character CR2 says, "By the way, the best car to drive has been released."

In this way, by having the characters talk to each other and introducing the product, the user can naturally become interested in the product.

[Specific example (3-2)]
For example, the automatic response device 40B acquires preference information such as a user's preference, preference, and action schedule based on a conversation between the first character and the user. The preference information is, for example, information such as a user's hobby, a facility or place frequently used, a product frequently purchased, a product or service desired to be purchased, and the like.

FIG. 17 is a diagram (No. 2) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. The provision control unit 50 asks the user whether or not to reflect the conversation information included in the conversation between the user and the first character in the content of the specific information output by the second character, and grants the user permission. When is obtained, the conversation information is reflected in the content of the specific information.

For example, as shown in FIG. 17, (1) the first character CR1 utters "User A. Can I tell other people that User A is thinking about purchasing a car?" It is assumed that user A replies "OK" to this utterance. (2) The first character CR1 responds, "I like it. I'll tell other people. I'm sure you'll find a good car!" In this way, the first character provides the second character with information such as the user's interests and tendencies, thereby optimizing the information output by the second character.

FIG. 18 is a diagram (No. 3) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. After the response of (2) of FIG. 17 described above, the following conversation is performed at a predetermined timing. (1) The second character CR2 utters "I'm sorry." In a state where it is not displayed on the display unit 15, for example. Next, (2) the first character CR1 responds with "Who is it?". Next, (3) the second character CR2 utters, "Are you sure you have some time?" Next, (4) the first character CR1 says, "User A, someone has asked. Can I put it in?" It is assumed that User A replied, "You can put it in." To this utterance. Next, (5) the first character CR1 utters "Please enter" in response to the utterance of user A. After that, the image shown in FIG. 19 is displayed on the display unit 15.

FIG. 19 is a diagram (No. 4) showing an example of an image displayed on the conversation and display unit 15 of the third embodiment. (1) The second character CR2, for example, is displayed on the display unit 15, and says, "I heard that you are looking for a car, so I came to introduce you." Next, (2) the first character CR1 responds with "User A, would you like to hear from us?" When the user makes a positive utterance in response to this response, for example, the second character CR2 introduces the product. When the user makes a negative utterance in response to this response, for example, the second character CR2 disappears without introducing the product.

In this way, regarding the handling of preference information, when permission is obtained, advertisement information according to the user's preference information is output, so that the user can be used while suppressing the annoyance. It is possible to improve the convenience for the person.

In the above-mentioned example, an example in which the first character CR1 and the second character CR2 talk to each other has been described, but instead (or in addition), the second character CR2 and the third character may talk to each other. Good. The third character is, for example, a character that promotes a product (or service) that competes with (or is related to) a product (or service) recommended by the second character CR2.

According to the third embodiment described above, the provision control unit 50 causes the output unit to output the response content according to the output mode according to the first character, and outputs the specific information according to the output mode according to the second character. By outputting to the output unit and outputting the conversation between the first character and the second character to the output unit, it is possible to further arouse the user's interest in information.

Although the information processing system 1 of each of the above-described embodiments has been described as having one terminal device 10, two or more terminal devices 10 may be provided. In this case, the automatic response device 40 acquires voice data input to the terminal device 10 together with identification information of the device from, for example, the first terminal device 10 or the second terminal device 10. Then, the automatic response device 40 refers to the acquired identification information, causes the first terminal device 10 to output the response content in the output mode of the first character, and causes the second terminal device 10 to output the response content in the output mode of the second character. To output specific information with.

According to the embodiment described above, the response unit for outputting the response content to the voice emitted by the user and the specific information different from the response content to the output unit, and the voice are the targets of input or output. By providing a control unit that controls the output mode of the specific information according to the degree of use of the user device, it is possible to provide information so as not to give a sense of discomfort to the user.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1,1A, 1B ... Information processing system, 10 ... Terminal device, 12 ... Microphone, 14 ... Speaker, 15 ... Display unit, 16 ... Voice recognition unit, 18 ... Voice generation unit, 19 ... Image generation unit, 40, 40A, 40B ... Automatic response device, 42 ... User identification unit, 43 ... Environmental analysis unit, 46 ... Interpretation unit, 48 ... Response unit, 49 ... Image provision unit, 50 ... Provision control unit, 52 ... Learning unit, 80 ... Advertisement provision Equipment, 82 ... Information provision department

Claims (20)

A response unit that outputs the response content to the voice emitted by the user and an advertisement that is specific information different from the response content to the output unit that outputs the voice as voice .
The voice stored in the storage device refers to information indicating the usage degree of the user device whose input or output is the target, and the response unit outputs the advertisement to the output unit according to the reference usage degree . Equipped with a control unit that controls the output frequency
When the control unit acquires information indicating that the volume included in the voice data corresponding to the voice emitted to the user is increased while the response unit is outputting the advertisement to the output unit. , Instruct the response unit to output the advertisement from the beginning,
The response unit outputs the advertisement to the output unit from the beginning.
Information processing system. When the control unit acquires the information indicating that the volume is to be increased, in addition to instructing the response unit to output the advertisement from the beginning, the volume of the advertisement to be output from the beginning is set to the volume. Instruct the response unit to raise the volume higher than before acquiring the information indicating that the volume should be raised.
The response unit raises the volume of the voice when the advertisement is output to the output unit from the beginning, and outputs the advertisement to the output unit from the beginning.
The information processing system according to claim 1. The control unit refers to the degree of use of the user device whose voice stored in the storage device is the target of input or output, and the higher the degree of use, the greater the degree of output of the specific information.
The information processing system according to claim 1 or 2 . The usage includes a value based on the number or frequency of voice inputs to the user device by the user.
The information processing system according to any one of claims 1 to 3 . The degree of utilization includes a value based on the number of times or frequency that the user has made the user device output voice.
The information processing system according to any one of claims 1 to 4 . The use degree is a degree of speech is output to the user device for advertising,
The information processing system according to any one of claims 1 to 5 . The use degree, except for degree of music is output to the user device,
The information processing system according to any one of claims 1 to 5. The advertisement is information on a product having the same meaning as the voice uttered by the user or the words contained in the response content, or information on the product related to the voice or the words contained in the response content.
The information processing system according to any one of claims 1 to 7 . The control unit controls the output mode of the specific information based on the usage degree of the user device whose voice by the user stored in the storage device is the target of input or output and the time zone.
The information processing system according to any one of claims 1 to 8. The control unit receives the specific information based on the degree of use of the user device whose voice by the user stored in the storage device is the target of input or output and the environment in which the user exists. Control the output mode,
The information processing system according to any one of claims 1 to 9. The control unit sets the specific information based on the degree of use of the user device whose voice by the user stored in the storage device is the target of input or output and the position where the user exists. Control the output mode,
The information processing system according to any one of claims 1 to 10. The control unit identifies the user device based on the degree of use of the user device for which the voice of the user stored in the storage device is input or output, and a person existing in the vicinity of the user. Control the output mode of information,
The information processing system according to any one of claims 1 to 11. The control unit determines an output mode of the specific information based on the degree of use of the user device whose voice by the user is input or output stored in the storage device and the schedule information of the user. Control,
The information processing system according to any one of claims 1 to 12. The control unit has the specific information based on the degree of use of the user device whose voice by the user is input or output stored in the storage device and the environmental sound of the environment in which the user exists. Control the output mode of
The information processing system according to any one of claims 1 to 13. The control unit makes the degree of change in the output mode of the specific information smaller when the environmental sound in the environment in which the user is present is equal to or larger than a predetermined loudness, as compared with the case where the environmental sound is less than a predetermined loudness. ,
The information processing system according to claim 14. The control unit controls the output mode of the specific information so that the user can easily hear the output mode of the specific information as the degree of use of the user device whose voice by the user is input or output is higher.
The information processing system according to any one of claims 1 to 15. The output mode of the specific information is the loudness of the sound, the pitch of the sound, or the tempo at which the specific information is output.
The information processing system according to any one of claims 9 to 16. The control unit outputs the response content to the output unit according to the output mode according to the first character, and outputs the specific information to the output unit according to the output mode according to the second character.
The information processing system according to any one of claims 1 to 17. One or more computers
A process in which the voice stored in the storage device refers to information indicating the usage level of the user device whose input or output is the target, and
Depending on the degree of use referred to above, the response unit that outputs the response content to the voice emitted by the user and the advertisement that is specific information different from the response content to the output unit that outputs the voice is controlled. The process of controlling the output frequency of the advertisement to be output to the output unit ,
When the response unit acquires information indicating that the volume included in the voice data corresponding to the voice emitted to the user is increased while the advertisement is being output to the output unit, the response unit receives information. A process of instructing the output of the advertisement from the beginning and outputting the advertisement from the beginning to the output unit by controlling the response unit.
Information processing method comprising a. On one or more computers
A process in which the voice stored in the storage device refers to information indicating the usage level of the user device whose input or output is the target, and
Depending on the degree of use referred to above, the response unit that outputs the response content to the voice emitted by the user and the advertisement that is specific information different from the response content to the output unit that outputs the voice is controlled. The process of controlling the output frequency of the advertisement to be output to the output unit ,
When the response unit acquires information indicating that the volume included in the voice data corresponding to the voice emitted to the user is increased while the advertisement is being output to the output unit, the response unit receives information. A process of instructing the output of the advertisement from the beginning and outputting the advertisement from the beginning to the output unit by controlling the response unit.
A program that executes .

Images