JP2021047507A - Notification system, notification control device, notification control method, and notification control program - Google Patents

Abstract

To provide a notification system, a notification control device, a notification control method, and a notification control program which enable content preferred by an occupant of a vehicle to be provided according to an ambient environment of the occupant of the vehicle.SOLUTION: A notification system comprises: a first acquisition unit which acquires voice data showing a voice uttered by an occupant; an utterance content interpretation unit which interprets a content of an utterance of the occupant on the basis of the voice data; a second acquisition unit which acquires ambient environment information; a third acquisition unit which outputs the content of the utterance and the ambient environment information to recommendation systems which make content recommendation to the occupant to acquire recommendation information showing one or more pieces of content; a notification control unit which causes an output unit provided in a moving body to perform notification of a plurality of pieces of recommendation information acquired by the third acquisition unit from each of the recommendation systems; and a provision unit which, when selection operation is accepted by an operation unit, causes the output unit to output the content corresponding to any selected recommendation information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a notification system, a notification control device, a notification control method, and a notification control program.

Conventionally, a technique for selecting a content system that provides user-favorite content according to information relating to the user has been disclosed (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2014-115897

Here, the user's preference may change depending on the user's surrounding environment and the place where the user is. However, with the conventional technology, it has been difficult to provide user-favorite content according to the user's surrounding environment.

The present invention has been made in consideration of such circumstances, and is a notification system, a notification control device, and a notification control method capable of providing favorite contents of a vehicle occupant according to the surrounding environment of the vehicle occupant. , And one of the purposes is to provide a notification control program.

The notification system, the notification control device, the notification control method, and the notification control program according to the present invention have adopted the following configurations.
(1) The notification system according to one aspect of the present invention includes a first acquisition unit that acquires voice data indicating voice spoken by an occupant on a moving body, and a first acquisition unit that acquires voice data of the occupant based on the voice data. The content of the utterance and the surroundings are provided to the utterance content interpretation unit that interprets the content, the second acquisition unit that acquires the ambient environment information related to the occupant's surrounding environment, and one or more recommendation systems that recommend the content to the occupant. The third acquisition unit that outputs environmental information and acquires recommendation information indicating one or more contents from each of the recommendation systems, and the recommendation unit acquired by the third acquisition unit to the output unit included in the moving body. When the operation of selecting one of the notification control unit for notifying a plurality of the recommendation information for each system and the recommendation information notified by the output unit is accepted by the operation unit included in the moving body, it is selected. It is provided with a providing unit for outputting the content corresponding to any of the recommended information to the output unit.

In the notification system according to the aspect (1), the notification control unit has a preference of showing the recommendation information and the preference of the occupant among the one or more contents corresponding to the recommendation information. Based on the information, the output unit notifies the content that matches the taste of the occupant.

The aspect (3) is a history in which the notification system according to the aspect (1) or (2) associates the recommendation information notified by the output unit with the ambient environment information related to the recommendation information. Based on the generation unit that generates information, the ambient environment information newly acquired by the second acquisition unit, and the history information generated by the generation unit, new content for recommending to the occupant is newly provided. The notification control unit further includes a selection unit to be selected, and causes the output unit to newly notify the content selected by the selection unit.

The aspect (4) is the case where in the notification system according to the aspect (3), the operation unit receives an operation in which the providing unit does not select one or more of the contents notified by the output unit. The output unit is not provided with one or more of the contents indicated by the recommendation information, and the generation unit generates historical information to the effect that the output unit did not provide the contents.

The aspect (5) is the notification system according to any one of the above (1) to (4), wherein the surrounding environment information includes information targeting the environment around the moving body and the occupant. Information that targets the surrounding environment is included, and the information that targets the environment around the moving body includes the weather, climate, temperature or humidity at the point where the moving body exists, or the information of the moving body. The POI information of the points existing in the vicinity is included, and the information targeting the environment around the occupant includes the date and time or day of the speech, the number of the occupants by age group, the number of persons by gender, or the total number of persons. Or, the situation of the occupants is included.

The aspect (6) is the notification system according to any one of the above aspects (1) to (5), wherein the third acquisition unit further bases the recommendation system on a query similar to the content of the utterance. The recommendation information that recommends the content is acquired.

The aspect (7) is a notification system according to any one of the above aspects (1) to (6), in which one or more of the recommendation systems are operated by operators different from each other.

(8) The notification control device of another aspect of the present invention includes a first acquisition unit that acquires voice data indicating voice spoken by an occupant on a moving body, and the occupant based on the voice data. The content of the utterance and the content of the utterance are added to the utterance content interpretation unit that interprets the content of the utterance, the second acquisition unit that acquires the ambient environment information related to the occupant's surrounding environment, and one or more recommendation systems that recommend the content to the occupant. The third acquisition unit acquired the surrounding environment information and acquired the recommendation information indicating one or more contents from each of the recommendation systems, and the output unit included in the moving body. The operation unit included in the moving body accepts an operation of selecting the recommendation information from the notification control unit for notifying a plurality of the recommendation information for each recommendation system and the recommendation information notified by the output unit. If so, it includes a providing unit that outputs the content corresponding to any of the selected recommendation information to the output unit.

(9) The notification control method of another aspect of the present invention includes a first acquisition process of acquiring voice data indicating voice spoken by an occupant aboard a moving body, which is executed by one or more computers. The utterance content interpretation process that interprets the utterance content of the occupant based on the voice data, the second acquisition process that acquires the ambient environment information related to the occupant's surrounding environment, and one or more that recommends the content to the occupant. A third acquisition process including an output step of outputting the content of the utterance and the surrounding environment information to the recommendation system, and an acquisition step of acquiring recommendation information indicating one or more contents from each of the recommendation systems, and the above-mentioned. A notification process for notifying the output unit of the moving body of a plurality of the recommendation information for each of the recommendation systems acquired by the third acquisition process, and the recommendation of any one of the recommendation information notified by the output unit. When the operation of selecting information is accepted by the operation unit included in the moving body, it has an output process for outputting the content corresponding to any of the selected recommendation information to the output unit.

(10) The notification control program of another aspect of the present invention is a program to be installed on one or more computers, and acquires voice data indicating voice spoken by an occupant on a moving body. The first acquisition process, the utterance content interpretation process that interprets the utterance content of the occupant based on the voice data, the second acquisition process that acquires the ambient environment information related to the occupant's surrounding environment, and the occupant Includes an output step of outputting the content of the utterance and the surrounding environment information to one or more recommendation systems that recommend the content, and an acquisition step of acquiring recommendation information indicating one or more contents from each of the recommendation systems. A notification process for notifying a third acquisition process, a plurality of the recommendation information for each recommendation system acquired by the third acquisition process, and an output unit included in the moving body, and the recommendation information notified by the output unit. When the operation of selecting one of the recommended information is accepted by the operation unit included in the moving body, the output process for outputting the content corresponding to the selected recommended information to the output unit, and the output process. Is to be executed by the computer.

According to (1) to (10), it is possible to provide the favorite content of the vehicle occupant according to the surrounding environment of the vehicle occupant.

According to (2), it is possible to provide more favorite contents of the occupants of the vehicle.

According to (3), it is possible to provide content according to the preference of the occupants of the vehicle so far.

According to (4), it is possible to prevent the provision of content that does not suit the tastes of the occupants of the vehicle.

According to (5) to (6), many content options can be presented to the occupants of the vehicle.

It is a block diagram of the agent system 1 including the agent apparatus 100. It is a figure which shows the structure of the agent apparatus 100 which concerns on embodiment, and the apparatus mounted on the vehicle M. It is a figure which shows the structure of the agent server 200 which concerns on embodiment, and a part of the structure of agent apparatus 100. It is a figure which shows an example of the structure of the content server 300-1. It is a figure which shows an example of the 1st notification image IMa1 scene used for the notification of the recommendation information. It is a figure which shows an example of the 2nd notification image IMa2 scene used for the notification of the recommendation information. It is a flowchart which shows an example of a series of operations of the agent system 1 which concerns on embodiment. It is a flowchart which shows an example of a series of operations of the agent system 1 which concerns on embodiment. It is a figure which shows an example of the contents of the history information 152. It is a figure which shows an example of the configuration of the agent apparatus 100A which has the function of the agent server 200.

Hereinafter, embodiments of the notification system, the notification control device, the notification control method, and the notification control program of the present invention will be described with reference to the drawings.

<Embodiment>
The agent device is a device that realizes a part or all of the agent system 1 including the notification system of the present embodiment. Hereinafter, as an example of the agent device, an agent device mounted on a vehicle (hereinafter, vehicle M) on which a occupant (an example of a user) is boarded and having an agent function will be described. For the application of the present invention, the agent device does not necessarily have to have an agent function. Further, the agent device may be a portable terminal device (general-purpose terminal) such as a smartphone, but the following description will be made on the premise of an agent device having an agent function mounted on a vehicle. The agent function is, for example, providing various information based on a request (command) included in the utterance of the occupant, controlling various devices, and mediating a network service while interacting with the occupant of the vehicle M. It is a function. When the agent device has a plurality of agent functions, the agent functions may have different functions, processing procedures, controls, and output modes / contents. In addition, some of the agent functions may have a function of controlling devices in the vehicle (for example, devices related to driving control and vehicle body control). The vehicle M is an example of a “moving body”.

Agent functions include, for example, a voice recognition function that recognizes the voice of an occupant (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network. It is realized by using a network search function that searches for another device or a predetermined database owned by the own device in an integrated manner. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. In addition, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) is an agent server capable of communicating with the in-vehicle communication device of the vehicle M or the general-purpose communication device brought into the vehicle M. It may be mounted on (external device). In the following description, it is assumed that a part of the configuration is mounted on the agent server, and the agent device and the agent server cooperate to realize the agent system. Further, a service provider (service entity) in which an agent device and an agent server cooperate to appear virtually is called an agent.

<Overall configuration>
FIG. 1 is a configuration diagram of an agent system 1 including an agent device 100. The agent system 1 includes, for example, an agent device 100 and one or more agent servers 200. Providers that provide the agent system 1 in the present embodiment include, for example, automobile manufacturers, network service providers, electronic commerce businesses, sellers and manufacturers of mobile terminals, and any other entity (corporation, organization, etc.). An individual, etc.) can be the provider of the agent system 1. Note that FIG. 1 has described the case where there is only one agent server 200, but the present invention is not limited to this, and the agent system 1 may include two or more agent servers 200. In this case, each agent server 200 may be provided by any entity different from each other.

The agent device 100 communicates with the agent server 200 via the network NW. The network NW includes, for example, a part or all of communication networks such as the Internet, cellular network, Wi-Fi network, WAN (Wide Area Network), LAN (Local Area Network), public line, telephone line, and wireless base station. Including. One or more content servers 300 (content servers 300-1 to 300-3 in the figure) are connected to the network NW, and the agent server 200 or the agent device 100 transmits content from the content server 300 via the network NW. Can be obtained.

Hereinafter, a case where the content servers 300-1 to 300-2 are server devices for providing video content and the content server 300-3 is a server device for providing music content will be described. Further, the content server 300-1 and the content server 300-2 may be provided by arbitrary entities (operators) different from each other. Hereinafter, the content server 300-1 is operated by "○○ Video", the content server 300-2 is operated by "△△ Video", and the content server 300-3 is operated by "○○ Music". It shall be. The content server 300 is an example of a “recommendation system”.

The agent device 100 interacts with the occupant of the vehicle M, transmits the voice from the occupant to the agent server 200, and presents the answer obtained from the agent server 200 to the occupant in the form of voice output or image display.

[vehicle]
FIG. 2 is a diagram showing the configuration of the agent device 100 according to the embodiment and the equipment mounted on the vehicle M. The vehicle M is equipped with, for example, one or more microphones 10, a speaker 20, a display / operation device 30, an in-vehicle communication device 40, and an agent device 100. These devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The microphone 10 is a sound collecting unit that collects sounds emitted in the vehicle interior. The speaker 20 includes, for example, a speaker (sound output unit) arranged in the vehicle interior. The display / operation device 30 is a device (or device group) capable of displaying an image and accepting an input operation. The display / operation device 30 includes, for example, a display device configured as a touch panel. The display / operation device 30 may further include a HUD (Head Up Display) or a mechanical input device. The display / operation device 30 may be shared by the agent device 100 and the navigation device (not shown). The in-vehicle communication device 40 is, for example, a wireless communication device that can access the network NW using a cellular network or a Wi-Fi network. The touch panel of the display / operation device 30 is an example of an “operation unit”.

[Agent device]
The agent device 100 includes a management unit 110, an agent function unit 130, an in-vehicle communication unit 140, and a storage unit 150. The management unit 110 includes, for example, an audio processing unit 111, an ambient environment information acquisition unit 112, an agent WU (Wake Up) determination unit 113, a communication control unit 114, and an output control unit 120. The software layout shown in FIG. 2 is simply shown for the sake of explanation, and is actually modified arbitrarily so that, for example, the management unit 110 may intervene between the agent function unit 130 and the in-vehicle communication device 40. can do. Further, in the following, an agent caused by the agent function unit 130 and the agent server 200 to appear in cooperation with each other may be simply referred to as an “agent”.

Each component of the agent device 100 is realized, for example, by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). Some or all of these components are hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by (including circuits), or it may be realized by the cooperation of software and hardware. The storage unit 150 may be realized by a storage device (a storage device including a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, and is a removable storage medium such as a DVD or a CD-ROM. It may be realized by (non-transient storage medium), or it may be a storage medium mounted on a drive device. Further, a part or all of the storage unit 150 may be an external device such as NAS or an external storage server that can be accessed by the agent device 100. The storage unit 150 stores, for example, information such as history information 152 in addition to a program executed by the agent device 100. The details of the history information 152 will be described later.

The management unit 110 functions by executing a program such as an OS (Operating System) or middleware.

The sound processing unit 111 of the management unit 110 receives the sound collected from the microphone 10, recognizes the wake-up word preset for each agent for the received sound, and recognizes other utterance contents. Perform sound processing to make it suitable for. The wakeup word is, for example, a word or phrase for activating the target agent. The wakeup word may start a single agent or may start a plurality of agents. The acoustic processing is, for example, noise removal by filtering such as a bandpass filter, sound amplification, and the like. Further, the sound processing unit 111 outputs the sound-processed voice (hereinafter, voice stream) to the agent WU determination unit 113 and the activated agent function unit 130. The sound processing unit 111 is an example of the “first acquisition unit”. The audio stream is an example of "audio data".

The surrounding environment information acquisition unit 112 acquires information related to the location where the occupant (vehicle M) exists (hereinafter referred to as ambient environment information) from an external device (not shown) connected to the network NW via the vehicle-mounted communication device 40. To do. Specifically, the surrounding environment information acquisition unit 112 transmits, for example, information indicating the position of the vehicle M specified by the GNSS (Global Navigation Satellite System) receiver (not shown) included in the vehicle M to the external device. .. Then, the external device receives the weather, climate (season), temperature, humidity of the position where the vehicle M exists, and the POI (POI) existing around the position where the vehicle M exists, based on the received information indicating the position of the vehicle M. Point of Interest) Surrounding environment information indicating information and the like is transmitted to the agent device 100.

The surrounding environment information acquisition unit 112 identifies the weather, climate, temperature, humidity, etc. at the position where the vehicle M exists based on the detection result of the sensor (not shown) that detects the outside air of the vehicle M, and the surrounding environment. Information may be generated. Further, the surrounding environment information acquisition unit 112 may generate the surrounding environment information by acquiring the POI information from the navigation device (not shown) included in the vehicle M.

Further, the vehicle M is provided with a camera that captures the surrounding environment of the vehicle M and generates an image, and the surrounding environment information acquisition unit 112 is a learning model learned by deep learning using an image showing the surrounding environment of the vehicle M. May be used to derive ambient environment information including weather, climate (season), temperature, humidity at the position where the vehicle M exists, POI information existing around the position where the vehicle M exists, and the like.

Further, the information included in the surrounding environment information is an example and is not limited to this. The surrounding environment information acquisition unit 112 may generate, for example, information related to the environment inside the vehicle interior of the vehicle M as the surrounding environment information. In this case, the vehicle M may include a camera (not shown) that images the interior of the vehicle M. The camera is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera is attached to an arbitrary place where the vehicle interior of the vehicle M can be imaged. Then, the camera periodically and repeatedly images the periphery of the vehicle M, for example. For example, the camera may be a stereo camera. In the ambient environment information acquisition unit 112, the camera images the interior of the vehicle M, processes the generated image, and the number of occupants of the vehicle M, the age of the occupants, and the situation of the occupants (whether it is exciting or enthusiastic about talking). Identify whether you are bored, bored, the mood of the occupants, etc.). For example, the ambient environment information acquisition unit 112 recognizes the occupant's face by image processing, specifies the number of occupants by the number of recognized faces, and identifies the occupant's age based on the feature amount of the recognized face area. To do. In addition, the surrounding environment information acquisition unit 112 recognizes the occupant's face by image processing, and identifies the occupant's situation by the recognized facial expression and mouth movement. Then, the surrounding environment information acquisition unit 112 generates the surrounding environment information including the specified information.

Further, the surrounding environment information acquisition unit 112 uses, for example, a learning model learned by deep learning using an image showing the interior of the vehicle M, such as the number of occupants of the vehicle M, the age of the occupants, the situation of the occupants, and the like. The surrounding environment information including may be derived.

Hereinafter, the surrounding environment information includes the weather, climate (season), temperature, humidity at the position where the vehicle M exists, POI information existing around the position where the vehicle M exists, the number of occupants of the vehicle M, and the age of the occupants. , At least one or more of the information such as the situation of the occupants shall be included. The ambient environment information acquisition unit 112 acquires (generates) the ambient environment information in response to an instruction from the agent server 200, and transmits the ambient environment information to the agent server 200 by the vehicle-mounted communication device 40, for example. The ambient environment information acquisition unit 112 is an example of the “second acquisition unit”.

The agent WU determination unit 113 recognizes a wakeup word predetermined for the agent. The agent WU determination unit 113 recognizes the voice uttered from the voice stream that has been sound-processed by the sound processing unit 111. First, the agent WU determination unit 113 detects a voice section based on the amplitude and zero intersection of the voice waveform in the voice stream. The agent WU determination unit 113 may perform frame-by-frame speech recognition based on a Gaussian mixture model (GMM) and section detection based on non-speech discrimination.

Next, the agent WU determination unit 113 converts the voice in the detected voice section into text and converts it into character information. Then, the agent WU determination unit 113 determines whether or not the textualized character information corresponds to the wakeup word. When it is determined that the wakeup word is used, the agent WU determination unit 113 activates the agent function unit 130 corresponding to the wakeup word. The agent server 200 may be equipped with a function corresponding to the agent WU determination unit 113. In this case, when the management unit 110 transmits the voice stream to which the sound processing has been performed by the sound processing unit 111 to the agent server 200 and determines that the agent server 200 is a wakeup word, the management unit 110 follows an instruction from the agent server 200. The agent function unit 130 is started. Further, each agent function unit 130 may be always activated and may determine the wakeup word by itself. In this case, the management unit 110 does not need to include the agent WU determination unit 113.

Further, the agent WU determination unit 113 recognizes the end word included in the spoken voice by the same procedure as the above procedure, and the agent corresponding to the end word is activated (hereinafter, , If necessary, it is referred to as "starting"), the running agent function unit is terminated (stopped). The start and end of the agent may be executed, for example, by accepting a predetermined operation from the display / operation device 30, but the following describes an example of starting and stopping by voice. Further, the activated agent may be stopped when the voice input is not received for a predetermined time or more.

The communication control unit 114 controls the agent function unit 130 so that it can be connected to the network NW. For example, the communication control unit 114 controls the connection state and the like when the agent function unit 130 communicates with an external device (for example, the agent server 200) via the network. Further, the communication control unit 114 controls such as reconnection when communication is interrupted and switching of the connection state.

The output control unit 120 provides the occupants with services and the like by causing the display unit or the speaker 20 to output information such as response contents in response to an instruction from the communication control unit 114 or the agent function unit 130 or the like. Specifically, the output control unit 120 displays, such as the speaker 20, information indicating one or more contents (hereinafter, recommended information) acquired (recommended) from the content server 300 by the agent function unit 130 described later. Notify the operating device 30. The content is, for example, a moving image, music, or the like.

Further, when the operation of selecting one of the one or more contents included in the notified recommendation information is received by the touch panel of the display / operation device 30, the output control unit 120 has an agent function according to the operation. The content is provided to the occupants by outputting the sound of the content acquired by the unit 130 to the speaker 20 and displaying the image of the content on the display device of the display / operation device 30.

The voice control unit 122 transmits a voice used for notifying the occupant of the vehicle M of the response content in which the agent responds to the utterance of the occupant of the vehicle M based on the information acquired by the agent function unit 130 from the agent server 200. Output to the speaker 20.

The display control unit 124 displays an image used for notifying the occupant of the vehicle M of the response content in which the agent responds to the utterance of the occupant of the vehicle M based on the information acquired by the agent function unit 130 from the agent server 200. It is displayed on the display device of the display / operation device 30.

The agent function unit 130 cooperates with the agent server 200 to provide a service including a response by voice and an image in response to an utterance of a vehicle occupant. For example, the agent function unit 130 is granted the authority to control the vehicle M or the in-vehicle device mounted on the vehicle M, and the utterance content of the vehicle M recognized by the agent server 200 by the process described later is the vehicle. In the case of a command for instructing the operation of the vehicle equipment mounted on the M, the agent function unit 130 controls those vehicle equipment based on the command. Based on the control of the communication control unit 114, the agent function unit 130 communicates with the agent server 200 by the vehicle-mounted communication unit 140 via the vehicle-mounted communication device 40.

The agent function unit 130 may be assigned the authority to control vehicle equipment according to laws, ordinances, contracts between businesses that provide agents, and the like.

The vehicle-mounted communication unit 140 communicates via the vehicle-mounted communication device 40, for example, when the agent function unit 130 connects to the network NW. The vehicle-mounted communication unit 140 outputs the information from the agent function unit 130 to the agent server 200 and other external devices via the vehicle-mounted communication device 40. Further, the vehicle-mounted communication unit 140 outputs the information input via the vehicle-mounted communication device 40 to the agent function unit 130.

The agent function unit 130 is activated based on an activation instruction by the agent WU determination unit 113, and generates and generates a response content to a request included in the voice of the occupant's utterance via the agent server 200 in response to the occupant's utterance. The response content is output to the output control unit 120. Further, when communicating with the agent server 200, the agent function unit 130 communicates according to the connection state controlled by the communication control unit 114. Further, the agent function unit 130 may stop the agent based on the control by the agent WU determination unit 113.

[Agent server]
FIG. 3 is a diagram showing a configuration of the agent server 200 and a part of the configuration of the agent device 100 according to the embodiment. Hereinafter, the operation of the agent function unit 130 and the like will be described together with the configuration of the agent server 200. Here, the description of the physical communication from the agent device 100 to the network NW will be omitted.

The agent server 200 includes a communication unit 210. The communication unit 210 is, for example, a network interface such as a NIC (Network Interface Card). Further, the agent server 200 includes, for example, a voice recognition unit 220, a natural language processing unit 221, a dialogue management unit 222, a network search unit 223, and a response content generation unit 224. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by mounting the storage medium in a drive device. The combination of the voice recognition unit 220 and the natural language processing unit 221 is an example of the "utterance content interpretation unit".

Further, the agent server 200 includes a storage unit 250. The storage unit 250 is realized by the same device as the various storage devices that realize the storage unit 150. The storage unit 250 stores, for example, data and programs such as a dictionary DB 252, a personal profile 254, a knowledge base DB 256, and a response rule DB 258.

In the agent device 100, the agent function unit 130 transmits, for example, a voice stream input from the sound processing unit 111 or the like, or a voice stream that has undergone processing such as compression or coding to the agent server 200. When the agent function unit 130 can recognize a command (request content) capable of local processing (processing that does not go through the agent server 200), the agent function unit 130 may execute the processing requested by the command. The command capable of local processing is, for example, a command that can be responded to by referring to the storage unit 150 included in the agent device 100. More specifically, the command capable of local processing is, for example, a telephone directory associated with a matching name by searching for the name of a specific person from the telephone directory data (not shown) existing in the storage unit 150. It is a command to make a call to (call the other party). Therefore, the agent function unit 130 may have a part of the functions provided in the agent server 200.

When the voice stream is acquired, the voice recognition unit 220 performs voice recognition and outputs the textualized character information, and the natural language processing unit 221 interprets the meaning of the character information while referring to the dictionary DB 252. The dictionary DB 252 is, for example, associated with abstract semantic information with respect to character information. The dictionary DB 252 includes, for example, a functional dictionary 252A and a general-purpose dictionary 252B.

The function dictionary 252A is a dictionary for covering the functions (services) provided by the agent realized by the agent server 200 in cooperation with the agent function unit 130. For example, when the agent provides a function to control an in-vehicle air conditioner, the function dictionary 252A contains "air conditioner", "air conditioning", "turn on", "turn off", "temperature", "raise", "lower", and "lower". Words such as "inside air" and "outside air" are registered in association with word types such as verbs and objects, and abstracted meanings. In addition, the functional dictionary 252A may include inter-word link information indicating that they can be used at the same time.

The general-purpose dictionary 252B is not limited to the functions provided by the agent, but is a dictionary in which general events are associated with abstracted meanings. Each of the functional dictionary 252A and the general-purpose dictionary 252B may include list information of synonyms and synonyms. The function dictionary 252A and the general-purpose dictionary 252B may be prepared corresponding to each of a plurality of languages, in which case the voice recognition unit 220 and the natural language processing unit 221 have functions corresponding to preset language settings. A dictionary 252A, a general-purpose dictionary 252B, and grammatical information (not shown) are used. The processing of the voice recognition unit 220 and the processing of the natural language processing unit 221 are not clearly separated in stages, and the voice recognition unit 220 corrects the recognition result in response to the processing result of the natural language processing unit 221. It may be done by influencing each other.

The natural language processing unit 221 acquires information on functions required for responding to a request for a service included in voice (hereinafter referred to as function necessary information) as one of semantic analysis based on the recognition result by the voice recognition unit 220. .. For example, when a text such as "open the window" or "raise the temperature of the air conditioner" instructing the control of the in-vehicle device of the vehicle M is recognized as the recognition result, the natural language processing unit 221 may use the dictionary DB252 or the like. To acquire the target device / function type called "vehicle device control". Then, the natural language processing unit 221 outputs the acquired function necessary information to the agent function unit 130. The natural language processing unit 221 acquires the determination result of whether or not the service request can be executed based on the function required information. The natural language processing unit 221 generates a command corresponding to the interpreted utterance content as being able to respond to the service request when the requested function can be executed.

In addition, when the meanings such as "introduce pop content" and "play some classic content" that request the content server 300 to provide the content are recognized as the recognition result, the natural language processing unit 221 Refers to the dictionary DB252 and the like, acquires the function type of "content recommendation control", and transmits the information of the function type of "content recommendation control" to the dialogue management unit 222.

The dialogue management unit 222 responds to the occupant of the vehicle M (for example, the utterance content to the occupant) while referring to the personal profile 254, the knowledge base DB 256, and the response rule DB 258 based on the command generated by the natural language processing unit 221. And the image and sound to be output from the output section). The knowledge base DB 256 is information that defines the relationships between things. The response rule DB 258 is information that defines the actions (answers, device control contents, etc.) that the agent should perform in response to the command. When the dialogue management unit 222 transmits the information of the function type "content recommendation control" from the natural language processing unit 221, the dialogue management unit 222 acquires the surrounding environment information from the agent device 100 and outputs the acquired surrounding environment information to the content server 300. To do. The content server 300 outputs the recommendation information to the agent server 200 by a process described later. The dialogue management unit 222 transmits the recommendation information acquired from the content server 300 to the agent device 100. Regarding the process related to the acquisition of the recommendation information, the dialogue management unit 222 is an example of the “third acquisition unit”.

Further, the dialogue management unit 222 may identify the occupant by collating with the personal profile 254 using the feature information obtained from the voice stream. In this case, for example, voice feature information is further attached to the personal profile 254. Voice feature information is, for example, information on speaking characteristics such as voice pitch, intonation, and rhythm (sound pitch pattern), and feature quantities based on Mel Frequency Cepstrum Coefficients and the like. .. The voice feature information is, for example, information obtained by having the occupant utter a predetermined word or sentence at the time of initial registration of the occupant and recognizing the uttered voice.

The dialogue management unit 222 causes the network search unit 223 to perform a search when the command requests information that can be searched via the network NW. The network search unit 223 accesses an external device such as the content server 300 via the network NW and acquires desired information.

The response content generation unit 224 generates a response sentence so that the content of the utterance determined by the dialogue management unit 222 can be understood by the occupants of the vehicle M, and transmits the generated response sentence to the agent device 100. Further, the response content generation unit 224 acquires the recognition result of recognizing the occupant of the vehicle M based on the image captured by the camera interior from the agent device 100, and the occupant who made an utterance including a command based on the acquired recognition result. When is identified as a occupant registered in the personal profile 254, a response sentence may be generated that calls the occupant's name or makes the occupant's speech similar to that of the occupant. In the present embodiment, the response sentence is, for example, a sentence such as "We have prepared the content recommended for you."

When the agent function unit 130 acquires the response sentence, the agent function unit 130 instructs the voice control unit 122 to perform voice synthesis and output the voice. Further, the agent function unit 130 instructs the display control unit 124 to display an image or the like including a response sentence.

[Content Server 300]
FIG. 4 is a diagram showing an example of the configuration of the content server 300-1. Since the content servers 300-1 to 300-3 have the same configuration, the content server 300-1 will be described below, and the description of the content servers 300-2 to 300-3 will be omitted.

The content server 300-1 includes a communication unit 310. The communication unit 310 is, for example, a network interface such as a NIC. Further, the content server 300-1 includes a functional unit of an acquisition unit 320 and a recommendation information acquisition unit 322. These components are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium such as a DVD or a CD-ROM (non-transient). It is stored in a sex storage medium) and may be installed by mounting the storage medium in a drive device.

Further, the content server 300-1 includes a storage unit 350 and a content DB 360. The storage unit 350 and the content DB 360 are realized by the same devices as the various storage devices that realize the storage unit 150 and the storage unit 250, respectively. The storage unit 350 stores, for example, data or a program such as a learning model 352. Further, the content DB 360 stores the content provided by the content server 300-1.

The learning model 352 is, for example, a learned model learned by machine learning using ambient environment information as input data and recommendation information as output data. The learning model 352 has an input layer, a hidden layer, and an output layer. The input layer of the learning model 352 includes the date and time when the occupant speaks, the day of the day, the weather at the point where the vehicle M exists, the climate, the temperature, the humidity, the POI information existing around the vehicle M, and the vehicle, which are included in the surrounding environment information. Information indicating the total number of M occupants, the number of occupants by age group, the number of occupants by gender, and the status of occupants is input. Recommendation information is output from the output layer. The recommendation information includes one or more pieces of information indicating a storage location (link) of the content in the content DB 360. The hidden layer has a multi-layer neural network connecting the input layer and the output layer. The parameters of the hidden layer are optimized by performing machine learning using the input to the input layer as training data and the data to be output from the output layer as teacher data. In this case, the learning data is the surrounding environment information, and the teacher data is the content selected by the occupant of the vehicle M existing in the environment indicated by the surrounding environment information according to the recommendation information.

The content server 300-1 may include a learning unit for learning the model at the learning stage. In this case, the learning unit trains the learning stage model based on the surrounding environment information and the information indicating the content selected by the occupant of the vehicle M existing in the environment indicated by the surrounding environment information according to the recommendation information. When the learning stage model is properly learned, the learning stage model is stored in the storage unit 350 as the learning model 352.

The acquisition unit 320 acquires (receives) ambient environment information from the agent server 200 by the communication unit 310. The recommendation information acquisition unit 322 inputs the ambient environment information acquired by the acquisition unit 320 into the learning model 352, and acquires the recommendation information as an output. The recommendation information acquisition unit 322 transmits the acquired recommendation information to the agent server 200. The agent server 200 receives recommendation information from the content server 300 for each content server 300, and transmits the received recommendation information to the agent device 100.

[Example of notification of recommendation information]
FIG. 5 is a diagram showing an example of a first notification image IMa1 scene used for notification of recommendation information. The agent function unit 130 receives the recommendation information from the agent server 200, and causes the output control unit 120 to notify the occupants of the vehicle M. The output control unit 120 generates, for example, the first notification image IMa1 including the recommendation information, displays it on the image of the display / operation device 30, and causes the occupant of the vehicle M to notify the recommendation information. As shown in FIG. 5, the first notification image IMa1 includes one or more images (hereinafter, content image IMc) indicating the contents included in the recommendation information. Further, the first notification image IMa1 includes a button B1 for selecting that the occupant of the vehicle M does not receive the content corresponding to the content image IMc. An example of the "notification control unit" is a combination of the output control unit 120 and the agent function unit 130 in the process related to the notification of the recommendation information.

In the first notification image IMa1 of the example of FIG. 5, a plurality of recommendation information for each content server 300 is notified. Specifically, the first notification image IMa1 is a content image IMc1 (illustrated content image IMc1-) showing the content included in the recommendation information output from the content server 300-1 operated by "OO Video". Content image IMc2 (illustrated content image IMc2-1 to IMC2-3) showing the content included in the recommendation information output from the content server 300-2 operated by 1 to IMc1-3) and "△△ Video". ) And the content image IMc3 (illustrated content images IMc3-1 to IMc3-3) indicating the content included in the recommendation information output from the content server 300-3 operated by "○○ Music". ..

The agent function unit 130 selects one of the one or more content image IMc included in the first notification image IMa1 in response to the display of the first notification image IMa1 by the occupant of the vehicle M. (That is, the operation of selecting the content) is determined. When the operation is accepted, the agent function unit 130 specifies information related to the content corresponding to the selected content image IMc (for example, a content storage location in the content DB 360) based on the recommendation information. The agent function unit 130 acquires the content stored in the specified storage location and causes the output control unit 120 to provide the content to the occupant of the vehicle M. Specifically, the voice control unit 122 causes the speaker 20 to output the voice of the content acquired by the agent function unit 130, and the display control unit 124 displays and operates an image of the content acquired by the agent function unit 130. It is displayed on the display device of the device 30. In the process related to the provision of contents, a combination of the output control unit 120 and the agent function unit 130 is an example of the “providing unit”.

Further, when the operation of the occupant of the vehicle M selected by the button B1 is accepted in response to the display of the notification image IMa, the agent function unit 130 does not acquire the content from the content DB 360, and the output control unit 120 does not acquire the content. Does not allow content to be provided by.

[When acquiring recommendation information from the selected content server 300]
In the above description, the dialogue management unit 222 of the agent server 200 outputs the surrounding environment information to any one or more content servers 300 connected to the network NW, and recommends information from each of the one or more content servers 300. The case of acquisition has been described, but the present invention is not limited to this. For example, when one or more content servers 300 are connected to the network NW, the dialogue management unit 222 selects an appropriate content server 300 (for example, only one) from the one or more content servers 300, and selects the selected content server. The surrounding environment information may be output to the 300.

In this case, the dialogue management unit 222 may select the content server 300 that provides the content that matches the preference of the occupant of the vehicle M as the appropriate content server 300, and the provision history that provides the content to the occupant of the vehicle M. The content server 300 may be selected, or the content server 300 registered in advance by the occupant of the vehicle M as a favorite may be selected, or the content server 300 matching the utterance content of the occupant of the vehicle M may be selected. You may.

When selecting the content server 300 that matches the preference of the occupant of the vehicle M, the dialogue management unit 222 refers to the personal profile 254 and identifies and identifies the preference (or preference, tendency, etc.) of the occupant of the vehicle M. Select the content server 300 that provides a large number of contents that match the tastes of the occupants of the vehicle M. Further, when the content server 300 having the provision history of providing the content to the occupant of the vehicle M is selected, the storage unit 250 (or the storage unit 150) can identify the occupant of the vehicle M who has received the content. The provision history information in which the information and the information that can identify the content server 300 that provided the content are associated with each other is stored, and the dialogue management unit 222 selects the content server 300 with reference to the provision history information. Further, when the content server 300 registered as a favorite by the occupant of the vehicle M in advance is selected, the storage unit 250 (or the storage unit 150) contains information that can identify the occupant of the vehicle M and the occupant's favorite. The favorite information associated with the information that can identify the content server 300 of the above is stored, and the dialogue management unit 222 selects the content server 300 with reference to the favorite information. The information indicating the content server 300 registered as a favorite by the occupant of the vehicle M in advance and the personal profile 254 are examples of "preference information".

FIG. 6 is a diagram showing an example of a second notification image IMa2 scene used for notification of recommendation information. The agent function unit 130 receives the recommendation information acquired from one content server 300 having the storage unit 350 in which the learning model 352 is stored, and causes the output control unit 120 to notify the occupant of the vehicle M. In the example of FIG. 6, the second notification image IMa2 is a content image IMc1 (illustrated content image IMc1) indicating the content included in the recommendation information output from the content server 300-1 operated by "○○ Video". -1 to IMc1-6) are included.

[Operation flow]
7 to 8 are flowcharts showing an example of a series of operations of the agent system 1 according to the embodiment. The processing of this flowchart may be repeatedly executed, for example, at a predetermined cycle or a predetermined timing. Further, in the examples of FIGS. 7 to 8, among the processes executed by the agent device 100, the process of activating the agent mainly by the wakeup word and responding to the request included in the utterance of the occupant shall be described. The processing of the agent server 200 executed in cooperation with the agent function unit 130 of the agent device 100 will also be described.

First, the agent WU determination unit 113 activates the agent (agent function unit 130) associated with the wakeup word (step S100). Next, the sound processing unit 111 determines whether or not the voice is received by the microphone 10 or the like (step S102). When the voice is received, the sound processing unit 111 performs sound processing so that the received voice is in a state suitable for recognizing the utterance content (step S104). The sound-processed voice is transmitted from the agent function unit 130 to the agent server 200.

The voice recognition unit 220 of the agent server 200 recognizes the received voice and converts the voice into text (step S106). Next, the natural language processing unit 221 executes natural language processing on the textualized character information and interprets the meaning of the character information (step S108). Next, the natural language processing unit 221 determines whether or not the content of the utterance of the occupant of the vehicle M whose meaning is interpreted is the content of the utterance requesting the provision of the content (step S110). When the natural language processing unit 221 determines that the content is not the content of the utterance requesting the provision of the content, the process proceeds to step S118. When the natural language processing unit 221 determines that the content is the content of the utterance requesting the provision of the content, the natural language processing unit 221 transmits information of the function type "content recommendation control" to the dialogue management unit 222, and proceeds to step S112.

The dialogue management unit 222 acquires the ambient environment information acquired by the ambient environment information acquisition unit 112 of the agent device 100 (step S112). The dialogue management unit 222 outputs the acquired surrounding environment information to the content server 300 and acquires the recommendation information (step S114). The acquisition unit 320 of the content server 300 acquires (receives) ambient environment information from the agent server 200 by the communication unit 310. The recommendation information acquisition unit 322 inputs the ambient environment information acquired by the acquisition unit 320 into the learning model 352, and acquires the recommendation information as an output. The recommendation information acquisition unit 322 transmits the acquired recommendation information to the agent server 200. The agent function unit 130 causes the output control unit 120 to notify the occupant of the vehicle M of the recommendation information acquired by the agent server 200 from the content server 300 (step S116). Specifically, the agent function unit 130 causes the display device of the display / operation device 30 to display the first notification image IMa1 and the second notification image IMa2 indicating the recommendation information.

The response content generation unit 224 of the agent server 200 generates the response content so that the occupant of the vehicle M can understand the content of the utterance determined by the dialogue management unit 222 (step S118). The agent function unit 130 causes the output unit to output the response result acquired from the agent server 200 (step S120). Specifically, the agent function unit 130 causes the voice control unit 122 to output the response content acquired from the agent server 200 as voice from the speaker 20, and displays it as an image on the display control unit 124 on the display device of the operation device 30. Display it. In this case, the response content is a message such as "We have prepared the recommended content for you."

Next, the agent function unit 130 determines whether or not the operation of the occupant of the vehicle M is accepted by the touch panel of the display / operation device 30 in response to the display of the first notification image IMa1 and the second notification image IMa2. Is determined (step S124). The agent function unit 130 waits until the operation is accepted. When the agent function unit 130 determines that the operation of the occupant of the vehicle M has been accepted, the agent function unit 130 determines that the content image IMc is one of the first notification image IMa1 and one or more content image IMc included in the second notification image IMa2. It is determined whether or not the operation of the occupant of the vehicle M for selecting the content (that is, the operation for selecting the content) has been accepted (step S126). The agent function unit 130 waits until the operation is accepted. When the operation of selecting the content is accepted, the agent function unit 130 acquires the content corresponding to the content image IMc from the content server 300 based on the recommendation information (step S128). The agent function unit 130 causes the occupant of the vehicle M to provide the acquired content by the output control unit 120 (step S130). When the operation for selecting the content is not accepted (that is, when the operation for selecting not to receive the content is accepted), the agent function unit 130 does not acquire the content from the content server 300 and controls the output. The content is not provided to the unit 120 (step S132).

After the processing of step S130 or S132, the agent function unit 130 determines whether or not to terminate the processing of the agent (step S134). If it is determined that the agent is not terminated, the process returns to step S102. If it is determined to terminate the agent, the management unit 110 terminates the agent (step S136). When terminating the agent, for example, when the voice of the end word for terminating the agent is received, when the in-vehicle switch for terminating the agent is pressed, or when the state in which the microphone 10 does not accept the voice continues for a predetermined time or longer. Etc. are included. The processing of this flowchart ends.

[Summary of Embodiment]
Here, the preference of the occupant of the vehicle M may change depending on the surrounding environment of the occupant inside the vehicle M and the place where the vehicle M exists. For example, the occupant of the vehicle M may want to be provided with sea-related content when the vehicle M is located at the beach, or may be in the rain when the vehicle M is located in a rainy area. You may want to be provided with related content. The agent function unit 130 and the output control unit 120 recommend the content based on the recommendation information derived by the content server 300 using the learning model 352 learned based on the surrounding environment information. Therefore, according to the agent system 1 of the present embodiment, the content is recommended using the recommendation information obtained according to the surrounding environment information, and the favorite content of the vehicle occupant is selected according to the surrounding environment of the vehicle occupant. Can be provided.

[Example of notification without recommendation information]
In the above description, the case where the agent function unit 130 recommends the content to the occupant of the vehicle M using the recommendation information acquired from the natural language processing unit 221 has been described, but the present invention is not limited to this. For example, the agent function unit 130 may select the content that recommends the content to the occupant of the vehicle M based on the history information. In this case, the agent function unit 130 refers to the history information 152 stored in the storage unit 150 and determines the content to be recommended.

FIG. 9 is a diagram showing an example of the contents of the history information 152. The history information 152 includes, for example, the surrounding environment information acquired by the surrounding environment information acquisition unit 112 and information that can identify the content even when the occupant of the vehicle M receives or does not receive the content. , Information indicating a storage location of the content in the content DB 360 is information associated with each other. When the content is provided to the occupants of the vehicle M based on the recommendation information, the agent function unit 130 includes the surrounding environment information related to the recommendation information, the information that can identify the content, and the content shown in the recommendation information. The history information 152 is generated (updated) by the records in which the information indicating the storage location of the above is associated with each other, and is stored in the storage unit 150.

Further, when the agent function unit 130 recognizes that the content of the utterance of the occupant of the vehicle M interpreted by the natural language processing unit 221 is the content requesting the provision of the content, the agent function unit 130 is acquired by the surrounding environment information acquisition unit 112. The history information 152 is searched using the surrounding environment information as a search key, and the content associated with the surrounding environment information that matches (or has a high degree of matching) with the surrounding environment information of the search key is selected. Then, the agent function unit 130 notifies the output control unit 120 of the provision of the selected content. In the process of selecting the content based on the history information 152, the agent function unit 130 is an example of the “selection unit”.

[Filtering recommendation information]
Further, the agent function unit 130 has described the case where the output control unit 120 is notified of one or more contents included in the recommendation information acquired from the content server 300, but the present invention is not limited to this. The agent function unit 130 may extract one of the one or more contents included in the recommendation information acquired from the content server 300 and notify the output control unit 120 of the content. In this case, the agent function unit 130 uses the content included in the recommendation information based on the history information 152 described above, the personal profile 254, the query (inquiry phrase) included in the content of the utterance of the occupant of the vehicle M, and the like. One of the contents is extracted (in other words, the content that does not suit the occupant's preference of the vehicle M is excluded), and the output control unit 120 is notified. For example, when the utterance content of the occupant of the vehicle M includes a "query" such as "play something" music "" or "play something" movie "", the agent function unit 130 recommends information. Of the contents included in, the contents that match the "query" such as "music" and "movie" are extracted, notified, and provided.

Further, the agent function unit 130 extracts the query and the content of the query similar to the query from the one or more contents included in the recommendation information acquired from the content server 300 (in other words, the query and the query-like content). The output control unit 120 may be notified (excluding the contents other than the contents of the query to be executed). In this case, the agent function unit 130 outputs a query to the agent server 200. Then, the natural language processing unit 221 identifies a query similar to the query specified based on the thesaurus (not shown) stored in the storage unit 250. A thesaurus is a dictionary in which a word and a thesaurus of the word are associated with each other. The agent function unit 130 extracts and notifies the query or the content matching the similar query from the contents included in the recommendation information based on the query and the similar query specified by the natural language processing unit 221. , And provide.

Instead of the configuration in which the agent function unit 130 excludes the content included in the recommendation information, the learning model 352 may input a query included in the utterance content of the occupant of the vehicle M. For example, when the content of the utterance of the occupant of the vehicle M includes a query such as "play the song of" singer A "", the agent function unit 130 provides the surrounding environment information and the information indicating the query to the content server 300. Output to. The recommendation information acquisition unit 322 inputs the surrounding environment information and the query into the learning model 352, and acquires the output recommendation information. At this time, the learning model 352 is a learned model in which the surrounding environment information and the query are input as input data and the recommendation information is used as output data.

In this case, the input layer of the learning model 352 includes the date and time when the occupant speaks, the day of the day, the weather at the point where the vehicle M exists, the climate, the temperature, the humidity, and the POI existing around the vehicle M included in the surrounding environment information. Information, the total number of occupants of the vehicle M, the number of occupants by age, the number of occupants by gender, the information indicating the occupant status, and the information indicating the query (and the information indicating the entity matching the query) are input. .. Recommendation information is output from the output layer. Since the learning method of the learning model 352 is the same as the method described above, the description thereof will be omitted.

[Configuration of Agent Device 100 and Agent Server 200]
In the above description, the case where the agent device 100 and the agent server 200 are separately configured has been described, but the present invention is not limited to this. The agent device 100 and the agent server 200 may be integrally configured. FIG. 10 is a diagram showing an example of the configuration of the agent device 100A having the function of the agent server 200. The agent device 100A includes an agent function unit 130A in place of (or in addition to) the agent function unit 130 included in the agent device 100. The agent function unit 130A includes, for example, a voice recognition unit 220, a natural language processing unit 221, a dialogue management unit 222, a network search unit 223, and a response content generation unit 224 as its functional units. Since the processing executed by these functional units is the same as the processing described above, the description thereof will be omitted. Further, the agent device 100A includes a storage unit 150A in place of (or in addition to) the storage unit 150 included in the agent device 100. In the storage unit 150A, for example, in addition to the program executed in the agent device 100A, the history information 152, the dictionary DB 252 (including the functional dictionary 252A, the general-purpose dictionary 252B, and the alias dictionary 252C), the personal profile 254, and knowledge. Information on the base DB 256 and the response rule DB 258 is stored. According to the agent device 100A, the above-described processing can be executed without performing communication via the network.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Agent system, 10 ... Microphone, 20 ... Speaker, 30 ... Display / operation device, 40 ... In-vehicle communication device, 100 ... Agent device, 110 ... Management unit, 111 ... Sound processing unit, 112 ... Surrounding environment information acquisition unit, 113 ... Agent WU determination unit, 114 ... Communication control unit, 120 ... Output control unit, 122 ... Voice control unit, 124 ... Display control unit, 130 ... Agent function unit, 140 ... In-vehicle communication unit, 150 ... Storage unit, 152 ... History information, 200 ... agent server, 210 ... communication unit, 220 ... voice recognition unit, 221 ... natural language processing unit, 222 ... dialogue management unit, 223 ... network search unit, 224 ... response content generation unit, 250 ... storage unit, 252A ... Functional dictionary, 252B ... General-purpose dictionary, 252 ... Dictionary DB, 256 ... Knowledge base DB, 258 ... Response rule DB, 254 ... Personal profile, 300, 300-1, 300-2, 300-3 ... Content server, 310 ... communication unit, 320 ... acquisition unit, 322 ... recommendation information acquisition unit, 350 ... storage unit, 352 ... learning model, 360 ... content DB, IMa ... notification image, IMa1 ... first notification image, IMa2 ... second notification image

Claims (10)

The first acquisition unit that acquires voice data indicating the voice spoken by the occupants on the moving object, and
An utterance content interpretation unit that interprets the utterance content of the occupant based on the voice data,
The second acquisition unit that acquires the surrounding environment information related to the surrounding environment of the occupant, and
With a third acquisition unit that outputs the content of the utterance and the surrounding environment information to one or more recommendation systems that recommend content to the occupant, and acquires recommendation information indicating one or more content from each of the recommendation systems. ,
A notification control unit that causes an output unit included in the mobile body to notify a plurality of the recommendation information for each recommendation system acquired by the third acquisition unit.
When the operation of selecting one of the recommended information notified by the output unit is accepted by the operation unit included in the moving body, the content corresponding to the selected recommended information is sent to the output unit. The provider to output and
Notification system with. The notification control unit uses the output unit to output content that matches the occupant's preference based on the recommendation information and the preference information indicating the occupant's preference among one or more of the contents corresponding to the recommendation information. Notify me
The notification system according to claim 1. A generation unit that generates history information in which the recommendation information notified by the output unit and the surrounding environment information related to the recommendation information are associated with each other.
A selection unit for newly selecting content for recommending to the occupant based on the ambient environment information newly acquired by the second acquisition unit and the history information generated by the generation unit is further provided.
The notification control unit causes the output unit to newly notify the content selected by the selection unit.
The notification system according to claim 1 or 2. When the operation unit does not select one or more of the contents notified by the output unit, the providing unit does not provide the output unit with one or more of the contents indicated by the recommendation information. ,
The generation unit generates historical information indicating that the output unit did not provide the content.
The notification system according to claim 3. The surrounding environment information includes information that targets the environment around the moving body and information that targets the environment around the occupant.
The information targeting the environment around the moving body includes the weather, climate, temperature or humidity of the point where the moving body exists, or POI information of the point existing around the moving body.
The information targeting the environment around the occupant includes the date and time or day of the week at the time of utterance, the number of the occupant by age group, the number of persons by gender or the total number of persons, or the situation of the occupant.
The notification system according to any one of claims 1 to 4. The third acquisition unit acquires the recommendation information in which the recommendation system recommends the content based on a query similar to the content of the utterance.
The notification system according to any one of claims 1 to 5. One or more of the recommendation systems are systems operated by different operators.
The notification system according to any one of claims 1 to 6. The first acquisition unit that acquires voice data indicating the voice spoken by the occupants on the moving object, and
An utterance content interpretation unit that interprets the utterance content of the occupant based on the voice data,
The second acquisition unit that acquires the surrounding environment information related to the surrounding environment of the occupant, and
With a third acquisition unit that outputs the content of the utterance and the surrounding environment information to one or more recommendation systems that recommend content to the occupant, and acquires recommendation information indicating one or more content from each of the recommendation systems. ,
A notification control unit that causes an output unit included in the mobile body to notify a plurality of the recommendation information for each recommendation system acquired by the third acquisition unit.
When the operation of selecting one of the recommended information among the recommended information notified by the output unit is accepted by the operation unit included in the moving body, the content corresponding to the selected recommended information is displayed. A providing unit that outputs to the output unit and
Notification control device. Runs by one or more computers,
The first acquisition process to acquire voice data indicating the voice spoken by the occupants on the moving object, and
An utterance content interpretation process that interprets the utterance content of the occupant based on the voice data,
The second acquisition process for acquiring the surrounding environment information related to the occupant's surrounding environment, and
An output step that outputs the content of the utterance and the surrounding environment information to one or more recommendation systems that recommend the content to the occupant, and an acquisition step that acquires recommendation information indicating one or more contents from each of the recommendation systems. And the third acquisition process, including
A notification process for causing the output unit included in the mobile body to notify a plurality of the recommendation information for each recommendation system acquired by the third acquisition process.
When the operation of selecting one of the recommended information among the recommended information notified by the output unit is accepted by the operation unit included in the moving body, the content corresponding to the selected recommended information is displayed. The output process to be output to the output unit and
Notification control method having. A program to be installed on one or more computers
The first acquisition process to acquire voice data indicating the voice spoken by the occupants on the moving object, and
An utterance content interpretation process that interprets the utterance content of the occupant based on the voice data,
The second acquisition process for acquiring the surrounding environment information related to the occupant's surrounding environment, and
An output step that outputs the content of the utterance and the surrounding environment information to one or more recommendation systems that recommend the content to the occupant, and an acquisition step that acquires recommendation information indicating one or more contents from each of the recommendation systems. And the third acquisition process, including
A notification process for causing the output unit included in the mobile body to notify a plurality of the recommendation information for each recommendation system acquired by the third acquisition process.
When the operation of selecting one of the recommended information among the recommended information notified by the output unit is accepted by the operation unit included in the moving body, the content corresponding to the selected recommended information is displayed. The output process to be output to the output unit and
A notification control program that causes the computer to execute.

Images