JP2020154922A - Service provision apparatus, settlement information processing apparatus and service provision method - Google Patents

Abstract

To provide a service provision apparatus, a settlement information processing apparatus and a service provision method that can perform handling considering a factor causing change of feeling with respect to the change of the feeling of a person.SOLUTION: A service provision apparatus 100 comprises: a person detection unit 132 that detects a plurality of persons in a target range; a feeling estimation unit 133 that estimates personal feeling of each person; a factor estimation unit 134 that estimates an occurrence factor of change of a total feeling state of the target range or a feeling state of any one or more persons on the basis of an estimation result of the feeling estimation unit 133; and a service determination unit 135 that determines a service provided to one or more persons in the target range corresponding to the occurrence factor estimated by the factor estimation unit 134.SELECTED DRAWING: Figure 2

Description

The present invention relates to a service providing device, a payment information processing device, and a service providing method.

Conventionally, a device that estimates a person's emotion and provides a service according to the estimated emotion is known (see, for example, Patent Document 1). The device described in Patent Document 1 estimates and quantifies the individual emotions of a plurality of occupants in the vehicle, and sets the sum of the individual emotion values as the total emotion value. This device provides services such as playing music from a speaker according to the value of the whole emotion. For example, it is disclosed to provide a music service that plays music suitable for a quiet environment in a car when there is no conversation in the car and the value of overall emotion is low.

JP-A-2018-63486

The device described in Patent Document 1 determines the service corresponding to the estimated total emotional value itself.
By the way, when trying to deal with a person's emotional change, it is effective to consider the factors that caused the emotional change. However, a method for taking such a countermeasure has not been proposed conventionally.
The present invention has been made in view of the above background, and is a service providing device, a payment information processing device, and a payment information processing device capable of coping with a change in a person's emotion in consideration of a factor causing the change in emotion. The purpose is to provide a service provision method.

As an embodiment for achieving the above object, based on a person detection unit that detects a plurality of persons in a target range, an emotion estimation unit that estimates the individual emotion of each person, and an estimation result of the emotion estimation unit. , The factor estimation unit that estimates the cause of the change in the emotional state of the person or one or more of the target range, and the target range corresponding to the cause estimated by the factor estimation unit. A service providing device including a service determining unit that determines a service to be provided to one or more of the persons in the room.

In the service providing device, the emotion estimation unit obtains an emotion estimation value representing the individual emotion of each person, and the factor estimation unit determines when the emotion estimation value of any person exceeds the first threshold value. When the sum of the emotional estimates exceeds the first threshold value, or when the amount of change in the sum of the emotional estimates exceeds the second threshold value, the cause of the change may be estimated. ..

In the service providing device, the factor estimation unit generates the change in the person whose emotion estimation value exceeds the third threshold value earlier than the other person among the plurality of persons in the target range. It may be configured to be presumed to be the factor person who caused the factor.

In the service providing device, the factor estimation unit estimates that the person whose emotion estimation value first exceeds the third threshold value is a temporary factor person, sets a detection period, and in the detection period, the factor estimation unit sets the detection period. When the emotion estimation value of the person different from the temporary factor person exceeds the fourth threshold value, the temporary factor person may be estimated as the factor person who is the cause of the change. ..

In the service providing device, the factor estimation unit determines that the emotion estimation value of the person different from the temporary factor person does not exceed the fourth threshold value after the temporary factor person in the detection period. It may be configured to presume that the emotion of the tentative factor person is in a suppressed state.

In the service providing device, the service determining unit may be configured to determine a service to be provided to the temporary factor person whose emotion is presumed to be in the suppressed state.

The service providing device includes an environment detection unit that acquires detection values related to the environment in the target range, and the factor estimation unit is based on a change in the emotion estimation value of one or more of the persons and the environment detection unit. Based on the acquired change in the detected value, it may be configured to estimate whether or not the change in the environment is the cause of the change in the estimated emotion value.

The service providing device includes a behavior detection unit that detects conversations or actions of a plurality of the persons in the target range, and the factor estimation unit includes changes in the emotion estimation value of any one or more of the persons and the above-mentioned. Based on the conversation or behavior detected by the behavior detection unit, it may be configured to estimate whether or not the conversation or behavior of any one or more of the persons is the cause of the change in the emotion estimation value.

In the service providing device, the behavior detection unit detects conversations or actions of a plurality of the persons in the target range, and obtains a numerical behavior detection value of the detected conversations or actions, and the emotion estimation unit. A correction unit that corrects the emotion estimation value estimated by the above based on the behavior detection value is provided, and the factor estimation unit estimates the cause of the change based on the emotion estimation value corrected by the correction unit. It may be a configuration.

The service providing device includes a service providing unit that provides a service determined by the service determining unit, and the service providing unit is a factor person whose factor estimation unit causes any of the persons to cause the change. If it is presumed that the factor person is different from the factor person, the service may be provided to notify the factor person.

In the service providing device, when the factor estimation unit estimates that any of the persons is the factor person who is the cause of the change, the service determination unit has a hobby of the factor person with respect to the factor person. Alternatively, it may be configured to determine to provide a service corresponding to the taste.

The service providing device includes a notification unit that notifies information to one or a plurality of the persons in the target range, and the notification unit is a factor that causes the change in any one of the persons by the factor estimation unit. When it is presumed to be a factor person, the person different from the factor person may be notified of a guide for urging the person to leave the factor person.

As another aspect for achieving the above object, a voice acquisition unit that acquires utterance voices of a plurality of persons in a target range, an image acquisition unit that acquires an image captured by capturing one or a plurality of the persons, and an image acquisition unit. A digitization processing unit that digitizes the information acquired by the voice acquisition unit and the captured image acquisition unit to generate numerical information, a change in the numerical information generated by the numerical processing unit, and a change in the numerical information. The event information storage unit that stores the correspondence with the event information related to the event that causes the occurrence of the above, and the numerical information based on the information acquired by the voice acquisition unit and the captured image acquisition unit during the detection target period determine the detection conditions. When the change is satisfied, the operation is based on the event information acquisition unit that acquires the event information corresponding to the change from the event information storage unit and the event information acquired by the event information acquisition unit. Examples thereof include a service providing device including an operation determination unit for determining the motion determination unit and an execution unit for executing the operation determined by the operation determination unit.

As another aspect for achieving the above object, a registration unit that registers a person who is a target of service provision by a service providing device or estimation of personal feelings, and evaluation acquisition for acquiring the evaluation of the person for the provision of the service. Examples thereof include a payment information processing device including a unit and a consideration determination unit that determines the consideration for providing the service based on the evaluation acquired by the evaluation acquisition unit.

In the payment information processing device, the consideration determination unit provides the service based on the evaluation acquired by the evaluation acquisition unit and the change status in which the personal emotion of the person changes in response to the provision of the service. It may be configured to determine the consideration.

The service providing device detects a plurality of persons in the target range, estimates the individual emotions of each person, and based on the estimation result of the emotion estimation unit of the individual emotions, the entire emotions of the target range. Providing a service that estimates the cause of a change in the emotional state of the person or one or more of the persons, and determines a service to be provided to the person or a plurality of the persons in the target range according to the estimated cause. Method.

According to the service providing device, it is possible to provide a service to a person in the target range in response to an occurrence factor that causes a change in the overall emotion or a change in the individual emotion in the target range. Therefore, it is possible to deal with the emotional change more effectively in consideration of the factors that changed the emotion. For example, it can be expected to effectively promote improvement of emotions and relaxation of specific emotional states. Similar effects can be obtained even when the above service provision method is used.
Further, according to the above-mentioned payment information processing device, a user and a service provider are required to use a device that determines a service in response to a factor that causes a change in overall emotion or a change in individual emotion in the target range. Appropriate consideration can be set for both parties.

The explanatory view which shows the example of the use state of the service providing apparatus of 1st Embodiment. A block diagram showing a functional configuration of a service providing device. A flowchart showing the operation of the service providing device. The schematic diagram which shows the example of the emotion value estimated by the service providing device. The schematic diagram which shows the example of the emotion value estimated by the service providing device. A block diagram showing a functional configuration of a service providing device. The explanatory view which shows the schematic structure of the service provision system of 2nd Embodiment. A block diagram showing a functional configuration of a service providing server and a payment information processing device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[1. Outline of the first embodiment]
FIG. 1 is a diagram showing an example of a usage state of the service providing device 100 of the first embodiment to which the service providing device of the present invention is applied. FIG. 2 is a block diagram showing a functional configuration of the service providing device 100.

FIG. 1 shows a living room RM capable of accommodating a plurality of persons as an example of an environment in which the service providing device 100 is used. The service providing device 100 sets the internal space of the living room RM as the target range A, and performs emotion estimation and service provision to the person in the target range A. In FIG. 1, a plurality of persons in the living room RM are referred to as users U1, U2, and U3 below.

The target range A does not need to be clearly separated from the space other than the target range A. For example, the target range A may be the entire living room RM or a part thereof. It is also possible to consider the range in which the service providing device 100 functions as the target range A. For example, the range that can be imaged by the camera 111, which will be described later, may be the target range A, the range that can be collected by the microphone 112 may be the target range A, and the range that the person can hear the sound output by the speaker 113 is targeted. It may be in the range A.

As shown in FIG. 2, the service providing device 100 includes a CPU (Central Processing Unit) 130 and a memory 160, and controls each part of the service providing device 100 by the function of the CPU 130. The service providing device 100 includes a camera 111, a microphone 112, a speaker 113, a motor 114, an infrared communication unit 115, and a wireless communication unit 116 that operate under the control of the CPU 130.

The camera 111 is a digital camera that captures an image of a range including at least a part of the target range A and outputs captured image data.
The microphone 112 collects the sound in the target range A and outputs the voice data.
The speaker 113 outputs sound according to the control of the CPU 130.
The motor 114 operates a drive mechanism provided in the lower part of the main body of the service providing device 100. The drive mechanism of the service providing device 100 is a mechanism for moving the service providing device 100 inside and outside the living room RM, and is composed of, for example, a plurality of wheels and a gear for transmitting the power of the motor 114 to the wheels. The motor 114 moves the service providing device 100 by operating the drive mechanism under the control of the CPU 130.

The infrared communication unit 115 transmits or transmits an infrared signal. The infrared communication unit 115 controls the operation of the air conditioner 250 by, for example, transmitting an infrared signal to the air conditioner 250 that air-conditions the living room RM. In this case, the service providing device 100 can start the operation of the air conditioner 250, stop the operation, set the target temperature for air conditioning, the air volume, and the like. Further, the service providing device 100 may acquire information on the room temperature of the living room RM detected by the air conditioner 250 by receiving an infrared signal from the air conditioner 250 by the infrared communication unit 115.

Further, the living room RM may be provided with a switchgear (not shown) for opening and closing the window W. The switchgear may be configured to operate by receiving an infrared signal from the service providing device 100. In this case, the service providing device 100 can open and close the window W by transmitting an infrared signal to the opening / closing device.

The wireless communication unit 116 executes short-range wireless communication such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). For example, the wireless communication unit 116 communicates with the biometric information detection device 220 worn by the user U1 on the body and the user terminal device 230 possessed by the user U1. The biometric information detection device 220 detects biometric information such as pulse, blood pressure, blood oxygen concentration, body temperature, and sweating of the user U1, and the user terminal device 230 that transmits the detection result to the service providing device 100 is, for example, a personal computer. It is a computer, smartphone, etc., and has voice call and screen display functions.
In the example shown in FIG. 1, the user U1 uses the biometric information detection device 220 and the user terminal device 230, but it is of course possible for the users U2 and U3 to use similar devices.
The wireless communication unit 116 may be configured to execute communication via a mobile wireless communication line or a public line network.

By reading and executing the control program 161, the CPU 130 reads and executes the data acquisition unit 131, the person detection unit 132, the emotion estimation unit 133, the factor estimation unit 134, the service determination unit 135, the service provision unit 136, the environment detection unit 137, and the like. It functions as a behavior detection unit 138, a correction unit 139, a notification unit 140, and a payment information processing unit 150. The payment information processing unit 150 is composed of a registration unit 151, an evaluation acquisition unit 152, a consideration determination unit 153, and a payment processing unit 154, and each of these functional units is also realized by the CPU 130 executing the control program 161. ..

The CPU 130 may be composed of another processor such as a microcomputer, or may be a device in which a processor, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like are integrated.

In addition to the control program 161, the memory 160 stores captured image data 162, audio data 163, target information 164, user information 165, emotion estimation value 166, overall emotion value 167, behavior detection value 168, and payment information 170. .. Further, a threshold storage unit 169 is provided in the storage area of the memory 160.

The data acquisition unit 131 acquires data regarding the users U1, U2, U3 or the target range A who are the persons in the target range A. The data acquisition unit 131 acquires the captured image data of the camera 111 and stores the captured image data 162 in the memory 160. The data acquisition unit 131 acquires the voice data collected by the microphone 112 and stores the voice data 163 in the memory 160. Further, the data acquisition unit 131 may acquire data on the room temperature and the like of the living room RM from the air conditioner 250 by the infrared communication unit 115. Further, the data acquisition unit 131 may acquire the data of the detection result of detecting the biometric information of the user U1 from the biometric information detection device 220. These data are also stored in the memory 160.

The person detection unit 132 detects a person in the target range A based on the data acquired by the data acquisition unit 131. For example, the person detection unit 132 detects the users U1, U2, and U3 by extracting a human image from the captured image data 162.
The person detection unit 132 may personally identify the detected person. For example, the user information 165 stored in the memory 160 stores the user who uses the service providing device 100 and the identification information for identifying each user. The data acquisition unit 131 may perform a process of specifying the user identification information of the person detected in the target range A with reference to the user information 165.

The emotion estimation unit 133 estimates the emotions of the users U1, U2, and U3 based on the data acquired by the data acquisition unit 131. Each emotion of the users U1, U2, and U3 is called an individual emotion. The emotion estimation unit 133 uses, for example, the sum of emotions of Plutchik to divide emotions into eight basic emotions (expectation, joy, acceptance, anxiety, surprise, sadness, disgust, and anger) for each of users U1, U2, and U3. , It is classified as an applied emotion by a combination of two adjacent emotions. The emotion estimation unit 133 quantifies individual emotions according to the fit to the emotion circle.

The emotion estimation unit 133 extracts, for example, an image of the face of the user U1 from the captured image data 162, determines the facial expression of the user U1 from the shape of the corner of the mouth, the shape of the eyebrows, the shape of the wrinkles of the face, and the like, and based on the determination result. Estimate the personal emotion of user U1. The emotion estimation unit 133 estimates the personal emotion of the user U1 based on the voice data 163 in addition to the captured image data 162. For example, the voice data of the user U1 is extracted from the voice data 163, and the individual emotion of the user U1 is estimated based on the utterance content of the user U1, the accent of the voice, the pronunciation, the voice volume, and the like. Further, the emotion estimation unit 133 estimates the personal emotion of the user U1 based on the biometric information of the user U1 acquired by the data acquisition unit 131 from the biometric information detection device 220. The emotion estimation unit 133 integrates the individual emotions of the user U1 estimated based on various data, and estimates the comprehensive personal emotions.

In the process of quantifying individual emotions, the emotion estimation unit 133 expresses favorable emotions such as joy as positive emotions with a positive value, and unfavorable emotions such as sadness as negative emotions with a negative value. It is represented by. In addition, the stronger the emotion, that is, the closer to the inside of the emotional circle, the larger the absolute value of the value representing the emotion.

Further, for example, the emotion estimation unit 133 determines whether or not the user U1 is moving based on the captured image data 162 and the detection result of the biological information detection device 220, and when the user U1 is not moving, the user U1 is sleeping. May be determined. In this case, the emotion estimation unit 133 sets the personal emotion of the sleeping user U1 to, for example, 0. The emotion estimation unit 133 estimates and quantifies individual emotions for users U2 and U3 by the same processing as for user U1. The emotion estimation unit 133 stores the numerical value of the individual emotion as the emotion estimation value 166 in the memory 160.

Further, the emotion estimation unit 133 estimates the emotions of the entire person in the target range A based on the individual emotions of the users U1, U2, and U3. The emotions of the entire person in the target range A are called overall emotions. The overall emotion represents the atmosphere of the target range A felt by the users U1, U2, and U3. For example, the emotion estimation unit 133 sets the sum of the emotion estimation values 166 of the person in the target range A as the total emotion of the target range A. That is, the total value of the emotion estimation value 166 of the user U1, the emotion estimation value 166 of the user U2, and the emotion estimation value 166 of the user U3 is defined as the total emotion of the target range A. The emotion estimation unit 133 obtains the total emotion of the target range A as a numerical value, and stores the obtained numerical value as the total emotion value 167 in the memory 160.

The emotion estimation value 166 is represented by a positive value or a negative value with the reference value as 0. When the emotion estimation value 166 is 0, it corresponds to a state where the individual emotion is calm or when the corresponding person is sleeping as described above.

The factor estimation unit 134 monitors the emotion estimation value 166 and the total emotion value 167, and changes the emotion estimation value 166 of any of the users U1, U2, and U3 or the total emotion value 167 in the target range A. Estimate the cause of the occurrence.
For example, the factor estimation unit 134 determines whether or not the change needs to be dealt with when the emotion estimation value 166 of the user U1, U2, or U3 changes, or when the overall emotion value 167 changes. judge. If the change needs to be dealt with, estimate the cause of the change. The factor estimation unit 134 refers to the threshold value stored in the threshold value storage unit 169 in a process of determining whether or not the change needs to be dealt with. The threshold storage unit 169 stores one or a plurality of preset threshold values.

The service determination unit 135 determines the service to be executed by the service providing device 100 according to the occurrence factor estimated by the factor estimation unit 134. For example, the service determination unit 135 determines to provide a service that mitigates changes in emotional estimates 166 and / or overall emotional values 167. Alternatively, the service determination unit 135 determines to provide a service for improving the emotion estimation value 166 and / or the overall emotion value 167 by changing it in the positive direction.

The service providing unit 136 provides the service determined by the service determining unit 135. Details of the service provided by the service providing unit 136 will be described later.

The environment detection unit 137 detects the environment in the target range A based on the captured image data 162 and / or the audio data 163.
The behavior detection unit 138 detects the behavior of each of the users U1, U2, and U3 based on the captured image data 162 and / or the voice data 163. The behavior detection unit 138 digitizes the behavior of each of the users U1, U2, and U3, generates a behavior detection value, and stores the behavior detection value in the memory 160 as the behavior detection value 168.

The correction unit 139 corrects the emotion estimation values 166 of the users U1, U2, and U3 estimated by the emotion estimation unit 133 based on the behavior detection value 168.

The notification unit 140 selects one or more of the users U1, U2, and U3 to notify the information. For example, the notification unit 140 outputs directional voice from the camera 111 to output the voice so that it can be heard only by a specific person among the users U1, U2, and U3, and notifies the information. In addition, the notification unit 140 notifies the user U1 of information by transmitting a voice call or transmitting a text message to the user terminal device 230. When the users U2 and U3 use the user terminal device 230, the same notification can be given to the users U2 and U3. When the notification unit 140 is controlled by the service providing unit 136, the notification by the notification unit 140 is a kind of service provision by the service providing unit 136.

The payment information processing unit 150 determines the consideration for the provision of the service by the service providing device 100 and executes the payment.
The registration unit 151 registers the person to be settled in consideration. Information about a person registered by the registration unit 151 is included in, for example, user information 165. In the present embodiment, as an example, users U1, U2, and U3 are registered by the registration unit 151.

The evaluation acquisition unit 152 acquires an evaluation for the service provided by the service providing unit 136. For example, the evaluation acquisition unit 152 generates an evaluation based on changes in the emotion estimation value 166 and the overall emotion value 167 after the service is provided by the service providing unit 136.

Further, for example, the evaluation acquisition unit 152 acquires the evaluation by the users U1, U2, and U3 registered by the registration unit 151 after the service is provided by the service providing unit 136 or at each preset period. .. In this case, the evaluation acquisition unit 152 may acquire evaluation information transmitted from a device other than the service providing device 100 such as the user terminal device 230. Further, the evaluation acquisition unit 152 may perform an interactive process with the users U1, U2, U3 by the microphone 112 and the speaker 113, and acquire the evaluation from the utterances of the users U1, U2, U3.

The consideration determination unit 153 determines the consideration for the provision of the service by the service providing unit 136. Examples of the mode of consideration include (1) flat rate system, (2) quantitative system, and (3) success fee type. The settlement information 170 includes the consideration imposed by the service providing unit 136 and the information indicating the mode of imposing the consideration. The consideration determination unit 153 may be capable of executing all of (1) flat rate system, (2) quantitative system, and (3) success fee type described below, or may execute only a part of the system. May be good.

(1) In the case of a flat rate system, a flat rate is set for the service providing device 100 to provide the service on a monthly, yearly, or other hourly basis. In this case, a fixed amount of consideration may be imposed on each of the users U1, U2, and U3 registered by the registration unit 151, for each group including a plurality of persons, for each place where the service providing device 100 is used, or for each place where the service providing device 100 is used. , A consideration may be levied for each service providing device 100.

(2) In the case of a quantitative system, a preset consideration is levied each time the service providing unit 136 provides a service. In this case, before the service providing unit 136 provides the service, all or a part of the users U1, U2, and U3 registered by the registration unit 151 may be confirmed whether or not the service can be used. The quantitative consideration may be paid only to the factor person described later.

(3) In the case of the success fee type, the consideration determination unit 153 determines the consideration for the service provided by the service providing unit 136 based on the evaluation acquired by the evaluation acquisition unit 152. The consideration determined by the consideration determination unit 153 is included in the settlement information 170. The consideration determination unit 153 updates the settlement information 170 every time the consideration is determined. The consideration determined by the consideration determination unit 153 is levied on all of the users U1, U2, U3, or a factor person described later.

The payment processing unit 154 executes payment based on the payment information 170. For example, the service providing device 100 may give valuable points to the users U1, U2, and U3 registered by the registration unit 151. In this configuration, the user information 165 stores information indicating the possessed points of the users U1, U2, and U3. In this case, the settlement processing unit 154 can receive the consideration by subtracting the points held by the users U1, U2, and U3.

The payment processing unit 154 may connect to a payment server operated by a financial institution or a payment service provider by the wireless communication unit 116 to perform payment processing for consideration. For example, the evaluation acquisition unit 152 may execute the withdrawal from the bank account, the credit card payment, the payment by electronic money or virtual currency, and the like by the payment server.

[2. Operation of the service providing device of the first embodiment]
FIG. 3 is a flowchart showing the operation of the service providing device 100.
The data acquisition unit 131 starts acquiring data (step S101). After that, the captured image data 162 and the audio data 163 are stored in the memory 160 at a preset cycle. Since the captured image data 162 and the audio data 163 are accumulated in the memory 160 within the set data capacity, the CPU 130 can refer to the captured image data 162 and the audio data 163 acquired in the past.

The person detection unit 132 detects a person in the target range A (step S102) and identifies the detected person (step S103). The process of step S103 can be omitted.

The emotion estimation unit 133 estimates individual emotions based on the captured image data 162 and the voice data 163, and calculates the emotion estimation value 166 (step S104). In step S104, the emotion estimation value 166 is calculated using the data acquired by the data acquisition unit 131 during the predetermined period. Subsequently, the emotion estimation unit 133 calculates the total emotion value 167 indicating the total emotion of the target range A by obtaining the sum of the emotion estimation values 166 (step S105).

The factor estimation unit 134 performs processing for determining the emotional change to be dealt with and estimating the cause of the emotional change at the emotional estimation value 166 and / or the total emotional value 167 (steps S106-S114). The factor estimation unit 134 refers to the first threshold value TH1, the second threshold value TH2, the third threshold value TH3, and the fourth threshold value TH4 stored in the threshold value storage unit 169. The first threshold value TH1, the third threshold value TH3, and the fourth threshold value TH4 are values to be compared with the absolute values of the emotion estimation value 166 and the total emotion value 167. Further, the second threshold value TH2 is a threshold value to be compared with the amount of change in the total emotion value 167. The change amount of the total emotion value 167 is described as, for example, an integral value of the change of the value in the set period, but may be the difference between the maximum value and the minimum value in the set period.

The causative factor, in a broad sense, refers to the factor that caused the change in the emotional estimate value 166 or the total emotional value 167. In a narrow sense, it refers to the factors that changed the overall emotional value 167. For example, when a change in the individual emotion of any of the users U1, U2, and U3 affects the individual emotion of another person and changes the overall emotion of the target range A, the change in the overall emotion of the target range A The factor of is one person. In this case, the cause of the change is the person who caused the change, and can also be called an influencer. This person is called a factor person.

In the following description of the operation of the factor estimation unit 134, specific examples of emotional changes in FIGS. 4 and 5 will be referred to.
4 and 5 are schematic views showing an example of an emotion estimation value 166 and an overall emotion value 167 estimated by the service providing device 100.
4 (A), (B), (C) and 5 (A), (B) are charts in which the horizontal axis is the passage of time and the vertical axis is the emotion estimation value 166, and FIG. 4 (A). And the curve F1 in FIG. 5 (A) shows the emotion estimate value 166 of the user U1. The curve F2 in FIG. 4 (B) shows the emotion estimation value 166 of the user U2, and the curve F3 in FIG. 4 (C) shows the emotion estimation value 166 of the user U3. Curve F5 in FIG. 5B shows another example of user U2's emotional estimate 166.
FIG. 4D is a chart in which the horizontal axis is the passage of time and the vertical axis is the total emotion value 167, and the curve FU shows the total emotion value 167. The curve FU is the sum of the emotional estimates 166 indicated by the curves F1, F2, and F3.
4 (A)-(D) and 5 (A) and 5 (B) show positive and negative values corresponding to the first threshold TH1, the third threshold TH3, and the fourth threshold TH4.

The outline of the emotional change illustrated in FIG. 4 is as follows.
The emotion estimation value 166 of the user U1 shown by the curve F1 is lowered by some factor generated at the time T1 and becomes a negative value. The emotion estimation value 166 of the user U1 fluctuates violently in the range of negative values, and it appears that the user U1 is emotional.
The change in the personal emotion of the user U1 also affects the users U2 and U3 in the target range A together with the user U1. For example, the user U2 senses a change in the personal emotion of the user U1 from the facial expression of the user U1, and the personal emotion of the user U2 is changed under the influence of the change. Therefore, the emotion estimation value 166 of the user U2 shown by the curve F2 decreases from the time T2 when the state of the user U1 is detected, and becomes a negative value.
Further, the overall emotion value 167 changes due to the change in the individual emotions of the user U1 and the user U2. The total emotion value 167 shown by the curve FU decreases following the change of the emotion estimation value 166 of the user U1 and the user U2, and becomes a negative value.
The user U3 senses that the overall emotion value 167 of the target range A has decreased at the time T3. Therefore, the emotion estimation value 166 of the user U3 is lowered to a negative value.

In step S106 and step S107, the factor estimation unit 134 determines whether or not an emotional change to be dealt with has occurred. The criteria for judgment are, for example, (1) that the absolute value of any one or more emotion estimation values 166 exceeds the first threshold value TH1, and (2) that the absolute value of the overall emotion value 167 exceeds the first threshold value TH1. (3) The amount of change in the overall emotion value 167 exceeds the second threshold value TH2.

Specifically, the factor estimation unit 134 first determines whether or not any of the emotion estimation value 166 and the total emotion value 167 exceeds the first threshold value TH1 (step S106). When it is determined that the first threshold value TH1 is not exceeded (step S106; NO), the factor estimation unit 134 determines whether or not the amount of change in the overall emotion value 167 exceeds the second threshold value TH2 (step S107). In step S107, the factor estimation unit 134 determines a detection period retroactively from the latest total emotion value 167 as an end point, and compares the amount of change in the total emotion value 167 within the detection period with the second threshold value TH2. When the amount of change in the overall emotion value 167 does not exceed the second threshold value TH2 (step S107; NO), the processing of the CPU 130 returns to step S104.

When either the emotion estimation value 166 or the total emotion value 167 exceeds the first threshold value TH1 (step S106; YES), and the amount of change in the total emotion value 167 exceeds the second threshold value TH2 (step S107; YES). ), The factor estimation unit 134 shifts to step S108.

In step S108, the factor estimation unit 134 identifies the emotion estimation value 166 that first exceeds the third threshold value TH3 among the emotion estimation values 166 of the person in the target range A, and corresponds to the identified emotion estimation value 166. Let the person be a tentative factor person (step S108). The tentative factor person may be a person corresponding to the emotion estimation value 166 exceeding the first threshold value TH1, or may be another person. Further, the third threshold value TH3 is the same value as the first threshold value TH1 or a value smaller than the first threshold value TH1.

In the example of FIG. 4, since the absolute value of the emotion estimation value 166 of the user U1 exceeds the first threshold value TH1 at the time T11, the factor estimation unit 134 makes an affirmative determination in step S106. Further, since the absolute value of the emotion estimation value 166 of the user U1 exceeds the third threshold value TH3 at the time T12, the factor estimation unit 134 sets the user U1 as a temporary factor person in step S108. A tentative factor person is a person who is likely to be a factor person and is a candidate for a factor person.

The factor estimation unit 134 sets the first detection period TE1 (step S109). The length of the first detection period TE1 is preset. The start point of the first detection period TE1 is the time when affirmative determination is made in step S106 or step S107, and is the time T11 in the example of FIG.

The factor estimation unit 134 determines whether or not the emotion estimation value 166 of the tentative factor person is a negative value in the first detection period TE1 (step S110). If the emotional estimated value 166 of the tentative factor person becomes a positive value in the first detection period TE1 (step S110; NO), the process of the CPU 130 returns to step S104.

When the emotion estimation value 166 of the tentative factor person maintains a negative value in the first detection period TE1 (step S110; YES), the factor estimation unit 134 sets the second detection period TE2 (step S111). The length of the second detection period TE2 is set in advance, and may be the same as or different from that of the first detection period TE1. The second detection period TE2 corresponds to an example of the detection period of the present invention. The start point of the second detection period is the time when affirmative determination is made in step S106 or step S107, and is the time T11 in FIG.

The factor estimation unit 134 determines in the second detection period TE2 whether or not the emotion estimation value 166 of the person excluding the temporary factor person exceeds the fourth threshold value TH4 (step S112). When the emotion estimation value 166 of a person other than the temporary factor person does not exceed the fourth threshold value TH4 (step S112; NO), the factor estimation unit 134 determines that the temporary factor person suppresses the emotion (step). S113). That is, it is determined that the user U1 is in the emotionally suppressed state.

The emotion estimation value 166 shown in FIG. 5B maintains a state in which the fourth threshold value TH4 is not exceeded during the second detection period TE2. This example shows that the emotional change of user U1 has not propagated to user U2. That is, the user U1 is in a state of suppressing emotional expression so that his / her emotion does not affect another person. In this case, the target range A is not affected by the emotional change of the user U1, but it is desirable to alleviate the emotional change of the user U1 or improve the emotion.

On the other hand, when the emotion estimation value 166 other than the temporary factor person exceeds the fourth threshold value TH4 in the second detection period TE2 (step S112; YES), the factor estimation unit 134 determines the temporary factor person as the factor person. (Step S114). In the example of FIG. 4, since the absolute value of the emotion estimation value 166 of the user U2 exceeds the fourth threshold value TH4 at the time T13, the user U1 is used as the factor person.
When the user U1 is a factor person, the change in the emotion of the user U1 can be alleviated, or the emotion of the user U1 can be improved, thereby improving the overall emotion of the target range A or alleviating the change.

After the suppression state is detected in step S113 and after the factor person is estimated in step S114, the service determination unit 135 determines the service (step S115). The service providing unit 136 executes the service determined by the service determining unit 135 (step S116).

The service determined by the service determination unit 135 and executed by the service provision unit 136 is, for example, a service that works on the entire target range A or a service that works on a specific person.

As a service that works on the entire target range A, there is an operation of softening the field of the target range A. Specific examples thereof include an operation of playing music from the speaker 113 and making a joke. When the service providing unit 136 is provided with a mechanism for supplying the odor to the target range A, the odor of a soothing aroma may be discharged to the target range A. These music, jokes, odors, and the like may be selected to suit the taste of the person who causes the music. Information about the preferences of users U1, U2, and U3 is included in, for example, user information 165.

Further, when the factor estimation unit 134 can estimate that the environment or the change in the environment detected by the environment detection unit 137 is the cause of the change in personal emotion, the service providing unit 136 eliminates or suppresses the factor. You may perform the operation. For example, when it is determined that the loudness of the noise from the window W is a factor of the change in personal emotion, the service providing unit 136 transmits a signal from the infrared communication unit 115 to the opening / closing device of the window W, and the window W You may control to close. In this case, the speaker 113 may notify a voice instructing the window to be closed, and the voice may be used to improve the overall emotion of the target range A.

As a service that works on a specific person, there is an operation of outputting a voice saying "Are you in trouble?" To the person who caused the occurrence or the person in the emotionally suppressed state.

Further, the service providing unit 136 may execute the pronunciation of the voice message by the speaker 113 and the operation by the motor 114 to guide the factor person from the exit D to the outside of the living room RM. For example, the service providing device 100 moves toward the exit D by pronouncing a voice message "Let's go outside" to the factor person. As a result, the individual emotions of the factor person and the overall emotion of the target range A can be calmed down by physically separating the factor person from the other person.

Further, the service providing unit 136 may execute the notification by the notification unit 140. For example, an operation of notifying a person other than the person who caused the occurrence of the person who caused the occurrence by the notification unit 140 and outputting a message prompting a solution by voice can be mentioned. In this case, if the factor estimation unit 134 can estimate the factor that changed the personal emotion of the person who caused the occurrence, the factor that changed the personal emotion may also be notified.

The service determination unit 135 preferentially determines a service that improves individual emotions or alleviates changes in individual emotions for all persons in the target range A. If it is difficult to execute such a service, or if it is judged that the effect of the service is weak, it is decided to execute the service targeting some people.

After the service is provided, the evaluation acquisition unit 152 acquires the evaluation for the service, and the consideration determination unit 153 determines the consideration (step S117). After that, the settlement processing unit 154 executes the settlement of consideration (step S118), and ends this processing. The operations of steps S117 and S118 may not be performed immediately after the service providing unit 136 has provided the service, and may be executed every preset period as described above.

As described above, in the service providing device 100 to which the present invention is applied, the person detection unit 132 that detects a plurality of persons in the target range A, the emotion estimation unit 133 that estimates the individual emotions of each person, and the emotion estimation unit. Based on the estimation result of 133, the factor estimation unit 134 that estimates the cause of the change in the individual emotion of any one or more persons or the overall emotion of the target range, and the factor estimation unit 134 correspond to the cause estimated by the factor estimation unit 134. , A service determination unit 135 for determining a service to be provided to one or more persons in the target range A.

According to this configuration, it is possible to provide a service to a person in the target range A in response to the cause of the change in the overall emotion or the change in the individual emotion in the target range A. Therefore, it is possible to deal with the emotional change more effectively in consideration of the factors that changed the emotion. For example, it can be expected that the emotion estimation value 166 and the behavior detection value 168 are improved and the relaxation of a specific emotional state is effectively promoted. Similar effects can be obtained even when the service providing method executed by the service providing device 100 is used.

Further, the payment information processing unit 150 corresponds to an example of the payment information processing device of the present invention.
The payment information processing unit 150 acquires a registration unit 151 that registers a person who is a target of service provision or personal emotion estimation by the service providing device 100, and evaluations of users U1, U2, and U3 for the service provision. It includes an evaluation acquisition unit 152 and a consideration determination unit 153 that determines the consideration for providing the service based on the evaluation acquired by the evaluation acquisition unit 152.
With this configuration, it is possible to set an appropriate consideration for both the users U1, U2, U3 and the service provider for using the service providing device 100.

Further, the consideration determination unit 153 may determine the consideration for the provision of the service based on the evaluation acquired by the evaluation acquisition unit 152 and the change situation in which the individual emotion of the person changes according to the provision of the service. For example, in the example of FIG. 4, as a result of the service being provided at time T12, the curves F1, F2, F3, and FU are all increased and changed from a negative value to a positive value. The consideration determination unit 153 may determine the effect of the service and determine the consideration based on the emotion estimation value 166 and the total emotion value 167 of the users U1, U2, and U3. In this case, the consideration for the service can be appropriately determined by reflecting not only the subjective evaluation of the users U1, U2, and U3 but also the objective effect.

In the service providing device 100, the emotion estimation unit 133 obtains the emotion estimation value 166 representing the individual emotion of each person, and the factor estimation unit 134 determines the case where the emotion estimation value 166 of any person exceeds the first threshold value TH1. , When the sum of the emotion estimation values 166 exceeds the first threshold value TH1, or when the amount of change in the sum of the emotion estimation values 166 exceeds the second threshold value TH2, the cause of the change is estimated. According to this configuration, changes in emotions can be appropriately detected based on the quantified overall emotions of the target range A and the individual emotions of the users U1, U2, and U3.

The factor estimation unit 134 considers a person whose emotional estimation value 166 exceeds the third threshold value TH3 earlier than other persons among a plurality of persons in the target range A to be the factor person who caused the change. presume. According to this configuration, the factor person who influenced the personal feelings of another person can be quickly estimated.

The factor estimation unit 134 sets the second detection period TE2, estimates that the person whose emotion estimation value 166 first exceeds the third threshold value TH3 is a tentative factor person, and tentative factors in the second detection period TE2. When the emotion estimation value 166 of a person different from the person exceeds the fourth threshold value TH4, the tentative factor person is estimated as the factor person who is the cause of the change.
According to this configuration, after estimating the temporary factor person, it is determined whether or not the temporary factor person has influenced the personal emotions of another person, so that the factor person can be estimated accurately and promptly. it can.

In the second detection period TE2, when the emotion estimation value 166 of a person different from the temporary factor person does not exceed the fourth threshold value TH4 after the temporary factor person, the factor estimation unit 134 causes the emotion of the temporary factor person. Presumed to be in a suppressed state. According to this configuration, it is possible to estimate a person who suppresses emotional expression while his emotional state is changing.

The service determination unit 135 determines the service to be provided to the tentative factor person whose emotion is presumed to be in a suppressed state. According to this configuration, it is possible to improve the emotional state by providing a service to a person who suppresses the emotional expression while the emotional state is changing.

The service providing device 100 includes a service providing unit 136 that provides a service determined by the service determining unit 135, and the service providing unit 136 is a factor person whose factor estimation unit 134 is a factor causing the change. When it is estimated, a service is provided to notify the factor person to a person different from the factor person. According to this configuration, the overall emotion value 167 and the emotion estimation value 166 are improved by transmitting the factor of the change in the overall emotion value 167 to a person who does not notice that the overall emotion value 167 has changed due to the influence of the factor person. Can be planned.

When the factor estimation unit 134 estimates that any person is the factor person who is the cause of the change, the service determination unit 135 provides the factor person with a service corresponding to the hobby or preference of the factor person. To determine. According to this configuration, the individual emotions of the factor person can be effectively dealt with, and the overall emotion value 167 and the emotion estimation value 166 can be improved more effectively.

The service providing device 100 includes a notification unit 140 that notifies information to one or more persons in the target range A.
When the factor estimation unit 134 estimates that any person is the factor person who is the cause of the change, the notification unit 140 notifies a person different from the factor person of guidance to urge them to leave the factor person. You may. For example, when the user U1 is a factor person, the notification unit 140 may notify the users U2 and U3 of leaving the user U1. In this case, deterioration of personal feelings of a person other than the factor person can be suppressed.

The service providing device 100 includes a behavior detection unit 138 that detects conversations or actions of a plurality of persons in the target range A. In the factor estimation unit 134, the conversation or behavior of any one or more persons is the emotion estimation value 166 based on the change of the emotion estimation value 166 of any one or more persons and the conversation or behavior detected by the behavior detection unit 138. It may be estimated whether or not it is the cause of the change in.

In this configuration, the speech / behavior detection unit 138 may be configured to detect conversations or actions of a plurality of persons in the target range A and obtain a speech / behavior detection value 168 obtained by quantifying the detected conversations or actions. The service providing device 100 includes a correction unit 139 that corrects the emotion estimation value 166 estimated by the emotion estimation unit 133 based on the behavior detection value 168. The factor estimation unit 134 estimates the cause of the change based on the emotion estimation value 166 corrected by the correction unit 139.

For example, in the operation shown in FIG. 3, the factor estimation unit 134 may analyze the words and actions of the users U1, U2, and U3 to determine the emotion estimation value 166 and the overall emotion value 167. Specifically, before the process of estimating the temporary factor person in step S108, the behavior detection unit 138 generates the behavior detection values 168 of the users U1, U2, and U3, and the correction unit 139 corrects the emotion estimation value 166. You may. The speech / behavior detection value 168 is an index indicating the loudness of the speech / behavior of the users U1, U2, U3, and the loudness of the voice of the speech, the length of the speech, the loudness of the motion, the loudness of the sound generated by the motion, etc. The larger the value, the larger the behavior detection value 168. The behavior detection value 168 is calculated for each of the users U1, U2, and U3. Therefore, it can be said that the behavior detection value 168 of each user U1, U2, U3 indicates the magnitude of the influence of the behavior of the users U1, U2, U3 on the target range A. When the speech / behavior detection value 168 of any of the users U1, U2, and U3 exceeds the threshold value, the threshold storage unit 169 corrects the emotion estimation value 166 after the speech / behavior detection value 168 exceeds the threshold value. For example, for a person different from the person corresponding to the behavior detection value 168 exceeding the threshold value, the change of the emotion estimation value 166 is corrected to be large.

In this case, the emotion estimation unit 133 recalculates the total emotion value 167 based on the emotion estimation value 166 corrected by the correction unit 139. The factor estimation unit 134 re-executes the process from step S106 with respect to the corrected emotion estimation value 166 and the total emotion value 167.
According to this configuration, the influence of any of the words and actions of the users U1, U2, and U3 on the emotion estimation value 166 of another person can be greatly evaluated, and the cause of the occurrence can be estimated and the service can be determined. ..

The service providing device 100 includes an environment detection unit 137 that acquires a detection value related to the environment of the target range A, and the factor estimation unit 134 uses the change of the emotion estimation value 166 of any one or more persons and the environment detection unit 137. Based on the change in the acquired detected value, it may be estimated whether or not the change in the environment is the cause of the change in the emotion estimation value 166. In this case, it is possible to improve the emotion estimation value 166 and the overall emotion value 167 in response to the change in the emotion estimation value 166 due to the change in the environment.

From another point of view, the service providing device 100 can have the configuration shown in FIG.
FIG. 6 is a block diagram showing a functional configuration when the service providing device 100 is viewed from another viewpoint. The CPU 130 includes a voice acquisition unit 191, a captured image acquisition unit 192, a digitization processing unit 193, an event information acquisition unit 194, an operation determination unit 195, and an execution unit 196. Further, the memory 160 includes an event information storage unit 197.

The service providing device 100 of FIG. 6 includes a voice acquisition unit 191 that acquires utterance voices of a plurality of persons in the target range A, an image capture image acquisition unit 192 that acquires an image of one or a plurality of persons, and a voice. A digitization processing unit 193 that digitizes the information acquired by the acquisition unit 191 and the captured image acquisition unit 192 to generate numerical information, a change in the numerical information generated by the numerical processing unit 193, and a change in the numerical information. The event information storage unit 197, which stores the correspondence with the event information related to the causative event, and the numerical information based on the information acquired by the voice acquisition unit and the captured image acquisition unit during the detection target period have changed to satisfy the detection conditions. In this case, the event information acquisition unit 194 that acquires the event information corresponding to the change from the event information storage unit 197, the operation determination unit 195 that determines the operation based on the event information acquired by the event information acquisition unit 194, and the operation. It includes an execution unit 196 that executes an operation determined by the determination unit.

The voice acquisition unit 191 acquires the utterance sound collected by the microphone 112, and the captured image acquisition unit 192 acquires the captured image data of the camera 111. These correspond to the above-mentioned data acquisition unit 131. The digitization processing unit 193 digitizes the information acquired by the data acquisition unit 131 to generate numerical information such as an emotion estimation value 166 and an overall emotion value 167. The digitization processing unit 193 corresponds to the emotion estimation unit 133.

The event information indicates the cause of the change in the emotional estimate 166 and / or the overall emotional value 167. The event information storage unit 197 stores information that associates the mode of change of the emotion estimation value 166 and / or the total emotion value 167 with the occurrence factor, and the event information acquisition unit 194 estimates the occurrence factor. That is, the event information acquisition unit 194 and the event information storage unit 197 correspond to an example in which the operation of the factor estimation unit 134 is executed based on the data stored in the memory 160.
The operation determination unit 195 corresponds to the service determination unit 135, and the execution unit 196 corresponds to the service provision unit 136. These operations are as described above.

Further, the configuration of each part included in the CPU 130 may be configured by artificial intelligence (AI: Artificial Intelligence). For example, the functions of the person detection unit 132, the emotion estimation unit 133, the factor estimation unit 134, the service determination unit 135, the environment detection unit 137, and the correction unit 139 may be executed by the AI mounted on the service providing device 100. The AI mounted on the service providing device 100 may be the programs and data constituting the trained model. Further, the AI mounted on the service providing device 100 may be configured to continue learning as the service providing device 100 operates.

Further, the service providing device 100 described above is not limited to a device that independently executes each function. An example in which a plurality of devices cooperate to function in the same manner as the service providing device 100 will be described as a second embodiment.

[3. Second Embodiment]
FIG. 7 is an explanatory diagram showing a schematic configuration of the service providing system 1 of the second embodiment, and also shows the functional configuration of the service providing device 100a installed in the target range A. FIG. 8 is a block diagram showing a functional configuration of the service providing server 300 and the payment information processing device 400. The service providing system 1 corresponds to the service providing device of the present invention as a whole.

The service providing device 100 is connected to the communication network 500 via the communication device 200. The communication network 500 may be a wide area network composed of the Internet or a public network, or may be a network in a specific area such as a LAN (Local Area Network).

A service providing server 300 and a payment information processing device 400 are connected to the communication network 500. The service providing device 100a, the service providing server 300, and the payment information processing device 400 constituting the service providing system 1 are connected to each other so as to be capable of data communication.

Like the service providing device 100, the service providing device 100a is installed in the living room RM corresponding to the target range A. The service providing device 100a has a CPU 130a as a processor similar to the CPU 130. The CPU 130a has the functions of a data acquisition unit 131, a service providing unit 136, and a notification unit 140, and further has a communication control unit 142.

Further, the service providing server 300 includes a CPU 310 and a memory 330. The CPU 310 is a computer that executes the control program 331, and by the function of the control program 331, the communication control unit 311, the person detection unit 312, the emotion estimation unit 313, the factor estimation unit 314, the service determination unit 315, and the environment detection unit. It operates as 317, a behavior detection unit 318, and a correction unit 319.

Similar to the data acquisition unit 131, the communication control unit 142 acquires data from the camera 111, the microphone 112, the infrared communication unit 115, and the like. The communication control unit 142 transmits the acquired data to the service providing server 300.
The communication control unit 311 of the service providing server 300 receives the data transmitted by the service providing device 100a, and stores the captured image data 332 and the voice data 333.

The person detection unit 312 functions in the same manner as the person detection unit 132 of the service providing device 100 shown in FIG. The emotion estimation unit 313 functions in the same manner as the emotion estimation unit 133, and the factor estimation unit 314 functions in the same manner as the factor estimation unit 134. Further, the service determination unit 315 functions in the same manner as the service determination unit 135. The environment detection unit 317 functions in the same manner as the environment detection unit 137, and the behavior detection unit 318 functions in the same manner as the behavior detection unit 138. Further, the correction unit 319 functions in the same manner as the correction unit 139.

The captured image data 332 and the audio data 333 stored in the memory 330 correspond to the captured image data 162 and the audio data 163 of the service providing device 100 shown in FIG. 2, respectively. The target information 334 corresponds to the target information 164. The user information 335 corresponds to the user information 165, the emotion estimation value 336 corresponds to the emotion estimation value 166, the overall emotion value 337 corresponds to the overall emotion value 167, and the behavior detection value 338 corresponds to the behavior detection value 168. The threshold storage unit 339 is the same as the threshold storage unit 169.

The payment information processing device 400 includes a CPU 410 and a memory 430. The CPU 410 is a computer that executes the control program 431, and operates as a communication control unit 411, a registration unit 412, an evaluation acquisition unit 413, a consideration determination unit 414, and a payment processing unit 415 by the function of the control program 431. .. In addition, the memory 430 stores the payment information 432.

The registration unit 412 functions in the same manner as the registration unit 151 of the service providing device 100 shown in FIG. The evaluation acquisition unit 413 functions in the same manner as the evaluation acquisition unit 152, the consideration determination unit 414 functions in the same manner as the consideration determination unit 153, and the settlement processing unit 415 functions in the same manner as the settlement processing unit 154. It is a thing. The payment information 432 corresponds to the payment information 170 stored in the service providing device 100.

The communication control unit 311 transmits data related to the service determined by the service determination unit 315 to the service providing device 100a.
The communication control unit 142 receives the data transmitted from the service providing server 300, and causes the service providing unit 136 to execute the service determined by the service determining unit 315. Further, the communication control unit 142 causes the notification unit 140 to execute the notification when the data received from the service providing server 300 is the data instructing the notification by the notification unit 140.

The communication control unit 311 transmits data related to the service determined by the service determination unit 315 to the payment information processing device 400. Further, the communication control unit 142 transmits data regarding the execution state of the service executed by the service providing unit 136 to the payment information processing device 400.

The communication control unit 411 causes the evaluation acquisition unit 413 to acquire the evaluation based on the data transmitted from the service providing device 100a and the service providing server 300, causes the consideration determination unit 414 to determine the consideration, and makes the payment by the payment processing unit 415. Let it run.

According to the service providing system 1 of the second embodiment, similarly to the service providing device 100, a person who is in the target range A in response to the cause of the change in the overall emotion or the change in the individual emotion in the target range A. Can provide services to. Therefore, it is possible to deal with the emotional change more effectively in consideration of the factors that changed the emotion. For example, it can be expected to effectively promote the improvement of the individual emotions of the users U1, U2, U3 and the overall emotion of the target range A, or the relaxation of a specific emotional state. Similar effects can be obtained even when the service providing method executed by the service providing system 1 is used.

[4. Other embodiments]
In the above embodiment, when the factor estimation unit 134 cannot estimate the factor person by the process of step S108-step S114 of FIG. 3, the factor estimation unit 134 may work on the users U1, U2, and U3 to determine the factor person. .. For example, the case where the emotion estimation value 166 of a plurality of persons exceeds the first threshold value TH1 at the same time or the case where the third threshold value TH3 is exceeded at the same time is applicable. The factor estimation unit 134 makes a voice call or a text message to the user terminal device 230, or makes a voice sound by the speaker 113. By these methods, the factor estimation unit 134 outputs, for example, a message asking the users U1, U2, and U3 what happened, and the data acquisition unit 131 acquires and analyzes the data regarding the state of the response. , Estimate the cause. In this case, the data acquisition unit 131 can collect more information and estimate the cause.

Further, as shown in FIG. 5, when the emotion estimation value 166 of the user U2 maintains a positive value while the emotion estimation value 166 of the user U1 is a negative value, the user U1 is in the suppressed state. Alternatively, it is conceivable that the user U1 changes his / her personal feelings due to circumstances unrelated to the users U2 and U3. Even in such a case, the factor estimation unit 134 may work on the users U1, U2, and U3 to determine the factor person or the cause.

Further, in the above embodiment, the process corresponding to the change such that the emotion estimation value 166 and the total emotion value 167 become negative values has been described, but the process corresponding to the change such that the total emotion value 167 becomes a positive value is supported. You may. For example, the service providing unit 136 may perform an operation such as playing music or joining the conversation of the users U1, U2, and U3 so as to further increase the overall emotion value 167 of the target range A.

Further, although the service providing device 100 and the service providing server 300 of the above-described embodiment are configured to include the environment detection units 137 and 317, they may not be provided. The same applies to the behavior detection unit 138, 318 and the correction unit 139, 319.

Further, although the service providing devices 100 and 100a of the above embodiment are configured to include the notification unit 140, the service providing devices 100 and 100a may not include the notification unit 140. The same applies to the motor 114 and the infrared communication unit 115. For example, the service providing devices 100 and 100a may be devices fixedly installed in the target range A, or may be so-called smart speakers.

Further, although the service providing device 100 and the payment information processing device 400 of the above embodiment are configured to include the evaluation acquisition units 152 and 413, they may not be provided.

In addition, FIG. 2, FIG. 6, FIG. 7, and FIG. 8 show the functional configurations of the service providing device 100 and the service providing system 1 according to the main processing contents in order to facilitate the understanding of the present invention. It is a schematic diagram, and these configurations may be configured by other divisions. Further, the processing of each component may be executed by one hardware unit, or may be executed by a plurality of hardware units. Further, the processing of each component may be executed by one program or may be executed by a plurality of programs.

1 ... Service providing system (service providing device), 100, 100a ... Service providing device, 130, 130a ... CPU, 131 ... Data acquisition unit, 132, 312 ... Person detection unit, 133, 313 ... Emotion estimation unit, 134, 314 ... Factor estimation unit, 135, 315 ... Service determination unit 136 ... Service provision unit, 137, 317 ... Environment detection unit, 138, 318 ... Word and behavior detection unit, 139, 319 ... Correction unit, 140 ... Notification unit, 142, 311 411 ... Communication control unit, 150 ... Payment information processing unit, 151, 412 ... Registration unit, 152, 413 ... Evaluation acquisition unit, 153, 414 ... Consideration determination unit, 154, 415 ... Payment processing unit, 160 ... Memory, 161 ... Control program, 162, 332 ... Captured image data, 163, 333 ... Audio data, 164, 334 ... Target information, 165, 335 ... User information, 166, 336 ... Emotion estimated value, 167, 337 ... Overall emotion value, 168, 338 ... Word-and-action detection value, 169, 339 ... Threshold storage unit, 170, 432 ... Payment information, 191 ... Voice acquisition unit, 192 ... Captured image acquisition unit, 193 ... Digitization processing unit, 194 ... Event information acquisition unit, 195 ... Operation determination unit, 196 ... Execution unit, 197 ... Event information storage unit, 300 ... Service providing server, 310 ... CPU, 311 ... Communication control unit, 330 ... Memory, 331 ... Control program, 400 ... Payment information processing device , 410 ... CPU, 411 ... Communication control unit, 430 ... Memory, 431 ... Control program, 500 ... Communication network, A ... Target range.

Claims (16)

A person detection unit that detects multiple people in the target range,
An emotion estimation unit that estimates the individual emotions of each person,
Based on the estimation result of the emotion estimation unit, a factor estimation unit that estimates the cause of the change in the emotional state of the entire target range or one or more of the emotional states of the person, and the factor estimation unit.
A service providing device including a service determining unit that determines a service to be provided to one or a plurality of the persons in the target range in response to an occurrence factor estimated by the factor estimating unit. The emotion estimation unit obtains an emotion estimation value representing the individual emotion of each person, and obtains an emotion estimation value.
The factor estimation unit determines that the emotion estimation value of any person exceeds the first threshold value, the sum total of the emotion estimation values exceeds the first threshold value, or the amount of change in the total sum of the emotion estimation values. The service providing device according to claim 1, wherein when the value exceeds the second threshold value, the cause of the change is estimated. Among the plurality of the persons in the target range, the factor estimation unit selects the person whose emotional estimation value exceeds the third threshold value earlier than the other persons as the factor person who caused the change. The service providing device according to claim 2, which is presumed to be. The factor estimation unit estimates that the person whose emotion estimation value first exceeds the third threshold value is a temporary factor person.
Set the detection period and
A claim that estimates the provisional factor person as the factor person who is the cause of the change when the emotion estimation value of the person different from the provisional factor person exceeds the fourth threshold value in the detection period. Item 3. The service providing device according to item 3. When the emotion estimation value of the person different from the temporary factor person does not exceed the fourth threshold value after the temporary factor person in the detection period, the factor estimation unit determines the emotion of the temporary factor person. The service providing device according to claim 4, wherein is presumed to be in a suppressed state. The service providing device according to claim 5, wherein the service determining unit determines a service to be provided to the provisional factor person whose emotion is presumed to be in the suppressed state. It is equipped with an environment detection unit that acquires detection values related to the environment in the target range.
In the factor estimation unit, the change in the environment is the change in the emotion estimation value based on the change in the emotion estimation value of the person of any one or more and the change in the detection value acquired by the environment detection unit. The service providing device according to any one of claims 2 to 6, which estimates whether or not it is the cause. It is provided with a behavior detection unit that detects conversations or actions of a plurality of the persons in the target range.
In the factor estimation unit, based on the change in the emotion estimation value of the person of any one or more and the conversation or action detected by the behavior detection unit, the conversation or action of the person of any one or more is the emotion. The service providing device according to any one of claims 2 to 7, which estimates whether or not it is a cause of change in the estimated value. The behavior detection unit is configured to detect conversations or actions of a plurality of the persons in the target range and obtain a speech / behavior detection value obtained by quantifying the detected conversations or actions.
A correction unit for correcting the emotion estimation value estimated by the emotion estimation unit based on the behavior detection value is provided.
The service providing device according to claim 8, wherein the factor estimation unit estimates the cause of the change based on the emotion estimation value corrected by the correction unit. A service providing unit that provides a service determined by the service determining unit is provided.
When any one of the persons is presumed to be the factor person who is the cause of the change by the factor estimation unit, the service providing unit notifies the factor person different from the factor person. The service providing device according to any one of claims 1 to 9, which provides a service. When the factor estimation unit estimates that any of the persons is a factor person who is the cause of the change, the service determination unit provides the factor person with a service corresponding to the hobby or preference of the factor person. The service providing device according to any one of claims 1 to 10, wherein the service providing device is determined to be provided. A notification unit that notifies information to one or more of the persons in the target range is provided.
When the factor estimation unit estimates that any of the persons is the factor person who is the cause of the change, the notification unit causes the person different from the factor person to move away from the factor person. The service providing device according to any one of claims 1 to 11, which notifies the prompting guidance. A voice acquisition unit that acquires utterances of multiple people in the target range,
An image acquisition unit that acquires an image of one or a plurality of the persons, and an image acquisition unit.
A digitization processing unit that digitizes the information acquired by the voice acquisition unit and the captured image acquisition unit to generate numerical information, and
An event information storage unit that stores the correspondence between the change in the numerical information generated by the numerical processing unit and the event information related to the event that causes the change in the numerical information.
When the numerical information based on the information acquired by the voice acquisition unit and the captured image acquisition unit changes during the detection target period, the event information corresponding to the change is used as the event information. The event information acquisition unit acquired from the storage unit and
An action determination unit that determines an operation based on the event information acquired by the event information acquisition unit,
A service providing device including an execution unit that executes the operation determined by the operation determination unit. A registration unit that registers a person who is the target of providing services or estimating personal feelings by a service providing device,
An evaluation acquisition unit that acquires the evaluation of the person for the provision of the service,
A payment information processing device including a consideration determination unit that determines the consideration for providing the service based on the evaluation acquired by the evaluation acquisition unit. Claim 14 that the consideration determination unit determines the consideration for the provision of the service based on the evaluation acquired by the evaluation acquisition unit and the change situation in which the personal emotion of the person changes in response to the provision of the service. The payment information processing device described in. Detects multiple people in the target range and
Estimate the individual emotions of each person,
Based on the estimation result of the individual emotion, the cause of the change in the entire emotional state of the target range or the emotional state of any one or more of the persons is estimated.
A service providing method for determining a service to be provided to one or more of the persons in the target range according to an estimated occurrence factor.

Images