JP5939493B1 - Service evaluation device, service evaluation system and service evaluation method provided with the same - Google Patents

Abstract

It is possible to correctly evaluate a customer service attitude in consideration of the presence or absence of a customer utterance. A service evaluation device 5 receives an image input unit 21 to which an image captured in a predetermined service area is input, and a face image extraction unit 22 extracts a face image of the customer from the image. A smile degree determination unit 23 that determines a degree of smile of the customer based on a face image of the customer, and an utterance determination unit 25 that determines the presence or absence of the utterance of the customer, and the utterance determination unit 25 If the customer service is determined not to speak, the customer service attitude of the customer service is evaluated based on the smile level of the customer service determined by the smile level determination unit 23. [Selection] Figure 3

Description

  The present invention relates to a customer service evaluation apparatus for evaluating a customer service attitude based on a smile level, a customer service evaluation system including the service evaluation system, and a customer service evaluation method.

  In the service industry such as retail, restaurant, hotel, etc., it is known that good customer service attitudes of customers such as store staff and employees lead to customer satisfaction, and as a result, the customer collection rate and sales are improved. As a method for evaluating customer service attitudes, customer opinion surveys are generally conducted. However, since such customer service evaluation methods are performed with a large number of personnel, they are inefficient. There is a problem that it is both objective and poor in objectivity.

  Therefore, for example, an apparatus is known that evaluates the customer service attitude by acquiring a customer service facial image and determining the customer smile level (smile level) based on the facial image ( Patent Document 1).

JP 2010-186373 A

  However, the conventional technique described in Patent Document 1 has a problem in that the customer service attitude of the customer service cannot be correctly evaluated because the customer service is not considered. During the utterance, the customer's mouth shape changes depending on the spoken language, so it is difficult to determine the smile level of the customer during the utterance based on the customer's face image, and the smile level of the customer during the utterance Will be appreciated. Therefore, even when a similar smile is made, the degree of smile of the customer who is speaking is lower than that of the customer who is not speaking. In addition, when obtaining the average smile level per predetermined customer service time, a customer who has a lot of utterances has a lower evaluation of a smile level than a customer who has a low number of utterances. Thus, unless the presence or absence of the customer's utterance is considered, the customer service attitude of the customer cannot be correctly evaluated.

  In addition, since the customer does not smile except when the customer is serving, the evaluation of the degree of smile during the non-customer is naturally low. Therefore, when calculating the average smile level per predetermined working hours, a customer who has a short customer service time has a lower smile rating than a customer who has a long customer service time. Therefore, in order to evaluate the customer service attitudes of a plurality of customers fairly, it is necessary to correctly evaluate the customer service attitudes according to the actual situation.

  The present invention has been devised in view of such problems of the prior art, and a customer service evaluation device capable of correctly evaluating a customer service attitude in consideration of the presence or absence of a customer service utterance and The main purpose is to provide a customer service evaluation system and a customer service evaluation method provided with this.

  The customer service evaluation device of the present invention is a customer service evaluation device for evaluating the customer service attitude based on the degree of smile, and is a customer service evaluation device for evaluating the customer service attitude based on the smile level. An image input unit for inputting an image captured in a predetermined customer service area, a face image extraction unit for extracting the customer's facial image from the image, and the facial image of the customer A smile degree determination unit that determines the degree of smile of the customer and an utterance determination unit that determines whether or not the customer speaks, and the utterance determination unit determines that the customer is not speaking In this case, the customer service attitude of the customer service is evaluated based on the smile level of the customer service determined by the smile level determination unit.

  According to the present invention, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer utterance.

Overall configuration diagram of the customer service evaluation system according to the first embodiment of the present invention Explanatory drawing which shows the example of application of the customer service evaluation system which concerns on 1st Embodiment. The block diagram which shows schematic structure of the customer service evaluation apparatus in FIG. The figure for demonstrating the determination method of the customer service smile degree The figure which shows the example of a customer service information table The figure which shows the example of a smile degree evaluation value table (A) is a figure which shows the modification of a customer service information table, (B) is a figure which shows the example of a face direction evaluation value table The flowchart which shows the flow of the customer service evaluation process by the customer service evaluation apparatus which concerns on 1st Embodiment. The block diagram which shows schematic structure of the customer service evaluation apparatus which concerns on 2nd Embodiment. The flowchart which shows the flow of the customer service evaluation process by the customer service evaluation apparatus which concerns on 2nd Embodiment. Overall configuration diagram of a customer service evaluation system according to a third embodiment of the present invention Explanatory drawing which shows the example of application of the customer service evaluation system which concerns on 3rd Embodiment The block diagram which shows schematic structure of the customer service evaluation apparatus which concerns on 3rd Embodiment. The flowchart which shows the flow of the customer service evaluation process by the customer service evaluation apparatus which concerns on 3rd Embodiment. The block diagram which shows schematic structure of the customer service evaluation apparatus which concerns on 4th Embodiment. The flowchart which shows the flow of the customer service evaluation process by the customer service evaluation apparatus which concerns on 4th Embodiment. Explanatory drawing which shows the other application example of the customer service evaluation system of this invention

  A first invention made to solve the above problems is a customer service evaluation device for evaluating a customer service attitude, and an image input in which an image captured in a predetermined customer service area is input A facial image extraction unit that extracts the facial image of the customer service from the image, a smile degree determination unit that determines a degree of smile of the customer service based on the facial image of the customer service, and An utterance determination unit that determines the presence or absence of utterance, and when the utterance determination unit determines that the customer is not speaking, the degree of smile of the customer determined by the smile degree determination unit Based on the above, the customer service attitude of the customer is evaluated.

  According to the customer service evaluation device according to the first aspect of the present invention, the presence or absence of the customer's utterance is determined. When it is determined that the customer does not speak, the customer's smile is determined based on the degree of the customer's smile. Since the configuration is such that the customer service attitude is evaluated, the customer service attitude while the customer is not speaking can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service.

  A second invention made to solve the above problems is a customer service evaluation device for evaluating a customer service attitude, and an image input in which an image captured in a predetermined customer service area is input A face image extracting unit that extracts a face image of the customer from the image, an utterance determining unit that determines whether or not the customer utters, and a face that outputs a face image of the customer who is not speaking And an image output unit.

  According to the customer service evaluation device according to the second aspect of the present invention, the user is configured to output a non-uttering face image of the customer, so that the user can receive the non-speaking customer service based on the output face image. Attitude can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service.

  In the third invention, in the first or second invention, the utterance determination unit detects a movement of the customer's mouth from the facial image of the customer, and based on the movement of the mouth, It is characterized by determining the presence or absence of a customer service utterance.

  According to the service evaluation device according to the third aspect of the present invention, the presence / absence of the customer's utterance is determined in order to determine the presence / absence of the customer's utterance based on the movement of the customer's mouth detected from the customer's face image. It is possible to easily and reliably determine.

  According to a fourth aspect of the present invention, in the first or second aspect of the present invention, the apparatus further includes a voice input unit for inputting the voice of the customer service, and the utterance determination unit is configured to receive the customer service based on the voice of the customer service. It is characterized by determining the presence or absence of a person's utterance.

  According to the customer service evaluation apparatus according to the fourth aspect of the present invention, since the presence / absence of the customer's utterance is determined based on the customer's voice, the presence / absence of the customer's utterance can be easily and reliably determined. .

  According to a fifth aspect of the present invention, in the first aspect of the present invention, a detection information input unit to which detection information of a customer is input, and a customer service state indicating whether or not the customer is in service based on the detection information And a customer service state determination unit that evaluates the customer service attitude when the customer service state determination unit determines that the customer service is in service.

  According to the customer service evaluation device according to the fifth aspect of the present invention, it is determined whether or not the customer is in service based on detection information of the customer, and it is determined that the customer is in service. In this case, since the customer service attitude of the customer service is evaluated, it is possible to evaluate the customer service attitude during the customer service. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the customer service status.

  According to a sixth aspect, in the second aspect, a detection information input unit to which detection information of a customer is input, and a customer service state indicating whether or not the customer is in service based on the detection information And a customer service state determination unit, wherein the face image output unit outputs a face image of the customer who is not speaking and in service.

  According to the customer service evaluation device according to the sixth aspect of the present invention, since the customer's non-speaking and customer service face image is output, the user is receiving the customer service based on the output face image. The customer service attitude can be evaluated. Therefore, it is possible to correctly evaluate the customer service attitude in consideration of the customer service status.

  In the seventh invention, the customer service evaluation device according to any of the first to fourth inventions, and an image imaging device that captures an image in the customer service area and inputs the captured image to the customer service evaluation device. It is a customer service evaluation system characterized by comprising.

  According to an eighth aspect of the present invention, the customer service evaluation device according to the fifth or sixth invention, an image pickup device that captures an image in the customer service area, and inputs the captured image to the customer service evaluation device; A customer service evaluation system comprising: a customer service partner detection device that detects a customer and inputs detection information of the service customer to the customer service evaluation device.

  The ninth invention is characterized in that, in the eighth invention, the customer-facing party detection device is an image pickup device, a sound acquisition device, or a proximity sensor.

  According to a tenth aspect of the present invention, there is provided a customer service evaluation method for evaluating a customer service attitude of a customer, an image acquisition step of acquiring an image captured in a customer service area set in advance, and the customer service from the image. A face image extracting step for extracting a person's face image, a smile degree determining step for determining a degree of smile of the customer based on the face image of the customer, and an utterance determination for determining whether or not the customer speaks And determining the customer service attitude of the customer based on the degree of smile of the customer determined in the smile level determination step when the customer is determined not to speak in the step and the speech determination step. And a customer service attitude evaluation step for evaluation.

  An eleventh aspect of the present invention is a customer service evaluation method for evaluating a customer service attitude of a customer with a smile, an image acquisition step of acquiring an image captured in a predetermined customer service area, and the image A face image extracting step for extracting the face image of the customer, an utterance determining step for determining the presence or absence of the utterance of the customer, and a face image outputting step for outputting a face image of the customer who is not speaking. It is characterized by that.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram of the customer service evaluation system according to the first embodiment of the present invention, FIG. 2 is an explanatory diagram showing an application example of the customer service evaluation system according to the first embodiment, and FIG. It is a block diagram which shows schematic structure of the customer service evaluation apparatus in FIG.

  As shown in FIG. 1, the customer service evaluation system 1 is constructed in a store 2 or the like, and evaluates the customer service attitude of a customer service (here, a store clerk) to a customer service customer (here, a customer who visited the store) based on the degree of smile. Make it possible to do. In addition, the customer service evaluation system 1 enables a user (for example, a store manager of the store 2 or a headquarters manager) to evaluate a customer service attitude based on a customer's face image. In the store 2, as a component of the customer service evaluation system 1, there is a camera (image capturing device) 3 that captures a customer service area 13 (see FIG. 2) set inside a cashier counter 11 (see FIG. 2) in the store. The customer service partner detection device 4 for detecting the customer service partner in the customer side area 14 (see FIG. 2) set on the outer front of the cashier counter 11 and the customer service attitude of the store clerk are installed in the backyard of the store. A customer service evaluation device 5 for evaluating based on the degree of smile is provided.

  The camera 3 and the customer partner detection device 4 can communicate directly or indirectly with the customer service evaluation device 5 via a communication line 6 such as a LAN (Local Area Network). In the customer service evaluation system 1, the camera 3, the service partner detection device 4, and the customer service evaluation device 5 are connected to a wide area network 8 such as the Internet based on a public line or a dedicated line by a relay device 7 provided on the communication line 6. Via the headquarters management apparatus 9.

  In the present embodiment, the store 2 to which the customer service evaluation system 1 is applied is described as an example of a retail store such as a convenience store, but is not limited thereto. As shown in FIG. 2 (plan view inside the store), in the store 2, store clerks S1 and S2 (hereinafter referred to as “S” when referring generically) are located in the customer service area 13 set inside the cashier counter 11. The customers C1 and C2 (hereinafter referred to as “C” in the generic name) lined up in the customer-side area 14 set in front of the cashier counter 11 settles the purchased product. The customer service area 13 and the customer side area 14 can be arbitrarily set, and both the areas may be arbitrary positions in the store. The two areas may overlap each other. Note that the number of salesclerks S is not limited to two. For example, there may be one salesclerk S in a time zone where customers C are few. Further, in a time zone where there are many customers C, there may be three or more clerk S.

  The camera 3 is a CCD camera, for example, and is installed, for example, around the cash register 12 on the cash register counter 11 and continuously photographs the inside of the customer service area 13. That is, the customer service area 13 is a virtual range in which the camera 3 can shoot. The video imaged by the camera 3 is transmitted as a video signal to the customer service evaluation device 5 via the communication line 6. As long as the camera 3 can capture at least the face image of the clerk S, the function, arrangement, quantity, and the like are not particularly limited, and various changes can be made. For example, the camera 3 may be a known omnidirectional network camera installed on the ceiling above the cashier counter 11.

  The customer service partner detection device 4 may be, for example, an image pickup device (camera), a sound acquisition device (microphone), a proximity sensor, or the like. When a camera is used as the customer service partner detection device 4, for example, a known omnidirectional network camera installed on the ceiling in the store or a CCD camera installed around the cash register 12 on the cash register counter 11 can be used. The camera 4 continuously shoots the situation in the customer side area 14 in order to recognize the presence or absence of the customer in the customer side area 14. When a microphone is used as the customer service partner detection device 4, for example, a known omnidirectional network microphone installed on the ceiling in the store or a known directional microphone installed around the cash register 12 on the cash register counter 11 is used. Can be used. The microphone 4 continuously acquires (collects sound) in the customer side area 14 in order to recognize the presence or absence of the customer in the customer side area 14. Moreover, when using a proximity sensor as the customer service partner detection apparatus 4, for example, a known optical proximity sensor can be used. The sensor 4 detects the customer C in the customer side area 14.

  Detection information (video, audio, detection result) of the customer C in the customer area 14 acquired or detected by the camera, microphone, and proximity sensor that is the customer partner detection device 4 is the customer service evaluation device via the communication line 6. 5 is transmitted. In addition, as long as at least the customer C can be detected, the function, arrangement, quantity, and the like of the camera, microphone, and proximity sensor are not particularly limited, and various changes can be made.

  The customer service evaluation device 5 is a PC (Personal Computer) that is installed in the backyard of the store 2 and used by a user (store manager, etc.). As will be described later, the customer service evaluation device 5 acquires the video from the camera 3 and the detection information from the customer partner detection device 4, and executes the evaluation process of the customer service attitude of the clerk S based on the acquired video and the detection information. To do.

  Although details are not shown, the customer service evaluation device 5 is a central processing unit (CPU) that centrally executes various information processing and control of peripheral devices based on a predetermined control program, and a RAM (functioning as a work area of the CPU). Random Access Memory), ROM (Read Only Memory) for storing control programs executed by the CPU and data, network interface for executing communication processing via the network, monitor (image output device), speaker, input device, and HDD ( Hard disk drive) and the like, and the CPU performs predetermined control on at least some of the various functions of the customer service evaluation device 5 (e.g., evaluation process of customer service attitude of the store clerk S) described later. This can be realized by executing a program (for example, an image recognition program). The customer service evaluation device 5 is not limited to a PC, and other information processing devices (such as a server) capable of performing the same function can also be used. Moreover, you may substitute at least one part of the function of the customer service evaluation apparatus 5 by the process by other well-known hardware.

  The headquarters management device 9 is a PC having a configuration similar to that of the customer service evaluation device 5, and can execute processing similar to that of the customer service evaluation device 5. The headquarters management device 9 is used by a headquarters administrator who manages a plurality of stores 2 similar to the stores 2. A configuration in which the head office management device 9 shares part of the evaluation process of the customer service attitude of the clerk S by the customer service evaluation device 5 is also possible.

  As shown in FIG. 3, the customer service evaluation device 5 includes an image input unit 21 to which video is input from an image pickup device (camera) 3, a face image extraction unit 22 that extracts a face image of the clerk S from the image, A smile degree determination unit 23 that determines the degree of smile of the clerk S based on the face image of the clerk S (hereinafter referred to as “smile degree”), and customer service information that stores the smile degree determined by the smile degree determination unit 23 A storage unit 24; an utterance determination unit 25 that determines the presence or absence of an utterance of the clerk S based on the face image of the clerk S; a detection information input unit 26 that receives detection information of the customer C from the customer service partner detection device 4; A customer service state determination unit 27 that determines whether or not the store clerk S is in service based on the detection information, and an evaluation of the customer service attitude of the store clerk S based on the smile level determined by the smile level determination unit 23 Evaluation value calculation unit 28 for calculating a value and evaluation value calculation unit 28 And a evaluation value storage unit 29 for storing the evaluation value of customer attitudes calculated clerk S. The customer service information storage unit 24 also stores the determination results of the utterance determination unit 25 and the customer service state determination unit 27. Each unit is controlled by a control unit (not shown).

  A face image of the clerk S located in the customer service area 13 photographed by the camera 3 is input to the image input unit 21 as a video signal. The video data of the input video signal includes the shooting time of the face image of the clerk S.

  The face image extraction unit 22 performs a face image extraction process for extracting the face image of the clerk S from the image (a plurality of image frames) input to the image input unit 21 using a known face image recognition technique. At this time, the process of determining the clerk ID of the clerk S from which the face image is extracted is also performed. Specifically, referring to the face image DB (not shown) of the clerk S, the clerk ID of the clerk S from which the face image is extracted is determined.

  The smile level determination unit 23 determines the smile level of the clerk S based on the shape of the mouth and eyes using a known face image recognition technique. The degree of smile is determined as a percentage (%). First, the facial expression of the clerk S is detected from the facial image of the clerk S using a known face image recognition technique. And as shown in FIG. 4, when it is determined that the facial expression of the clerk S is (A), the smile degree is determined to be 80%, and the facial expression of the clerk S is determined to be (B) The smile level is determined to be 60%, and when the salesperson S's facial expression is determined to be (C), the smile level is determined to be 20%. Note that the smile degree determination method shown in FIG. 4 is merely an example, and actually, it may be determined in more detail.

  The customer service information storage unit 24 stores the smile level of the clerk S determined by the smile level determination unit 23. The smile level of the clerk S is recorded in the customer service information table (see FIG. 5) stored in the customer service information storage unit 24 together with the shooting time and the clerk ID of the clerk S. In the customer service information table of FIG. 5, the smile level of the clerk S1 is 80 points at 8:12:24, 70 points at 8:12:25, and 60 at 8:12:26. It is shown that it was 70 points at 8:12:27 and 20 points at 8:12:28. In addition, the smile degree of the clerk S2 was 30 points at 8:12:32, 60 points at 8:12:33, and 80 points at 8:12:34. Has been. In addition, the determination results of the utterance determination unit 25 and the customer service state determination unit 27 are also input to the customer service information storage unit 24 and recorded in the customer service information table in association with the smile level of the clerk S.

  The utterance determination unit 25 determines the presence or absence of the utterance of the clerk S from which the face image is extracted using a known face image recognition technique. Specifically, the utterance determination unit 25 detects the movement of the clerk's S mouth from the face image of the clerk S, and determines the presence or absence of the utterance of the clerk S based on the movement of the mouth. If a predetermined mouth movement is detected, the clerk S is determined to be speaking, and if no predetermined mouth movement is detected within a predetermined number of image frames, the clerk S is determined not to speak. The The determination result in the utterance determination unit 25 is input to the customer service information storage unit 24 and is recorded as “speaking” or “not speaking” in the customer service information table (see FIG. 5).

  The detection information input unit 26 receives the detection information of the customer C in the customer area 14 acquired by the customer partner detection device 4, that is, information indicating the presence or absence of the customer C in the customer area 14. The This detection information includes the time when the customer service partner detection device 4 acquires the information.

  The customer service status determination unit 27 determines the customer service status as to whether or not the clerk S is in customer service based on the detection information of the customer C input to the detection information input unit 26. When the customer C exists in the customer side area 14, it is determined that the customer is in service, and when the customer C does not exist in the customer side area 14, it is determined that the customer C is not in service. The determination result in the customer service state determination unit 27 is input to the customer service information storage unit 24 and is recorded in the customer service information table (see FIG. 5) as “serving customers” or “not receiving customers”. When the customer service state determination unit 27 determines that the customer service is in progress, there may be a case where the clerk S does not exist in the customer service area 13, but in such a case, the customer clerk S is not evaluated for the customer service attitude. .

  Based on the customer service information table (see FIG. 5) stored in the customer service information storage unit 24, the evaluation value calculator 28 calculates an evaluation value of the customer service attitude of each clerk S during customer service and during non-utterance. Specifically, the average value of the smile levels while each clerk S is in customer service and not speaking is calculated. The calculated average value of the smile level of each clerk S during customer service and during non-speaking is recorded in a smile level evaluation value table (see FIG. 6) stored in the evaluation value storage unit 29.

  In the customer service information table shown in FIG. 5, the store clerk S1 is in service / not speaking at 8:12:24 and is in service from 8:12:25 to 8:12:27.・ Speaking, it is shown that at 8:12:28, the customer was not serving or talking. In addition, the store clerk S2 was not in service / speaking at 8:12:32 and was in service / not speaking at 8:12:33 to 8:12:34. It is shown. In addition, as a case of non-serving customers / speaking, a conversation with another store clerk or a greeting to a customer outside the customer side area 14 can be considered.

  And the evaluation value calculation part 28 calculates the average value of the smile degree of each store clerk S during customer service and not speaking based on the customer service information table shown in FIG. The store clerk S1 is in customer service and is not speaking at 8:12:24, and the smile level at that time is 80, so the average value of the smile level of the store clerk S1 during customer service and not speaking is 80. . In addition, the store clerk S2 is in service and is not speaking from 8:12:33 to 8:12:34, and the smile level at that time is 60 and 80, so the store clerk S2 is in service and is not speaking. The average smile level is 70. In this way, the evaluation value calculation unit 28 can calculate the average value of the smile level while each store clerk S is serving customers and not speaking. In the smile degree evaluation value table shown in FIG. 6, the average value of the smile degree during the customer service and non-speaking of the clerk S1 is 80, and the average value of the smile degree during the customer service and non-speaking of the clerk S2 is 70. It is shown that there is.

  Further, the utterance determination unit 25 may determine the face direction or the line-of-sight direction of the clerk S using a known face image recognition technique. During the customer service, it is preferable that the face of the clerk S is facing the customer C. Therefore, the evaluation value of the clerk S's face or line of sight is added to the evaluation of the clerk S's attitude to the customer. Good. In this case, the determination result in the speech determination unit 25 is input to the customer service information storage unit 24 and added to the customer service information table shown in FIG. FIG. 7A is obtained by adding the determination result of the face direction of the clerk S to the customer service information table of FIG. The face orientation of the clerk S is recorded as “customer direction” or “other than customer direction”. Then, the evaluation value calculation unit 28 calculates an evaluation value of the orientation of the face of the clerk S during customer service based on the customer service information table shown in FIG. The calculated evaluation value of the face direction during customer service is recorded in a face direction evaluation value table (see FIG. 7B) stored in the evaluation value storage unit 29.

  In the customer service information table shown in FIG. 7 (A), the direction of the face of the clerk S1 is the customer direction at 8:12:24, and the time at 8:12:25 is other than the customer direction. It is indicated that the time from 8:12:26 to 8:12:27 is the customer direction, and the time of 8:12:28 is other than the customer direction. The direction of the face of store clerk S2 is 8:12:32 except for the customer direction, 8:12:33 is the customer direction, and 8:12:34 is not the customer direction. It was shown that it was.

  Then, the evaluation value calculation unit 28 calculates the evaluation value of the orientation of the face of each clerk S during customer service based on the customer service information table shown in FIG. The evaluation value of the face direction is represented by a ratio (percent (%)) that the face direction of the clerk S who is serving the customer is facing the customer direction. The store clerk S1 is in service from 8:12:24 to 8:12:27, and the direction of the face of the store clerk S1 at that time is 8:12:24, 8:12:26, The time at 8:12:27 was for the customer, and the time at 8:12:25 was not for the customer. Therefore, the evaluation value of the direction of the face of the store clerk S1 during the customer service is 75%. Also, the store clerk S2 is in service from 8:12:33 to 8:12:34, and the direction of the face of the store clerk S2 at that time is 8:12:33. , 8:12:34 was not in the direction of the customer. Therefore, the evaluation value of the direction of the face of the store clerk S2 during customer service is 50%. In this way, the evaluation value calculation unit 28 can calculate the evaluation value of the face orientation of each clerk S during customer service. In the face direction evaluation value table shown in FIG. 7B, the evaluation value of the face direction during the customer service of the clerk S1 is 75%, and the evaluation value of the face direction during the customer service of the clerk S2 is 50%. It is shown that there is.

  Furthermore, the customer service evaluation device 5 includes an evaluation value output unit 30 that outputs the evaluation value of the customer service attitude of the store clerk S stored in the evaluation value storage unit 29 as desired by the user. As described above, the evaluation value of the customer service attitude of the clerk S is the average value of the smile degree during the customer service and non-speaking of the clerk S recorded in the smile evaluation value table (see FIG. 6), or It is an evaluation value of the direction of the face of the store clerk S during customer service recorded in the direction evaluation value table (see FIG. 7A).

  Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the first embodiment shown in FIG. 3 will be described with reference to FIG.

  First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image pickup device (camera) 3 arranged in the store (ST101). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST102). Next, the smile level determination unit 23 determines the smile level of the clerk S based on the face image of the clerk S extracted by the face image extraction unit 22 (ST103).

  In subsequent ST104, the utterance determination unit 25 determines the presence or absence of the clerk S's utterance based on the face image of the clerk S extracted by the face image extraction unit 22. The presence / absence of the clerk's utterance is recorded in the customer service information table (see FIG. 5). If it is determined that the clerk S is speaking (speaking), the process returns to step S101. If it is determined that the clerk S is not speaking (not speaking), the process proceeds to step S105.

  In step S <b> 105, the detection information input unit 26 acquires detection information of the customer C in the customer side area 14 from the customer partner detection device 4 arranged in the store. As described above, the detection information of the customer C is information indicating whether or not there is a customer service partner in the customer area 14.

  In subsequent ST 106, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service, and when the customer C is not present in the customer side area 14, it is determined that the customer is not in service. The customer service status of the clerk S is recorded in the customer service information table (see FIG. 5). If it is determined that the customer is in service, the process proceeds to step S107. If it is determined that the customer is not in service, the process returns to step S101.

  In ST107, the evaluation value calculation unit 28 calculates the evaluation value of the customer service attitude of the clerk S based on the smile level determined by the smile level determination unit 23. The evaluation value of the customer service attitude of the clerk S calculated by the evaluation value calculation unit 28 is stored in the evaluation value storage unit 29 and is output from the evaluation value output unit 30 according to the user's desire.

  In this manner, according to the customer service evaluation device 5 according to the first embodiment, when the store clerk S is not speaking and serving customers, the evaluation value of the customer service attitude of the store clerk S can be calculated. Therefore, it is possible to correctly evaluate the customer service attitude of the store clerk S in consideration of the presence / absence of the customer's utterance and the customer service status.

(Second Embodiment)
FIG. 9 is a block diagram showing a schematic configuration of the customer service evaluation device according to the second embodiment of the present invention, and FIG. 10 is a flowchart showing the flow of customer service evaluation processing by the customer service evaluation device according to the second embodiment. is there. The customer service evaluation device according to the second embodiment is used in the customer service evaluation system 1 shown in FIG.

  The customer service evaluation device 5 according to the second embodiment illustrated in FIG. 9 is similar to the customer service evaluation device 5 according to the first embodiment illustrated in FIG. 3, and includes an image input unit 21, a face image extraction unit 22, and an utterance determination. Unit 25, detection information input unit 26, and customer service state determination unit 27. The customer service evaluation device 5 according to the second embodiment further includes a face image information storage unit 32 and a face image output unit 33 that are not provided in the customer service evaluation device 5 according to the first embodiment. The same parts as those in the customer service evaluation device 5 according to the first embodiment shown in FIG.

  The face image information storage unit 32 stores the face image of the clerk S acquired by the face image extraction unit 22. The face image of the clerk S is stored in the face image information storage unit 32 together with the shooting time and the clerk ID of the clerk S. The face image information storage unit 32 also receives the determination results of the utterance determination unit 25 and the customer service state determination unit 27 and records them in association with the face image of the clerk S.

  The face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in accordance with an instruction from the user. At this time, only a face image in which the customer service state is “busy” and the utterance state is “not speaking” may be read and output. The face image of the clerk S may be output to a known image output device such as a liquid crystal monitor connected to the customer service evaluation device 5, or as a video signal (video data) via the wide area network 8, the head office management device 9. May be sent to.

  Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the second embodiment shown in FIG. 9 will be described with reference to FIG.

  First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image capturing device (camera) 3 arranged in the store (ST201). Subsequently, the face image extraction unit 22 extracts the face image of the customer service (store clerk S) from the image input to the image input unit 21 (ST202). The face image of the clerk S extracted by the face image extraction unit 22 is stored in the face image information storage unit 32 (ST203).

  Next, in ST204, the utterance determination unit 25 determines the presence or absence of the clerk S's utterance based on the face image of the clerk S extracted by the face image extraction unit 22. Whether or not the clerk S speaks is recorded in the face image information storage unit 32. If it is determined that the clerk S is speaking, the process returns to step S201. If it is determined that the clerk S is not speaking, the process proceeds to step S205.

  In step 205, the detection information input part 26 acquires the detection information of the customer C in the customer side area 14 from the customer service partner detection apparatus 4 (refer FIG. 1) arrange | positioned in the shop. In subsequent ST 206, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service, and when the customer C is not present in the customer side area 14, it is determined that the customer is not in service. The customer service state of the clerk S is recorded in the face image information storage unit 32. If it is determined that the customer is in service, the process proceeds to step S207. If it is determined that the customer is not in service, the process returns to step S201.

  In ST207, the face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in response to an instruction from the user. At this time, only the face image of the store clerk S who is in customer service and not speaking may be read and output.

  In this way, according to the customer service evaluation device 5 according to the second embodiment, it is possible to output a face image of the clerk S who is not speaking and in service. Then, the user can evaluate the customer service attitude while the customer is not speaking and during customer service based on the output face image. Therefore, it is possible to correctly evaluate the customer service attitude of the store clerk S in consideration of the presence / absence of the customer's utterance and the customer service status.

(Third embodiment)
FIG. 11 is an overall configuration diagram of the customer service evaluation system according to the third embodiment of the present invention, FIG. 12 is an explanatory diagram showing an application example of the customer service evaluation system according to the third embodiment, and FIG. FIG. 14 is a block diagram illustrating a schematic configuration of a customer service evaluation device according to a third embodiment of the present invention, and FIG. 14 is a flowchart illustrating a flow of customer service evaluation processing performed by the customer service evaluation device according to the third embodiment.

  The customer service evaluation system according to the third embodiment shown in FIG. 11 is different from the customer service evaluation system 1 according to the first embodiment shown in FIG. Since other points are the same as the customer service evaluation system 1 according to the first embodiment shown in FIG. 1, the same reference numerals are given and the description thereof is omitted.

  The voice acquisition device 10 is a general directional microphone, and is arranged around the cash register 12 of the cashier counter 11 as shown in FIG. The microphone 10 continuously acquires (collects sound) in the customer service area 13. In addition, as long as the microphone 10 can acquire at least the voice of the clerk S, the function, arrangement, quantity, and the like are not particularly limited, and various changes can be made. For example, a configuration in which microphones are arranged at a plurality of locations according to the arrangement of the store clerk S in the store, or a configuration in which microphones are attached to the clothes of the store clerk S is also possible.

  The customer service evaluation device according to the third embodiment shown in FIG. 13 includes a voice input unit 31 and a point that the speech determination unit 25 is connected to the voice input unit 31 instead of the face image extraction unit 22. However, it is different from the customer service evaluation device 5 according to the first embodiment shown in FIG. Since the other points are the same as those in the customer service evaluation device 5 according to the first embodiment shown in FIG. 3, the same reference numerals are given and the description thereof is omitted.

  The voice of the clerk S in the customer service area 13 acquired by the microphone 10 is input to the voice input unit 31. This audio information includes the time when the microphone 10 acquired the audio. The utterance determination unit 25 is connected to the voice input unit 31 and determines the presence or absence of the utterance of the clerk S based on the voice input to the voice input unit 31.

  Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the third embodiment shown in FIG. 13 will be described with reference to FIG.

  First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image capturing device (camera) 3 arranged in the store (ST301). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST302). Next, the smile level determination unit 23 determines the smile level of the clerk S based on the face image of the clerk S extracted by the face image extraction unit 22 (ST303).

  In subsequent ST304, the voice input unit 31 acquires the voice of the clerk S in the customer service area 13 from the voice acquisition device (microphone) 10 disposed in the store. In continuing ST305, the speech determination part 25 determines the presence or absence of the speech of the salesclerk S based on the voice of the salesclerk S which the voice input part 31 acquired. When the clerk's S voice is acquired, it is determined that the utterance is being performed, and when the clerk's S voice is not acquired, it is determined that the utterance is not being spoken. The presence / absence of the clerk's utterance is recorded in the customer service information table (see FIG. 5). If it is determined that the clerk S is speaking, the process returns to step S301. If it is determined that the clerk S is not speaking, the process proceeds to step S306.

  In step S306, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer partner detection device 4 arranged in the store. In subsequent ST307, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. When the customer C is present in the customer side area 14, it is determined that the customer is in service, and when the customer C is not present in the customer side area 14, it is determined that the customer is not in service. The customer service status of the clerk S is recorded in the customer service information table (see FIG. 5). If it is determined that the customer is in service, the process proceeds to step S308. If it is determined that the customer is not in service, the process returns to step S301.

  In ST308, the evaluation value calculation unit 28 calculates the evaluation value of the customer service attitude of the clerk S based on the smile level determined by the smile level determination unit 23. The evaluation value of the customer service attitude of the clerk S calculated by the evaluation value calculation unit 28 is stored in the evaluation value storage unit 29 and is output from the evaluation value output unit 30 according to the user's desire.

  In this way, according to the customer service evaluation device 5 according to the third embodiment, since the presence or absence of the clerk S can be determined based on the voice of the clerk S, the presence or absence of the clerk S can be more easily and easily determined. It is possible to make a reliable determination. Since the evaluation value of the customer service attitude of the store clerk S can be calculated when the store clerk S is not speaking and customer service, the customer service attitude of the store clerk S is considered in consideration of the presence or absence of the customer utterance and the customer service status. Can be correctly evaluated.

(Fourth embodiment)
FIG. 15 is a block diagram showing a schematic configuration of a customer service evaluation device according to the fourth embodiment of the present invention, and FIG. 16 is a flowchart showing a flow of customer service evaluation processing by the customer service evaluation device according to the fourth embodiment. is there. The customer service evaluation device according to the fifth embodiment of the present invention is used in the customer service evaluation system 1 shown in FIG.

  The customer service evaluation device 5 according to the fourth embodiment has a point that the voice input unit 31 is provided and that the speech determination unit 25 is connected to the voice input unit 31 instead of the face image extraction unit 22. It is different from the customer service evaluation device 5 according to the second embodiment shown in FIG. Since the other points are the same as those in the customer service evaluation device 5 according to the second embodiment shown in FIG. 9, the same reference numerals are given and description thereof is omitted.

  The voice input unit 31 receives the voice of the clerk S in the customer service area 13 acquired by the microphone 10. This audio information includes the time when the microphone 10 acquired the audio. The utterance determination unit 25 is connected to the voice input unit 31 and determines the presence or absence of the utterance of the clerk S based on the voice input to the voice input unit 31.

  Next, the flow of customer service evaluation processing by the customer service evaluation device 5 according to the fourth embodiment shown in FIG. 15 will be described with reference to FIG.

  First, the image input unit 21 of the customer service evaluation device 5 acquires an image in the customer service area 13 from the image pickup device (camera) 3 arranged in the store (ST401). Subsequently, the face image extraction unit 22 extracts the face image of the clerk S from the image input to the image input unit 21 (ST402). The face image of the clerk S extracted by the face image extraction unit 22 is stored in the face image information storage unit 32 (ST403).

  In subsequent ST404, the voice input unit 31 acquires the voice of the clerk S in the customer service area 13 from the voice acquisition device (microphone) 10 arranged in the store. In continuing ST405, the speech determination part 25 determines the presence or absence of the speech of the salesclerk S based on the voice of the saleswoman S which the voice input part 31 acquired. Whether or not the clerk S speaks is recorded in the face image information storage unit 32. If it is determined that the clerk S is speaking, the process returns to step S401. If it is determined that the clerk S is not speaking, the process proceeds to step S406.

  In step S406, the detection information input unit 26 acquires the detection information of the customer C in the customer side area 14 from the customer partner detection device 4 arranged in the store. In subsequent ST407, the customer service state of the clerk S is determined based on the detection information input to the detection information input unit 26. The customer service state of the clerk S is recorded in the face image information storage unit 32. If it is determined that the customer is in service, the process proceeds to step S408. If it is determined that the customer is not in service, the process returns to step S401.

  In ST408, the face image output unit 33 outputs the face image of the clerk S stored in the face image information storage unit 32 in response to an instruction from the user. At this time, only the face image of the store clerk S who is in customer service and not speaking may be read and output.

  In this way, according to the customer service evaluation device 5 according to the fourth embodiment, the presence or absence of the clerk S can be determined based on the voice of the clerk S. It is possible to make a reliable determination. Since the face image of the clerk S who is not speaking and during the customer service can be output, the customer service attitude of the clerk S can be correctly evaluated in consideration of the presence or absence of the clerk S and the customer service state. .

  Further, the customer service area is not limited to the vicinity of the cashier counter 11 and can be arbitrarily set. For example, as shown in FIG. 17, the entire store can be a customer service area. In this case, the camera 3 and the customer partner detection device 4 are preferably installed on a ceiling at an appropriate place in the store. Moreover, when using the microphone 10, the microphone 10 is good to attach to the clothes of each store clerk S1-S3.

  The customer service evaluation system, customer service evaluation device, and customer service evaluation method of the present invention can also be applied to commercial facilities such as restaurants and department stores, and customer service windows such as banks, travel agencies, and real estate companies. However, the customer service area can be set arbitrarily. For example, in the case of a restaurant, it is possible to set the area around the sales counter or cash register counter as a customer service area. In the case of a department store, the entire sales area can be used as a customer service area. Can set a counter facing the customer in the customer service area.

  As mentioned above, although this invention was demonstrated based on specific embodiment, these embodiment is an illustration to the last, Comprising: This invention is not limited by these embodiment. In addition, as for each component of the customer service evaluation apparatus which concerns on this invention shown in the said embodiment, the customer service evaluation system provided with this, and a customer service evaluation method, all are not necessarily essential, unless it deviates at least from the scope of the present invention. It is possible to select appropriately.

  For example, in the above embodiment, the presence / absence of the customer's utterance and the customer service status are determined, and the customer service attitude of the customer service is evaluated when the customer is not speaking and customer service. The customer service attitude of the customer may be evaluated only when the customer is not speaking.

  The customer service evaluation apparatus, the customer service evaluation system and the customer service evaluation method according to the present invention make it possible to correctly evaluate the customer service attitude in consideration of the presence or absence of the customer service and the customer service state. It is useful as a customer service evaluation apparatus according to the above, a customer service evaluation system equipped with the same, and a customer service evaluation method.

1 Customer Service Evaluation System 2 Store 3 Camera (Image Capture Device)
4 Service partner detection device 5 Service evaluation device 21 Image input unit 22 Face image extraction unit 23 Smile degree determination unit 24 Service information storage unit 25 Speech determination unit 26 Detection information input unit 27 Service state determination unit 28 Evaluation value calculation unit 29 Evaluation value Storage unit 30 Evaluation value output unit

Claims (11)

A customer service evaluation device for evaluating customer service attitudes,
An image input unit for inputting an image captured in a predetermined customer service area;
A face image extracting unit for extracting the face image of the customer from the image;
A smile degree determination unit that determines the degree of smile of the customer based on the customer's face image;
An utterance determination unit for determining the presence or absence of the utterance of the customer,
When the utterance determination unit determines that the customer is not speaking, evaluating the customer service attitude of the customer based on the degree of smile of the customer determined by the smile degree determination unit Characteristic customer service evaluation device. A customer service evaluation device for evaluating customer service attitudes,
An image input unit for inputting an image captured in a predetermined customer service area;
A face image extracting unit for extracting the face image of the customer from the image;
An utterance determination unit for determining the presence or absence of the utterance of the customer;
A customer service evaluation apparatus comprising: a face image output unit that outputs a face image of the customer who is not speaking.   The utterance determination unit detects movement of the customer's mouth from the facial image of the customer, and determines whether or not the customer's utterance is based on the movement of the mouth. The customer service evaluation apparatus according to claim 2. A voice input unit for receiving the voice of the customer;
The customer service evaluation device according to claim 1, wherein the utterance determination unit determines the presence or absence of the customer utterance based on the voice of the customer service. A detection information input unit for inputting the detection information of the service partner;
A customer service state determination unit that determines whether or not the customer service is in service based on the detection information;
The customer service evaluation device according to claim 1, wherein when the customer service state determination unit determines that the customer service is in service, the customer service attitude of the customer service is evaluated. A detection information input unit for inputting the detection information of the service partner;
A customer service state determination unit that determines whether or not the customer service is in service based on the detection information;
3. The customer service evaluation device according to claim 2, wherein the face image output unit outputs a face image when the customer is not speaking and during customer service. The customer service evaluation device according to any one of claims 1 to 4,
A service evaluation system comprising: an image capturing device that captures an image in the service area and inputs the captured image to the service evaluation device. The customer service evaluation device according to claim 5 or 6,
An image capturing device that captures an image in the customer service area and inputs the captured image to the customer service evaluation device;
A customer service evaluation system comprising: a customer service partner detection device that detects a customer service partner and inputs detection information of the customer service customer to the customer service evaluation device.   The customer service evaluation system according to claim 8, wherein the customer partner detection device is an image pickup device, a sound acquisition device, or a proximity sensor. A customer service evaluation method for evaluating customer service attitudes,
An image acquisition step of acquiring an image captured in a preset service area;
A face image extraction step of extracting the customer's face image from the image;
A smile degree determination step for determining a degree of smile of the customer based on the customer's face image;
An utterance determination step for determining the presence or absence of the utterance of the customer;
A customer who evaluates the customer service attitude of the customer based on the smile level of the customer determined in the smile degree determination step when the customer is determined not to speak in the speech determination step A customer service evaluation method comprising: an attitude evaluation step. A customer service evaluation method for evaluating customer service attitudes,
An image acquisition step of acquiring an image captured in a preset service area;
A face image extraction step of extracting the customer's face image from the image;
An utterance determination step for determining the presence or absence of the utterance of the customer;
A customer service evaluation method comprising: a facial image output step of outputting a facial image of the customer who is not speaking.

Images