JP2013081516A - Device and method for determining sleepiness - Google Patents

Abstract

The present invention provides a drowsiness determination device capable of accurately determining a drowsiness level even when facial expression variation occurs.
A drowsiness determination apparatus includes a facial expression variation acquisition unit that acquires a size and speed of a facial expression variation of a person, and a facial expression variation determination that determines whether the facial expression variation is less than a predetermined threshold. And a sleepiness determination unit 130 that determines the sleepiness level of a person based on the speed of the expression change on the condition that the magnitude of the expression change is less than a predetermined threshold.
[Selection] Figure 1

Description

  The present invention relates to a drowsiness determination apparatus and a drowsiness determination method for determining a human drowsiness level.

  2. Description of the Related Art Conventionally, a technique for automatically determining a person's sleepiness level has been proposed for the purpose of, for example, an alarm for preventing drowsy driving (see, for example, Patent Document 1).

  The technology described in Patent Document 1 (hereinafter referred to as “conventional technology”) is based on the distance between the driver's facial feature points (left and right) from the image of the driver's face taken by an imaging device installed in the vehicle interior. The distance between the corners of the mouth, the distance between the eyebrows and the eyes, etc.) are sequentially detected. Then, the conventional technology detects the driver's drowsiness sign by comparing the detected distance between feature points with the distance between feature points when the driver is in an awake state. More specifically, in the conventional technology, the drowsiness level of the driver is determined from the magnitude relationship between the distance between feature points in the awake state and the distance between feature points at the determination target time. Such a conventional technique can automatically determine a person's sleepiness level.

JP 2009-45418 A

Hiroki Kitajima, 3 others, “Study on prediction method of sleepiness when driving a car”, Transactions of the Japan Society of Mechanical Engineers (C), Japan Society of Mechanical Engineers, 1997, 63, 613, p. 93-100

  However, people often change their facial expressions even when the level of drowsiness is relatively high, and while the facial expressions are changing, the distance between feature points may be the same value as in the awake state. Therefore, the conventional technique has a problem that the drowsiness level cannot be accurately determined when facial expression variation occurs.

  An object of the present invention is to provide a drowsiness determination device and a drowsiness determination method capable of accurately determining a drowsiness level even when facial expression variation occurs.

  The drowsiness determination device of the present invention includes a facial expression variation acquisition unit that acquires the magnitude and speed of a facial expression variation of a person, a facial expression variation determination unit that determines whether the facial expression variation is less than a predetermined threshold value, and And a drowsiness determination unit that determines the drowsiness level of the person based on the speed of the expression variation on condition that the magnitude of the expression variation is less than the predetermined threshold.

  The drowsiness determination method of the present invention includes a step of acquiring the magnitude and speed of a human facial expression variation, a step of determining whether the facial expression variation magnitude is less than a predetermined threshold, and the facial expression variation magnitude. And a step of determining the sleepiness level of the person based on the speed of the expression change on the condition that the length is less than the predetermined threshold.

  According to the present invention, the drowsiness level can be accurately determined even when facial expression variation occurs.

The block diagram which shows an example of a structure of the drowsiness determination apparatus which concerns on Embodiment 1 of this invention. The block diagram which shows an example of a structure of the drowsiness determination apparatus and safe driving assistance apparatus which concern on Embodiment 2 of this invention. The figure which shows an example of the content of the 1st drowsiness determination table in Embodiment 2 of this invention. The flowchart which shows an example of operation | movement of the safe driving assistance apparatus which concerns on Embodiment 2 of this invention. The figure for demonstrating the acquisition method of the expression variation in Embodiment 2 of this invention The block diagram which shows an example of a structure of the drowsiness determination apparatus and safe driving assistance apparatus which concern on Embodiment 3 of this invention. The flowchart which shows an example of operation | movement of the safe driving assistance apparatus which concerns on Embodiment 3 of this invention. The block diagram which shows an example of a structure of the drowsiness determination apparatus and safe driving assistance apparatus which concern on Embodiment 4 of this invention. An example of the content of the 2nd sleepiness determination table in Embodiment 4 of the present invention is shown. The flowchart which shows an example of operation | movement of the safe driving assistance apparatus which concerns on Embodiment 4 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
Embodiment 1 of the present invention is an example of a basic aspect of the present invention.

  FIG. 1 is a block diagram showing an example of the configuration of the drowsiness determination apparatus according to Embodiment 1 of the present invention.

  In FIG. 1, the drowsiness determination device 100 includes a facial expression variation acquisition unit 110, a facial expression variation determination unit 120, and a sleepiness determination unit 130.

  The facial expression variation acquisition unit 110 acquires the magnitude and speed of human facial expression variation.

  The facial expression variation determination unit 120 determines whether the facial expression variation is less than a predetermined threshold.

  The sleepiness determination unit 130 determines the sleepiness level of the person based on the speed of the facial expression variation on the condition that the magnitude of the facial expression variation is less than a predetermined threshold.

  The sleepiness determination apparatus 100 may have a computer configuration including, for example, a storage medium such as a central processing unit (CPU) and a random access memory (RAM). In this case, the drowsiness determination device 100 operates when the CPU executes a control program to be stored.

  Such a drowsiness determination apparatus 100 can determine a human drowsiness level based on the speed of expression change (hereinafter, referred to as “expression change speed” as appropriate).

  Human facial expression fluctuations, such as pulling the mouth corner to the left and right, raising eyebrows, etc., are not only when awake (for example, when the sleepiness level is lowest. Sometimes seen. On the other hand, as one of the signs of drowsiness, there is a tendency that the facial expression fluctuation speed becomes slow. Accordingly, it can be considered that the drowsiness level is determined to be high when the facial expression variation speed is slow.

  However, the inventor has found through experiments and evaluations that rapid facial expression variation can occur even in a state where the drowsiness level is high. Therefore, if the drowsiness level is simply determined based on the facial expression fluctuation speed, there is a risk of omission of determination.

  On the other hand, the inventor has found through experiments and evaluations that, in a state in which the drowsiness level is high, fast facial expression fluctuations occur remarkably in large facial expression fluctuations and hardly occur in small facial expression fluctuations.

  Therefore, the sleepiness determination apparatus 100 excludes large facial expression variation from the determination target, and performs a human sleepiness level determination (hereinafter referred to as “sleepiness determination”) based on whether the facial expression variation speed is slow. Thereby, the drowsiness determination apparatus 100 can implement | achieve the determination of the drowsiness level with the above-mentioned high precision, preventing a misjudgment. That is, the drowsiness determination device 100 can accurately determine the drowsiness level even when facial expression variation occurs.

  In addition, since the decrease in facial expression fluctuation speed occurs even in the initial stage of sleepiness, the sleepiness determination apparatus 100 can accurately determine the sleepiness level even in the initial stage of sleepiness.

(Embodiment 2)
The second embodiment of the present invention is an example of a specific mode when the present invention is applied to a safe driving support device that issues a warning to the driver according to the drowsiness level of the vehicle driver. That is, in the present embodiment, the subject of sleepiness determination according to the present invention is a vehicle driver.

  First, the configuration of a drowsiness determination device according to the present embodiment and a safe driving support device including the device will be described.

  FIG. 2 is a block diagram illustrating an example of the configuration of the drowsiness determination device and the safe driving support device according to the present embodiment.

  In FIG. 2, the safe driving support device 200 includes a facial expression variation detection unit 210, a drowsiness determination device 100, and an output unit 220.

  The facial expression variation detection unit 210 inputs time-series data (image data, video) of a driver's image taken by a camera (not shown) installed on a column cover of the vehicle, for example. The facial expression variation detection unit 210 detects the magnitude and speed of the facial expression variation of the driver at a predetermined period (for example, a period of 0.03 seconds) from the input image data.

  Then, facial expression variation detection section 210 sequentially outputs a set of the facial expression variation magnitude and the driver's facial expression variation speed acquired at the same timing to facial expression variation acquisition section 110 as facial expression variation information. That is, facial expression fluctuation detection section 210 outputs time series data of facial expression fluctuation information acquired at a predetermined cycle to drowsiness determination device 100.

  The driver's facial expression variation is a change caused by expansion / contraction of the driver's facial expression muscle, for example, a facial movement such as drawing a corner of the mouth.

  In the present embodiment, the driver's facial expression variation is a variation in the distance between two points on the driver's face. These two points are the left and right mouth corners, which are two points that expand and contract in conjunction with the movement of the facial expression muscle. “Facial expression variation” is the variation of the distance between the driver's left and right mouth corners (hereinafter referred to as “mouth angular distance”), and “Facial expression variation speed” is the variation speed of the mouth corner distance. .

In the present embodiment, facial expression variation detection section 210 extracts the image feature points of the driver's face from the captured image, and extracts the driver's left and right mouth angles from the extraction result.
A method for acquiring facial expression variation will be described later.

  The drowsiness determination device 100 determines a human drowsiness level based on facial expression variation information input from the facial expression variation detection unit 210. The drowsiness determination device 100 includes a facial expression variation acquisition unit 110, a facial expression variation determination unit 120, and a sleepiness determination unit 130.

  The expression variation acquisition unit 110 acquires the magnitude and speed of the driver's expression variation. More specifically, facial expression variation acquisition section 110 acquires facial expression variation information from facial expression variation detection section 210 and outputs it to facial expression variation determination section 120.

  The facial expression variation determination unit 120 determines whether the facial expression variation is less than a predetermined threshold. Then, facial expression variation determination section 120 outputs facial expression variation speed included in facial expression variation information to sleepiness determination section 130 only for facial expression variation information whose facial expression variation is less than a predetermined threshold. That is, the facial expression variation determination unit 120 filters the facial expression variation information input from the facial expression variation acquisition unit 110 based on the facial expression variation included in the facial expression variation information.

  The drowsiness determination unit 130 determines the driver's drowsiness based on the facial expression variation speed on condition that the facial expression variation is less than a predetermined threshold. More specifically, the drowsiness determination unit 130 refers to the first drowsiness determination table and determines the drowsiness level based on the facial expression variation speed input from the facial expression variation determination unit 120. Then, the sleepiness determination unit 130 notifies the driver of the determination result of the driver's sleepiness level using the output unit 220 described later.

  The first drowsiness determination table is a table that describes the range of the facial expression fluctuation speed at each drowsiness level for each of a plurality of predetermined drowsiness levels.

  FIG. 3 is a diagram illustrating an example of the contents of the first sleepiness determination table.

  As shown in FIG. 3, the first sleepiness determination table 310 describes, for example, a facial expression fluctuation speed 311 and a sleepiness level 312 in association with each other. The expression variation speed 311 describes the range of the expression variation speed S (t). The sleepiness level 312 describes a predetermined sleepiness level.

Here, the parameters Th _{a1 to} Th _a4 described in the facial expression variation speed 311 are threshold values that define each range of the facial expression variation speed S (t), and satisfy the relationship represented by the following expression (1). That is, the parameter Th _a1 is the lowest threshold value and corresponds to a faster expression variation speed S (t). The parameter Th _a4 is the highest threshold value and corresponds to the slower expression variation speed S (t).
Th _a1 <Th _a2 <Th _a3 <Th _a4 (1)

  “Level 1” to “Level 5” described in the sleepiness level 312 are five levels of sleepiness defined in Non-Patent Document 1. “Level 1” corresponds to a state that does not seem sleepy at all. “Level 2” corresponds to a slightly sleepy state. “Level 3” corresponds to a sleepy state. "Level 4" corresponds to a state that seems quite sleepy. “Level 5” corresponds to a very sleepy state. That is, “level 1” corresponds to the arousal state, and “level 2” and “level 3” correspond to the initial somnolence.

  That is, the first sleepiness determination table 310 associates higher sleepiness levels with slower facial expression fluctuation speeds.

  The output unit 220 outputs notification information corresponding to the sleepiness level determined by the sleepiness determination unit 130 and notifies the driver of the determination result. The notification information takes the form of information that can be perceived by a person, such as sound, light, image, fragrance, wind, and temperature.

  In the present embodiment, the notification information is output for preventing accidents caused by driver drowsiness.

  Therefore, for example, at the drowsiness level 1, the output unit 220 outputs nothing.

  Further, for example, at the drowsiness level 2, the output unit 220 uses at least one of an emphasis sound, an aroma, a cold wind, and light that emphasizes an environmental sound present in the vehicle interior as the notification information. Thereby, it is possible to promptly prevent the driver from drowsiness. In this case, the output unit 220 is realized by a speaker, a ventilator, LED lighting, or the like.

  For example, at the drowsiness level 3, the output unit 220 uses at least one of alarm sound (beep sound, voice, ultrasonic wave), seat vibration, steering vibration, and tightening by the seat belt as the notification information.

  Further, for example, at the drowsiness level 4 and the drowsiness level 5, the output unit 220 uses at least one of a loud alarm sound, a large seat vibration, and a large steering vibration as the notification information. Further, for example, the output unit 220 may output information that promptly encourages the driver to take a break together with an alarm. Further, for example, when the drowsiness level is high and the inter-vehicle distance from the preceding vehicle is very close, the output unit 220 may intervene in the control of the vehicle and stop the vehicle urgently.

  The decrease in facial expression variation speed due to small facial expression variation occurs even in the initial stage of sleepiness. For this reason, the drowsiness determination device 100 can accurately determine the drowsiness level even in the initial stage of drowsiness. It is said that in order to prevent accidents, it is effective to prompt the driver to be alerted appropriately and to support safe driving, particularly at the initial sleepiness level up to sleepiness level 3. Therefore, the output unit 220 can improve driving safety by performing the above-described alarm in the vicinity of the drowsiness level 2 and the drowsiness level 3.

  Above, description about the structure of the drowsiness determination apparatus 100 and the safe driving assistance apparatus 200 is finished.

  Next, the operation of the safe driving support apparatus 200 including the drowsiness determination apparatus 100 will be described.

  FIG. 4 is a flowchart showing an example of the operation of the safe driving support apparatus 200.

  First, in step S2000, facial expression variation detection section 210 detects the facial expression variation magnitude D (t) and facial expression variation speed S (t) of the driver. Then, facial expression variation acquisition section 110 acquires the facial expression variation magnitude D (t) and facial expression variation speed S (t) and outputs them to facial expression variation determination section 120.

  FIG. 5 is a diagram for explaining a facial expression variation acquisition method. Here, it is assumed that the latest time (current time) is t, and the past time that is back by n is t−n. For example, the driver is in an awake state at time t-n, and the driver is in a somnolence state at time t.

As shown in FIG. 5A, immediately after time t−1, the facial expression variation detection unit 210 obtains the coordinates of the driver's right mouth corner 301 and the left mouth corner 302 from the image 300 at time t−1. , Detected as F _R (t−1) and F _L (t−1), respectively. More specifically, the facial expression variation detection unit 210 uses, for example, face feature point detection processing such as AAM (Active Appearance Model) or ASM (Active Shape Model) to perform the coordinate F _R (t−1) and the coordinate F _L (t-1) is detected as a feature point of the face in the two-dimensional coordinate system on the image.

Further, as shown in FIG. 5B, the facial expression variation detection unit 210 also applies the coordinates of the right mouth corner 303 and the left mouth corner 304 of the image 300 at the time t immediately after the time t. Coordinates are detected as F _R (t) and F _L (t), respectively.

  The expression variation detection unit 210 detects the expression variation magnitude D (t) and the expression variation speed S (t) at time t, for example, as follows.

The expression variation detection unit 210 calculates the magnitude D _d (t) of the variation in the distance between the left and right mouth corners using, for example, the following equation (2).
D _d (t)
_{= || F L (t) -} F R (t) || F L (t-n) - F R (t-n) ||
(2)

The expression variation detection unit 210 calculates the expression variation speed S (t) at time t using, for example, the following equation (3).
S (t) = | D _d (t) | / n (3)

  In addition, facial expression variation detection section 210 uses the moving average of the speeds calculated by the above equation (3) at a plurality of times as the final facial expression variation speed so that detection errors and the like are absorbed. Also good.

  Then, in step S3000 of FIG. 4, facial expression variation determination section 120 determines whether or not the facial expression variation magnitude D (t) input from facial expression variation acquisition section 110 is less than a preset threshold value α. To do. If the expression variation magnitude D (t) is not less than the threshold value α (S3000: NO), the expression variation determination unit 120 proceeds to step S4000. This is a situation where the variation in facial expression is large, and the drowsiness determination cannot be accurately detected from the decrease in the facial expression variation speed. If facial expression variation magnitude D (t) is less than threshold value α (S3000: YES), facial expression variation determination section 120 proceeds to step S5000. This is a situation in which facial expression variation is small and drowsiness determination can be accurately detected from a decrease in facial expression variation speed.

  In step S5000, facial expression variation determination section 120 outputs facial expression variation speed S (t) to drowsiness determination section 130. As a result, the drowsiness determination unit 130 determines the drowsiness level of the driver based on the expression variation speed S (t) input from the expression variation determination unit 120. More specifically, the drowsiness determination unit 130 refers to the first drowsiness determination table (see FIG. 3) to determine the drowsiness level corresponding to the facial expression variation speed S (t). Then, the sleepiness determination unit 130 outputs the determination result to the output unit 220.

  In step S6000, the output unit 220 performs processing according to the determination result input from the drowsiness determination unit 130, and proceeds to step S4000. As described above, the process according to the determination result is a process for giving a warning by sound or the like when the sleepiness level is 2 to 5, for example.

  In step S4000, facial expression variation detection section 210 determines whether or not an instruction to end the process is given by the driver's operation or the like. If the facial expression variation detection unit 210 is not instructed to end the process (S4000: NO), the facial expression variation detection unit 210 returns to step S2000 and repeats the detection of facial expression variation in the above-described predetermined cycle. In addition, facial expression variation detection section 210 ends a series of processing when it is not instructed to end processing (S4000: YES).

  By such processing, the safe driving support apparatus 200 determines that the drowsiness level is high when the magnitude of the driver's facial expression variation is small and the speed is slow, and issues a warning to the driver. be able to.

  Above, description about operation | movement of the safe driving assistance apparatus 200 containing the drowsiness determination apparatus 100 is finished.

  As described above, the drowsiness determination device 100 according to the present embodiment provides the driver's drowsiness level based on the driver's facial expression fluctuation speed on the condition that the magnitude of the facial expression fluctuation of the driver is less than a predetermined threshold. Judgment is made. As a result, the drowsiness determination device 100 can accurately determine the drowsiness level even when facial expression variation occurs.

  In addition, since the safe driving support apparatus 200 according to the present embodiment issues a warning based on such a drowsiness level determination result by the drowsiness determination apparatus 100, it is possible to more surely prevent drowsy driving while preventing false alarms. Can do.

(Embodiment 3)
The third embodiment of the present invention is an example of a specific aspect in the case where the threshold value α of the expression variation magnitude is set for each driver or for each environment.

  There are individual differences in the magnitude of facial expression fluctuations during arousal. Further, even for the same person, the magnitude of facial expression fluctuation (size on the image) at awakening varies depending on the shooting environment (distance from the camera). Therefore, the safe driving support apparatus according to the present embodiment takes a history of the expression variation magnitude before starting the drowsiness determination, and sets the threshold value α of the expression variation magnitude based on the appearance tendency of the expression variation. Make adjustments.

  FIG. 6 is a block diagram illustrating an example of the configuration of the drowsiness determination device and the safe driving support device according to the present embodiment, and corresponds to FIG. 2 of the second embodiment. The same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted.

  In FIG. 6, the safe driving support apparatus 200a according to the present embodiment includes a drowsiness determination apparatus 100a including a facial expression variation determination unit 120a different from the second embodiment. The expression variation determination unit 120a includes a threshold value determination unit 121a.

  The threshold value determination unit 121a stores a history of the magnitude of facial expression variation, and based on the history, determines the above-described threshold value α used for comparison of the magnitude of facial expression variation.

  FIG. 7 is a flowchart showing an example of the operation of the safe driving support apparatus 200a according to the present embodiment, and corresponds to FIG. 4 of the second embodiment. The same parts as those in FIG. 4 are denoted by the same step numbers, and description thereof will be omitted.

  Prior to the processing of steps S2000 to S6000 in FIG. 4, the safe driving support device 200a performs the processing of steps S1100a to 1400a described below.

  First, in step S1100a, facial expression variation detection section 210 detects the magnitude D (t) of the driver's facial expression variation. Then, facial expression variation acquisition section 110 acquires the facial expression variation magnitude D (t) and outputs it to facial expression variation determination section 120a.

In step S1200a, the threshold value determination unit 121a determines whether or not a predetermined time _TL necessary for determining the threshold value α has elapsed since the start of processing. The predetermined time _TL is a predetermined time as a time required for accumulating a history necessary for accurately adjusting the threshold value α. If the predetermined time _TL has not elapsed (S1200a: NO), the threshold determination unit 121a proceeds to step S1300a. Moreover, the threshold value determination part 121a will progress to step S1400a, when predetermined | prescribed time _TL has passed (S1200a: YES).

In step S1300a, the threshold value determination unit 121a records (accumulates) the input facial expression variation magnitude D (t), and returns to step S1100a. As a result, the threshold determination unit 121a obtains a set _{D U} size D (t) of the expression change.

In step 1400a, the threshold determination unit 121a, based on the set _{D U} size D (t) of the accumulated expression variation performs processing for determining a threshold value alpha. More specifically, the threshold value determination unit 121a determines, for example, an intermediate value or an average value of facial expression fluctuation magnitudes D acquired in the past as the threshold value α.

  Accordingly, when the driver's facial expression variation is relatively large, or when the distance between the driver and the camera is short, a larger value can be determined as the threshold value α. When the driver's facial expression variation is relatively small, or when the distance between the driver and the camera is long, a smaller value can be determined as the threshold value α.

  Such a safe driving support apparatus 200a can determine drowsiness by setting an appropriate threshold value α by adjustment. Therefore, the drowsiness level can be determined with higher accuracy, and driving safety can be further improved.

(Embodiment 4)
Embodiment 4 of the present invention is an example of a specific aspect in the case of determining a drowsiness level based on the amount of change in facial expression variation speed.

  The expression change speed during somnolence has a tendency that the amount of change is smaller than that during awakening. That is, a small amount of change is one of the signs of sleepiness. Therefore, drowsiness determination is performed using facial expression fluctuation (hereinafter referred to as “low conversion facial expression fluctuation”) with a small amount of change in speed.

  FIG. 8 is a block diagram illustrating an example of the configuration of the drowsiness determination device and the safe driving support device according to the present embodiment, and corresponds to FIG. 2 of the second embodiment. The same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted.

  The safe driving support apparatus 200b according to the present embodiment includes a drowsiness determination apparatus 100b including a drowsiness determination unit 130b different from that of the second embodiment. The sleepiness determination unit 130b includes a speed change amount calculation unit 131b.

  The speed change amount calculation unit 131b calculates the change amount of the facial expression variation speed based on the facial expression variation speed input from the facial expression variation determination unit 120 (that is, the facial expression variation speed targeted for sleepiness determination).

  In addition, the sleepiness determination unit 130b performs the driver's sleepiness determination based on the amount of change in the facial expression variation speed on condition that the facial expression variation is less than a predetermined threshold. More specifically, the drowsiness determination unit 130b refers to the second drowsiness determination table, and determines the drowsiness level based on the change amount of the expression variation speed calculated by the speed change amount calculation unit 131b. Then, the drowsiness determination unit 130b notifies the driver of the determination result of the driver's drowsiness level using the output unit 220 described later.

  The second drowsiness determination table is a table that describes the range of change in facial expression variation speed at each drowsiness level for each of a plurality of predetermined drowsiness levels.

  FIG. 9 is a diagram illustrating an example of the contents of the second drowsiness determination table.

  As shown in FIG. 9, the second drowsiness determination table 320b describes, for example, a change amount 321b of the expression fluctuation speed and a drowsiness level 322b in association with each other. The range of the variation amount ΔS (t) of the facial expression variation speed is described in the variation amount 321b of the facial expression variation speed. The sleepiness level 322b describes a predetermined sleepiness level.

Here, the parameters Th _{c1 to} Th _c4 described in the change amount 321b of the facial expression variation speed are threshold values that define each range of the facial expression variation speed ΔS (t), and satisfy the relationship shown in the following formula (4). . That is, the parameter Th _c1 is the lowest threshold value and corresponds to a faster change amount ΔS (t) of the expression variation speed. The parameter Th _c4 is the highest threshold value and corresponds to a smaller change amount ΔS (t).
Th _c1 <Th _c2 <Th _c3 <Th _c4 (4)

  Further, “level 1” to “level 5” described in the sleepiness level 322b are five levels of sleepiness levels defined in Non-Patent Document 1, as in the second embodiment.

  That is, the drowsiness determination device 100b compares the expression variation speed S (t−n) at the past time t−n with the expression variation speed S (t) at the time t. Then, the drowsiness determination device 100b determines that drowsiness is stronger as the facial expression variation speed S (t) becomes slower from time t-n to time t.

  FIG. 10 is a flowchart showing an example of the operation of the safe driving support apparatus 200b, and corresponds to FIG. 4 of the second embodiment. The same parts as those in FIG. 4 are denoted by the same step numbers, and description thereof will be omitted.

  When the expression variation magnitude D (t) is less than the threshold value α (S3000: YES), the expression variation determination unit 120 outputs the expression variation speed S (t) to the drowsiness determination unit 130b and proceeds to step S5100b. .

  In step S5100b, the speed variation calculation unit 131b records the facial expression variation speed S (t) input from the facial expression variation acquisition unit 110. As a result of repeating this step, for example, facial expression fluctuation speeds S (0) to S (t) from time 0 to time t are recorded.

  Then, in step S5200b, speed change amount calculation section 131b determines a reference speed (hereinafter referred to as “reference speed for facial expression variation”) S (B) from the facial expression variation speed recorded in the past. To do.

  Specifically, the speed change amount calculation unit 131b calculates, for example, an average value of facial expression variation speeds S (0) to S (tn) from time 0 to time t−n as a reference speed S (B of facial expression variation. ) May be determined. Further, the speed change amount calculation unit 131b may determine, for example, the maximum value of the expression variation speeds S (0) to S (t) from time 0 to time t as the reference speed S (B) for expression variation. . It should be noted that the method for determining the reference speed S (B) for facial expression variation is not limited to these, and may be any value that represents the past facial expression variation speed.

  In step S5300b, the speed change amount calculation unit 131b determines the difference of the latest expression change speed S (t) from the determined reference speed S (B) of the expression change to the change amount ΔS (t of the expression change speed. ).

Specifically, the speed change amount calculation unit 131b calculates the change amount ΔS (t) of the facial expression variation speed using, for example, the following equation (5).
ΔS (t) = | S (t) −S (B) | (5)

  In step S5400b, drowsiness determination unit 130 determines the drowsiness level of the driver based on facial expression variation speed change amount ΔS (t) calculated by speed change amount calculation unit 131b. More specifically, the drowsiness determination unit 130b refers to the second drowsiness determination table (see FIG. 9) to determine the drowsiness level corresponding to the change amount ΔS (t) of the expression variation speed. Then, the drowsiness determination unit 130b outputs the determination result to the output unit 220, and proceeds to step S6000.

  Such a safe driving support apparatus 200b can perform drowsiness determination using the low change amount fluctuation while excluding the low change amount fluctuation at the time of awakening from the determination target. Driving safety can be improved.

  In the safe driving support apparatus 200b according to the present embodiment, the threshold value α may be adjusted as in the third embodiment.

In the second to fourth embodiments described above, the method for calculating the magnitude D _d (t + n) of the variation in the distance between the left and right mouth corners is not limited to the above example.

For example, the facial expression variation detection unit obtains the expression variation magnitude D _R (t + n) of the right mouth corner and the expression variation D _L (t + n) of the left mouth corner at time t + n by the following expressions (6) and (7): ) To calculate.
_{D R (t + n) =} | F R (t + n) - F R (t) | ······ (6)
D _L (t + n) = | F _L (t + n) −F _L (t) | (7)

Then, the expression variation detection unit uses the following expression (8) to calculate the magnitude D of the distance between the left and right mouth corners from the magnitudes D _R (t + n) and D _L (t + n) of the expression variations at the left and right mouth corners. _d (t + n) is calculated as the magnitude of facial expression variation.
D _d (t + n) = D _R (t + n) −D _L (t + n) (8)

  In Embodiments 2 to 4 described above, the magnitude and speed of facial expression variation in the two-dimensional coordinate system on the image are used, but the magnitude of facial expression variation in the real space three-dimensional coordinate system. And speed may be used.

  Further, the sleepiness level to be determined is not limited to the above five levels. The sleepiness level to be determined may be, for example, three levels: “no sleepiness”, “weak sleepiness”, and “strong sleepiness”.

In this case, in the second embodiment and the third embodiment, for example, the facial expression variation speed S (t) is divided into three ranges using the two parameters Th _b1 and Th _b2 as threshold values, and sleepiness determination is performed. In this case, the parameters Th _b1 and Th _b2 satisfy the relationship represented by the following formula (9), for example.
Th _b1 <Th _b2 (9)

In the fourth embodiment, for example, the change amount ΔS (t) of the facial expression variation speed is divided into three ranges using the two parameters Th _d1 and Th _d2 as threshold values, and sleepiness determination is performed. In this case, the parameters Th _d1 and Th _d2 satisfy the relationship represented by the following formula (10), for example.
Th _d1 <Th _d2 (10)

  In addition, although the facial expression variation detection unit detects the magnitude and speed of the facial expression variation of the driver based on the image data, the present invention is not limited to this. For example, the facial expression fluctuation detection unit may acquire myoelectric information from the driver's face and detect the driver's facial expression fluctuation based on the myoelectric information. In addition, the facial expression variation detection unit may detect the magnitude and speed of the facial expression variation using values obtained by quantifying other various image features such as a change in the texture shape around the mouth.

  The present invention is useful as a drowsiness determination device and a drowsiness determination method that can accurately determine a drowsiness level even when facial expression variation occurs.

100, 100a, 100b Drowsiness determination device 110 Expression variation acquisition unit 120, 120a Expression variation determination unit 121a Threshold determination unit 130, 130b Drowsiness determination unit 131b Speed change amount calculation unit 200, 200a, 200b Safe driving support device 210 Expression variation detection unit 220 Output unit

Claims (13)

A facial expression variation acquisition unit that acquires the magnitude and speed of the facial expression variation of a person;
A facial expression variation determination unit that determines whether or not the magnitude of the facial expression variation is less than a predetermined threshold;
A drowsiness determination unit that determines the drowsiness level of the person based on the speed of the facial expression variation on the condition that the magnitude of the facial expression variation is less than the predetermined threshold;
Sleepiness determination device. The sleepiness determination unit
When the speed of the expression change is slower, a higher sleepiness level is determined as the sleepiness level of the person.
The sleepiness determination apparatus according to claim 1. The magnitude of the expression fluctuation is a value corresponding to the magnitude of the distance fluctuation between two points on the person's face, and the speed of the expression fluctuation is a value corresponding to the speed of the distance fluctuation. Is,
The sleepiness determination apparatus according to claim 2. The facial expression variation determination unit
A value corresponding to a difference with respect to a reference distance that is a distance between the two points when the human sleepiness level of the acquired distance is low is set as the magnitude of the variation.
The sleepiness determination apparatus according to claim 3. The sleepiness determination unit
With respect to each of a plurality of predetermined sleepiness levels, the sleepiness level of the person is determined with reference to a first sleepiness determination table that describes a range of the speed of expression change at the sleepiness level.
The sleepiness determination apparatus according to claim 2. The facial expression variation determination unit
Storing a history of the magnitude of the facial expression variation, and determining the predetermined threshold based on the history;
The sleepiness determination apparatus according to claim 2. The sleepiness determination unit
Determining the sleepiness level of the person based on the acquired amount of change in the speed of the facial expression variation;
The sleepiness determination apparatus according to claim 2. The sleepiness determination unit
With respect to each of a plurality of predetermined sleepiness levels, the sleepiness level of the person is determined with reference to a second sleepiness determination table that describes a range of the rate of change in the speed of expression change at the sleepiness level.
The sleepiness determination apparatus according to claim 7. The two points are two points where the distance expands and contracts in conjunction with the movement of the facial expression muscle of the person.
The sleepiness determination apparatus according to claim 3. The two points are the left and right mouth corners of the person,
The sleepiness determination apparatus according to claim 9. The two points are image feature points of the person's face extracted from an image obtained by photographing the person.
The sleepiness determination apparatus according to claim 9. The sleepiness determination unit
Notifying the person of the determination result of the person's sleepiness level using an output unit that outputs information,
The sleepiness determination apparatus according to claim 2. Obtaining the magnitude and speed of human facial expression variation;
Determining whether the magnitude of the facial expression variation is less than a predetermined threshold;
Determining the sleepiness level of the person based on the speed of the facial expression variation, provided that the facial expression variation is less than the predetermined threshold.
Drowsiness determination method.

Images