JP7531782B2 - Driver state estimation system - Google Patents

Description

The present invention relates to a driver state estimation system, and in particular to a driver state estimation system that estimates the state of a driver who drives a mobile object.

This type of technology is disclosed, for example, in the following patent documents. Patent documents 1 and 2 disclose a human state estimation device that determines which of the eye movement and head movement occurs first when a rider of a saddle-type vehicle moves their gaze toward a visual target, and estimates the internal state of the person based on the result of this determination. Patent documents 1 and 2 propose the idea that when a visual target is present within the visual field, eye movement is likely to occur before head movement, and when a visual target is not present within the visual field, head movement is likely to occur before eye movement. Based on this idea, the devices in Patent documents 1 and 2 estimate that the more head movement occurs before eye movement, the narrower the rider's visual field is, i.e., the lower the level of attention.

Patent Document 3 also discloses a method for determining signs of drowsiness before a vehicle driver, machine operator, or other person becomes aware of drowsiness, based on reflex eye movement (vestibulo-ocular reflex) that rotates the eyes in the opposite direction to head movement at approximately the same speed to obtain a clear field of vision.

Patent No. 6068964 specification Patent No. 6344747 specification Patent No. 6432702 specification

However, the devices described in Patent Documents 1 and 2 estimate a person's internal state based on the sequence of eye and head movements when moving the gaze towards a visual target, and therefore require active action of moving the gaze towards a visual target different from the current one as a prerequisite for estimating the internal state. Therefore, in situations where there is no gaze movement, such as when the driver continues to gaze at one object, the driver's state cannot be estimated. Furthermore, with the technology in Patent Document 3, it is not possible to determine the driver's signs of drowsiness unless the vestibulo-ocular reflex occurs. Thus, there is a problem with conventional technology in that it is not possible to estimate the driver's state at the desired timing.

The present invention has been made to solve the problems of the conventional technology described above, and aims to provide a driver state estimation system that can estimate the driver's state at any desired timing.

In order to achieve the above object, the present invention provides a driver state estimation system for estimating the state of a driver who drives a moving body, the system comprising: a camera for photographing the face of the driver; a deceleration device for causing deceleration in the moving body; a memory for storing a program; and a processor for executing the program, the processor detects the pitch angle of the driver's head and the pitch angle of the eyes based on an image of the face photographed by the camera, causes a constant deceleration in the moving body by the deceleration device independent of the driver's intention, and maintains the constant deceleration for a data acquisition time required to estimate the driver's state. If, while the reduction gear device is maintaining the deceleration of the moving body at a constant deceleration , the eye pitch angle reaches its maximum value while the head pitch angle increases from its minimum value to its maximum value, and the eye pitch angle reaches its minimum value while the head pitch angle decreases from its maximum value to its minimum value, the driver's condition is presumed to be normal; if, while the reduction gear device is causing deceleration to the moving body, the times at which the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle appear do not satisfy the conditions for presuming the driver's condition to be normal, the driver's condition is presumed to be abnormal.
In the present invention configured as described above, the driver's state can be estimated at a desired timing without waiting for the driver's active behavior or the natural occurrence of the vestibulo-ocular reflex. Furthermore, since the driver's state is estimated based on the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values, rather than on the simple order in which the head and eye movements occur, the estimation can reflect disturbances in the coordinated movement between the head pitch angle change and the eye pitch angle change due to abnormalities in the vestibulo-ocular reflex, thereby improving the reliability of the estimation of the driver's state.

In addition , it is possible to prevent the head pitch angle from changing unnecessarily in response to fluctuations in deceleration, which would affect the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values, thereby ensuring the reliability of the estimation of the driver's state.

In the present invention, the processor preferably estimates that the driver's condition is normal when, while the deceleration device is decelerating the moving body , the sequence of appearance of the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle is repeated, such that the eye pitch angle reaches a maximum value while the head pitch angle increases from a minimum value to a maximum value, and the eye pitch angle reaches a minimum value while the head pitch angle decreases from the maximum value to a minimum value.
In the present invention configured in this manner, the disruption of the coordinated movement between the change in head pitch angle and the change in eye pitch angle due to an abnormality in the vestibulo-ocular reflex can be more accurately reflected in the estimation, thereby improving the reliability of the estimation of the driver's condition.

In the present invention, the reduction gear is preferably a brake gear.
In the present invention thus configured, deceleration is generated by the brake device, so that the deceleration required for estimating the driver's state can be reliably generated.

In the present invention, the moving body is preferably an automobile, and the reduction gear is an accessory that utilizes power output from a power source of the automobile.
In the present invention thus configured, the deceleration of the vehicle is generated by the auxiliary device, so that the deceleration required for estimating the driver's state can be generated while effectively utilizing the power output from the power source.

The driver state estimation system of the present invention can estimate the driver's state at any desired time.

1 is a plan view showing a schematic configuration of a vehicle to which a driver's state estimation system according to an embodiment of the present invention is applied. 1 is a block diagram showing a functional configuration of a driver's state estimation system according to an embodiment of the present invention. 4 is a flowchart of a main routine of a driver's state estimation process according to the embodiment of the present invention. 5 is a flowchart of a subroutine of a driver's state estimation process according to the embodiment of the present invention. 5 is a graph illustrating a change in a head pitch angle and an eye pitch angle when a driver's state estimation system according to an embodiment of the present invention estimates a driver's state.

The driver state estimation system according to an embodiment of the present invention will be described below with reference to the attached drawings.

<System Configuration>
First, the configuration of a driver's state estimating system according to the present embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a plan view showing a schematic configuration of a vehicle to which the driver's state estimating system according to the present embodiment is applied, and Fig. 2 is a block diagram showing the functional configuration of the driver's state estimating system according to the present embodiment.

In FIG. 1, reference numeral 1 denotes a vehicle equipped with a driver state estimation system according to this embodiment. A power source 4 that drives drive wheels (left and right front wheels 2 in the example of FIG. 1) is mounted on the front of the vehicle body of the vehicle 1. The power source 4 is, for example, an internal combustion engine such as a gasoline engine or a diesel engine, or an electric motor, and in this embodiment, it is a gasoline engine. An air conditioner compressor 6 and an alternator 8 are provided near the power source 4 as accessories driven by the power source 4.

The vehicle 1 also includes a brake system 10 for generating braking force. The brake system 10 includes brake devices 12 provided on each wheel, a hydraulic pump 14 for generating brake fluid pressure required to generate braking force in the brake devices 12, and a brake controller 16 for controlling the hydraulic pressure supplied from the hydraulic pump 14 to the brake devices 12. The hydraulic pump 14 is driven by power supplied from a battery, for example, and is capable of generating the brake fluid pressure required to generate braking force in each brake device 12 even when the brake pedal is not depressed (i.e., when the driver has no intention of decelerating the vehicle 1). The brake controller 16 is electrically connected to an ECU (Electronic Control Unit) 18, allowing for mutual input and output of signals.

A camera 20 is also provided in the cabin of the vehicle 1 to capture an image of the driver's face. The camera 20 is installed, for example, above the front window of the vehicle 1, and captures the situation inside the cabin including the driver. The image captured by the camera 20 is sent to the ECU 18.

A display 22 is provided in front of the driver in the vehicle cabin, which displays various information based on the control of the ECU 18. For example, a liquid crystal display or a head-up display is used as the display 22. In addition, a speaker 24 is provided in the vehicle cabin, which outputs various information as audio based on the control of the ECU 18.

Furthermore, the vehicle 1 is equipped with an acceleration sensor 26 that detects the acceleration and deceleration of the vehicle 1. The acceleration sensor 26 outputs the detection value to the ECU 18.

As shown in FIG. 2, the ECU 18 has a processor 28 and memory 30 (ROM, RAM, etc.) for storing various programs (including basic control programs such as the OS and application programs that are run on the OS to realize specific functions) that are interpreted and executed on the processor 28, and various data.

In this vehicle 1, the ECU 18 performs various controls. In this embodiment, in addition to controlling the power source 4, the brake controller 16, the air conditioner compressor 6, the alternator 8, etc., the ECU 18 detects the pitch angle of the driver's head and the pitch angle of the eyes using an image of the face captured by the camera 20, and estimates the driver's state based on these head and eye pitch angles.

<Control Processing>
Next, the driver's state estimation process according to this embodiment will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a flowchart of a main routine of the driver's state estimation process according to this embodiment, and Fig. 4 is a flowchart of a subroutine of the driver's state estimation process. The flow of Fig. 3 is started by the ECU 18 when the power supply of the vehicle 1 is turned on, and is repeatedly executed at a predetermined cycle.

First, in step S1, ECU 18 determines whether it is time to perform driver state estimation. Specifically, ECU 18 determines that it is time to perform driver state estimation when, for example, a shaky steering operation is detected within a recent predetermined time period using known technology, when drowsiness is detected from the degree to which the driver's eyelids are open, or at every predetermined time. As a result, if it is determined that it is not time to perform driver state estimation, ECU 18 ends the driver state estimation process.

On the other hand, if it is determined that it is time to perform driver state estimation, the process proceeds to step S2, where the ECU 18 guides the driver's gaze to an object in front of the driver. Specifically, for example, the ECU 18 causes the display 22 or speaker 24 to output a message urging the driver to look at the vehicle ahead, the display 22, the meter panel, or the like. The object to which the driver's gaze is guided, such as the vehicle ahead, the display 22, or the meter panel, can be determined according to the driving conditions, such as the vehicle speed and the road on which the vehicle is traveling.

Next, in step S3, the ECU 18 starts detecting the pitch angle of the driver's head and the pitch angle of the eyes based on the image of the face captured by the camera 20. The detection of the pitch angle of the head and the pitch angle of the eyes can be realized by existing image processing technology. The pitch angle of the head and the pitch angle of the eyes are acquired as time series data.

Next, in step S4, the ECU 18 starts decelerating the vehicle 1. Specifically, the ECU 18 outputs a deceleration command to the brake controller 16, instructing a predetermined deceleration (e.g., 1 m/ ^s2 ) for estimating the driver's state. The brake controller 16 pre-stores a map that specifies the relationship between the deceleration to be generated in the vehicle 1 and the rotation speed of the hydraulic pump 14, and by referring to this map, operates the hydraulic pump 14 at the rotation speed corresponding to the deceleration instructed by the ECU 18. The control of the brake system 10 in step S4 is performed regardless of the operation of the brake pedal by the driver.

After the start of deceleration in step S4, in step S5, the ECU 18 waits until the deceleration of the vehicle 1 reaches a value (e.g., 1 m/ ^s2 ) instructed to the brake controller 16. When the deceleration of the vehicle 1 reaches the value instructed by the ECU 18, the brake controller 16 controls the hydraulic pump 14 based on the detection value of the acceleration sensor 26 so as to maintain the deceleration constant.

In step S5, when the deceleration of vehicle 1 reaches a value instructed to the brake controller 16 (e.g., 1 m/ ^s2 ) and the deceleration is maintained constant by the brake controller 16, the process proceeds to step S6, and the ECU 18 obtains the times when the maximum and minimum values of the head pitch angle and eye pitch angle appear after the deceleration of vehicle 1 becomes constant, based on the time series data of the head pitch angle and eye pitch angle.

Next, in step S7, the ECU 18 determines whether the data acquisition time (e.g., 2 seconds; hereinafter, referred to as the "estimated time" as necessary) required to estimate the driver's state has elapsed after the deceleration of the vehicle 1 has become constant. If the result shows that the estimated time has not elapsed, the process returns to step S6. Thereafter, the process of step S6 is repeated until the estimated time has elapsed.

If it is determined in step S7 that the estimated time has elapsed since the deceleration of vehicle 1 became constant, the process proceeds to step S8, where ECU 18 ends guiding the driver's gaze to the object ahead, detecting the pitch angle of the head and the pitch angle of the eyes, and decelerating vehicle 1.

Next, in step S9, the ECU 18 estimates the driver's state based on the time series data of the head pitch angle and eye pitch angle acquired during the estimated time after the deceleration of the vehicle 1 becomes constant. The subroutine for estimating the driver's state in step S9 will be described with reference to FIG. 4.

As shown in Fig. 4, first, in step S11, the ECU 18 judges whether the order of appearance of the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle after the deceleration of the vehicle 1 becomes constant is a predetermined order. Specifically, if the maximum and minimum values of the head pitch angle and the eye pitch angle appear in the order of minimum head pitch angle → maximum eye pitch angle → maximum head pitch angle → minimum eye pitch angle → minimum head pitch angle (the same order is repeated hereafter) after the deceleration of the vehicle 1 becomes constant, the ECU 18 estimates that the driver's condition is normal. In this case, the process proceeds to step S12, where 2 points are added to the score representing the driver's condition. The initial value of this score is set to 0 points.
On the other hand, if the maximum and minimum values of the head pitch angle and eye pitch angle do not appear in the above order, the ECU 18 estimates that the driver's condition is abnormal and skips step S12 (i.e., does not add points).

Next, in step S13, the ECU 18 judges whether the time from the minimum value of the head pitch angle to the maximum value of the eye pitch angle and the time from the maximum value of the head pitch angle to the minimum value of the eye pitch angle are within a predetermined range. Specifically, if the time from the minimum value of the head pitch angle to the maximum value of the eye pitch angle and the time from the maximum value of the head pitch angle to the minimum value of the eye pitch angle are each within a predetermined time (specifically, for example, from 0 seconds to less than 0.05 seconds. This numerical range is experimentally obtained), the ECU 18 estimates that the driver's condition is normal. In this case, the process proceeds to step S14, where one point is added to the score representing the driver's condition.
On the other hand, if at least one of the time from the minimum value of the head pitch angle to the maximum value of the eye pitch angle and the time from the maximum value of the head pitch angle to the minimum value of the eye pitch angle is outside the above range, the ECU 18 estimates that the driver's condition is abnormal. In this case, the ECU 18 skips step S14 (i.e., does not add points).

Next, in step S15, the ECU 18 judges whether the time from when the deceleration of the vehicle 1 becomes constant to when the head pitch angle becomes minimum is equal to or longer than a predetermined value. Specifically, if the time from when the deceleration of the vehicle 1 becomes constant to when the head pitch angle becomes minimum is equal to or longer than 0.2 seconds, the ECU 18 estimates that the driver's condition is normal. In this case, the process proceeds to step S16, where one point is added to the score representing the driver's condition.
On the other hand, if the time from when the deceleration of the vehicle 1 becomes constant to when the head pitch angle reaches the minimum value is less than 0.2 seconds, the ECU 18 estimates that the driver's condition is abnormal. In this case, the ECU 18 skips step S16 (i.e., does not add points).

Next, in step S17, the ECU 18 performs a final estimation of the driver's state based on the total score added during steps S11 to S16. Specifically, if the total score is 3 points or more, the driver's state is estimated to be normal, and if the total score is 2 points or less, the driver's state is estimated to be abnormal. After step S17, the process returns to the main routine, and the ECU 18 ends the driver's state estimation process.

<Example>
Next, an example of driver's state estimation by the driver's state estimation system of this embodiment will be described with reference to FIG.
The inventors of the present application conducted a driving experiment simulating a situation in which a driver's state estimation system estimates a driver's state. Figure 5 is a graph illustrating changes in the head pitch angle and eye pitch angle of the driver in the driving experiment, where Figure 5(a) shows a case in which the driver's state is normal (case 1), and Figure 5(b) shows a case in which the driver's state is abnormal (case 2). In addition, for case 2 in which the driver's state is abnormal, an abnormality in the autonomic nervous system was reproduced by intentionally making the sympathetic nervous system dominant by giving the subject a mental arithmetic load during the experiment.

5(a) and 5(b), the horizontal axis indicates time [s], the vertical axis on the left indicates acceleration [m/ ^s2 ], and the vertical axis on the right indicates the pitch angle [deg] of the head and eyes. A negative acceleration value means that the vehicle 1 is decelerating. The pitch angles of the head and eyes increase as the head and eyes point upward, and decrease as the head and eyes point downward. In the graphs, the solid line indicates acceleration, the dotted line indicates the head pitch angle, and the dashed-dotted line indicates the eye pitch angle, with ○ indicating the minimum value of the head pitch angle, ● indicating the maximum value of the head pitch angle, △ indicating the maximum value of the eye pitch angle, and ▲ indicating the minimum value of the eye pitch angle.

Case 1 shown in Figure 5(a) illustrates the change in pitch angle of the head and eyes when the driver's state is normal. A normal driver state means that the balance function that keeps the head stable and the coordinated movement of the head and eyes are normal.

Specifically, first, as vehicle 1 starts to decelerate (step S3 in FIG. 3), the head pitch angle starts to decrease. This is because inertial forces act to rotate the driver's head downward. At this time, the driver's line of sight is guided to an object ahead (the vehicle ahead or the display 22) (step S2 in FIG. 3), so an ocular reaction (vestibulo-ocular reflex) is triggered that rotates the eyes in the opposite direction to the head rotation to keep the object in the center of the field of view. In other words, the eye pitch angle increases.

Thereafter, the head pitch angle gradually converges while repeatedly increasing and decreasing as the balancing function operates to stabilize the head against the inertial force caused by the deceleration of the vehicle 1. In case 1 shown in Fig. 5(a), after the deceleration of the vehicle 1 starts to be maintained constant at a predetermined value (e.g., 1 m/ ^s2 ) at time _Ts , the head pitch angle starts to increase at time T1, which is 0.2 seconds or more after the deceleration of the vehicle ₁ starts to be maintained constant at time Ts, as the muscle force trying to suppress the downward rotation of the head exceeds the inertial force. That is, the head pitch angle reaches its first minimum value at time _T1 . Then, at time _T2 , the head pitch angle starts to decrease again. That is, at time _T2 , the head pitch angle reaches its maximum value. Thereafter, until the deceleration of the vehicle 1 ends at time _T8 (step S9 in Fig. 3), the head pitch angle repeatedly increases and decreases, reaching a minimum value at time _T3 , a maximum value at time _T4 , a minimum value at time _T5 , and a maximum value at time _T6 .

As described above, when the head pitch angle increases or decreases, the eye pitch angle also increases or decreases due to the vestibular ocular reflex. That is, when the head pitch angle starts to increase at time _T1 , the eye pitch angle reaches a maximum value _ΔT1 after time _T1 and starts to decrease. Thereafter, the eye pitch angle reaches a minimum value from time _T2 to _ΔT2 , a maximum value from time _T3 to _ΔT3 , a minimum value from time _T4 to _ΔT4 , a delayed maximum value from time _T5 , and a minimum value from time _T6 to _ΔT6 .

In this case 1, after the deceleration of the vehicle 1 becomes constant at time _Ts , the maximum and minimum values of the head pitch angle and eye pitch angle appear in the following order: minimum head pitch angle → maximum eye pitch angle → minimum head pitch angle → minimum head pitch angle (the same order is repeated hereafter), so 2 points are added to the score representing the driver's state (step S12 in FIG. 4).
Furthermore, since the time from the minimum value of the head pitch angle to the maximum value of the eye pitch angle ( _ΔT1 , _ΔT3 , _ΔT5 ) and the time from the maximum value of the head pitch angle to the minimum value of the eye pitch angle ( _ΔT2 , _ΔT4 , _ΔT6 ) are each within a predetermined time (specifically, within a range of 0 seconds or more and less than 0.05 seconds), one point is added to the score representing the driver's condition (step S14 in Figure 4).
Furthermore, since the time from time T _S when the deceleration of the vehicle 1 becomes constant to time T ₁ when the head pitch angle reaches its first minimum value is 0.2 seconds or more, one point is added to the score (step S16 in FIG. 4).
As a result of the above, the total added points is 4 points, which is more than 3 points, so that the driver's condition is correctly estimated to be normal (step S17 in FIG. 4).

On the other hand, Case 2 shown in Figure 5(b) illustrates the change in pitch angle of the head and eyes when the driver's condition is abnormal. An abnormal driver condition means that the balance function that tries to keep the head stable and the coordinated movement of the head and eyes are not functioning normally.

Specifically, the balance function that stabilizes the head does not function normally, so the head is more susceptible to the influence of the inertial force caused by the deceleration of the vehicle 1 than under normal circumstances. As a result, in Case 2, the head pitch angle reaches a minimum value at time T1 _' , 0.2 seconds before the deceleration of the vehicle 1 starts to be maintained constant at a predetermined value (e.g., 1 m/ ^s2 ) at time Ts _' . Therefore, step S16 in Figure 4 is skipped and no points are added.

Furthermore, the vestibulo-ocular reflex does not function normally, disrupting the coordinated movement between the change in head pitch angle and the change in eye pitch angle. In Case 2, the maximum value of the head pitch angle and the minimum value of the eye pitch angle appear almost simultaneously (i.e., with a time difference of less than X seconds) at times T2 _' and _T4 '. Also, the time difference between the minimum value of the head pitch angle at time T3 _' and the subsequent maximum value of the eye pitch angle is greater than Y seconds. Therefore, step S14 in FIG. 4 is skipped and no points are added.

However, also in this case 2, after the deceleration of the vehicle 1 becomes constant at time _Ts' , the maximum and minimum values of the head pitch angle and eye pitch angle appear in the following order: minimum head pitch angle → maximum eye pitch angle → minimum head pitch angle → minimum head pitch angle (the same order is repeated hereafter), so two points are added to the score in step S12 of FIG. 4.
As a result of the above, the total added points is 2 points, which is less than 3 points, so that the driver's condition is correctly estimated to be abnormal (step S17 in FIG. 4).

<Modification>
Next, a modified example of the present invention will be described.
In the above-described embodiment, the driver's state is estimated based on (1) the order in which the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle appear, (2) the time from when the head pitch angle reaches its minimum value until the eye pitch angle reaches its maximum value and the time from when the head pitch angle reaches its maximum value until the eye pitch angle reaches its minimum value, and (3) the time from when the deceleration of the vehicle 1 reaches a constant value until the head pitch angle reaches its minimum value. However, the driver's state may be estimated based on any one of these three parameters or a combination of any two of them.

In the above embodiment, the ECU 18 controls the brake system 10 in step S4 in Fig. 3 to decelerate the vehicle 1 regardless of the driver's operation of the brake pedal. However, the vehicle 1 may be decelerated by controlling auxiliary devices including the air conditioner compressor 6 and the alternator 8. Specifically, in step S4 in Fig. 3, the ECU 18 outputs a control command to drive the air conditioner compressor 6 and the alternator 8 by the power source 4. As a result, a part of the driving force output from the power source 4 is used to drive the air conditioner compressor 6 and the alternator 8, so that the driving force for driving the drive wheels 2 is reduced, and the vehicle 1 decelerates independently of the driver's intention. At this time, the ECU 18 controls the driving load of the air conditioner compressor 6 and the alternator 8 to generate a predetermined deceleration (e.g., 1 m/ ^s2 ) for estimating the driver's state.

In the above embodiment, if the order of appearance of the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle after the deceleration of the vehicle 1 becomes constant is a predetermined order, two points are added to the score representing the state of the driver (step S12 in FIG. 4), if the time from the minimum value of the head pitch angle to the maximum value of the eye pitch angle and the time from the maximum value of the head pitch angle to the minimum value of the eye pitch angle are within a predetermined range, one point is added to the score (step S14 in FIG. 4), and if the time from when the deceleration of the vehicle 1 becomes constant to when the head pitch angle becomes the minimum value is equal to or greater than a predetermined value, one point is added to the score (step S16 in FIG. 4). However, the number of points added in each case may be different.

In addition, while the above-described embodiment shows an example in which the driver state estimation system is applied to a four-wheeled automobile, the driver state estimation system according to this embodiment can also be applied to vehicles other than four-wheeled automobiles and moving bodies other than vehicles (e.g., trains, airplanes, ships).

<Action and effect>
Next, the effects of the driver's state estimating system according to the above-described embodiment and modified example will be described.

First, while the vehicle 1 is being decelerated by the brake device 12 or accessories independent of the driver's will, if the eye pitch angle reaches a maximum value while the head pitch angle increases from a minimum value to a maximum value, and if the eye pitch angle reaches a minimum value while the head pitch angle decreases from the maximum value to the minimum value, the ECU 18 presumes that the driver's condition is normal, and if this is not the case, the ECU 18 presumes that the driver's condition is abnormal.
Therefore, the driver's state can be estimated at a desired timing without waiting for the driver's active behavior or the natural occurrence of the vestibulo-ocular reflex. Furthermore, since the driver's state is estimated based on the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values, rather than on the simple order in which the head and eye movements occur, the estimation can reflect disturbances in the coordinated movement between the head pitch angle change and the eye pitch angle change due to an abnormality in the vestibulo-ocular reflex, thereby improving the reliability of the estimation of the driver's state.

In addition, the ECU 18 causes a constant deceleration of the vehicle 1 using the brake device 12 and auxiliary equipment, independent of the driver's intentions, and therefore prevents the head pitch angle from changing unnecessarily in response to fluctuations in deceleration, which would affect the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values. This ensures the reliability of the estimation of the driver's state.

In addition, while the vehicle 1 is being decelerated, the ECU 18 estimates the driver's state based on the timing of appearance of multiple maximum and minimum values of the head pitch angle and multiple maximum and minimum values of the eye pitch angle, so that the estimation can more accurately reflect disruptions in the coordinated movement between changes in the head pitch angle and changes in the eye pitch angle due to abnormalities in the vestibulo-ocular reflex, thereby improving the reliability of the estimation of the driver's state.

In addition, while the brake device 12 and auxiliary equipment are causing the vehicle 1 to decelerate, the ECU 18 adds a score in each of the following cases: (1) when the order in which the head pitch angle's maximum and minimum values and the eye pitch angle's maximum and minimum values appear is a predetermined order; (2) when the time from when the head pitch angle reaches its minimum value to when the eye pitch angle reaches its maximum value and the time from when the head pitch angle reaches its maximum value to when the eye pitch angle reaches its minimum value are within a predetermined range; or (3) when the time from when the deceleration of the vehicle 1 reaches a constant value to when the head pitch angle reaches its minimum value is equal to or greater than a predetermined value; and if the total score is equal to or greater than a threshold value, the ECU 18 presumes that the driver's condition is normal.
Therefore, the driver's state can be estimated at a desired timing without waiting for the driver's active behavior or the natural occurrence of the vestibulo-ocular reflex. Furthermore, the driver's state is estimated using multiple parameters related to the balance function that tries to keep the head stable and the coordinated movement of the head and eyes, rather than simply the occurrence sequence of the head and eye movements, so that the reliability of the estimation of the driver's state can be improved.

In addition, (1) the points added when the order of appearance of the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle is a predetermined order is greater than the points added when (2) the time from when the head pitch angle reaches its minimum value to when the eye pitch angle reaches its maximum value and the time from when the head pitch angle reaches its maximum value to when the eye pitch angle reaches its minimum value are within a predetermined range, and (3) the time from when the deceleration of vehicle 1 reaches a constant value to when the head pitch angle reaches its minimum value is equal to or greater than a predetermined value.
That is, the weighting of the parameters that are likely to cause a difference between normal and abnormal driver states is increased, so that the reliability of the estimation of the driver state can be further improved.

In addition, after the ECU 18 guides the driver's gaze to an object in front of the driver by outputting information from the display 22 and speaker 24, while decelerating the vehicle 1 using the brake device 12 and auxiliary equipment independently of the driver's will, if the order in which the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle appear is a predetermined order, the ECU 18 presumes that the driver's condition is normal, and if this is not the case, the ECU 18 presumes that the driver's condition is abnormal.
Therefore, the driver's state can be estimated at a desired timing without waiting for the driver's active behavior or the natural occurrence of the vestibulo-ocular reflex. Furthermore, since the driver's state is estimated based on the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values, rather than on the simple order in which the head and eye movements occur, the estimation can reflect disturbances in the coordinated movement between the change in head pitch angle and the change in eye pitch angle due to an abnormality in the vestibulo-ocular reflex, improving the reliability of the estimation of the driver's state. Furthermore, since the driver's line of sight is guided to an object in front of the driver, it is possible to prevent unnecessary changes in the line of sight from affecting the order in which the head pitch angle and eye pitch angle reach their maximum and minimum values. This ensures the reliability of the estimation of the driver's state.

In addition, because the driver's gaze is guided to the display 22 in front of the driver, the position and timing for guiding the driver's gaze can be precisely controlled, further improving the reliability of the estimation of the driver's state.

In addition, the driver's gaze is guided to other vehicles ahead, so the driver's gaze can be guided without interfering with driving operations.

In addition, the ECU 18 generates a constant deceleration of the vehicle 1 using the brake device 12 and auxiliary equipment independently of the driver's intentions, and if the time from when the deceleration reaches a constant value to when the head pitch angle reaches a minimum value is equal to or greater than a threshold value, it presumes that the driver's condition is normal, and if this is not the case, it presumes that the driver's condition is abnormal.
Therefore, the driver's state can be estimated at a desired timing without waiting for the driver's active behavior or the natural occurrence of the vestibulo-ocular reflex. Furthermore, since the driver's state is estimated based on the time from when the deceleration reaches a constant value to when the head pitch angle reaches a minimum value, rather than simply on the sequence of occurrence of the head and eye movements, the estimation can reflect disturbances in the equilibrium function that tries to stabilize the head against the inertial force caused by the deceleration of the vehicle 1, thereby improving the reliability of the estimation of the driver's state.

In particular, by setting the threshold value for estimating the driver's state to 0.2 seconds, it is possible to appropriately estimate whether the driver's state is normal or abnormal.

In addition, the brake device 12 decelerates the vehicle 1, so the deceleration required to estimate the driver's state can be reliably generated.

In addition, the auxiliary equipment generates deceleration in the vehicle 1, so it is possible to generate the deceleration required to estimate the driver's state while effectively utilizing the power output from the power source 4.

1 Vehicle 2 Drive wheels (front wheels)
4. Power source (engine)
6 Air conditioner compressor 8 Alternator 10 Brake system 12 Brake device 14 Hydraulic pump 16 Brake controller 18 ECU
20 Camera 22 Display 24 Speaker 26 Acceleration sensor 28 Processor 30 Memory

Claims (4)

A driver state estimation system that estimates a state of a driver who drives a moving object,
A camera for photographing the face of the driver;
a deceleration device that generates a deceleration on the moving body;
A memory for storing programs;
A processor for executing the program;
The above processor is
Detecting a pitch angle of the driver's head and a pitch angle of the driver's eyes based on an image of the driver's face captured by the camera;
causing a constant deceleration of the moving body by the deceleration device independent of the intention of the driver, and maintaining the constant deceleration for a data acquisition time required to estimate the state of the driver;
while the deceleration device is maintaining the deceleration of the moving body at the constant deceleration , if the pitch angle of the eyeballs reaches a maximum value while the pitch angle of the head increases from a minimum value to a maximum value, and if the pitch angle of the eyeballs reaches a minimum value while the pitch angle of the head decreases from the maximum value to the minimum value, the driver's condition is estimated to be normal,
while the deceleration device is decelerating the moving body, if the times at which the maximum and minimum values of the pitch angle of the head and the maximum and minimum values of the pitch angle of the eyeballs appear do not satisfy the conditions for estimating the state of the driver to be normal, the state of the driver is estimated to be abnormal.
Driver state estimation system. 2. The driver state estimation system of claim 1, wherein the processor estimates that the driver's state is normal when a sequence of appearance of the maximum and minimum values of the head pitch angle and the maximum and minimum values of the eye pitch angle is repeated such that the eye pitch angle reaches a maximum value while the head pitch angle increases from a minimum value to a maximum value, and the eye pitch angle reaches a minimum value while the head pitch angle decreases from the maximum value to the minimum value, while the deceleration device is maintaining the deceleration of the moving body at the constant deceleration. 3. The driver's state estimating system according to claim 1 , wherein the reduction gear is a brake gear. 3. The driver's state estimation system according to claim 1, wherein the moving body is an automobile, and the reduction gear is an accessory that utilizes power output from a power source of the automobile.

Images