JP6852407B2 - Driver status detector - Google Patents

Description

The present invention relates to a driver state detection device.

If the driver of the vehicle suffers severe pain due to a sudden illness or the like, the driver cannot drive the vehicle properly. Therefore, it is desirable to be able to monitor the driver status and detect that the driver is in an inoperable state.

Conventionally, several methods have been proposed for detecting that the driver is inoperable. For example, Patent Document 1 discloses that an inoperable state of a driver is detected based on the magnitude of the amplitude of the shaking of the driver's head detected after an external force is applied to a moving vehicle. ..

International Publication No. 2015/198542

However, in this method, the inoperable state of the driver can be detected only when an external force is applied to the moving vehicle, and the inoperable state of the driver cannot be detected quickly.

Therefore, an object of the present invention is to provide a driver state detection device capable of quickly detecting an inoperable state of a driver.

In order to solve the above problems, a driver monitor camera that photographs the driver's face of the vehicle and generates the driver's face image, an eye information detection unit that detects the driver's eye information based on the face image, and the like. A driver state determination unit that determines the state of the driver based on the eye information is provided, and the driver state determination unit has a first difference in the degree of eye opening of both eyes of the driver detected by the eye information detection unit. A driver state in which the driver is determined to be in an inoperable state when the value is equal to or greater than the threshold value, or when the difference in height between the eyes of the driver detected by the eye information detection unit is equal to or greater than the second threshold value. A detector is provided.

According to the present invention, there is provided a driver state detection device capable of quickly detecting an inoperable state of a driver.

FIG. 1 is a block diagram showing a configuration of a driver state detection device according to an embodiment of the present invention. FIG. 2 is a diagram schematically showing the inside of a vehicle equipped with a driver state detection device. FIG. 3 is a diagram showing an example of a driver's face image. FIG. 4 is a flowchart showing a control routine of the eye information detection process in the present embodiment. FIG. 5 is a flowchart showing a control routine of the driver state determination process in the present embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a driver state detection device according to an embodiment of the present invention. The driver state detection device 1 is mounted on the vehicle and detects the state of the driver of the vehicle. The driver state detection device 1 includes a driver monitor camera 10 and an electronic control unit (ECU) 20.

FIG. 2 is a diagram schematically showing the inside of a vehicle equipped with a driver state detection device. The driver monitor camera 10 captures the face of the driver of the vehicle 80 and generates a driver image. The driver monitor camera 10 is provided in the vehicle. Specifically, as shown in FIG. 2, the driver monitor camera 10 is provided above the steering column 81 of the vehicle 80. The driver monitor camera 10 may be provided at another position as long as the driver's face of the vehicle 80 can be photographed. For example, the driver monitor camera 10 may be provided on the steering wheel 82, the rearview mirror 83, the meter panel, the meter hood, or the like of the vehicle 80. Further, the driver state detection device 1 may include a plurality of driver monitor cameras 10.

The driver monitor camera 10 is composed of a camera and a floodlight. For example, the camera is a CMOS (complementary metal oxide semiconductor) camera or a CCD (charge-coupled device) camera, and the floodlight is an LED (light emitting diode). Further, the floodlight is preferably a near-infrared LED so that the driver's face can be photographed without causing discomfort to the driver even in low illuminance such as at night. For example, the floodlight is two near-infrared LEDs located on either side of the camera. Further, the camera may be provided with a filter such as a visible light cut filter.

The ECU 20 is a microcomputer provided with a central arithmetic unit (CPU), a read-only memory (ROM), a random access memory (RAM), an input port, and an output port connected to each other by a bidirectional bus. In the present embodiment, one ECU 20 is provided, but a plurality of ECUs may be provided for each function. The ECU 20 controls an eye information detection unit 21 that detects eye information, a driver state determination unit 22 that determines the driver status, a vehicle control unit 23 that controls various actuators 40 of the vehicle 80, and a driver monitor camera 10. Includes a camera control unit 24.

The camera control unit 24 of the ECU 20 controls the driver monitor camera 10 to photograph the driver's face. The driver's face image generated by the driver monitor camera 10 is transmitted from the driver monitor camera 10 to the ECU 20. The eye information detection unit 21 of the ECU 20 detects the driver's eye information based on the driver's face image generated by the driver monitor camera 10. Specifically, the eye information detection unit 21 detects the driver's line of sight, eye opening degree, difference in eye opening degree between both eyes, and difference in height between both eyes from the driver's face image. Such detection is performed by, for example, the following image processing.

The eye information detection unit 21 identifies a face region from the driver's face image and detects the face parts by extracting feature points of the face parts such as eyes, nose, and mouth. After that, the eye information detection unit 21 detects the position of the Purkinje image (corneal reflex image) and the position of the center of the pupil, and detects the line of sight of the driver from the positional relationship between the Purkinje image and the center of the pupil. Further, the eye information detection unit 21 detects the degree of opening of both eyes by extracting the feature points of the upper and lower eyelids of the eye, and detects the difference in the degree of opening of both eyes from the degree of opening of both eyes. Further, the eye information detection unit 21 detects the central axis of the face from the positions of the face region and the facial component, and detects the distance between the eyes in the direction of the central axis of the face as the difference in height between the eyes. For example, the center of the pupil is detected as the position of the eye. For reference, FIG. 3 shows the difference Hd between the center axis X of the face and the height of both eyes in the driver's face image.

The driver state determination unit 22 of the ECU 20 determines the driver state based on the eye information detected by the eye information detection unit 21. Specifically, the driver state determination unit 22 determines that the driver is looking aside when the driver's line of sight is out of the predetermined range for a predetermined time or longer. Further, the driver state determination unit 22 determines that the driver is dozing when both eyes of the driver are closed for a predetermined time or longer.

By the way, when the driver of the vehicle 80 suffers severe pain due to a sudden illness such as subarachnoid hemorrhage, the driver cannot drive the vehicle 80 properly. Also, when a person suffers from severe pain, the pain distorts his face. When the face is distorted, typically at least one of the difference in the degree of eye opening and the difference in the height of both eyes is larger than usual. Therefore, the driver state determination unit 22 uses the driver when the difference in the degree of opening of both eyes of the driver is equal to or greater than the first threshold value or when the difference in height between the driver's eyes is equal to or greater than the second threshold value. Determined to be inoperable. As a result, the driver state detection device 1 can quickly detect the inoperable state of the driver.

When the driver status determination unit 22 determines that the driver is looking aside or taking a nap, the driver status determination unit 22 gives an alarm to the driver. For example, the driver state determination unit 22 gives an alarm to the driver visually or audibly via a human machine interface (HMI) 50. The HMI 50 is an interface for inputting / outputting information between the driver and the vehicle 80. The HMI 50 is composed of, for example, a display for displaying character or image information, a speaker for generating sound, an operation button for a driver to perform an input operation, a touch panel, a microphone, and the like.

Further, the vehicle control unit 23 of the ECU 20 controls various actuators 40 of the vehicle 80 to retract the vehicle 80 when it is determined that the driver is in an inoperable state. For example, the vehicle control unit 23 turns on the hazard lamp of the vehicle 80 and gradually reduces the vehicle speed to stop the vehicle 80 on the shoulder.

Hereinafter, the control for detecting the driver state by using the driver state detection device 1 will be described in detail with reference to the flowcharts of FIGS. 4 and 5. FIG. 4 is a flowchart showing a control routine of the eye information detection process in the present embodiment. This control routine is repeatedly executed by the ECU 20 at predetermined time intervals while the ignition switch of the vehicle 80 is turned on. In this control routine, the driver's eye information is detected.

First, in step S101, the camera control unit 24 controls the driver monitor camera 10 to photograph the driver's face. Next, in step S102, the eye information detection unit 21 detects the driver's eye information based on the driver's face image generated by the driver monitor camera 10. Specifically, the eye information detection unit 21 detects the driver's line of sight, the degree of opening of both eyes, the difference in the degree of opening of both eyes, and the difference in the height of both eyes by the method described above. After step S102, this control routine ends.

FIG. 5 is a flowchart showing a control routine of the driver state determination process in the present embodiment. This control routine is repeatedly executed by the ECU 20 while the ignition switch of the vehicle 80 is turned on. In this control routine, the driver state is determined based on the eye information detected by the eye information detection process.

First, in step S201, the driver state determination unit 22 determines whether or not the driver is looking aside. The driver state determination unit 22 determines that the driver is looking aside when the driver's line of sight is out of the predetermined range for a predetermined time or longer, and when the driver's line of sight is not out of the predetermined range for a predetermined time or longer, the driver Judge that you are not looking aside. The driver's line of sight is detected in step S102 of FIG.

If it is determined in step S201 that the driver is not looking aside, the control routine proceeds to step S202. In step S202, the driver state determination unit 22 determines whether or not the driver is dozing. The driver status determination unit 22 determines that the driver is dozing when both eyes of the driver are closed for a predetermined time or longer, and the driver is dozing when both eyes of the driver are not closed for a predetermined time or longer. Judge that it is not done. The degree of opening of both eyes of the driver is detected in step S102 of FIG.

If it is determined in step S202 that the driver has not fallen asleep, the control routine proceeds to step S203. In step S203, the driver state determination unit 22 determines whether or not the difference ESd in the degree of opening of both eyes of the driver is equal to or greater than the first threshold value TH1. The first threshold value TH1 is preset so as to be the minimum value of the difference in the degree of eye opening when the face is distorted. The difference in eye opening degree ESd is detected in step S102 of FIG.

If it is determined in step S203 that the difference ESd in eye opening degree is less than the first threshold value TH1, the control routine proceeds to step S204. In step S204, the driver state determination unit 22 determines whether or not the difference Hd in height between the driver's eyes is equal to or greater than the second threshold value TH2. The second threshold value TH2 is preset so as to be the minimum value of the difference in height between the eyes when the face is distorted. The height difference Hd between the eyes is detected in step S102 of FIG.

If it is determined in step S204 that the height difference Hd between the eyes is less than the second threshold TH2, the control routine proceeds to step S205. In step S205, the driver state determination unit 22 determines that the driver state is normal. After step S205, this control routine ends.

On the other hand, if it is determined in step S201 that the driver is looking aside, or if it is determined in step S202 that the driver is dozing, the control routine proceeds to step S206. In step S206, the driver status determination unit 22 gives an alarm to the driver. For example, the driver state determination unit 22 gives an alarm to the driver visually or audibly via the HMI 50. After step S206, this control routine ends.

Further, when it is determined in step S203 that the difference ESd in eye opening degree is equal to or greater than the first threshold value TH1, or when it is determined in step S204 that the difference Hd in height between both eyes is equal to or greater than the second threshold value TH2. This control routine proceeds to step S207. In step S207, the driver state determination unit 22 determines that the driver is in an inoperable state.

Next, in step S208, the vehicle control unit 23 controls various actuators 40 of the vehicle 80 to retract the vehicle 80. For example, the vehicle control unit 23 turns on the hazard lamp of the vehicle 80 and gradually reduces the vehicle speed to stop the vehicle 80 on the shoulder. If the vehicle 80 is not provided with an automatic driving system that automatically controls the traveling of the vehicle 80, only an operation such as lighting a hazard lamp is performed in step S208. After step S208, this control routine ends. Even if the driver is not clearly notified that the driver is in an inoperable state, it is determined that the driver is substantially in an inoperable state when a danger avoidance control such as evacuation of the vehicle 80 is executed. It can be said that it has been done.

The driver status determination unit 22 may give an alarm to the driver before step S207. In this case, if the driver does not respond to the alarm, steps S207 and S208 are executed, and if the driver responds to the alarm, this control routine ends.

Further, the first threshold value TH1 may be set in consideration of the difference in the degree of opening of both eyes when the driver is normal. In this case, the first threshold value Tth1 is set for each driver. For example, the difference in the degree of opening of both eyes when the driver is normal is acquired and updated immediately after the ignition switch of the vehicle 80 is turned on.

Similarly, the second threshold TH2 may be set in consideration of the difference in height between the eyes when the driver is normal. In this case, the second threshold TH2 is set for each driver. For example, the normal height difference between the eyes of the driver is acquired and updated immediately after the ignition switch of the vehicle 80 is turned on.

When it is detected that one eye of the driver is hidden by an eye patch or the like, the determination in step S201 and step S202 is performed based on the eye information of one eye, and steps S203 to S205, steps S207 and Step S208 is omitted.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the claims.

1 Driver status detector 10 Driver monitor camera 20 Electronic control unit (ECU)
21 Eye information detection unit 22 Driver status determination unit 80 Vehicle

Claims (1)

A driver monitor camera that takes a picture of the driver's face of the vehicle and generates a face image of the driver,
An eye information detection unit that detects eye information of the driver based on the face image,
It is provided with a driver state determination unit that determines the state of the driver based on the eye information.
In the driver state determination unit, when the difference in eye opening degree between the eyes of the driver detected by the eye information detection unit is equal to or greater than the first threshold value, or the eye information detection unit detects the driver in the direction of the central axis of the face. A driver state detection device that determines that the driver is in an inoperable state when the difference in height between the eyes of the driver detected as the distance between the eyes of the driver is equal to or greater than a second threshold value.

Images