CN101224113B - Motor vehicle driver state monitoring method and system - Google Patents

Abstract

The invention relates to motor vehicle control technology, in particular to a method and a device for monitoring the state of a driver by using characteristic data of vital activity. Aiming at the shortcomings of the driver monitoring method in the prior art, such as inconvenient detection, limited application range and ineffective monitoring, the invention discloses a motor vehicle driver state monitoring method and system for identifying and monitoring the driver. The present invention collects the driver's vital activity characteristic data, especially the heart rate, head movement distance and its change standard deviation, comprehensively processes various data, and finally makes an accurate judgment. The present invention can be used for driver's fatigue driving, other people's substitute driving, vehicle anti-theft monitoring and the like. It has the characteristics of accurate judgment and convenient use.

Description

Motor vehicle driver state monitoring method and system

technical field

The invention relates to motor vehicle control technology, in particular to a method and a device for monitoring the state of a driver by using characteristic data of vital activity.

Background technique

With the rapid development of the transportation industry, traffic accidents caused by theft of motor vehicles and abnormal driving by drivers are becoming more and more serious. The relevant departments of the state attach great importance to it, and many technologies for driver identification and status monitoring have emerged. . Judging from the existing scientific and technological literature and patents, there are mainly technologies based on image recognition, biometrics and monitoring the driver's state according to the state of the car. Among them, the representative patents are: application number 200410041994.8, published on May 11, 2005, the invention patent application titled "Method and System for Biological Recognition of Driver Fatigue Warning", which collects the driver's pupils, eyelids, and brain waves And other information is calculated and processed to determine whether the driver is in a state of fatigue, and perform corresponding processing. Application No. 200620120044.9, published on August 29, 2007, is a utility model patent titled "Active Vehicle Driver Fatigue Driving Warning Device", which uses the detection of vehicle motion status to determine whether the driver is in an abnormal state. The previous patent application requires the driver to wear a special brain wave detection sensor, which will make the driver feel uncomfortable; the detection of eye information such as the eyelids cannot be detected by the infrared image sensor when the driver wears glasses while driving. The latter patent is affected by many factors because the motion state of the car, especially the road surface state, is difficult to play a role under the existing domestic road conditions. In addition, the relevant traffic regulations of the People's Republic of China define fatigue driving, which refers to drivers who drive for more than 8 hours a day, drive motor vehicles for more than 4 hours without rest or stop for less than 20 minutes, and drivers engaged in road transport for more than 3 hours at a time. Hours, or due to lack of sleep, excessive physical exertion, etc., driving drowsiness, weakness of limbs, failure to detect and accurately deal with road traffic conditions and other conditions. Now many big city management departments use the method of driver's card swiping to monitor the driver's driving time, but the phenomenon of substitute driving is serious and cannot be effectively monitored.

Contents of the invention

The technical problem to be solved by the present invention is to provide a motor vehicle driver state monitoring method and system for the driver monitoring method of the prior art, which is inconvenient to detect, limited in scope of use and weak in monitoring. identification and monitoring.

The technical solution adopted by the present invention to solve the technical problem is that the motor vehicle driver's state monitoring method comprises the following steps:

a. Collect the driver's vital activity characteristic data during the initial period;

b. record the data collected in step a;

c. Real-time collection of driver vital activity characteristic data;

d. Compare the data collected in step c with the data recorded in step b; when the comparison result between the data collected in step c and the data recorded in step b exceeds the set value, it is considered that the driver is in a state of fatigue;

The feature data includes the driver's heart rate and heart rate standard deviation; the distance between the driver's head feature point and the reference point and the distance standard deviation.

The motor vehicle driver state monitoring system of the present invention includes an image acquisition system, a heart rate acquisition system, an alarm device and a computer control system;

The image acquisition system is used to collect the driver's facial image, and its composition includes an infrared camera and an image processing system, the infrared camera is connected with the image processing system, and the image processing system is connected with the computer control system;

The heart rate acquisition system is used to collect the driver's heart rate data, and its composition includes a sequentially connected heart rate sensor and a signal processing circuit, and the signal processing circuit is connected to a computer control system;

The alarm device is connected with the computer control system, and sends sound and light alarm information according to the alarm trigger signal output by the computer control system and reports to the designated communication terminal;

The computer control system processes the collected data, judges whether the driver is fatigued according to the heart rate and the standard deviation of the heart rate, the distance between the central point of the driver's forehead and the camera and the standard deviation of the distance, and outputs the judgment result to control the vehicle.

The beneficial effect of the present invention is that the driver's life activity characteristic data can be detected conveniently without the driver wearing any additional device, the infrared camera can avoid the interference of the complex lighting and background environment in the driver's cab, and the combined processing of various data reduces misjudgment rate. It has the advantages of convenient use and accurate detection.

Description of drawings

Fig. 1 is a schematic diagram of the system structure;

Figure 2 is a flow chart of signal processing.

Detailed ways

The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

The motor vehicle driver state monitoring method of the present invention uses fuzzy logic and neural network technology to collect and record the vital activity characteristic data of the driver in the initial period (assuming that it is in a non-fatigue normal state), and then collects the above vital activity in real time Feature data, according to the comparison of real-time data and recorded data, finally make a judgment on whether the driver is tired or not.

The motor vehicle driver state monitoring method of the present invention comprises the following steps:

a. Collect the driver's vital activity characteristic data during the initial period;

b. record the data collected in step a;

c. Real-time collection of driver vital activity characteristic data;

d. Compare the data collected in step c with the data recorded in step b; when the comparison result between the data collected in step c and the data recorded in step b exceeds the set value, it is considered that the driver is in a state of fatigue;

The characteristic data are the driver's heart rate and heart rate standard deviation.

It is also a good choice to use the distance and/or distance standard deviation between the driver's head feature point and the reference point as the driver's vital activity characteristic data.

Specifically, the feature point is the center of the driver's forehead, and the reference point is the camera.

Further, the aforementioned vital activity characteristic data may also be collected simultaneously for comprehensive judgment, so as to improve the accuracy of judgment.

Specifically, the initial period in step a means that the data collected in step a and the data collected in step c have a time difference of at least 10 minutes.

The motor vehicle driver state monitoring system of the present invention includes an image acquisition system, a heart rate acquisition system, an alarm device and a computer control system;

The image acquisition system is used to collect the driver's facial image, and its composition includes an infrared camera and an image processing system, the infrared camera is connected with the image processing system, and the image processing system is connected with the computer control system;

The heart rate acquisition system is used to collect the driver's heart rate data, and its composition includes a sequentially connected heart rate sensor and a signal processing circuit, and the signal processing circuit is connected to a computer control system;

The alarm device is connected with the computer control system, and sends sound and light alarm information according to the alarm trigger signal output by the computer control system and reports to the designated communication terminal;

The computer control system processes the collected data, judges the state of the driver according to the heart rate and the standard deviation of the heart rate, the distance between the central point of the driver's forehead and the camera and the standard deviation of the distance, and outputs the judgment result to control the vehicle.

Further, it also includes an alcohol concentration detection system, which is composed of sequentially connected alcohol sensors and signal processing circuits; the alcohol concentration detection system is connected with a computer control system for detecting the alcohol content of the driver's breath .

Furthermore, it also includes a card reader, which is connected with the computer control system and used to read the information of the driver's identification card.

Specifically, the heart rate sensor is a non-contact heart rate sensor.

More specifically, the infrared camera is installed directly in front of the driver, and the heart rate sensor is installed on the outer ring of the steering wheel.

Example

The system structure of this example is shown in Figure 1, by heart rate sensor, signal processing circuit; Alcohol sensor, signal processing circuit; Infrared camera, image processing system; Computer control system composition, comparison device; And card reader, GPRS/CDMA interface , Recorder composition. The heart rate sensor is installed on the outer ring of the steering wheel, which is convenient for the driver to hold. The collected heart rate data is sent to the computer control system through the signal processing circuit. If a non-contact heart rate acquisition method is used (see Chinese Patent Publication "Measurement Method for Non-Contact Key Physiological Parameters", Publication No. CN101006915A, Publication Date 2007.08.01), the installation position of the heart rate sensor may not be subject to the above restrictions. The alcohol sensor is installed directly in front of the steering wheel to collect the alcohol content of the driver's exhaled breath and then send it to the computer control system. The infrared camera is installed directly in front of the driver, and the images recorded by it are sent to the computer control system for analysis, processing and storage after the image processing system, and the driver's face information and forehead position information are obtained. The computer control system fuses all the input information, and judges whether the collected driver’s face information is consistent with the information read by the card reader, so as to prevent the phenomenon of substitute driving. If there is a substitute driving phenomenon, the computer control system starts the recorder to record the driving. Member information, and drive the GPRS/CDMA interface to send to the management department. If it is judged that the collected driver's facial information does not match all the stored images, it can be judged that a car theft has occurred, and the computer control system starts the recorder to record the image of the car thief, and drives the alarm device to alarm and transmit the image of the car thief . Through the alcohol sensor, it is judged whether it is drunk driving. If there is drunk driving, the computer control system starts the recorder, records the driver's information, and drives the GPRS/CDMA interface to send the information to the management department. The computer control system performs information fusion and judgment on driving based on the driver's heart rate, standard deviation of heart rate, face information, distance between forehead center and infrared camera, standard deviation of the distance, driver's face image, information read by card reader, driving time, etc. If the driver is fatigued, the computer control system will start the alarm device to remind the driver, and start the recorder to record the driver's information, and drive the GPRS/CDMA interface to send the information to the management department. The signal processing flow of this example is shown in Figure 2. The device makes full use of the driver's information, and can be extended to anti-theft, anti-fatigue, and anti-violation driving monitoring.

The driver state recognition monitoring method of this system is as follows:

After the system starts, the card reader collects the driver's information, which is provided by the existing driver's IC card. Including the driver's age, gender, driving experience, photo and other information, and sending the driver's information to the computer control system;

The heart rate sensor installed on the outer ring (handle) of the steering wheel collects the driver's heart rate information and sends it to the computer control system;

The alcohol sensor detects the driver's exhaled gas, and the detection information is processed and sent to the computer control system;

The infrared camera takes pictures of the driver's face, which is converted into digital signals by the image processing system and sent to the computer control system for further processing;

The computer control system fuses and analyzes the obtained information, and judges whether the collected driver's facial information matches the information provided by the card reader, so as to prevent substitute driving and car theft.

The image processing system of the present invention preprocesses the image through a median filter; segments the image through an adaptive threshold segmentation method; and performs position detection on the mouth, nose, eyes and forehead through multi-scale morphology.

和一个结构函数

并且定义结构函数g(x)对图像f(x)膨胀操作为：Let R and Z represent the real number set and the integer set respectively, given an image

and a constructor

And define the structure function g(x) to expand the image f(x) as:

$<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&CirclePlus;</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; \&cap;</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; D.</span>}}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>$

The corrosion operation of the structure function g(x) on the image f(x) is:

$<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f\&Theta;g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; \&cap;</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>$

max{f}和min{f}分别代表f的最大值和最小值，在离散的情况下函数为有限点集，因此使用的是最大值和最小值。通常膨胀腐蚀的结果与结构函数原点的选取有关，为了避免原点选取的影响，在这里我们规定：where D _x ={x+t:t∈D},

max{f} and min{f} represent the maximum and minimum values of f, respectively. In discrete cases, the function is a finite point set, so the maximum and minimum values are used. Usually the result of dilation and corrosion is related to the selection of the origin of the structure function. In order to avoid the influence of the selection of the origin, here we stipulate:

$\begin{array}{cc}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; D.</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}$

When the structure function is scale-dependent, that is, g _σ (x)=|σ|g(|σ| ^-1 x), x∈G _σ , σ≠0. Where σ is the scale factor, G _σ = {x: |σ| ^-1 x∈G}, ||x||≤|σ|, and the commonly used structural functions are as follows:

Planar structure function: g _σ (x) = 0, x∈G, where G = {x:||x||≤|σ|}

Hemispherical structure function:

${\mathrm{<span\; class="notranslate">\; g</span>}}_{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; (</span>\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; |</span>}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&le;</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; |</span>$

Parabolic structure function:

g _σ (x)＝-|σ|(||x||/|σ|) ² , ||x||≤|σ|

Expand the region to include all the head region A (including most or all of the hair and the entire face), then use multi-thresholding in region A to binarize and merge, and use region labels in the resulting binary image B The algorithm labels different "white" connected regions, and finds out the region formed by two eyes, nose and mouth according to certain prior knowledge. In the obtained binary image B of the L, R, and N regions and the binarized edge image E, the high-frequency characteristics of the four regions are used, which is determined by the principle of image acquisition formed by the infrared image sensor through the temperature difference. There is a temperature difference between the temperature of each region and the surrounding temperature, and the region point sets that form two eyes are described as:

Thus, two rough point sets of the eyes, nose and mouth regions are obtained, and the center of gravity of the point sets of these four regions is calculated respectively, and some candidate points are obtained with the center of gravity as the center, and for each candidate point P, the following is calculated on the edge image Support functions:

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; A</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; E.</span>}}_{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}$

Where E is the edge image, A is the ring area determined by the radius range R ₁ ≤ r ≤ _{R 2} , and N is the number of points in it. Take the point with the largest support function in the three candidate point sets as the center point of the L, R, and N regions:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; Left\; Iris</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; Rng</span>}}_{\mathrm{<span\; class="notranslate">\; left</span>}}}{\mathrm{<span\; class="notranslate">\; MAX</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; Right\; Iris</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; Png</span>}}_{\mathrm{<span\; class="notranslate">\; Right</span>}}}{\mathrm{<span\; class="notranslate">\; MAX</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; Nose</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; Rng</span>}}_{\mathrm{<span\; class="notranslate">\; the\; nose</span>}}}{\mathrm{<span\; class="notranslate">\; MAX</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; mouth</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; Rng</span>}}_{\mathrm{<span\; class="notranslate">\; mouth</span>}}}{\mathrm{<span\; class="notranslate">\; MAX</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; )</span>\end{array}\right.$

In this way, the rough positions of the eyes, nose and mouth regions on the image can be detected, and then the position P _forehead of the central point of the forehead region can be estimated. Based on the above processing, the computer control system can calculate the distance l between the forehead center and the reference point camera and the standard deviation l _δ of the change. Usually, the distance l is 45cm and the standard deviation l _δ is 1.5. When the data collected in real time exceeds the set value (l=45cm, _lδ =1.5), the computer control system judges that the driver is in a fatigue state and starts an alarm program.

The processing of the heart rhythm data is to collect the driver's heart rate through the heart rate sensor. Since the heart rate benchmarks of different people are different at different times (some have a heart rate of 60 times/s, and some have a heart rate of 70 times/s), In order to better reflect the fluctuation of the subject's heart rate, the actual heart rate value X(t ₀ ) at a certain moment can be used to subtract the average value X of the heart rate in a certain period of time to obtain the heart rate fluctuation value D_X(t ₀ ) at this moment ,Right now:

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; \&Delta;t</span>}}$

The fluctuation value sequence D_X(t) of heart rate in three mental states can be calculated:

In the normal state, the heart rate of the subjects fluctuated the most;

In the critical state, the fluctuation of the heart rate of the subjects is the smallest and almost unchanged;

In the fatigue state, the subjects' heart rate fluctuated more gently, and the heart rate value tended to decrease with time.

In order to express the mathematical characteristics of the trend of heart rate variation, it can be reflected by finding the standard deviation of heart rate:

${\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; h</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

The standard deviation of the heart rate in the critical state is obviously smaller than that of the other two mental states with σ _x <0.2, and the standard deviation of the heart rate can be used to identify the critical state, which is a factor for judging driver fatigue.

In the center of the four areas detected by the driver's facial image, a vector relationship model {x, y, z, w} is established, which makes it possible to extract features well even in the face rotation state, and finally for {x, y, z , w} establishes a set of observation vectors, o _i = J(xi _, y _i , _zi , w _i ), and uses the hidden Markov model to obtain face recognition information; Whether the obtained driver information is consistent, to prevent substitute driving and car theft.

Finally, according to the driver's heart rate, heart rate standard deviation, face information, distance between the forehead and the reference point, distance standard deviation, driver information, driving time and other information, fuzzy neural network information fusion is performed to determine whether the driver is fatigued. The control system activates the alarm device to remind the driver of fatigue driving, and activates the recorder to record the driver's information and send the information to the management department. It is assumed here that the driver is normal during the initial period of driving the car, so that in the first ten minutes, the fuzzy neural network is in the learning stage and remembers the driver's state. After ten minutes, the fuzzy neural network is in offline self-learning, realizing online driving Staff status check.

Due to the use of fuzzy logic and neural network technology, the heart rhythm data and image data input to the computer control system are comprehensively processed, which can not only judge the driver's fatigue state, but also identify the driver's identity. A comprehensive monitoring system for motor vehicle drivers. All these image processing technologies have public literatures that can be referred to, and will not be repeated here. Using the standard deviation (including the standard deviation of the heart rate and the standard deviation of the distance between the forehead center point and the infrared camera) can eliminate the influence of accidental errors and improve the accuracy of judgment.

Claims (6)

1. The motor vehicle driver state monitoring method comprises the following steps: a. Collect the driver's vital activity characteristic data during the initial period when the driver is in a non-fatigue state; b. record the data collected in step a; c. Start real-time collection of driver vital activity characteristic data after the initial period; d. Compare the data collected in step c with the data recorded in step b. When the comparison result between the data collected in step c and the data recorded in step b exceeds the set value, it is considered that the driver is in a state of fatigue; The vital activity characteristic data includes the driver's heart rate and heart rate standard deviation, and the distance and distance standard deviation between the center point of the driver's forehead and the infrared camera fixedly installed directly in front of the driver; Wherein the data collected in step a and the data collected in step c have a time difference of at least 10 minutes. 2. Motor vehicle driver status monitoring system, including image acquisition system, heart rate acquisition system, alarm device and computer control system; The image acquisition system is used to collect the driver's facial image, and its composition includes a fixedly installed infrared camera and an image processing system, the infrared camera is connected with the image processing system, and the image processing system is connected with the computer control system; The heart rate collection system is used to collect the driver's heart rate data, and its composition includes a sequentially connected heart rate sensor and a first signal processing circuit, and the first signal processing circuit is connected to a computer control system; The alarm device is connected with the computer control system, and sends sound and light alarm information according to the alarm trigger signal output by the computer control system and reports to the designated communication terminal; The computer control system processes the collected heart rate data to calculate the driver's heart rate and heart rate standard deviation, and processes the collected driver's facial image to calculate the distance and distance standard deviation between the driver's forehead center point and the infrared camera , and according to the heart rate and heart rate standard deviation and the distance and distance standard deviation between the center point of the driver's forehead and the infrared camera, fuzzy neural network information fusion is performed to judge the fatigue state of the driver, and the judgment result is output to the vehicle. control. 3. The motor vehicle driver's state monitoring system according to claim 2, is characterized in that, also includes an alcohol concentration detection system, and the alcohol concentration detection system is made of sequentially connected alcohol sensors and a second signal processing circuit; The alcohol concentration detection system is connected with the computer control system and is used to detect the alcohol content of the driver's exhaled gas. 4. The motor vehicle driver's status monitoring system according to claim 2, further comprising a card reader connected to a computer control system for reading information on the driver's identification card. 5. The motor vehicle driver state monitoring system according to claim 2, wherein the heart rate sensor is a non-contact heart rate sensor. 6. The motor vehicle driver state monitoring system according to claim 2, 3, 4 or 5, wherein the infrared camera is fixedly installed directly in front of the driver, and the heart rate sensor is installed on the outer ring of the steering wheel.

Images