CN101872171B - Driver fatigue state recognition method and system based on information fusion - Google Patents

Abstract

The invention discloses a driver fatigue state recognition method and system based on information fusion, wherein the method includes: collecting driver's eye state information, steering wheel manipulation state information and vehicle trajectory driving state information; The eye state information, the steering wheel manipulation state information and the vehicle track driving state information are processed to obtain the driver’s fatigue performance information; the fatigue performance information is weighted and fused to obtain the fatigue membership degree corresponding to the driver’s fatigue level, and according to The fatigue membership yields the driver's fatigue level. The invention improves the reliability and accuracy of driver fatigue state monitoring, can reduce the number of traffic accidents caused by driver fatigue driving, and ensures road traffic safety; and the acquisition of the multi-source information does not interfere with the driver's driving, and has strong practicability .

Description

Driver Fatigue State Recognition Method and System Based on Information Fusion

technical field

The invention relates to traffic engineering, in particular to a driver fatigue state recognition method and system based on information fusion.

Background technique

With the development of the economy and the sharp increase in the number of cars, the incidence of road traffic accidents has gradually increased, and driver fatigue driving is an important factor leading to road traffic accidents. Therefore, scientific and effective monitoring of driver fatigue is important to ensure Road traffic safety is of great significance.

The monitoring of the driver's fatigue state in the prior art mainly adopts single-source information monitoring, for example, by monitoring the driver's physiological condition to determine whether the driver is in a fatigue driving state. EEG measurements, or electrocardiograms, have long been hailed as the "gold standard" for monitoring driver fatigue. The waves of different frequency bands in the brain wave will have corresponding changes in different periods of driver fatigue. In the initial fatigue, the slow wave delta and alpha will change, and the belta wave will increase in the middle fatigue, and the delta, theta and The alpha wave will increase significantly. Both the high-frequency component HF and the low-frequency component LF of heart rate variability in ECG indicators will change during fatigue.

However, in the prior art, the method of using single-source information to monitor the driver's fatigue state still has the following technical defects: on the one hand, the single-source information has defects such as single information, low recognition accuracy, stability and reliability, etc., if The inaccuracy of the single-source information will make the above-mentioned driver fatigue state identification method invalid, and it is difficult to monitor the driver's driving fatigue state, so that the driver's driving has a great potential safety hazard; on the other hand, the single-source information Measuring equipment (such as equipment for measuring EEG or equipment for measuring ECG) will cause greater interference to the driver's driving, and is not suitable for monitoring the actual driver's fatigue state, and has poor practicability.

Contents of the invention

The purpose of the present invention is to provide a driver fatigue state recognition method and system based on information fusion, which can effectively overcome the defects of the prior art that are not suitable for practical application, single information, and low recognition accuracy and reliability.

The invention provides a driver fatigue state recognition method based on information fusion, comprising:

Step 1. Collect the driver's eye status information, steering wheel manipulation status information and vehicle track driving status information;

Step 2. Process the driver's eye state information, steering wheel manipulation state information, and vehicle track driving state information to obtain driver fatigue performance information, which includes the eye closure degree exceeding 80% per unit time The proportion of time occupied, the average degree of eye opening, the longest eye-closed time, the proportion of time the steering wheel remains still, the angle change of the steering wheel suddenly turning, the standard deviation of the lateral displacement of the vehicle in a serpentine driving state, and the lateral displacement in The time ratio of the safety distance range;

Step 3. Perform weighted fusion processing on the fatigue performance information to obtain a fatigue membership degree corresponding to the driver's fatigue level, and obtain the driver's fatigue level according to the fatigue membership degree.

The invention provides a driver fatigue state recognition system based on information fusion, comprising:

The collection module is used to collect the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information;

The processing module is used to process the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information to obtain driver fatigue performance information, and the fatigue performance information includes the degree of eye closure per unit time exceeding The proportion of time occupied by 80%, the average degree of eye opening, the longest eye-closed time, the proportion of time when the steering wheel remains still, the angle change of sudden steering wheel rotation, the standard deviation of the lateral displacement of the vehicle in a serpentine driving state, and the lateral displacement of the vehicle. The proportion of time that the displacement is within the safe distance range;

The fusion module is configured to perform weighted fusion processing on the fatigue performance information to obtain a fatigue membership degree corresponding to the driver's fatigue level, and obtain the driver's fatigue level according to the fatigue membership degree.

The present invention provides a driver fatigue state recognition method and system based on information fusion. Firstly, the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information are collected, and then the driver's eye state information is collected. information, steering wheel control state information and vehicle trajectory driving state information to obtain the driver’s fatigue performance information, and finally carry out weighted fusion processing on the fatigue performance information to obtain the fatigue membership degree corresponding to the driver’s fatigue level, and according to the The fatigue membership yields the driver's fatigue rating. Since the present invention evaluates the driver's driving fatigue by synthesizing the multi-source information reflecting the driver's driving fatigue, the reliability and accuracy of driver's fatigue state recognition are improved, and the fatigue caused by driver fatigue can be effectively reduced. The number of traffic accidents to ensure road traffic safety. Compared with the prior art, the implementation of the information fusion-based driver fatigue state identification method and system of the present invention will not cause interference to the driver's driving, has the advantages of strong practicability, easy implementation, etc., and has broad application prospects.

Description of drawings

Fig. 1 is the flowchart of the first embodiment of the driver's fatigue state recognition method based on information fusion in the present invention;

Fig. 2 is the flowchart of the second embodiment of the driver's fatigue state identification method based on information fusion in the present invention;

Fig. 3 is the schematic structural diagram of the first embodiment of the driver's fatigue state recognition system based on information fusion in the present invention;

FIG. 4 is a schematic structural diagram of a second embodiment of the driver fatigue state recognition system based on information fusion in the present invention.

Detailed ways

The inventor found in the process of realizing the present invention that the driver's fatigue status can be reflected in various aspects, and the physiological state, manipulation behavior and vehicle status all have a certain correlation with the driver's fatigue status, for example, the eyelids in the facial status Movement, head movement, eye sight and facial expression will all have obvious changes when the driver is fatigued; in terms of vehicle manipulation, the time domain and frequency domain indicators of the steering wheel angle, the grip force of the steering wheel, angular velocity, vehicle speed, pedal The pedaling force of the board and the lateral deviation of the vehicle trajectory are related to the driver's fatigue state to a certain extent. Based on this, the present invention grasps the key external symptoms when monitoring the driver's fatigue state, fuses the information of the eye state, steering wheel manipulation and vehicle trajectory, and applies it to the monitoring of the driver's fatigue state to comprehensively evaluate the driver's fatigue state. The fatigue status of the driver can be eliminated when the reliability of the parameter data is low, and other information with high reliability can be used to make up for the lack of single-source information monitoring and improve the accuracy and stability of driver fatigue status monitoring.

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

Fig. 1 is the flowchart of the first embodiment of the driver's fatigue state identification method based on information fusion in the present invention, as shown in Fig. 1, this embodiment includes:

Step 101, collecting the driver's eye state information, steering wheel manipulation state information and vehicle trajectory driving state information.

The collection of the driver's eye state information in this step can use various image collection methods in the prior art, such as photography or video recording. Photography refers to collecting a driver's eye image at intervals of a set time (such as 1 minute), and photography refers to extracting still images according to a set time (such as 1 minute) after continuously collecting driver's eye images . A variety of sensor methods in the prior art can be used to collect the driver's steering wheel manipulation state information, for example, an encoder or an angle sensor is set on the steering wheel. A variety of measurement methods in the prior art can be used to collect vehicle trajectory driving state information, for example, a position measurement device is used to measure the lateral offset distance of the vehicle from the centerline of the road. The driver's eye parameters in this embodiment can be obtained through statistical processing of eye images collected by the camera; the steering wheel parameters can be obtained through statistical processing of the instantaneous steering wheel angle collected by the steering wheel encoder; the vehicle trajectory parameters can be collected through image processing. The lateral offset distance between the trajectory and the road centerline is obtained through statistical processing.

Step 102: Process the driver's eye state information, steering wheel manipulation state information, and vehicle trajectory driving state information to obtain driver fatigue performance information.

Statistically process the collected eye state information, steering wheel control state information, and vehicle trajectory driving state information to obtain driver fatigue performance information. The fatigue performance information includes the proportion of time in which the eyes are closed more than 80% per unit time, the average eye opening degree, the longest eye-closed time, the proportion of time when the steering wheel remains still, the angle change of the steering wheel suddenly turning, the vehicle snake The standard deviation of the lateral displacement in the normal driving state and the time ratio of the lateral displacement within the safe distance range.

Step 103, perform weighted fusion processing on the fatigue performance information to obtain a fatigue membership degree corresponding to the driver's fatigue level, and obtain the driver's fatigue level according to the fatigue membership degree.

This embodiment evaluates the driver's driving fatigue status by combining multiple sources of information reflecting the driver's driving fatigue status, which improves the reliability and accuracy of driver fatigue status monitoring, and can reduce the number of traffic accidents caused by driver fatigue driving , to ensure road traffic safety; and the acquisition of the multi-source information does not interfere with the driving of the driver, and has strong practicability.

Fig. 2 is the flowchart of the second embodiment of the driver's fatigue state recognition method based on information fusion in the present invention, as shown in Fig. 2, this embodiment includes:

Step 201. Obtain eye state information reflecting the driver's fatigue state, steering wheel manipulation state information, and vehicle trajectory driving state information, and obtain fatigue performance information through processing;

In the technical solution of this embodiment, the method of expert video scoring is firstly used to cut the video recording of the driver's facial state during the entire driving process into separate videos for each minute, and several experts perform effective statistical processing on the scores of each minute Get a fatigue score for the entire driving process. Here, the fatigue level is divided into 4 levels, namely awake, mild fatigue, moderate fatigue and severe fatigue. Using the above-mentioned expert scores as the standard, determine and obtain the required information types reflecting the driver's fatigue state, including eye state information, steering wheel manipulation state information, and vehicle trajectory driving state information. On this basis, statistical methods such as correlation analysis and variance analysis are used to statistically process the driver's eye state information, steering wheel control state information and vehicle trajectory driving state information collected through the sensor, and extract the most correlated with driver fatigue evaluation. Strong fatigue performance information. For example, the driver's eye parameters are obtained by statistically processing the degree of eye closure of a single frame obtained by processing the collected eye images. In a unit time, the driver's eye parameters include PERCLOS80 (unit The proportion of time when the eyes are closed more than 80%), the longest eye-closed time (closed more than 80% is judged as closed eyes) and the average eye opening (can be expressed as a proportional value); the steering wheel parameters are obtained by encoding the steering wheel The instantaneous steering wheel angle collected by the sensor is obtained by statistical processing. The steering wheel parameters include the time ratio of the steering wheel remaining still and the angle change of the steering wheel's sudden rotation in a unit time; The lateral offset distance of the road centerline is obtained through statistical processing. The vehicle trajectory parameters include the standard deviation of the lateral displacement of the vehicle in a serpentine state and the time ratio of the lateral displacement within the safe distance range. The safe distance is that the lateral displacement is less than 100mm/s.

Step 202: Input the acquired fatigue performance information into the artificial neural network to obtain primary fatigue membership degrees corresponding to different measurement periods in different parameter spaces.

The eye state parameters, steering wheel control state parameters and vehicle trajectory state parameters in the fatigue performance information are respectively input into the trained neural network model to obtain the primary evaluation results, that is, the primary membership degrees belonging to each fatigue level. The present embodiment can adopt three-layer BP feed-forward network, and the model formula of this three-layer BP feed-forward network is as follows formula (1):

y _k = f ₂ (ω ⁽²⁾ , f ₁ (ω ⁽¹⁾ , x _k , b ⁽¹⁾ ), b ⁽²⁾ ) (1)

Among them, f ₁ (), f ₂ ( ⁾ are the transfer functions ^of neurons in the hidden layer and output layer of the neural network respectively; The weight vector of the unit; b ⁽¹⁾ and b ⁽²⁾ are the bias vectors of the neurons in the hidden layer and the output layer respectively; x _k is the parameters of different parameter spaces, such as eye state parameters, steering wheel control state parameters or vehicle trajectory Driving state parameters, y _k is the primary membership degree corresponding to each fatigue level. The representation of the specific output layer y _k is shown in Table 1 below:

Table 1 Fatigue grade expression

For example, as shown in the first row of Table 1 above, 0.9, 0.1, 0.1, and 0.1 respectively indicate the membership degrees to fatigue levels of sobriety, mild fatigue, moderate fatigue, and severe fatigue. When the fatigue level of sober membership is the highest at 0.9, The fatigue level corresponding to the output result is "awake".

The neural network in this embodiment is classified in advance according to the driver's physiological characteristics and operating characteristics, and the BP neural network established is trained offline by using training samples of different types of drivers, and the weights and offsets are stored in the notepad. In real-time judgment, the stored parameters are called in, and according to the idea of template matching, the corresponding driver type training parameters are used for online judgment; therefore, the parameters in the neural network model include different types of parameters that match different drivers.

Step 203 , performing multi-period D_S evidence theory fusion of the fatigue grade membership degrees output by the neural network in different measurement periods, to obtain the basic reliability corresponding to different parameter spaces.

The measurement of the sensor includes multiple measurement cycles, and the membership degrees of different measurement cycles are theoretically fused with single-sensor multi-cycle D_S evidence. First, the primary fatigue membership of multiple measurement cycles output by the neural network is normalized and transformed into D_S evidence The basic reliability distribution parameters of the important basis of the theory; and then use the following formula (2) to fuse the basic reliability distribution parameters of multiple measurement cycles to obtain the basic reliability of different parameter spaces. The fusion formula of the D_S evidence theory that fuses multiple measurement cycles is as follows:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; c</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\underset{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; I</span>}& {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1,2</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

in

$\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\underset{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; I</span>}& {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&Phi;</span>}\end{array}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; I</span>}& \mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&phi;</span>}\end{array}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>$

The above formula is the fused a posteriori reliability distribution formula for k propositions based on the accumulative measurements of n measurement periods, where M ₁ (A _i ), M ₂ (A _i ),...M _n (A i _i ) is the posterior reliability distribution obtained by the sensor through continuous target situation and fixed prior reliability distribution in each measurement period, i=1, 2,..., k, M _j (A _i ) represents the credibility assignment to proposition A _i in the j-th period. The basic reliability of different parameter spaces can be obtained through the above formula 2, that is, the reliability of eye state parameters, the reliability of steering wheel control state parameters and the reliability of vehicle trajectory driving state parameters.

After the neural network and D_S evidence theory in steps 202-203 complete the fusion at the subsystem level, continue to execute steps 204-205 to perform system-level fusion.

Step 204, determining fusion weights for weighted fusion of the fatigue performance information.

Eye state parameters, steering wheel control state parameters, and vehicle trajectory state parameters all have a certain correlation with the driver's fatigue state, but the degree of correlation is different, so the fusion weight corresponding to the above three types of parameters should also be different. The fusion weight is determined by combining the constant weight with the parameter credibility obtained in the subsystem-level fusion.

First, the constant weight is determined by using the normalized result of the class separability index in pattern recognition. The larger the class separability index in pattern recognition, the better the class separability of the index. At the system level, the constant weights are determined by utilizing the fatigue discrimination ability of parameters in different parameter spaces. The constant weight is represented by J, and its determination method is to use the following formula (3):

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; i</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, J represents the classifiable ability of the parameter, for example, the constant weight J of the eye state parameter represents the classifiable ability of the eye state parameter. μ _i , μ _j represent the mean value of adjacent classes, σ _i , σ _j are the variances of adjacent classes, and M represents the total number of classes. By normalizing the J value as a constant weight. Adjacent classes refer to adjacent fatigue levels, for example, mild fatigue and moderate fatigue are two adjacent classes. Calculate the mean or variance of the parameters corresponding to the adjacent classes in the three parameter spaces, respectively, to obtain μi, μj and σi, σj. The fatigue level in this embodiment is level 4, so M is equal to 4. The corresponding constant weights of the three types of parameters can be obtained through the above formula 3, namely, the constant weights of the eye state parameters, the constant weights of the steering wheel manipulation state parameters, and the constant weights of the vehicle trajectory driving state parameters.

Then, the fusion weight is determined by combining the constant weight with the parameter credibility obtained in the subsystem-level fusion. An upper limit of reference reliability and a lower limit of reference reliability may be preset. In this embodiment, the upper limit of reference reliability is set to 0.8, and the lower limit of reference reliability is set to 0.5. Report the parameters and parameter credibility of each parameter space to the fusion center together. When the parameter reliability is less than 0.5, the fusion weight is zero, that is, the parameter space is not fused, and the parameters of the parameter space are discarded; when the parameter reliability is greater than 0.8, the weight is not changed at this time, and the constant weight is used for fusion ; When the parameter reliability is between 0.5-0.8, calculate the fusion weight according to the formula (4)-(9). At this time, the parameter credibility is linearly proportional to the fusion weight, and the constant weight ratio is used as the proportional coefficient. The relationship between the weights is as follows:

$\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}<span\; class="notranslate">\; \&times;</span>\frac{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

$\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}<span\; class="notranslate">\; \&times;</span>\frac{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

W ₁ +W ₂ +W ₃ =1 (6)

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}}{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

Among them, W ₁ , W ₂ , and W ₃ represent the fusion weights of eye state parameters, steering wheel control state parameters, and vehicle trajectory state parameters respectively; r ₁ , r ₂ , and r ₃ represent the credible eye state parameters w ₁₂ , w ₂₃ , and w ₁₃ represent the ratio of the constant weight of the eye state parameter to the constant weight of the steering wheel control state parameter, and the constant weight of the steering wheel control state parameter The ratio of the weight to the constant weight of the vehicle trajectory driving state parameter, the ratio of the constant weight of the eye state parameter to the constant weight of the vehicle trajectory driving state parameter.

Step 205 , perform self-D_S fusion with variable weights on the fatigue performance information to obtain the fatigue membership degree corresponding to the driver's fatigue level, and obtain the driver's fatigue level according to the fatigue membership degree.

According to the fusion weights of the three different parameters obtained in step 204, the eye state parameters, the steering wheel manipulation state parameters and the vehicle trajectory driving state parameters are weighted for their own D_S fusion, that is, the multi-sensor D_S evidence theory of different parameter spaces is carried out at the system level Fusion, the fatigue membership degree corresponding to the driver's fatigue level is obtained, and the driver's fatigue level is obtained according to the fatigue membership degree. The multi-sensor D_S evidence theory fusion in this step still adopts the formula (2), but the data fusion of multiple measurement periods is changed to the data fusion of multiple parameter spaces.

In this embodiment, the multi-source information reflecting the driver's driving fatigue is combined to evaluate the driver's driving fatigue. The three aspects of information can more comprehensively evaluate the fatigue. The fatigue state can be obtained, thereby improving the reliability and accuracy of driver fatigue state monitoring, reducing the number of traffic accidents caused by driver fatigue driving, and ensuring road traffic safety; and the acquisition of multi-source information does not interfere with the driver's driving , Strong practicability.

Figure 3 is a schematic structural diagram of the first embodiment of the driver fatigue state recognition system based on information fusion in the present invention, as shown in Figure 3, the driver fatigue monitoring system of this embodiment may include an acquisition module 301, a processing module 302 and a fusion module 303 , wherein the collection module 301 is used to collect the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information; the processing module 302 is used to collect the driver's eye state information, steering wheel manipulation state information and vehicle Trajectory driving state information is processed to obtain the fatigue performance information of the driver; the fusion module 303 is used to carry out weighted fusion processing on the fatigue performance information to obtain the fatigue membership degree corresponding to the driver fatigue level, and obtain the fatigue membership degree according to the fatigue membership degree The driver's fatigue level.

Specifically, the collection module 301 collects eye state information reflecting the driver's fatigue state, steering wheel manipulation state information, and vehicle trajectory driving state information, and the processing module 302 analyzes and processes the above information obtained by the collection module 301 to obtain the driver's fatigue performance The fatigue performance information includes the proportion of time when the eyes are closed more than 80% per unit time, the average eye-closed time, the longest eye-closed time, the proportion of time when the steering wheel remains still, the angle change of the steering wheel suddenly turned, the vehicle The standard deviation of the lateral displacement in the serpentine driving state and the time ratio of the lateral displacement within the safe distance range. The fusion module 303 performs weighted fusion processing on the above fatigue performance information to obtain the comprehensive fatigue membership degree corresponding to the fatigue level, and obtains the fatigue level of the driver according to the comprehensive fatigue membership degree.

The driver fatigue monitoring system of this embodiment evaluates the driver's driving fatigue status by integrating multiple sources of information reflecting the driver's driving fatigue status, which improves the reliability and accuracy of driver fatigue status monitoring, and can reduce driver fatigue driving. The number of traffic accidents caused can ensure road traffic safety; and the acquisition of the multi-source information does not interfere with the driver's driving, which is highly practical.

FIG. 4 is a schematic structural diagram of the second embodiment of the driver fatigue state recognition system based on information fusion in the present invention. As shown in FIG. Unit 3031 and system-level fusion unit 3032, wherein the subsystem-level fusion unit 3031 is used to perform weighted fusion processing on the eye state information, steering wheel manipulation state information, and vehicle trajectory driving state information in the fatigue performance information to obtain eye Credibility of eye state information, steering wheel manipulation state information credibility, and vehicle trajectory driving state information credibility; system-level fusion unit 3032 The reliability of the driving state information determines the fusion weight of the eye state information, the fusion weight of the steering wheel manipulation state information and the fusion weight of the vehicle trajectory driving state information respectively, and according to the fusion weight of the eye state information, the fusion weight of the steering wheel manipulation state information and the vehicle trajectory The driving state information fusion weight performs weighted fusion processing on the eye state information, steering wheel manipulation state information and vehicle trajectory driving state information to obtain the driver's fatigue level.

Further, the subsystem-level fusion unit 3031 may further include a first subunit and a second subunit, wherein the first subunit is used to combine the eye state information, steering wheel manipulation state information, and vehicle trajectory driving state information according to neural Performing weighted fusion on the network parameter definition relationship to obtain the primary fatigue membership degree corresponding to the driver fatigue level, the primary fatigue membership degree includes the primary fatigue membership degree corresponding to a plurality of measurement cycles of the eye state information, steering wheel manipulation state information, and vehicle trajectory driving state information. Fatigue membership degree; the second subunit is used to normalize the primary fatigue membership degree and transform it into a basic reliability distribution parameter, and will correspond to the plurality of eye state information, steering wheel manipulation state information and The basic reliability distribution parameters of the vehicle trajectory driving state information measurement cycle are fused with D_S evidence theory, and the eye state parameter credibility, steering wheel manipulation state parameter credibility and vehicle trajectory driving state parameter credibility are respectively obtained. The system-level fusion unit 3032 includes a weight determination subunit, which is used to determine the eye state parameter fusion weight, steering wheel Manipulation state parameter fusion weight and vehicle trajectory driving state parameter fusion weight.

Specifically, on the basis of the first embodiment, the process of the fusion module 303 performing weighted fusion processing on the above fatigue performance information can be divided into two steps: subsystem-level fusion and system-level fusion, and subsystem-level fusion can be divided into two steps: There are two steps of neural network fusion and D_S evidence theory fusion. The first subunit in the subsystem-level fusion unit 3031 first inputs the eye state parameters, steering wheel manipulation state parameters, and vehicle trajectory driving state parameters into the trained artificial neural network, and can obtain the three parameter spaces corresponding to different The primary fatigue membership of the fatigue class and the primary fatigue membership of the different measurement cycles. The second subunit normalizes the above-mentioned primary fatigue membership into basic reliability distribution parameters corresponding to different measurement periods, and then uses the D_S evidence theory to fuse the basic reliability distribution parameters of multiple measurement periods in a single space, The basic reliability of each type of parameter space can be obtained, that is, the reliability of eye state parameters, the reliability of steering wheel control state parameters and the reliability of vehicle trajectory driving state parameters. When the system-level fusion unit 3032 performs weighted fusion of the three types of parameters, the weight determination subunit needs to first determine the fusion weight of the three types of parameters, and the fusion weight is changed according to the basic credibility obtained by the second subunit. It is comprehensively determined through constant weights combined with the credibility of various parameters. For the specific determination method, please refer to the method embodiment. After the fusion weight is determined, the system-level fusion unit 3032 can perform weighted fusion of the three types of parameters and self-variable weight D_S evidence theory fusion according to the fusion weight to obtain the fatigue level result.

The driver fatigue monitoring system of this embodiment evaluates the driver's driving fatigue status by integrating multiple sources of information reflecting the driver's driving fatigue status, which improves the reliability and accuracy of driver fatigue status monitoring, and can reduce driver fatigue driving. The number of traffic accidents caused can ensure road traffic safety; and the acquisition of the multi-source information does not interfere with the driver's driving, which is highly practical.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that: it still Modifications or equivalent replacements can be made to the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims (6)

1. A driver's fatigue state recognition method based on information fusion, is characterized in that, comprises: Step 1. Collect the driver's eye state information, steering wheel manipulation state information and vehicle trajectory driving state information; Step 2. Process the driver's eye state information, steering wheel manipulation state information, and vehicle track driving state information to obtain driver fatigue performance information, which includes the eye closure degree exceeding 80% per unit time The proportion of time occupied, the average degree of eye opening, the longest eye-closed time, the proportion of time the steering wheel remains still, the angle change of the steering wheel suddenly turning, the standard deviation of the lateral displacement of the vehicle in a serpentine driving state, and the lateral displacement in The time ratio of the safety distance range; Step 3, performing weighted fusion processing on the fatigue performance information to obtain a fatigue membership degree corresponding to the driver's fatigue level, and obtaining the driver's fatigue level according to the fatigue membership degree; Wherein, said step 3 includes: Step 31. Fusion the proportion of time that the eye closure degree exceeds 80% per unit time, the average eye opening degree, and the longest eye closure time to obtain the eye state parameter reliability, and keep the steering wheel still The time ratio of the steering wheel is fused with the angle change of the sudden rotation of the steering wheel to obtain the reliability of the steering wheel manipulation state parameters, and the standard deviation of the lateral displacement of the vehicle in a serpentine driving state is fused with the time ratio of the lateral displacement in the safe distance range to obtain The reliability of vehicle trajectory driving state parameters; Step 32: Determine the fusion weight of the eye state parameters according to the credibility of the eye state parameters, determine the fusion weight of the steering wheel steering state parameters according to the credibility of the steering wheel steering state parameters, and determine the fusion weight of the steering wheel steering state parameters according to the credibility of the vehicle trajectory driving state parameters Obtaining the fusion weight of the vehicle trajectory driving state parameters; performing weighted fusion processing on the fatigue performance information according to the fusion weight of the eye state parameters, the fusion weight of the steering wheel manipulation state parameters and the fusion weight of the vehicle trajectory driving state parameters to obtain the corresponding driver fatigue level The fatigue membership degree, and according to the fatigue membership degree, the driver's fatigue level is obtained. 2. The method according to claim 1, wherein said step 31 specifically comprises: Step 311: Fuse the fatigue performance information according to the neural network model to obtain the primary fatigue membership degree of the eye state parameter, the primary fatigue membership degree of the steering wheel control state parameter, and the vehicle trajectory driving state parameter of different measurement periods corresponding to the fatigue performance information primary fatigue membership; Step 312: Convert the primary fatigue membership degrees of different measurement periods into basic reliability allocation parameters, and fuse the basic reliability allocation parameters of different measurement periods according to the D-S evidence theory model to obtain the eye state parameter credibility respectively. degree, the reliability of steering wheel control state parameters, and the reliability of vehicle trajectory driving state parameters. 3. method according to claim 1, it is characterized in that, parameter credibility comprises described eye state parameter credibility, steering wheel steering state parameter credibility and vehicle trajectory driving state parameter credibility, described step 32 includes: Step 321, extracting the pre-set upper limit of reference reliability and lower limit of reference reliability; Step 322: Compare the parameter credibility with the upper limit of reference credibility and the lower limit of reference credibility, if the parameter credibility is smaller than the lower limit of reference credibility, then it corresponds to the parameter credibility The parameter fusion weight is zero; if the parameter credibility is greater than the upper limit of the reference credibility, the parameter fusion weight corresponding to the parameter credibility is a constant weight used to identify the ability of the parameter to distinguish fatigue; If the parameter credibility is between the reference reliability lower limit and the reference reliability upper limit, then the parameter fusion weight corresponding to the parameter credibility is linearly proportional to the parameter credibility. 4. A driver fatigue state recognition system based on information fusion, characterized in that it comprises: The collection module is used to collect the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information; The processing module is used to process the driver's eye state information, steering wheel manipulation state information and vehicle track driving state information to obtain driver fatigue performance information, and the fatigue performance information includes the degree of eye closure per unit time exceeding The proportion of time occupied by 80%, the average degree of eye opening, the longest eye-closed time, the proportion of time when the steering wheel remains still, the angle change of sudden steering wheel rotation, the standard deviation of the lateral displacement of the vehicle in a serpentine driving state, and the lateral displacement of the vehicle. The proportion of time that the displacement is within the safe distance range; The fusion module is used to carry out weighted fusion processing on the fatigue performance information to obtain the fatigue membership degree corresponding to the driver's fatigue level, and obtain the driver's fatigue level according to the fatigue membership degree; Wherein, the fusion module includes: The subsystem-level fusion unit is used to fuse the time proportion of the eye closure degree exceeding 80% in the unit time, the average eye opening degree and the longest eye closure time to obtain the eye state parameter credibility, and combine the The time ratio of the steering wheel keeping still and the angle change of the sudden rotation of the steering wheel are fused to obtain the reliability of the steering wheel control state parameters, and the standard deviation of the lateral displacement of the vehicle in a serpentine driving state and the value of the lateral displacement within the safe distance range The time ratio is fused to obtain the reliability of the vehicle trajectory driving state parameters; The system-level fusion unit is used to fuse weights of eye state parameters and fusion weights of steering wheel manipulation state parameters related to the credibility of eye state parameters, steering wheel manipulation state parameters, and vehicle trajectory driving state parameters. Fusing the weight with the vehicle trajectory driving state parameters to perform weighted fusion processing on the fatigue performance information to obtain the fatigue membership degree corresponding to the driver fatigue level, and obtain the driver fatigue level according to the fatigue membership degree. 5. The system according to claim 4, wherein the subsystem-level fusion unit comprises: The first subunit is used to fuse the fatigue performance information according to the neural network model, and respectively obtain the primary fatigue membership degree of the eye state parameter, the primary fatigue membership degree of the steering wheel manipulation state parameter and the vehicle Trajectory driving state parameter primary fatigue membership degree; The second subunit is used to convert the primary fatigue membership degrees of different measurement periods into basic reliability distribution parameters, and fuse the basic reliability distribution parameters of different measurement periods according to the D-S evidence theory model to obtain eye State parameter credibility, steering wheel manipulation state parameter credibility and vehicle trajectory driving state parameter credibility. 6. The system according to claim 4, wherein the system-level fusion unit comprises: The weight determination subunit is used to determine the eye state parameter fusion weight, steering wheel manipulation state parameter fusion weight and Fusion weight of vehicle trajectory driving state parameters.

Images