CN110084220A - A kind of vehicle-mounted fatigue detection method based on multiple dimensioned binary mode - Google Patents

Abstract

The present invention relates to a kind of vehicle-mounted fatigue detection methods based on multiple dimensioned binary mode, what is solved is the low technical problem of detection accuracy, several unduplicated subregions are divided by using by training sample image, feature extraction is carried out using multiple dimensioned local binary pattern, obtains multiple dimensioned partial binary characteristics of image；Discrete fourier variation is carried out to multiple dimensioned partial binary characteristics of image, obtains histogram Fourier's feature vector of multiple dimensioned binary mode；Step 4, connect and compose histogram Fourier's feature vector of multiple dimensioned binary mode, for characterizing characteristics of image, select kernel function, classification based training is carried out with MLBP feature of the Nonlinear Support Vector Machines to sample image, and the technical solution of the svm classifier model after being trained and parameter, the problem is preferably resolved, can be used in vehicle-mounted fatigue detection.

Description

A Vehicle Fatigue Detection Method Based on Multi-scale Binary Patterns

technical field

The invention relates to the technical field of artificial intelligence, in particular to a vehicle fatigue detection method based on a multi-scale binary pattern.

Background technique

With the development of society and the progress of economy, cars have become a necessary means of transportation for people to expand their living space, improve their living efficiency and improve their quality of life. The rapid increase in the number of automobiles has brought many social problems while the society is prosperous. The primary issue is road safety, and road traffic accidents have become the primary factor causing abnormal deaths of human beings. Statistics show that in all road traffic accidents, human factors account for 80%, and fatigue driving is the most common human factor. Therefore, the recognition and early warning of fatigue driving will play a vital role in avoiding vicious traffic accidents and ensuring the safety of people's lives and property.

In the prior art, the LBP method has been applied in many methods. However, face recognition is affected by posture and illumination changes. Illumination changes and angle changes will hinder feature extraction, affect the detection rate, and cause fatigue detection to be inaccurate. The LBP operator has the following limitations in its application. First, single-scale LBP operators cannot detect dominant texture features when computing features that capture image structures, and they are not robust to image transformations and rotations. Second, in single-channel, such as grayscale image spaces, feature extraction only captures monochromatic intensity information, which limits recognition performance.

The present invention is an algorithm aimed at improving the accuracy of fatigue detection. With the fusion of computer vision, image analysis, and signal processing development functions derived from physical and biological vision, it can solve the above technical problems.

SUMMARY OF THE INVENTION

The technical problem to be solved by the invention is the technical problem of poor detection accuracy existing in the prior art. A new vehicle fatigue detection method based on multi-scale binary patterns is provided, and the vehicle fatigue detection method based on multi-scale binary patterns has the characteristics of high detection accuracy.

In order to solve the above technical problems, the technical scheme adopted is as follows:

A vehicle fatigue detection method based on a multi-scale binary pattern, the vehicle fatigue detection method based on a multi-scale binary pattern comprises:

Step 1: Manually classify and mark fatigue samples and non-fatigue samples on the driver image, select training samples and test samples respectively, and preprocess the training samples and test samples;

Step 2, divide the training sample image into several non-repetitive sub-regions, use multi-scale local binary patterns for feature extraction, and obtain multi-scale local binary image features;

Step 3, performing discrete Fourier transformation on the multi-scale local binary image features to obtain the histogram Fourier feature vector of the multi-scale binary pattern;

Step 4, connect the histogram Fourier feature vectors that constitute the multi-scale binary pattern to characterize the image features, select the kernel function, use the nonlinear support vector machine to classify and train the MLBP features of the sample image, and obtain the trained SVM Classification models and parameters;

Step 4: repeat steps 2 to 3 for the test sample, and use the trained SVM classification model and parameters to detect the histogram Fourier feature of the multi-scale binary pattern of the test sample.

Working principle of the present invention: the invention provides a vehicle fatigue detection method based on multi-scale binary pattern histogram Fourier feature (MLBP-HF) and support vector machine SVM. It is mainly based on the statistical learning theory and supports vector machines for data classification according to the supervised learning method.

First, feature extraction based on multi-scale local binary patterns:

Divide the image into multiple sub-regions of the same size but not overlapping, M ₀ , M ₁ ,...M _j-1 , the LBP histogram of each sub-region with different radii, calculated with the following formula:

B(v) is a Bool pointer, which is to calculate the histogram for each area of different scales (M _j ) to provide regional information, and L is the number of histogram bins.

Connect the histograms of the above sub-regions by the following formula to obtain a single vector, which is the final multi-scale feature description operator:

f _j =[h _P,1,j ,h _P,2,j ,...,h _P,R,j ].

The feature extraction method can effectively represent fatigue and achieve better detection results. The detection speed is fast and is less affected by illumination changes. However, it is found that the detection rate is very low when the driver's face is tilted or deflected, and the missed detection rate and high false positive rate. After extracting the MLBP features from the driver's face image, use Fourier transform to process the MLBP features, and perform discrete Fourier transform to correct the image with a certain deflection angle from the feature, thereby reducing the fatigue Detection of the impact, improve the detection rate.

In the above solution, for optimization, further, the preprocessing includes converting the RGB image into a grayscale image and image standardization.

Further, the feature extraction based on multi-scale local binary patterns includes:

Step 1, use LBP _8,1 , LBP _8,2 , LBP _8,3 three LBP operators to calculate the original image, and obtain images at different scales, respectively I1, I2, I3;

Step 2, divide I1, I2, and I3 into blocks, and the sub-blocks are as follows:

I1(1,n),I2(1,n),I3(1,n);

Step 3, splicing the corresponding sub-regions at different scales;

Step 4, calculating the LBP histogram for each splicing result in step 3;

Wherein, n is a positive integer.

Further, step three includes:

Step 3.1, use discrete Fourier transform to transform multi-scale local binary image features, let H _P,r (n, ) be the DFT of the nth row of the histogram h _P,r (U(n,c)) ,which is:

H _P,r (n,u)=∑h _P,r (U(n,c))e ^-i2πur/P

Step 3.2, for the phase shift in the DFT coefficients brought by the discrete Fourier transform input with an appropriate amount of cyclic shift, it is determined that h'P _,r (U(n,c))=hP _,r (U(n,ca )),

Then define H' _P,r (n,c)=H _P,r (n,u)e ^-i2πur/P ;

Definition For any 1≤n ₁ , n ₂ ≤P-1, there are:

in, is the complex conjugate of H _P,r (n ₂ ,u);

For any 1≤n ₁ , n ₂ ≤P-1 and histogram Fourier features of multiscale binary patterns, compute:

Define the Fourier magnitude spectrum as

definition

Then LBP ^r/u2 is included as a subset in the Fourier magnitude spectrum.

Use U(n,c) to represent the canonical LBP mode of the image. Among the 58 different canonical modes with the circular area (8,R), different rows represent the 1 in the bit sequence calculated by the canonical LBP operator for the central pixel. The number of different columns represent the rotation angle of the image.

Use n to represent the number of pattern bits, that is, the number of rows, and c is the rotation angle, that is, the number of columns. If the number of sample points is P, then 0≤n≤P+1, where n=P+1 represents all non-canonical modes, and when 1≤n≤P-1, the degree of rotation is 0≤c≤P-1. Add f'(x,y) to represent the deflected image of f(x,y), and the deflection angle is α. After the deflection, the point (x, y) moves to (x', y'), and it can be known that the two points are also deflected by α degrees.

According to the characteristics of multi-scale local binary operators, the deflection angle can be an integer multiple of the angle of two sample points, namely:

Then the canonical mode histogram of the rotated image corresponds to the cyclic shift of the histogram of the original image in each row.

h'P _,r (U(n,c+a))=hP _,r (U(n,c)).

To avoid the effect of cyclic shift on the rotated image histogram, a Fourier transform is applied to the image.

Use the discrete Fourier transform DFT to transform the previously obtained MLBP features:

Let H _P,r (n, ) be the DFT of the nth row of the histogram h _P,r (U(n,c)), namely:

H _P,r (n,u)=∑h _P,r (U(n,c))e ^-i2πur/P ;

The phase shift in the DFT coefficients brought about by an appropriate amount of cyclic shift for the DFT input if:

h' _P,r (U(n,c))=h _P,r (U(n,ca)), then H' _P,r (n,c)=H _P,r (n,u)e ^-i2πur/P ,

And for any 1≤n ₁ , n ₂ ≤P-1 have:

in is the complex conjugate of H _P,r (n ₂ ,u).

Show that for any 1≤n ₁ , n ₂ ≤P-1 and MLBP-HF, the features:

The cyclic transformation of the rows of h _P,r (U(n,c)) is invariant, and it is also invariant to the rotation of the input image f(x,y).

Fourier magnitude spectrum can be considered a special case of these characteristics.

Ultimately, since Then LBP ^r/u2 is included as a subset in the Fourier magnitude spectrum.

Therefore, after extracting MLBP features from the driver's face image, Fourier transform can be used to process the features. Fatigue detection effect, improve detection rate.

Beneficial effects of the present invention: the present invention proposes a fatigue detection method based on multi-scale binary pattern histogram Fourier feature (MLBP-HF) and SVM. The MLBP-HF features extracted in the method can not only overcome the influence of illumination changes, but also preserve the image details to the greatest extent. Moreover, the Fourier transform of MLBP features can overcome the influence of attitude changes and solve the problem of low detection rate for tilted faces. The invention effectively improves the recognition rate of face recognition, makes the recognition less affected by posture and illumination changes, and improves the fatigue detection rate to a certain extent.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

FIG. 1 is a flow chart of the vehicle fatigue detection method in Embodiment 1.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example 1

This embodiment provides a vehicle fatigue detection method based on multi-scale binary patterns, as shown in Figure 1, the vehicle fatigue detection method based on multi-scale binary patterns includes:

Step 1: Manually classify and mark fatigue samples and non-fatigue samples on the driver image, select training samples and test samples respectively, and preprocess the training samples and test samples;

Step 2, divide the training sample image into several non-repetitive sub-regions, use multi-scale local binary patterns for feature extraction, and obtain multi-scale local binary image features;

Step 3, performing discrete Fourier transformation on the multi-scale local binary image features to obtain the histogram Fourier feature vector of the multi-scale binary pattern;

Step 4, connect the histogram Fourier feature vectors that constitute the multi-scale binary pattern to characterize the image features, select the kernel function, use the nonlinear support vector machine to classify and train the MLBP features of the sample image, and obtain the trained SVM Classification models and parameters;

Step 4: repeat steps 2 to 3 for the test sample, and use the trained SVM classification model and parameters to detect the histogram Fourier feature of the multi-scale binary pattern of the test sample.

Specifically, the preprocessing includes converting the RGB image into a grayscale image and image standardization.

In detail, the feature extraction based on multi-scale local binary patterns includes:

Step 1, use LBP _8,1 , LBP _8,2 , LBP _8,3 three LBP operators to calculate the original image, and obtain images at different scales, respectively I1, I2, I3;

Step 2, divide I1, I2, and I3 into blocks, and the sub-blocks are as follows:

I1(1,n),I2(1,n),I3(1,n);

Step 3, splicing the corresponding sub-regions at different scales;

Step 4, calculate the LBP histogram for each splicing result in step 3.

In detail, step three includes:

Step 3.1, use discrete Fourier transform to transform multi-scale local binary image features, let H _P,r (n, ) be the DFT of the nth row of the histogram h _P,r (U(n,c)) ,which is:

H _P,r (n,u)=∑h _P,r (U(n,c))e ^-i2πur/P

Step 3.2, for the phase shift in the DFT coefficients brought by the discrete Fourier transform input with an appropriate amount of cyclic shift, it is determined that h'P _,r (U(n,c))=hP _,r (U(n,ca )),

Then define H' _P,r (n,c)=H _P,r (n,u)e ^-i2πur/P ;

Definition For any 1≤n ₁ , n ₂ ≤P-1, there are:

in, is the complex conjugate of H _P,r (n ₂ ,u);

For any 1≤n ₁ , n ₂ ≤P-1 and histogram Fourier features of multiscale binary patterns, compute:

Define the Fourier magnitude spectrum as

definition

Then LBP ^r/u2 is included as a subset in the Fourier magnitude spectrum.

As shown in Figure 1, feature extraction based on MLBP includes the following two steps:

① Divide the image into multiple sub-regions of the same size but not overlapping, M ₀ , M ₁ , ... M _j-1 , the LBP histogram of each sub-region with different radii is calculated by the formula;

② Connect the histograms of the above sub-regions through the formula to obtain a single vector, which is the final multi-scale feature description operator.

As shown in Figure 1, the MLBP feature acquisition process of the image includes the following four steps:

① Use three different LBP operators to calculate the original image to obtain images at different scales;

②Block images at different scales;

③ Splicing the corresponding sub-regions at different scales;

④ Calculate the LBP histogram for each splicing result.

As shown in Figure 1, the standard SVM classification is divided into two stages:

First, the training phase:

① Establish a training sample set;

② Select the kernel function and parameters to transform the low-dimensional space to the high-dimensional space;

③ Normalization of input samples;

④ Construct a kernel matrix;

⑤ Solve the Lagrangian coefficient;

⑥Calculate support vectors and solve classification hyperplane coefficients;

⑦ According to the obtained various coefficients, the optimal classification hyperplane of the training data is established to complete the training process.

Second, the detection phase:

① Read in the template data file trained in the learning phase;

② Calculate the output value of the new input characteristic data according to the formula;

③ Use the classification function to calculate and give the classification result.

This embodiment proposes a vehicle-mounted fatigue detection method based on MLBP features and SVM. Compared with the prior art, the present invention improves the problem that a face image with a certain degree of inclination will affect the detection rate. The feature extraction method is optimized.

The present invention is divided into training samples and test samples, and the experiment is divided into two processes of training and detection. In the training stage, the method based on the MLBP operator is first used to extract the features of the face training sample images, and then the feature vector is input into the SVM. Carry out classification training, and select a kernel function to obtain the SVM classification model; in the detection stage, first use the same method to extract features from the test samples, and then use the SVM classification model for fatigue detection. Image segmentation technology is used in extracting features. After comparing experiments with different segmentation types and analyzing the experimental results, the method is aimed at the problem of low detection rate of tilted faces.

This embodiment proposes a fatigue detection method based on multi-scale binary pattern histogram Fourier feature MLBP-HF and support vector machine SVM. The MLBP-HF feature extracted in this method can not only overcome the influence of illumination changes, but also preserve the image detail features to the maximum extent, and can overcome the influence of attitude changes after performing Fourier transformation on MLBP features. This embodiment has a better recognition rate in face recognition, is less affected by posture and illumination changes, and improves the fatigue detection rate to a certain extent. Therefore, this vehicle fatigue detection method based on MLBP features and SVM has important theoretical significance and application value in the field of vehicle fatigue detection.

Although the illustrative specific embodiments of the present invention have been described above, so that those skilled in the art can understand the present invention, the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as each As long as the changes are within the spirit and scope of the present invention defined and determined by the appended claims, all inventions and creations utilizing the concept of the present invention are included in the protection list.

Claims (4)

1. A vehicle-mounted fatigue detection method based on a multi-scale binary mode is characterized by comprising the following steps: the vehicle-mounted fatigue detection method based on the multi-scale binary mode comprises the following steps:

step one, marking a fatigue sample and a non-fatigue sample for manual classification of a driver image, respectively selecting a training sample and a testing sample, and preprocessing the training sample and the testing sample;

dividing the training sample image into a plurality of non-repetitive sub-regions, and performing feature extraction by using a multi-scale local binary mode to obtain multi-scale local binary image features;

performing discrete Fourier change on the multi-scale local binary image characteristics to obtain a histogram Fourier characteristic vector of the multi-scale binary mode;

connecting histogram Fourier feature vectors forming a multi-scale binary mode, representing image features, selecting a kernel function, carrying out classification training on MLBP (maximum likelihood back propagation) features of the sample image by using a nonlinear support vector machine, and obtaining a trained SVM classification model and parameters;

and step four, repeating the step two to the step three on the test sample, and detecting the histogram Fourier characteristics of the multi-scale binary mode of the test sample by utilizing the SVM classification model and the parameters obtained by training.

2. The vehicle-mounted fatigue detection method based on the multi-scale binary pattern according to claim 1, characterized in that: the preprocessing comprises converting the RGB image into a gray scale image and image standardization processing.

3. The vehicle-mounted fatigue detection method based on the multi-scale binary pattern according to claim 1, characterized in that: the feature extraction based on the multi-scale local binary pattern comprises the following steps:

step 1, with LBP_8,1,LBP_8,2,LBP_8,3Calculating the original image by using the three LBP operators to obtain images under different scales, namely I1, I2 and I3;

step 2, partitioning I1, I2 and I3 into sub-blocks which are respectively:

I1(1,n),I2(1,n),I3(1,n)；

step 3, splicing the corresponding sub-regions under different scales;

step 4, calculating an LBP histogram for each splicing result in the step 3;

wherein n is a positive integer.

4. The vehicle-mounted fatigue detection method based on the multi-scale binary pattern according to claim 1, characterized in that: the third step comprises:

step 3.1, transforming the multi-scale local binary image characteristics by using discrete Fourier transform, and setting H_P,r(n,. cndot.) is a histogram h_P,rDFT of the nth row of (U (n, c)), namely:

H_P,r(n,u)＝∑h_P,r(U(n,c))e^-i2πur/P

step 3.2, h 'is determined for the phase shift in DFT coefficient caused by inputting proper amount of cyclic shift in discrete Fourier transform'_P,r(U(n,c))＝h_P,r(U(n,c-a))，

Then define H'_P,r(n,c)＝H_P,r(n,u)e^-i2πur/P；

Definition for arbitrary 1 ≦ n₁,n₂P-1 is less than or equal to, and the following components are:

wherein,is H_P,r(n₂Complex conjugation of u);

for any 1 ≦ n₁,n₂And (3) calculating the histogram Fourier characteristics of the multi-scale binary mode and the P-1 or less, and calculating:

defining a Fourier magnitude spectrum as

Definition of

Then LBP^r/u2As a subset, to the fourier magnitude spectrum.