CN106874900A - A kind of tired driver detection method and detection means based on steering wheel image - Google Patents

Abstract

The present invention is a driver fatigue detection method and detection device based on a steering wheel image. The main steps of the method are: obtaining a template image, obtaining the coordinates of the center position of the steering wheel, obtaining feature points on the steering wheel, and obtaining feature points in the current frame and the next frame Coordinate position, calculate and store the steering wheel rotation angle; to calculate the zero speed percentage and angle standard deviation of the steering wheel within a certain period of time; use the support vector machine classification algorithm to train the sample set, and find out the decision function for judging whether the driver is in a state of fatigue. During monitoring, the steering wheel image is collected in real time, and the current zero-speed percentage and the value of the angle gauge deviation are obtained from the steering wheel angle, which is substituted into the decision function. The result is 1, which means that the current driver is in a fatigue state, and an alarm is given immediately. The camera of the device is installed on the top of the cab, and the camera is connected to the central processor; the central processor is also connected to the alarm module. The invention has simple operation, accurate measurement results, can calculate the steering wheel rotation angle larger than one cycle, and has a wider application range.

Description

A driver fatigue detection method and detection device based on steering wheel image

technical field

The invention relates to the field of vehicle driver fatigue detection, in particular to a driver fatigue detection method and detection device based on steering wheel images.

Background technique

As the number of motor vehicles increases, traffic accidents also continue to increase. Statistics show that the number of deaths caused by fatigue driving in my country is about 9,000 every year, which has brought huge loss of personnel and property to the country. Statistics show that traffic accidents caused by driver fatigue account for about 20% of the total number of traffic accidents and more than 40% of the number of serious traffic accidents. If early warning can be given at the initial stage of driver fatigue, the occurrence of traffic accidents will be reduced, which is of great significance to traffic safety.

Many of the existing driver fatigue detection is based on face technology, because it is necessary to identify and classify the characteristics of the driver's normal state and fatigue state, the hardware cost is high, the algorithm is complex, the real-time effect is poor, and it is difficult to realize, and because some drivers Fatigue is not particularly reflected on the face, and it is greatly affected by personal characteristics.

The study found that the steering wheel angle data can reflect the driver's operating status in real time. When the driver is in a state of fatigue, his ability to perceive the environment, judge ability and actual control ability will all decrease, and the steering wheel angle directly controlled by him will fluctuate abnormally.

By detecting the real-time change of the steering wheel angle, the fatigue state of the driver can be judged. The key is the measurement of the steering wheel angle. At present, the method for measuring the steering wheel angle is to install a rotation angle sensor to measure the rotation angle of the steering wheel. A more representative example is the invention patent application with application number 201310068176.6, titled "Fatigue Detection Method and Detection Device Based on Steering Wheel Angle Data" and application number 201510113007.9, titled "Driver Fatigue State Detection Method Based on Steering Wheel Angle Information" "Invention patent application, although their scheme effectively extracts the fatigue feature information of the steering wheel angle and effectively detects the driver's fatigue state, but there are also obvious problems:

1. The steering wheel angle sensor needs to be directly fixed on the steering wheel of the vehicle or on the steering wheel shaft. The angle sensor blocks part of the instrument panel, and the driver's view of the instrument is blocked. On the other hand, the sensor on the steering wheel or its shaft may also affect the steering wheel. rotation, affecting normal driving operation;

2. The existing steering wheel angle sensor has no angle memory function, so when the steering wheel turns more than one circle, the angle sensor cannot obtain correct angle data;

3. The steering wheel angle sensor is difficult to install. It usually needs to be fixed on the steering wheel or the rotating shaft before the car leaves the factory. It is inconvenient to disassemble and has poor portability. It cannot meet the needs of existing vehicle driver fatigue detection.

The application number that occurs again for this reason is that 201210343216.9 name is called " a kind of vehicle steering wheel angle detection method and detection device based on image " scheme, it does not need to install steering wheel angle sensor, sticks two kinds of marking stickers and marking line on the back side of steering wheel, with The initial state image of the steering wheel is compared with the image after rotation, and the steering wheel angle is calculated. However, the marking is too cumbersome. There are six marking lines to divide the steering wheel into six areas. In each area, a triangular marking and an arc marking should be pasted according to the regulations, which is difficult to implement; the marking is pasted on the back of the steering wheel, and the light is poor. Affecting the image clarity, the driver will cover part of the mark when holding the steering wheel, which also affects the detection effect. Therefore it is also difficult to be practical.

The traditional driver fatigue discrimination based on steering wheel angle information uses Fisher's linear discriminant algorithm. This method regards the vehicle's driving as straight-line driving, and uses the two indicators of angle standard deviation σ and zero speed percentage PNS to identify drivers with Fisher's linear discriminant algorithm. state of fatigue. However, the actual driving of the vehicle has situations such as taking an "S" shape, overtaking and changing lanes. It is obviously unreasonable to use the angle standard deviation σ greater than 1.2° as the criterion for judging fatigue. Therefore, the accuracy of the traditional driver fatigue discrimination method based on steering wheel angle information is not high.

An effective, simple and easy image-based driver fatigue state detection method has not been seen so far.

Contents of the invention

The purpose of the present invention is to propose a driver fatigue detection method based on the steering wheel image in order to overcome the defects in the prior art. The camera collects the steering wheel image in real time, uses a template matching algorithm to match the template image with the real-time image, and obtains the result by image processing. Steering wheel rotation angle, calculate the current zero-speed percentage and angle standard deviation, and use the support vector machine algorithm to obtain the current driver's state. The method is simple in operation, strong in practicability, and can accurately detect the state of the driver.

Another object of the present invention is to propose a driver fatigue detection device based on the steering wheel image, including a central processor, a camera and an alarm module, the camera is installed on the top of the cab, the steering wheel is in the viewfinder frame of the camera, and the video signal of the camera is output The line is connected with the central processor of the device; the central processor is also connected with the alarm module.

The main steps of a kind of driver fatigue detection method based on the steering wheel image designed by the present invention are as follows:

Step Ⅰ. Get the template image

The camera is installed on the top of the cab, and the video signal output line of the camera is connected to the central processor of the detection device; the camera collects the still steering wheel image and stores it as the initial image;

After the central processor performs denoising processing on the initial image, it is stored as a template image.

Points on the steering wheel circle correspond to points on the image ellipse captured by the camera, and the distance between a point m(x _m , y _m ) on the image ellipse and the corresponding point M(x _M , y _M ) on the steering wheel circle The corresponding relation is x _M ＝x _m 、 In the formula, the radius R is the semi-major axis of the ellipse, and the length of R remains unchanged.

Step Ⅱ. Obtain the coordinates o(x ₀ ,y ₀ ) of the center of the steering wheel

The steering wheel center position coordinates can be obtained on the steering wheel template image of any type of vehicle obtained in step I,

Ⅱ-1. Steering wheel outline

Adopt the findContours () function that the OpenCV open source visual library carries to the template image of step I gained, obtain the coordinates of each point on the outer contour of the steering wheel image by setting corresponding parameters;

Ⅱ-2. Position coordinates of the center of the steering wheel

From the 5 coordinate points (x _m1 , y _m1 , x _m2 , y _m2 ,..., x _m5 , y _m5 ) on the outer contour of the steering wheel image determined in step Ⅱ-1, determine the ellipse equation and obtain the position of the center of the ellipse Coordinate o(x ₀ ,y ₀ );

Step Ⅲ. Feature point tracking

This method uses the optical flow method to match each video sequence with the template image to detect the rotation angle of the current steering wheel image; take representative and easy-to-determined feature points on the steering wheel, and the representative and easy-to-determined feature points are A point that remains unambiguously located on the steering wheel image after the steering wheel has been turned. By iteratively obtaining the coordinate position of the feature point in the steering wheel image in the current frame and the coordinate position in the next frame, according to the coordinates of the feature point in the current frame and the coordinates of the feature point in the next frame, it is calculated from the current frame to the next frame. The rotation angle of the feature point in one frame, that is, the rotation angle of the steering wheel;

Step Ⅳ. Calculate the rotation angle θ of the steering wheel in two adjacent frames

The calculation of the following angle θ adopts the position coordinates of the characteristic point A on the steering wheel circle.

The center of the steering wheel o(x ₀ , y ₀ ) is determined in step II, and the coordinates of the position A ₀ (x _M01 , y _M01 ) of the feature point A in the current frame are determined in step III. After the steering wheel is rotated by an angle θ, the feature point is determined again The coordinates of the feature point position A ₁ (x _M11 , y _M11 ) of the next frame corresponding to the position A ₀ (x _M01 , y _M01 ).

The included angle between the center o(x ₀ , y ₀ ) and the line connecting A ₀ (x _M01 , y _M01 ), A ₁ (x _M11 , y _M11 ) is θ ₁ , according to the law of cosines That is, the rotation angle of feature point A,

Calculated by the distance formula between two points:

Step Ⅴ, the rotation angle of the feature point of the steering wheel image

Let the coordinates of the center of the steering wheel be o(x ₀ , y ₀ ), the coordinates of the feature points in the kth frame image and the k+1th frame image are D _k (x _Mk , y _Mk ), D _k+1 (x _{M( k+1)} , y _M(k+1) ), the rotation angle of feature points in two adjacent frames of images is less than 90°.

When y _Mk ≥ y ₀ , y _M(k+1) ≥ y ₀ : If x _M(k+1) ≥ x _Mk is satisfied, it means that the k+1th frame image rotates an angle clockwise relative to the kth frame image ; If x _M(k+1) ≤ x _Mk is satisfied, it means that the k+1 frame image is rotated counterclockwise by an angle relative to the k frame image.

When y _Mk ≤y ₀ , y _M(k+1) ≤y ₀ : if x _M(k+1) ≥x _Mk is satisfied, it means that the k+1th frame image rotates an angle counterclockwise relative to the kth frame image ; If it satisfies x _M(k+1) ≤ x _Mk , it means that the k+1th frame image rotates clockwise by an angle relative to the kth frame image.

When y _Mk ≥y ₀ , y _M(k+1) ≤y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ is satisfied, it means that the k+1th frame image is relative to the kth frame The image rotates an angle clockwise; if x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ is satisfied, it means that the k+1 frame image rotates an angle counterclockwise relative to the k frame image.

When y _Mk ≤y ₀ , y _M(k+1) ≥y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ are satisfied, it means that the k+1th frame image is relative to the kth frame The image is rotated by an angle counterclockwise; if x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ is satisfied, it means that the k+1th frame image is rotated by an angle clockwise relative to the kth frame image.

This method stipulates that the steering wheel in the next frame rotates clockwise with respect to the steering wheel in the current frame. The angle θ is negative, and the angle θ is positive when it is rotated counterclockwise. Accumulate the rotation angle of each frame of the steering wheel image in sequence. When the accumulated value is greater than 360° in a certain period of time, that is, the position of the steering wheel relative to the template image rotates counterclockwise for more than one week; Rotate the position clockwise for more than one revolution;

Calculate the zero-speed percentage and angle standard deviation of the steering wheel during the steering wheel rotation time period according to the feature point rotation angle data stored in each frame of image;

Step Ⅵ, extract the technical index zero speed percentage and angle standard deviation

When the frequency of steering wheel angle changes decreases within time t, that is, when the frequency of corrections to the steering wheel by the driver decreases, the time for which the steering wheel does not move increases. As one of the technical indicators of this method, the zero speed percentage PNS is used to characterize the selected The extent to which the steering wheel does not move within the time period, the t is 50 to 80 seconds.

The definition of zero speed percentage PNS is as follows:

The sampling frequency of the present invention is 8 to 12 frames per second, PNS _i represents the zero velocity percentage of the i frame, N _i represents the total number of sampling points of the angular velocity within t seconds before the i frame, and _ni represents the internal angular velocity of the i frame within t seconds Number of points between ±0.1°/s.

The definition of another technical index angle standard deviation of this method is shown in the following formula:

Among them: θ _j is the steering wheel rotation angle corresponding to the jth frame, and μ represents the average steering wheel rotation angle value of each frame in the m2 frame;

Step VII, Support Vector Machine Classification Algorithm

Combine the zero-speed percentage PNS and angle standard deviation σ obtained in step VI, and use the support vector machine classification algorithm to judge whether the driver is in a fatigue state;

The support vector machine classification algorithm first trains the sample set, and finds a hyperplane in the sample space that divides samples of different categories. The model corresponding to the hyperplane is ω ^T x + b = 0, and the model parameter ω is the normal vector and b is the displacement item , T is the transpose, x is the independent variable composed of zero speed percentage and angle standard deviation; the decision function obtained by the hyperplane model

f(x)＝sgn[(ω ^* ×x)+b ^* ]

Where sgn[(ω ^* ×x)+b ^* ] is a sign function, when ω ^* ×x+b>0 f(x)＝1, when ω ^* ×x+b＝0f(x)＝0, when ω ^* ×x+b<0 f(x)=-1;

According to the W training samples of the vehicle driving on the above-mentioned different road sections, W is 80-150, learned by the support vector machine classification algorithm, and obtains the decision function f(x) that distinguishes whether the driver is awake or fatigued;

During the actual driving process of the driver, the camera collects the steering wheel image in real time, and after processing, the steering wheel angle is obtained, and then the value of the current zero-speed percentage and angle standard deviation is calculated, and the current zero-speed percentage and angle standard difference are substituted into the calculated values Decision function f(x), if f(x)=1, it means that the current driver is in a state of fatigue, and the system will give an alarm immediately.

The specific execution of the step I includes the following steps:

Ⅰ-1. Camera installation

The camera is installed on the top of the cab, so that the entire steering wheel is in the frame of the camera, and the camera is adjusted to make the image clear and match the size of the frame; transmit the collected image data to the central processor;

Ⅰ-2. Acquisition of the initial image of the steering wheel

Before the vehicle starts, when the steering wheel is in the initial state, the camera collects the still steering wheel image and stores it as the initial image;

Ⅰ-3. Template image

The central processor uses a median filter with a kernel function window of 3×3 to denoise the initial image obtained in step I-2, and maintains the edge detail information of the image while removing impulse noise and salt and pepper noise.

In step I-2, before the camera captures the initial image of the steering wheel, a piece of white paper is placed under the steering wheel, the projection of the steering wheel is located on the white paper, and an initial image with a white background is collected.

The specific process of the step III feature point tracking is as follows:

Ⅲ-1. Processing of the current image

The shooting frequency of the camera in this method is 12-16 frames/second, and the sampling frequency of the central processor is 9-12 frames/second lower than the shooting frequency;

The central processor performs the same denoising processing as step Ⅰ-3 on the steering wheel image collected by the camera in real time;

Ⅲ-2. Feature points of the current frame

Use the existing Shi-Tomasi in OpenCV to perform feature point detection on the current image processed in step Ⅲ-1, and use the goodFeaturesToTrack() function of the OpenCV open source vision library to detect and obtain the converted position of the feature points of the current frame A ₀ (x _M01 , y _M01 );

Ⅲ-3. The feature points of the next frame

Call the cvCalcOpticalFlowPyrLK() function of the OpenCV open source vision library, input the feature point position A ₀ (x _M01 , y _M01 ) in the current frame image obtained in step III-2, and output the converted feature point in the next frame image Position A ₁ (x _M11 , y _M11 ).

In the step III, another representative and easy-to-determined feature point B is taken on the steering wheel, and the angle between A, B and the line connecting the center of the circle is greater than 30 degrees;

The III-2 simultaneously determines the current frame position B ₀ (x _M02 , y _M02 ) of the feature point B;

The same method as described in III-3 determines the position B ₁ (x _M12 , y _M12 ) of the feature point position B ₀ (x _M02 , y _M02 ) in the next frame image;

The included angle between the center o(x ₀ , y ₀ ) and the line connecting B ₀ (x _M02 , y _M02 ) and B ₁ (x _M12 , y _M12 ) is θ ₂ , which is determined according to the cosine That is, the rotation angle of feature point B, Calculated by the distance formula between two points:

The final steering wheel rotation angle θ is the average value of the rotation angles of the two feature points A and B,

Tracking the rotation angle of two feature points can make the detection more accurate; in addition, when one of the feature points fails to pick up the rotation result due to the driver's operation occlusion, that is, the tracking fails, the other feature point can also be used to achieve the tracking purpose. The rotation angle θ of the steering wheel is the rotation angle of the feature point.

The specific steps of the step VII support vector machine classification algorithm are as follows:

Ⅶ-1. Collection of training samples

Select at least 5 drivers with five or more years of driving experience and at least 5 vehicles in good condition. Each driver randomly selects a vehicle, and collects discrete data on different driving situations of straight-line driving, "S"-shaped route driving and overtaking. Next, the samples when the driver is awake and fatigued, (x ₁ , y ₁ ),..., (x _W , y _W ). Where x ₁ = (x ₁₁ , x ₁₂ ) ^T , x ₁₁ , x ₁₂ respectively represent the first index zero speed percentage PNS and the second index angle standard deviation σ of the first sample data; y _k1 ∈ Y = { 1, -1}, Y=1, Y=-1 respectively represent that the driver is awake and fatigued, W is 80-150, the driver’s fatigue state is judged by the driver’s facial video scoring method to evaluate the driver’s fatigue state ; The driver's facial video is divided into multiple sections with an interval of 12-20s. At least 5 trained experimenters independently score each section of video according to the currently commonly used driver state characteristics. Judgment result;

Ⅶ-2. Solving of parameters

The hyperplane that separates the training samples satisfies that the sum of the distances from the two types of support vectors to the hyperplane is the largest, and the support vector is the sample point that satisfies ω ^T x + b = ± 1, specifically to find the hyperplane model ω ^T x The parameters ω and b of +b=0 make the sum of the distances from different types of support vectors to the hyperplane the largest, which is transformed into:

ST y _t (ω ^T x _t +b)≥1, t=1, 2,..., n1

where x _t = (PNS _t , σ _t )y _t ∈ {1, -1}

This is a convex quadratic programming problem, and its dual is obtained by using the efficient Lagrange multiplier method. The above model is non-linearly separable. A Gaussian kernel function that realizes linear mapping is introduced, and the dual of the above formula is obtained after transformation, namely:

in: σ ₁ is the bandwidth of the Gaussian kernel,

x _t1 , x _t2 , y _t1 and y _t2 represent sample parameters;

C is a regularization constant, t1, t2=1, 2,..., n1;

Solve the optimal Lagrangian multiplier α ^* = (α ₁ ^* , α ₂ ^* ,..., α _n1 ^* ) ^T , and calculate the optimal normal vector Select a positive component α _t1 ^* of the optimal Lagrange multiplier α ^* that is smaller than the regularization constant C to calculate the optimal displacement term

VII-3. Construct a hyperplane (ω ^* ×x) + b ^* = 0, and the decision function obtained from it

f(x)=sgn[(ω ^* ×x)+b ^* ].

According to the driver fatigue detection method based on the steering wheel image of the present invention, the present invention designs a driver fatigue detection device based on the steering wheel image, including a central processor, a camera and an alarm module, and the camera is installed on the top of the cab, above the steering wheel , the whole steering wheel is in the viewfinder frame of the camera, and the video signal output line of the camera is connected to the central processor of the device; the central processor is equipped with a storage module and a power supply module, and is also connected to an alarm module, and the alarm module is an alarm light or buzzer;

The central processor is connected to the image processing module through the data transmission module, and the image processing module is connected to the camera, and the steering wheel image collected by the camera is sent to the image processing module of the central processor, and the image processing module processes the image and video data, and is sent to the image processing module through the data transmission module. The storage module stores and extracts the data and sends them to the central processor. The central processor will compare the current steering wheel image with the template image to obtain the current steering wheel rotation angle, obtain the current zero speed percentage and angle standard deviation, and substitute it into the classification according to the support vector machine. The decision function learned by the algorithm judges whether the current driver is in a state of fatigue; when the driver is detected to be in a state of fatigue, the central processor sends an instruction to the alarm module, and its alarm light flashes or the buzzer sounds.

The camera is installed at point N on the top of the cab, and point N is behind the vertical plane passing through the center of the steering wheel and perpendicular to the longitudinal axis of the vehicle body, and the distance from the vertical plane is less than or equal to 20cm. The distance between the center of the steering wheel and the vertical plane parallel to the longitudinal axis of the vehicle body is less than or equal to 15cm.

Compared with the prior art, the advantages of a driver fatigue detection method based on the steering wheel image and the detection device of the present invention are: 1. Simple operation, real-time detection of the steering wheel angle according to the real-time image of the steering wheel, accurate measurement results, and when the steering wheel When the angle of rotation is greater than one circle, the angle of rotation of the steering wheel can also be obtained; 2. The camera is installed on the top, which does not block the driver's sight of the instrument, nor does it affect the rotation of the steering wheel, and the driver's normal driving is not hindered; 3. The central processor The OpenCV open source vision library and support vector machine classification algorithm used are mature software, with good practicability and robustness; 4. The application range of the support vector machine classification algorithm is wider, and it is no longer limited to drivers driving in a straight line. Fatigue state discrimination; 5. The camera is easy to install and disassemble, has strong portability and good practicability, and can be installed on various existing vehicles to meet the needs of driver fatigue detection.

Description of drawings

Fig. 1 is the schematic diagram of the installation relationship between the camera and the steering wheel of the embodiment of the driver fatigue detection device based on the steering wheel image;

Fig. 2 is the structural block diagram of the embodiment of the driver's fatigue detection device based on the steering wheel image;

Fig. 3 is the embodiment flow chart of the driver's fatigue detection method based on the steering wheel image;

Fig. 4 is a schematic diagram of feature points A and B taken in step III of the embodiment of the driver fatigue detection method based on the steering wheel image;

5 is a schematic diagram of the actual steering wheel feature point positions and the corresponding feature point positions in the steering wheel image collected by the camera in the embodiment of the driver fatigue detection method based on the steering wheel image;

FIG. 6 is a schematic diagram of rotation angles of two feature points in the embodiment of the method for detecting driver fatigue based on the steering wheel image.

In the picture:

1-Steering wheel 2-Camera.

detailed description

The specific implementation manners of the present invention will be described in further detail below in conjunction with the accompanying drawings.

Embodiment of Driver Fatigue Detection Device Based on Steering Wheel Image

The embodiment of the driver fatigue detection device based on the steering wheel image includes a central processor, a camera and an alarm module. The video signal output line is connected with the central processing unit of the device; the collected image data is transmitted.

As shown in Figure 1, the camera is installed at point N on the top of the cab. In this example, point N is behind the vertical plane passing through the center of the steering wheel and perpendicular to the longitudinal axis of the vehicle body, and the distance from the vertical plane is 15cm. N The distance between the point and the vertical plane passing through the center of the steering wheel and parallel to the longitudinal axis of the vehicle body is 5cm.

As shown in Figure 2, the central processor in this example is equipped with a storage module and a power supply module, and is also connected to an alarm module, which uses a buzzer in this embodiment. In this example, the central processor is connected to the image processing module via the data transmission module. The image processing module is connected to the camera, and the steering wheel image collected by the camera is sent to the image processing module. The image processing module processes the image and video data, sends them to the storage module for storage through the data transmission module, and extracts the data and sends them to the central processor. The central processor compares the current steering wheel image with the template image to obtain the current steering wheel rotation angle θ, and obtain The current zero-speed percentage PNS and angle standard deviation σ are substituted into the decision function f(x)=ω ^T x+b learned from the support vector machine classification algorithm (SVM) to calculate. When f(x)=1, it means that the driver is fatigued state, the central processor sends instructions to the alarm module, and its buzzer sounds.

Embodiment of Driver Fatigue Detection Method Based on Steering Wheel Image

The embodiment of the driver fatigue detection method based on the steering wheel image, its flow process is as shown in Figure 3, and the specific steps are as follows:

Step Ⅰ. Get the template image

Ⅰ-1. Camera installation

After selecting the vehicle in this embodiment, the camera is installed on the top of the cab, so that the whole steering wheel is in the viewfinder of the camera, and the camera is adjusted to make the image clear and match the size of the viewfinder; the video signal output line of the camera and the detection device The central processor is connected; the collected image data is transmitted.

Ⅰ-2. Acquisition of the initial image of the steering wheel

Before the vehicle starts, put a piece of white paper under the steering wheel, the projection of the steering wheel is located on the white paper, and when the steering wheel is in a static initial state, the camera collects the steering wheel image in the initial state with a white background; transmits it to the central processor, The central processor stores this as the initial image;

Ⅰ-3. Template image

The central processor performs denoising processing on the initial image collected by the camera. This method uses a kernel function window 3×3 median filter for denoising, and maintains the edge details of the image while removing impulse noise and salt and pepper noise.

After the central processor completes denoising processing on the initial image in step I, it is stored as a template image. The template image effect is shown in Figure 4.

The steering wheel image is an ellipse, and the points on the steering wheel circle correspond to the points on the image ellipse as shown in Figure 5. The point m(x _m , y _m ) on the image ellipse corresponds to the corresponding point M(x _M , y _M ) is the correspondence between The radius R is the semi-major axis of the ellipse, and the length of R is constant.

Step Ⅱ, the coordinates of the center of the steering wheel o(x ₀ ,y ₀ )

According to the steering wheel template image of the vehicle obtained in step I, the position coordinates of the center of the steering wheel are obtained, as follows:

Ⅱ-1. Steering wheel outline

Under the integrated development environment VS2013 and OpenCV3.0.0, use the findContours() function in the OpenCV open source vision library for the template image obtained in step I, and obtain the coordinates of each point on the outer contour of the steering wheel image by setting the corresponding parameters. The function is:

Void findContours(InputArray image,

OutputArrayOfArrays contours, OutputArray hierarchy,

int mode, int method, Point offset = Point())

The parameters are as follows:

InputArray image represents the input image of InputArray,

OutputArrayOfArrays contours represents the point vector (x _m , y _m ) stored in the contour,

OutputArray hierarchy represents an optional output vector, which contains the topological information of the image,

int mode represents the retrieval method of the contour,

int method represents the approximation method of the contour,

Point offset = Point() represents an optional offset for each contour point, use the default value Point().

The function used in the present embodiment to find the coordinates of each point of the outer contour of the steering wheel is:

findContours(image, contours, hierarchy, RETR_EXTERNAL,

CHAIN_APPROX_NONE, Point offset = Point ()).

Ⅱ-2. Calculate the coordinates of the center of the steering wheel by the method of least squares

The steering wheel image collected in this example is an ellipse as shown in Figure 5, and the general formula of the ellipse quadratic curve is:

ax ² +bxy+cy ² +dx+ey+1＝0

From the five coordinate points (x _m1 , y _m1 , x _m2 , y _m2 ,..., x _m5 , y _m5 ) on the outer contour of the steering wheel image determined in step Ⅱ-1, the ellipse equation is determined, and in this example, The coordinates of the center of the ellipse are o(545,421).

Step Ⅲ. Feature point tracking

This method uses the optical flow method to match each video sequence with the template image to detect the rotation angle of the current steering wheel image; as shown in Figure 4, this example takes the intersection point between the two symmetrical edges in the inner contour of the steering wheel and the outside of the center As two feature points of A and B. No matter how the steering wheel turns, points A and B are easy to determine on its image.

The specific tracking process is as follows:

Ⅲ-1. Processing of the current image

In this embodiment, the imaging frequency of the camera is 15 frames/second, and the sampling frequency of the central processor is 10 frames/second.

The central processor performs the same denoising preprocessing as step Ⅰ-3 on the steering wheel image collected by the camera in real time;

Ⅲ-2. Feature points of the current frame

The current image processed in step III-1 adopts Shi-Tomasi in the existing OpenCV to perform feature point detection, as shown in Figure 6, in this embodiment, the goodFeaturesToTrack () function in the OpenCV open source vision library is used to detect the current The positions A ₀ (x _M01 , y _M01 ) and B ₀ (x _M02 , y _M02 ) of a set of feature points of the frame obtained after conversion; the parameters of this function are as follows:

Void goodFeaturesToTrack(InputArray image,OutputArray corners,

Int maxCorners, double qualityLevel, double minDistance,

InputArray mask, int blockSize, bool useHarrisDetector, double k)

The parameters in which are

InputArray image represents the source image,

OutputArray corners represent the output vector (x _m , y _m ) of the detected feature points,

Int maxCorners represents the maximum number of feature points taken by this method,

double qualityLevel represents the minimum feature value acceptable for feature point detection,

double minDistance represents the minimum distance between feature points,

InputArray mask is the set area of interest,

int blockSize represents the default value of the neighborhood range specified when calculating the derivative autocorrelation matrix is 3,

bool useHarrisDetector means not to use Harris feature point detection, the default value is false,

The double k represents the default value of the weight coefficient for setting the relative weight of the determinant of the hessian autocorrelation matrix to 0.04.

In this example, the specific parameters are set to:

goodFeaturesToTrack(g_grayImage, corners, 20, 0.01, 40, imageROI, 3, false, 0.04).

Ⅲ-3. The feature points of the next frame

Input a group of feature point positions A ₀ (x _M01 , y _M01 ) and B ₀ (x _M02 , y _M02 ) in the current frame image obtained in step III-2 into the CalcOpticalFlowPyrLK() function of the OpenCV open source vision library, and output the above _The positions A ₁ ( _{x M11} , _{y M11 )} and B _{1 ( x M12} , y _M12 ),

The meaning of each parameter of the function is:

Void cvCalcOpticalFlowPyrLK(const CvArr*prev,

const CvArr*curr, const CvPoint2D32f*prev_features,

Const CvPoint2D32f*curr_features,

char*status, float*track_error, CvSize win_size,

int count, CvTermCriter criteria, int flags)

The parameters are as follows:

const CvArr*prev represents the previous frame image,

const CvArr*curr represents the current frame image,

const CvPoint2D32f*prev_features represents the previous frame coordinates of feature points,

Const CvPoint2D32f*curr_features represents the current frame coordinates of the feature points,

char*status represents the output status vector,

float*track_error represents the output error vector,

CvSize win_size represents the search window size of each pyramid layer,

int count represents the maximum number of pyramid levels taken by this method,

CvTermCriter criteria represents the specified search algorithm iteration type,

int flags represents the default value of this parameter is 0.

In this embodiment, the specific parameters are set to:

CalcOpticalFlowPyrLK(pre_image, curr_image, points2D[0], points2D[1], status, err, winSize, 3, termcrit, 0).

Step Ⅳ. Calculate the rotation angle θ of the steering wheel in two adjacent frames

Step Ⅱ-2 determines the steering wheel center o(x ₀ , y ₀ ), step Ⅲ determines the current frame feature point positions A ₀ (x _M01 , y _M01 ) and B ₀ (x _M02 , y _M02 ); as shown in Figure 6 As shown, when the steering wheel turns an angle θ, the feature point positions A ₁ (x _M11 , y _M11 ) and B ₁ (x _M12 , y _M12 ) of the next frame are determined by step III-2.

The included angle between the center o(x ₀ , y ₀ ) and the line connecting A ₀ (x _M01 , y _M01 ), A ₁ (x _M11 , y _M11 ) is θ ₁ , according to the law of cosines That is, the rotation angle of feature point A,

The included angle between the center o(x ₀ , y ₀ ) and the line connecting B ₀ (x _M02 , y _M02 ) and B ₁ (x _M12 , y _M12 ) is θ ₂ , which is determined according to the cosine That is, the rotation angle of feature point B, Calculated by the distance formula between two points:

The final steering wheel rotation angle θ is the average of the rotation angles of the two feature points A and B,

Step Ⅴ, the rotation angle of the feature point of the steering wheel image

Let the coordinates of the center of the steering wheel be o(x ₀ , y ₀ ), the coordinates of the feature points in the kth frame image and the k+1th frame image are D _k (x _Mk , y _Mk ), D _k+1 (x _{M( k+1)} , y _M(k+1) ), the rotation angle of feature points in two adjacent frames of images is less than 90°;

When y _Mk ≥ y ₀ , y _M(k+1) ≥ y ₀ : If x _M(k+1) ≥ x _Mk is satisfied, it means that the k+1th frame image rotates an angle clockwise relative to the kth frame image ; If x _M(k+1) ≤ x _Mk is satisfied, it means that the k+1 frame image rotates an angle counterclockwise with respect to the k frame image;

When y _Mk ≤y ₀ , y _M(k+1) ≤y ₀ : if x _M(k+1) ≥x _Mk is satisfied, it means that the k+1th frame image rotates an angle counterclockwise relative to the kth frame image ; If x _M(k+1) ≤ x _Mk is satisfied, it means that the k+1th frame image rotates an angle clockwise with respect to the kth frame image;

When y _Mk ≥y ₀ , y _M(k+1) ≤y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ is satisfied, it means that the k+1th frame image is relative to the kth frame The image rotates an angle clockwise; if it satisfies x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ , it means that the image of frame k+1 rotates an angle counterclockwise relative to the image of frame k;

When y _Mk ≤y ₀ , y _M(k+1) ≥y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ are satisfied, it means that the k+1th frame image is relative to the kth frame The image rotates an angle counterclockwise; if it satisfies x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ , it means that the k+1th frame image is rotated clockwise by an angle relative to the kth frame image;

This example stipulates that the steering wheel rotation angle θ in the next frame relative to the steering wheel in the current frame is negative, and θ is positive when it rotates counterclockwise; the rotation angle of each frame of the steering wheel image is accumulated sequentially. When the accumulated value is greater than 360° in a certain period of time, that is, this time The position of the steering wheel relative to the template image in the segment rotates counterclockwise for more than one cycle; when the accumulated value is less than -360°, that is, the position of the steering wheel relative to the template image rotates clockwise for more than one cycle during this period;

Step Ⅵ, extract the technical index zero speed percentage and angle standard deviation

One of the technical indicators of this example, the percentage of non-steering, PNS (percentage of non-steering), represents the degree of steering wheel motion in the selected time period. In this embodiment, this time period is taken as 50 seconds.

The definition of zero speed percentage PNS is as follows:

The sampling frequency of the present invention is 10 frames per second, PNS _i represents the zero speed percentage of the i frame, N _i represents the total sampling points of the angular velocity in the first 50 seconds of the i frame, and n _i represents the angular velocity in the first 50 seconds of the i frame i Points between ±0.1°/s.

The definition of the angle standard deviation of another technical indicator is shown in the following formula:

Among them: θ _j is the steering wheel rotation angle corresponding to frame j, and μ represents the average steering wheel rotation angle value of each frame in m2 frame.

Step VII, Support Vector Machine Classification Algorithm

Combining the zero-speed percentage PNS and angle standard deviation σ obtained in step VI, the support vector machine (SVM) classification algorithm is used to judge whether the driver is in a fatigue state.

The support vector machine classification algorithm first trains the sample set, and finds a hyperplane in the sample space that divides samples of different categories. The model corresponding to the hyperplane is ω ^T x + b = 0, and the model parameter ω is the normal vector and b is the displacement item , T is the transpose, x is the independent variable composed of zero speed percentage and angle standard deviation. The specific steps of the support vector machine classification algorithm are as follows:

Ⅶ-1. Collection of training samples

In this example, 5 drivers with five or more years of driving experience and 5 vehicles in good condition are selected. Each driver randomly selects a vehicle, and discretely collects different driving in straight lines, "S"-shaped routes, and overtaking. 100 samples of the driver in a state of wakefulness and fatigue. The specific sample size is shown in Table 1 below:

Table 1 A list of the number of samples collected under different driving conditions in different states of the driver

(x ₁ , y ₁ ), . . . , (x ₁₀₀ , y ₁₀₀ ). Where x ₁ = (x ₁₁ , x ₁₂ ) ^T , x ₁₁ , x ₁₂ respectively represent the first index zero speed percentage PNS and the second index angle standard deviation σ of the first sample data; y _k1 ∈ Y = { 1, -1}, k1=1,...,100, Y=1, Y=-1 respectively represent the driver is awake and fatigued, the judgment of the driver’s state is to evaluate the driving driver by scoring the driver’s facial video state of fatigue. According to the currently commonly used driver state discrimination method, the state of the driver is divided into two levels: sobriety and fatigue. The facial features of the sober and fatigue states are shown in Table 2 below:

Table 2 List of facial features of drivers' status levels

In this example, the driver’s facial video is divided into multiple sections at intervals of 15s. Five trained experimenters independently rated each section of video according to the currently commonly used driver status characteristics in Table 2 above, and the average value of the scores of the five experimenters was used as the driver The judgment result of the state.

Ⅶ-2. Solving of parameters

In this example, the parameters ω and b of the hyperplane model ω ^T x + b = 0 are calculated, so that the sum of the distances from different types of support vectors to the hyperplane is the largest, namely:

ST y _t (ω ^T x _t +b)≥1, t=1, 2,..., n1

where x _t = (PNS _t , σ _t )y _t ∈ {1, -1}

In this example, the Gaussian kernel function is used, and the dual of the above formula is obtained after transformation, as follows:

in: σ ₁ is the bandwidth of the Gaussian kernel, x _t1 , x _t2 , y _t1 and y _t2 represent the sample parameters;

C is a regularization constant, t1, t2=1, 2,..., n1;

Solve the optimal Lagrangian multiplier α ^* = (α ₁ ^* , α ₂ ^* ,..., α _n1 ^* ) ^T , and calculate the optimal normal vector Select a positive component α _t1 ^* of the optimal Lagrange multiplier α ^* that is smaller than the regularization constant C to calculate the optimal displacement term

VII-3. Construct a hyperplane (ω ^* ×x) + b ^* = 0, and the decision function obtained from it

f(x)＝sgn[(ω ^* ×x)+b ^* ]

Where sgn[(ω ^* ×x)+b ^* ] is a sign function, when ω ^* ×x+b>0 f(x)＝1, when ω ^* ×x+b＝0f(x)＝0, when ω ^* ×x+b<0 f(x)=-1.

According to the 100 training samples of the vehicle driving on the above-mentioned different road sections, the support vector machine classification algorithm is used to learn, and the decision function f(x) for distinguishing whether the driver is awake or fatigued is obtained.

During the actual driving process, the camera collects the steering wheel image in real time, and after processing, the steering wheel angle is obtained, and then the value σ of the current zero speed percentage PNS and the angle deviation is calculated, and the current zero speed percentage and the angle deviation value are substituted into the calculated value The decision function f(x) of f(x), if f(x)=1, it means that the current driver is in a state of fatigue, and the system will give an alarm immediately.

The above-mentioned embodiments are only specific examples for further specifying the purpose, technical solutions and beneficial effects of the present invention, and the present invention is not limited thereto. Any modifications, equivalent replacements, improvements, etc. made within the disclosed scope of the present invention are included in the protection scope of the present invention.

Claims (8)

1. A driver fatigue detection method based on steering wheel image, the main steps are as follows: Step Ⅰ. Get the template image The camera is installed on the top of the cab, and the video signal output line of the camera is connected to the central processor of the detection device; the camera collects the still steering wheel image and stores it as the initial image; After the central processor performs denoising processing on the initial image, it is stored as a template image; Points on the steering wheel circle correspond to points on the image ellipse captured by the camera, and the distance between a point m(x _m , y _m ) on the image ellipse and the corresponding point M(x _M , y _M ) on the steering wheel circle The corresponding relation is x _M ＝x _m 、 In the formula, the radius R is the semi-major axis of the ellipse, and the length of R remains unchanged; Step Ⅱ. Obtain the coordinates o(x ₀ ,y ₀ ) of the center of the steering wheel Ⅱ-1. Steering wheel outline Adopt the findContours () function that the OpenCV open source visual library carries to the template image of step I gained, obtain the coordinates of each point on the outer contour of the steering wheel image by setting corresponding parameters; Ⅱ-2. Position coordinates of the center of the steering wheel From the 5 coordinate points (x _m1 , y _m1 , x _m2 , y _m2 ,..., x _m5 , y _m5 ) on the outer contour of the steering wheel image determined in step Ⅱ-1, determine the ellipse equation and obtain the position of the center of the ellipse Coordinate o(x ₀ ,y ₀ ); Step Ⅲ. Feature point tracking This method uses the optical flow method to match each video sequence with the template image to detect the rotation angle of the current steering wheel image; take representative and easy-to-determined feature points on the steering wheel, and the representative and easy-to-determined feature points are Points that can still be clearly located on the steering wheel image after the steering wheel is turned; through iteration to obtain the coordinate position of the feature point in the steering wheel image in the current frame and the coordinate position in the next frame, according to the coordinates of the feature point in the current frame and the feature The coordinates of the point in the next frame are calculated to obtain the rotation angle of the feature point from the current frame to the next frame, that is, the rotation angle of the steering wheel; Step Ⅳ. Calculate the rotation angle θ of the steering wheel in two adjacent frames The calculation of the following angle θ adopts the position coordinates of the characteristic point A on the steering wheel circle; The center of the steering wheel o(x ₀ , y ₀ ) is determined in step II, and the coordinates of the feature point position A ₀ (x _M01 , y _M01 ) of the current frame are determined in step III. After the steering wheel rotates an angle θ, the feature point A is determined ₀ (x _M01 , y _M01 ) corresponds to the coordinates of the next frame feature point position A ₁ (x _M11 , y _M11 ); The included angle between the center o(x ₀ , y ₀ ) and the line connecting A ₀ (x _M01 , y _M01 ), A ₁ (x _M11 , y _M11 ) is θ ₁ , which is the rotation angle of feature point A, ${\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oA</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oA</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oA</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oA</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; ;</span>$ Step Ⅴ, the rotation angle of the feature point of the steering wheel image Let the coordinates of the center of the steering wheel be o(x ₀ , y ₀ ), the coordinates of the feature points in the kth frame image and the k+1th frame image are D _k (x _Mk , y _Mk ), D _k+1 (x _{M( k+1)} , y _M(k+1) ), the rotation angle of feature points in two adjacent frames of images is less than 90°; When y _Mk ≥ y ₀ , y _M(k+1) ≥ y ₀ : If x _M(k+1) ≥ x _Mk is satisfied, it means that the k+1th frame image rotates an angle clockwise relative to the kth frame image ; If x _M(k+1) ≤ x _Mk is satisfied, it means that the k+1 frame image rotates an angle counterclockwise with respect to the k frame image; When y _Mk ≤y ₀ , y _M(k+1) ≤y ₀ : if x _M(k+1) ≥x _Mk is satisfied, it means that the k+1th frame image rotates an angle counterclockwise relative to the kth frame image ; If x _M(k+1) ≤ x _Mk is satisfied, it means that the k+1th frame image rotates an angle clockwise with respect to the kth frame image; When y _Mk ≥y ₀ , y _M(k+1) ≤y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ is satisfied, it means that the k+1th frame image is relative to the kth frame The image rotates an angle clockwise; if it satisfies x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ , it means that the image of frame k+1 rotates an angle counterclockwise relative to the image of frame k; When y _Mk ≤y ₀ , y _M(k+1) ≥y ₀ : if x _Mk ≥x ₀ , x _M(k+1) ≥x ₀ are satisfied, it means that the k+1th frame image is relative to the kth frame The image rotates an angle counterclockwise; if it satisfies x _Mk ≤ x ₀ , x _M(k+1) ≤ x ₀ , it means that the k+1th frame image is rotated clockwise by an angle relative to the kth frame image; This method stipulates that the rotation angle θ of the steering wheel in the next frame relative to the steering wheel in the current frame is negative, and θ is positive when it is rotated counterclockwise; the rotation angle of each frame of the steering wheel image is accumulated sequentially. When the accumulated value is greater than 360° in a certain period of time, that is The position of the steering wheel relative to the template image in the segment rotates counterclockwise for more than one cycle; when the accumulated value is less than -360°, that is, the position of the steering wheel relative to the template image rotates clockwise for more than one cycle during this period; Step Ⅵ, extract the technical index zero speed percentage and angle standard deviation When the frequency of steering wheel angle changes in a certain period of time t decreases, that is, when the driver’s correction frequency of the steering wheel decreases, the time the steering wheel does not move increases, as the technical index of this method—zero speed percentage PNS, characterized by The extent to which the steering wheel does not move within the selected time period, where the time period t is 50 to 80 seconds; The definition of zero speed percentage PNS is as follows: $\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; PNS</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}& \mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 500</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 501</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; ...</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 500</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1000</span>}<span\; class="notranslate">\; )</span>\end{array}$ The sampling frequency of the present invention is 8 to 12 frames per second, PNS _i represents the zero velocity percentage of the i frame, N _i represents the total number of sampling points of the angular velocity within t seconds before the i frame, and _ni represents the internal angular velocity of the i frame within t seconds Number of points between ±0.1°/s; The definition of the angle standard deviation of another technical indicator is shown in the following formula: $\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$ $\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}& \mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; ...</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 150</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 200</span>}<span\; class="notranslate">\; )</span>\end{array}$ Among them: θ _j is the steering wheel rotation angle corresponding to the jth frame, and μ represents the average steering wheel rotation angle value of each frame in the m2 frame; Step VII, Support Vector Machine Classification Algorithm Combine the zero-speed percentage PNS and angle standard deviation σ obtained in step VI, and use the support vector machine classification algorithm to judge whether the driver is in a fatigue state; The support vector machine classification algorithm first trains the sample set, and finds a hyperplane in the sample space that divides samples of different categories. The model corresponding to the hyperplane is ω ^T x + b = 0, and the model parameter ω is the normal vector and b is the displacement item , T is transpose, x is the independent variable composed of zero speed percentage and angle standard deviation; the decision function obtained by the hyperplane model f(x)＝sgn[(ω ^* ×x)+b ^* ] Where sgn[(ω ^* ×x)+b ^* ] is a sign function, when ω ^* ×x+b>0f(x)=1, when ω ^* ×x+b=0f(x)=0, when ω ^* ×x+b<0f(x)=-1; According to the W training samples of the vehicle running on the above-mentioned different road sections, W is 80-150, learned by the support vector machine classification algorithm, and obtains the decision function f(x) that distinguishes whether the driver is awake or fatigued; During the actual driving process of the driver, the camera collects the steering wheel image in real time, and after processing, the steering wheel angle is obtained, and then the value of the current zero-speed percentage and angle standard deviation is calculated, and the current zero-speed percentage and angle standard difference are substituted into the calculated values Decision function f(x), if f(x)=1, it means that the current driver is in a state of fatigue, and the system will give an alarm immediately. 2. the driver fatigue detection method based on steering wheel image according to claim 1, is characterized in that: Described step I comprises the following steps: Ⅰ-1. Camera installation The camera is installed on the top of the cab, so that the entire steering wheel is in the frame of the camera, and the camera is adjusted to make the image clear and match the size of the frame; transmit the collected image data to the central processor; Ⅰ-2. Acquisition of the initial image of the steering wheel Before the vehicle starts, when the steering wheel is in the initial state, the camera collects the still steering wheel image and stores it as the initial image; Ⅰ-3. Template image The central processor uses a median filter with a kernel function window of 3×3 to denoise the initial image obtained in step I-2, and maintains the edge detail information of the image while removing impulse noise and salt and pepper noise. 3. the driver fatigue detection method based on steering wheel image according to claim 2, is characterized in that: In step I-2, before the camera collects the initial image of the steering wheel, a piece of white paper is placed under the steering wheel, the projection of the steering wheel is located on the white paper, and an initial image with a white background is collected. 4. the driver fatigue detection method based on steering wheel image according to claim 1, is characterized in that: The specific process of the step III feature point tracking is as follows: Ⅲ-1. Processing of the current image The shooting frequency of the camera in this method is 12-16 frames/second, and the sampling frequency of the central processor is 9-12 frames/second lower than the shooting frequency; The central processor performs the same denoising processing as step Ⅰ-3 on the steering wheel image collected by the camera in real time; Ⅲ-2. Feature points of the current frame Use the existing Shi-Tomasi in OpenCV to perform feature point detection on the current image processed in step Ⅲ-1, and use the goodFeaturesToTrack() function of the OpenCV open source vision library to detect and obtain the converted position of the feature points of the current frame A ₀ (x _M01 , y _M01 ); Ⅲ-3. The feature points of the next frame Call the cvCalcOpticalFlowPyrLK() function of the OpenCV open source vision library, input the feature point position A ₀ (x _M01 , y _M01 ) in the current frame image obtained in step III-2, and output the converted feature point in the next frame image Position A ₁ (x _M11 , y _M11 ). 5. the driver fatigue detection method based on steering wheel image according to claim 4, is characterized in that: In the step III, another representative and easy-to-determined feature point B is taken on the steering wheel, and the angle between A, B and the line connecting the center of the circle is greater than 30 degrees; The III-2 simultaneously determines the position B ₀ (x _M02 , y _M02 ) of the feature point B of the current frame; The same method as described in III-3 determines the position B ₁ (x _M12 , y _M12 ) of the feature point position B ₀ (x _M02 , y _M02 ) in the next frame image; The included angle between the center o(x ₀ , y ₀ ) and the line connecting B ₀ (x _M02 , y _M02 ), B ₁ (x _M12 , y _M12 ) is θ ₂ , which is the rotation angle of feature point B, ${\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oB</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oB</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oB</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; oB</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; ;</span>$ The final steering wheel rotation angle θ is the average value of the rotation angles of the two feature points A and B, 6. the driver fatigue detection method based on steering wheel image according to claim 1, is characterized in that: Step VII The specific steps of the support vector machine classification algorithm are as follows: Ⅶ-1. Collection of training samples Select at least 5 drivers with five or more years of driving experience and at least 5 vehicles in good condition. Each driver randomly selects a vehicle, and collects discrete data on different driving situations of straight-line driving, "S"-shaped route driving and overtaking. Next, the samples when the driver is awake and fatigued, (x ₁ , y ₁ ), ..., (x _W , y _W ); where x ₁ = (x ₁₁ , x ₁₂ ) ^T , x ₁₁ , x ₁₂ respectively represent the first index zero-speed percentage PNS and the second index angle standard deviation σ of the first sample data; y _k1 ∈ Y={1,-1}, Y=1, Y=-1 respectively represent the driver In the state of wakefulness and fatigue, W is 80-150. The judgment of the driver’s fatigue state is to evaluate the driver’s fatigue state by scoring the driver’s facial video; A trained experimenter independently scores each section of video according to the currently commonly used driver state characteristics, and the average value of the scores of multiple experimenters is used as the discriminant result of the driver state; Ⅶ-2. Solving of parameters The hyperplane that separates the training samples satisfies that the sum of the distances from the two types of support vectors to the hyperplane is the largest, and the support vector is the sample point that satisfies ω ^T x + b = ± 1, specifically to find the hyperplane model ω ^T x The parameters ω and b of +b=0 make the sum of the distances from different types of support vectors to the hyperplane the largest, which is transformed into: $\begin{array}{cc}\underset{\mathrm{<span\; class="notranslate">\; \&omega;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}& \frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&omega;</span>}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}$ ST y _t (ω ^T x _t +b)≥1, t=1, 2,..., n1 where x _t = (PNS _t , σ _t )y _i ∈ {1, -1} This is a convex quadratic programming problem, and its dual is obtained by using the efficient Lagrange multiplier method. The above model is nonlinearly separable, and a Gaussian kernel function that realizes linear mapping is introduced, and the dual of the above formula is obtained after transformation, namely: $\begin{array}{cc}\underset{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}{\mathrm{<span\; class="notranslate">\; min</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}& \mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; ...</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}$ in: σ ₁ is the bandwidth of the Gaussian kernel, x _t1 , x _t2 , y _t1 and y _t2 represent sample parameters; C is a regularization constant, t1, t2=1, 2,..., n1; Solve the optimal Lagrangian multiplier α ^* = (α ₁ ^* , α ₂ ^* ,..., α _n1 ^* ) ^T , and calculate the optimal normal vector Select a positive component α _t1 ^* of the optimal Lagrange multiplier α ^* that is smaller than the regularization constant C to calculate the optimal displacement term VII-3. Construct a hyperplane (ω ^* ×x) + b ^* = 0, and the decision function obtained from it f(x)=sgn[(ω ^* ×x)+b ^* ]. 7. A kind of driver fatigue detection device based on the steering wheel image according to the driver fatigue detection method design based on the steering wheel image according to any one of claims 1 to 6, characterized in that: Including a central processor, a camera and an alarm module, the camera is installed on the top of the cab above the steering wheel, the entire steering wheel is in the viewfinder frame of the camera, and the video signal output line of the camera is connected to the central processor of the device; the central processing The device is equipped with a storage module and a power supply module, and is also connected to an alarm module, which is an alarm lamp or a buzzer; The central processor is connected to the image processing module through the data transmission module, and the image processing module is connected to the camera, and the steering wheel image collected by the camera is sent to the image processing module of the central processor, and the image processing module processes the image and video data, and is sent to the image processing module through the data transmission module. The storage module stores and extracts the data and sends them to the central processor. The central processor will compare the current steering wheel image with the template image to obtain the current steering wheel rotation angle, obtain the current zero speed percentage and angle standard deviation, and substitute it into the classification according to the support vector machine. The decision function learned by the algorithm judges whether the current driver is in a state of fatigue; when the driver is detected to be in a state of fatigue, the central processor sends an instruction to the alarm module, and its alarm light flashes or the buzzer sounds. 8. The driver fatigue detection device based on steering wheel image according to claim 7, characterized in that: The camera is installed at point N on the top of the cab, and point N is behind the vertical plane passing through the center of the steering wheel and perpendicular to the longitudinal axis of the vehicle body, and the distance from the vertical plane is less than or equal to 20cm. The distance between the center of the steering wheel and the vertical plane parallel to the longitudinal axis of the vehicle body is less than or equal to 15cm.