CN104318237A - Fatigue driving warning method based on face identification - Google Patents

Abstract

A fatigue driving early warning method based on face recognition, involving computer vision. Set up the application environment, use an infrared light source, add a filter to the camera, so that the human eye can produce the red-eye effect; the camera collects the image of the infrared environment, obtains the image frame, and performs differential processing, and the obtained differential image can highlight the pupil with the red-eye effect part; then perform image enhancement on the differenced image to highlight the area of interest, that is, the pupil part of the eye area; perform adaptive thresholding on the enhanced image, and binarize the image to adaptively highlight the pupil and the background image The difference; open the binarized image to eliminate the non-interest area noise or offset stripes that still exist in the image; extract eye features, locate the pupil, and use Kalman filter to reduce the eye area of the next frame Extract the range and accurately perform dynamic tracking and positioning of the human eye; calculate the eye feature parameters through the extracted eye features, and make a fatigue judgment based on the PERCLOS value.

Description

Fatigue driving warning method based on face recognition

technical field

The invention relates to computer vision, in particular to a fatigue driving early warning method based on face recognition.

Background technique

Globally, driver fatigue driving has become one of the important reasons leading to traffic safety accidents. When the driver's vision is blurred, slow to respond, and distracted due to factors such as lack of sleep, drinking, and illness, it will directly affect the driver's perception, thinking, judgment, decision-making, and exercise execution capabilities, especially when frequent For trucks working at any time in 24 hours, the catastrophic accidents caused by fatigue driving can reach 60% (China Traffic Yearbook [M]. China Communications Press. 2010). In the severe traffic accident situation, it is necessary to study the method of preventing driver's fatigue driving. Nowadays, judging the fatigue status of the driver's body through human body information to carry out early warning and provide corresponding protective measures is a research hotspot of experts and scholars at home and abroad, and the research results are also of great significance (Li Duhou, Liu Qun, Yuan Wei etc. Relationship between fatigue driving and traffic accidents [J]. Journal of Traffic and Transportation Engineering, 2010,10(2):104-109). The institutions currently researching fatigue driving detection and the fatigue detection methods under development mainly include:

Based on the physiological signal detection method (Li Zengyong, Jiao Kun, Chen Ming. Correlation analysis between car driver's simulated mental load and heart rate variability [J]. Beijing Biomedical Engineering, 2002, 3:49-51) is a combination of biometrics and Fatigue status is linked, mainly by using various final sensors to obtain various physiological indicators of the driver during driving, including brain waves, electrocardiograms, electromyography, body fluid secretion and other physiological responses.

The detection method based on vehicle behavior (A.Williamson, A.Feyer, R.Friswell.The impact of work practices on fatigue in long distance truck drivers Accident Analysis and Prevention,1996,28(6):709–719) is when the vehicle deviates from When the original driving route, the system will capture the information and send out an alarm signal, which is mainly used in the infrared monitoring of vehicles on expressways.

The detection method based on computer vision (Research on driver fatigue detection method based on eye recognition [D]. Dalian: Dalian Maritime University, 2013) is to capture some subtle actions related to fatigue of the driver through analysis, such as drooping head, dozing, closing Eyes or squints, whether the eyes are on the road, the frequency of yawning and other motion information to judge the driver's fatigue.

Therefore, improving the real-time performance, accuracy, and reliability of vehicle fatigue alarm methods, reducing production costs, and finding non-contact, information fusion fatigue detection methods will all be research directions for future fatigue driving.

Contents of the invention

The purpose of the present invention is to provide a vehicle-based, visual, real-time, fast and accurate detection method for fatigue driving warning based on face recognition.

The present invention comprises the following steps:

1) Set up the application environment, use infrared light source, add a filter to the camera to eliminate the influence of visible light, so that the human eye can produce the red-eye effect;

2) The camera collects the image of the infrared environment, obtains the image frame, and performs differential processing on the image, and the obtained differential image can highlight the pupil part with the red-eye effect; then perform image enhancement operation on the differential image to improve the visual effect of the image , to highlight the area of interest, that is, the pupil part of the eye area;

3) Perform adaptive thresholding on the enhanced image, and binarize the image to adaptively highlight the difference between the pupil and the background image;

4) Perform an open operation on the binarized image to eliminate the noise or offset stripes in the non-interest area still existing in the image;

5) Extract eye features, locate pupils, use Kalman filter to narrow the extraction range of eye area in the next frame, and accurately perform dynamic tracking and positioning of human eyes;

6) Calculate the eye feature parameters through the extracted eye features, and make a fatigue judgment based on the PERCLOS value.

In step 1), the setting application environment can adopt the design of inner and outer infrared apertures.

In step 2), the specific method for the camera to collect infrared images can be: perform differential calculation on the collected images, and use the red-eye effect generated by the human eye on the infrared light source to combine the two images of the bright and dark pupils adjacent to each other. After performing a difference, the obtained difference image removes the background interference, and highlights the part with a large difference in pixel values, that is, the pupil area.

The specific method for removing background interference from the differential image may be: filter the image to filter out noise generated during image acquisition or transmission conversion, and eliminate adverse effects on human eye feature extraction and target segmentation.

In step 3), the adaptive threshold is carried out to the enhanced image, which is to use the Otsu method to automatically select a suitable threshold for segmentation according to the difference image, so that the binarization of the image is adaptive and adaptable to the environment. enhanced.

In step 4), the opening operation on the binarized image is a process of corroding and then expanding the binary image. The opening operation can be used to eliminate small objects, disconnect some narrow connected pixels, and at the same time Being able to smooth the boundary of the pupil region of interest without significantly changing its area affects the computation.

In step 5), the specific method of extracting eye features and locating pupils can be: using the camera to obtain inter-frame images with different degrees of pupil brightness, combined with the geometric constraints of the human eye, the human eye area can be quickly and accurately found , and use Kalman filter to track and locate the pupil, predict the motion state of the next frame image through Kalman filter, narrow the extraction range of eye features, and accurately realize the dynamic tracking of the eye area;

The concrete steps of described Kalman filter tracking and positioning pupil can be:

(1) Initialize the system state.

(2) In the kth frame, the eye pupil is extracted according to the predicted range of the previous frame. If the pupil is extracted, the system state parameters of the Kalman filter are corrected. If not, the next frame search range Return to the whole image and restart the Kalman filtering process with the results of the next search.

(3) Use the Kalman filter to predict the motion state of the next frame image and narrow the range of feature extraction.

(4) Repeat steps (2) and (3).

In step 6), the calculation of the characteristic parameters of the eyes means that after the pupil area is located, it can be used to extract the outline and calculate the characteristic parameters of the eyes. The feature parameters of the eyes are selected from the aspect ratio of the eyes and the area of the eye, and the aspect ratio of the eyes is determined by the length and width of the circumscribed rectangular frame of the contour after the pupil positioning; the area of the eye is determined in addition to the number of statistical pixels. It can also be quickly calculated by multiplying the length and width of the circumscribed rectangular frame. By calculating the area and height-to-width ratio of the circumscribed rectangle of the outline, if the area is less than a certain threshold, it is considered closed eyes, otherwise it is regarded as open eyes; Judging the degree of fatigue is calculated by calculating the ratio of the closed state of the eyes to all states within a period of time. The longer the driver's eyes are closed, the greater the calculated PERCLOS value, which reflects the greater the driver's fatigue. Through Measure the ratio of the eyes closed state to judge the driver's fatigue degree, and obtain a numerical result for the driver's fatigue state. Once the driver is detected to be in a fatigue driving state, corresponding measures will be taken to issue an alarm.

The invention makes a scientific judgment on the driver's fatigue degree by extracting the features of the eye pupil and combining the fatigue judgment criterion. The beneficial effects of the present invention are as follows:

The present invention adopts the method of machine vision to judge the driver's fatigue state and has many advantages, and is one of the most mainstream technologies at present. The present invention unifies the day and night fatigue detection, adopts the same feature extraction method, is not sensitive to environmental changes, has better adaptability, and has high accuracy. Compared with the feature extraction of pattern recognition, this method is aimed at the system Using the environment to design, simple and efficient, and high accuracy, improved the shortcomings of pattern recognition, and laid the foundation for the large-scale promotion of the system.

Description of drawings

Fig. 1 is a functional block diagram of an embodiment of the present invention;

Fig. 2 is a structural diagram of an infrared lamp source according to an embodiment of the present invention;

Fig. 3 is a structural block diagram of the human eye tracking and positioning algorithm of an embodiment of the present invention;

Fig. 4 is a block diagram of fatigue discrimination in an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and technical effect of the present invention clearer, the present invention will be further described below in conjunction with specific embodiments and with reference to the accompanying drawings.

The invention is a fatigue driving early warning method based on face recognition.

As shown in Figure 1, the system extracts and locates eye features. Human eye positioning is a key step in judging whether the driver is fatigued. Whether the driver's eyes can be accurately positioned will directly affect the accuracy of fatigue detection. The 850nm infrared light source can be used to artificially generate the red-eye effect, so the camera can acquire two frames of images with different pupil brightness. The dark pupil picture is generated by the outer ring light, that is, the general illumination picture. Then the difference operation is performed on the two frames of images of the adjacent bright and dark pupils. This highlights areas where pixel values differ, especially the pupil area, and eliminates unwanted background noise. From the difference images, it can be observed that the image brightness of the pupil area is brighter than other areas, and the difference between images is obvious. The image enhancement operation is performed on the differenced image, and the enhanced image needs to be binarized through a threshold to generate a binary image for easy positioning. To perform threshold processing on the image, it is necessary to select a suitable threshold V to effectively highlight the eye area. The invention adopts the Otsu method to divide the image into two parts, the background and the target, according to the grayscale characteristics of the image, and calculates the threshold value to carry out image binarization, so that the variance between the classes of the foreground and background images is the largest. In this way, an appropriate threshold can be automatically selected for segmentation according to the difference image. Perform an open operation on the binary image to eliminate some non-interest area noise or offset stripes in the image. The Kalman filter is used to track and locate the pupil to realize the dynamic positioning of the eye area. Calculate the ratio of eye closure state to all states within a period of time, and use the PERCLOS criterion to determine the driver's fatigue degree.

As shown in Figure 2, the present invention uses an infrared light source to set the driver's application environment. The closer the infrared light source is to the optical axis of the camera lens, the easier it is to directly project the light reflected from the eyes into the sensor of the camera. When the camera is used to collect images, the infrared light source is irradiated on the driver's face, an infrared filter is attached to the lens to eliminate the influence of visible light, and the image of the driver's eye area is collected in real time through the camera.

As shown in Figure 3, after the camera acquires the image with the red-eye effect, it performs differential processing on the bright and dark pupil images to obtain an image that highlights the pupil. The binary image is obtained after using the Otsu method as an adaptive threshold, and the system uses the geometric characteristics of the eye as the relevant constraint conditions for the positioning method of the human eye to find the human eye area. The eye region is filtered by the following four constraints:

1. The eyes are in the upper half of the image, and the hair and forehead also occupy part of the space, that is, the line where the eyes are located should satisfy L/15<r<L/2. Among them, L is the total number of lines of the image, that is, the height of the image;

2. The aspect ratio or aspect ratio of the area determined by the shape of the pupil should be around 1.

3. The area occupied by the eye area should be greater than a certain threshold to eliminate the interference of noisy spots.

4. The area occupied by the eye area should be smaller than a certain threshold, so as to exclude the interference of large white spots caused by improper threshold value.

After the eye area is selected, the Kalman filter is used to track the pupil to realize the dynamic tracking and positioning of the eye area.

As shown in Figure 4, after locating the position of the pupil, it can be used to extract the contour and calculate the parameters of the eye features. Parameters such as eye aspect ratio and area are relatively simple, feasible and effective ways to analyze eye state recognition. The aspect ratio of the eye is determined by the length and width of the circumscribed rectangle of the pupil-located contour. In addition to counting the number of pixels, the area of the eye can also be quickly calculated by multiplying the length and width of the circumscribed rectangular frame.

The following method is used to determine whether the eyes are open or closed: the eyes are located by the difference image. In the case of closed eyes, the red eye effect cannot be produced. The ideal state after image processing is that the area of the pupil area is almost 0. The wider the eye is opened, the more pronounced the red-eye effect is, and the larger the area of the pupil area obtained. Therefore, by calculating the area and aspect ratio of the rectangle bounding the outline. If the area is smaller than a certain threshold, it is considered as closed eyes, otherwise it is considered as open eyes.

Computes the ratio of eye-closed states to all states over a period of time. The longer the driver's eyes are closed, the greater the calculated PERCLOS value, which reflects the more serious the driver's fatigue. Therefore, the driver's fatigue degree can be judged by measuring the ratio of eye closure state.

Claims (10)

1., based on the fatigue driving method for early warning of recognition of face, it is characterized in that comprising the following steps:

1) applied environment is set, adopts infrared light supply, add optical filter at camera, eliminate the impact of visible ray, enable human eye produce red-eye effect;

2) image of camera collection infrared environmental, obtain picture frame, and carry out difference processing to image, the difference image obtained can highlight the pupil portion with red-eye effect; Then differentiated image is carried out image enhancement operation, improve the visual effect of image, outstanding interested region, i.e. ocular pupil portion;

3) carry out adaptive threshold to the image after strengthening, by image binaryzation, being used for self-adaptation highlights the difference of pupil and background image;

4) opening operation is done to the image after binaryzation, the regions of non-interest noise that elimination picture still exists or skew striped;

5) extract eye feature, location pupil, adopts Kalman filtering, reduces next frame ocular and extracts scope, accurately carry out human eye and carry out dynamic track and localization;

6) eye feature by extracting, calculates eye feature parameter, makes degree of fatigue adjudicate by the size of PERCLOS value.

2., as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 1) in, the described design that the inside and outside infrared aperture of applied environment employing is set.

3. as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 2) in, the concrete grammar of described camera collection infrared image is: carry out calculus of differences to the image after gathering, by the red-eye effect that human eye produces infrared light supply, two width images of bright dark pupil adjacent for front and back are carried out difference, the difference image removing background interference obtained, highlights the part that value differences is larger, i.e. pupil region.

4. as claimed in claim 3 based on the fatigue driving method for early warning of recognition of face, it is characterized in that the concrete grammar of described difference image removing background interference is: filtering process is done to image, for the noise that filtering produces in image acquisition or transmission transfer process, eliminate the adverse effect to human eye feature extraction and Target Segmentation.

5. as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 3) in, described to strengthen after image carry out adaptive threshold, adopt Da-Jin algorithm automatically to select suitable threshold value to split according to difference image, make the binaryzation of image have adaptivity, the adaptability of environment is strengthened.

6., as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 4) in, describedly doing opening operation to the image after binaryzation, is the process of bianry image first being corroded to rear expansion.

7. as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 5) in, described extraction eye feature, the concrete grammar of location pupil is: obtain the different inter frame image of the bright dark degree of pupil by camera, geometry constraint conditions in conjunction with human eye can find human eye area fast and accurately, and adopt Kalman filter tracking to locate pupil, by the motion state of Kalman prediction next frame image, reduce the extraction scope of eye feature, accurately realize the dynamic tracking of ocular.

8., as claimed in claim 7 based on the fatigue driving method for early warning of recognition of face, it is characterized in that the concrete steps of described Kalman filter tracking location pupil are:

(1) initialization system state;

(2) in kth frame, the extraction of eye pupil is carried out according in the scope of previous frame prediction, if extract pupil, then revise the system status parameters of Kalman filter, if do not extract, then next frame hunting zone revert to picture in its entirety, and the result of next time searching for is restarted Kalman filtering process;

(3) use the motion state of Kalman filter prediction next frame image, reduce the extraction scope of feature;

(4) action of step (2) (3) is repeated.

9., as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 6) in, described calculating eye feature parameter is after navigating to pupil region, utilizes it to extract the characteristic parameter that profile calculates eyes; The characteristic parameter of described eyes selects the depth-width ratio of eyes and the area of eye, and the length and width of the boundary rectangle frame of the profile after the depth-width ratio employing Pupil diameter of eyes are determined; The area of eye also can carry out the long-pending of length and width by boundary rectangle frame and calculate fast except statistical pixel point number, by calculating area and the depth-width ratio of profile boundary rectangle, if its area is less than certain threshold value, then think to close one's eyes, otherwise be considered as opening eyes.

10. as claimed in claim 1 based on the fatigue driving method for early warning of recognition of face, it is characterized in that in step 6) in, described degree of fatigue judgement accounts for the ratio of all states by calculating eyes closed state in a period of time, the time that driver's eyes is closed is longer, the PERCLOS value calculated is larger, reflect that the degree of fatigue of driver also will be larger, the degree of fatigue of driver is judged by the ratio measuring eyes closed state, a result quantized is drawn to the fatigue state of driver, once detect that driver is in fatigue driving state, give the alarm with regard to taking corresponding measure.