CN106530623B

CN106530623B - A kind of fatigue driving detection device and detection method

Info

Publication number: CN106530623B
Application number: CN201611264042.1A
Authority: CN
Inventors: 曹兵; 李鹏; 王许生
Original assignee: Shenzhen Hongchang Up Power Technology Co Ltd; Nanjing University of Science and Technology
Current assignee: Shenzhen Hongchang Up Power Technology Co Ltd; Nanjing University of Science and Technology
Priority date: 2016-12-30
Filing date: 2016-12-30
Publication date: 2019-06-07
Anticipated expiration: 2036-12-30
Also published as: CN106530623A

Abstract

The invention discloses a kind of fatigue driving detection device and detection methods, including arm processor equipment, SD card, USB camera, data line, radiator fan and warning device, arm processor equipment further includes Face detection module, human eye state identification module, tired determination module；Detection method includes the following steps for fatigue driving detection: 1. initialization cameras；2. acquiring image, image information is conveyed to arm processor equipment；3. image preprocessing；4. judging whether driver is in a state of fatigue by the face that human eye feature classifier positions the detection driver of human eye area 5. by face characteristic classifier locating human face region；6. the eye of detection driver judges whether driver is in a state of fatigue；7. comprehensive descision driver fatigue state starts corresponding alarm.The ocular image binaryzation processing method that the present invention uses can preferably be partitioned into pupil and fringe region, and than the detection of single method, recognition accuracy is higher.

Description

A kind of fatigue driving detection device and detection method

Technical field

The invention belongs to vehicle security drive technical fields, belong to the technologies such as image procossing, pattern-recognition, neural network neck Domain, especially a kind of fatigue driving detecting system.

Background technique

With the rapid development of social economy, the quantity of automobile is also more and more next more while transportation develops, More and more multiple trend is presented in the traffic accident as caused by fatigue driving.For this phenomenon, produce various tired Please detection technique is sailed, contact measurement and non-contact detection are broadly divided into.Contact measurement is usually to measure driver's Electrocardiogram, electroencephalogram etc., this measurement method can not only interfere the driver behavior of driver, but also at high cost.When driver is tired Lao Shi can show to bow, eye closing frequency increases etc., and physiological characteristics, non-contact detection technology are exactly to be detected by monitoring device These physiological characteristics of driver.Contactless fatigue-driving detection technology has the characteristics that at low cost, accuracy is high, therefore, It is widely adopted in current fatigue driving detection device.

The currently existing contactless fatigue detection device based on physiological driver's feature passes through image procossing skill mostly Art, locating human face, then in the range of face analyze eyes state, judge whether fatigue.Chinese invention patent CN101593425A and CN201681470U discloses a kind of method for detecting fatigue driving, and is by single detection people The state of eye judges the fatigue state of people, although this single detection mode can have in fatigue detecting it is certain accurate Property, but the influence for being easy to be illuminated by the light, whether wear glasses etc. factors causes accidentally to survey.Wherein patent of invention CN101593425A is used Maximum variance between clusters, be easy by the eyelash shadow ring, be unfavorable for iris segmentation.

Summary of the invention

The purpose of the present invention is to provide a kind of fatigue driving detection device and detection methods, by face state and people It is corresponding to can be realized different alarms as a result, the fatigue state to driver carries out final judgement for the comprehensive descision of eye shape state Mode, and ocular image binaryzation processing adaptive approach used can preferably be partitioned into iris area in the present invention Domain.

The technical solution for realizing the aim of the invention is as follows:

A kind of fatigue driving detection device, including for storing embedded system SD card, for acquire driver front The USB camera of image, the warning device for prompting driver, is used for processor heat dissipation at the data line for equipment power supply Radiator fan；Further include: the report for the arm processor equipment type of alarm different with progress that image procossing and fatigue determine Alarm device；The warning device includes that can carry out different alarm sides according to the Different Results of comprehensive descision driver fatigue state The loudspeaker warning device and LED lamp alarm device of formula alarm；

The arm processor equipment includes Face detection module, human eye state identification module, tired determination module；Face After locating module receives pictorial information, face state is identified with Adaboost algorithm locating human face and by coordinate difference；Human eye State recognition module obtains the human face region image in Face detection module, positions human eye area with Adaboost algorithm, passes through Human eye state is identified using the integral projection of eye self-adaption binaryzation processing image；Tired determination module according to face state with Human eye state carries out tired judgement, and judging result is then converted to electric signal, is conveyed to warning device by I/O interface.

The Face detection module, human eye state identification module, tired determination module are specific as follows:

Face detection module: with trained face characteristic classifier locating human face region, and face center position is calculated Set and image center be in longitudinal deviation value, according to the vibration frequency of face center in the time of deviation oscillation and setting come Judge whether driver is in a state of fatigue；

Human eye state identification module: behind locating human face region, according to the face distribution proportion of people, by 3/5, face top It is allocated as interested region, positions human eye area with trained human eye classifier；Feature is carried out to the eye of driver to mention It takes, calculates separately the maximum value of the vertically and horizontally integral projection of the image of ocular binary conversion treatment and the width of integral domain Ratio is spent, in conjunction with two ratios, comprehensive judgement goes out the current state of human eye, i.e. human eye closure situation, and sentences by what is set It fixes, it is whether tired to carry out the current state of mind of driver；

Tired determination module: according to the judging result of face state and human eye state, the state of mind of driver is carried out Final judgement；

A kind of method for detecting fatigue driving, includes the following steps:

Step 1, camera is initialized, setting camera reads in the attribute value of picture；

Step 2, USB camera acquires image, and image information is conveyed to arm processor equipment；

Step 3, image preprocessing, i.e. image down, gray processing processing；

Step 4, existing face and human eye feature classifier in OpenCV machine vision library are loaded, training in advance is passed through Face characteristic classifier locating human face region；

Step 5, pass through face state determine fatigue state: using Adaboost algorithm detect driver face there are positions It sets, calculates face center position and image center in longitudinal deviation value, according to deviation value compared with given threshold and one It fixes time the vibration frequency of interior face center, that is, frequency of nodding judges whether driver is in a state of fatigue；

Step 6, pass through human eye state determine fatigue state: using Adaboost algorithm detect driver eye there are positions It sets, feature extraction is carried out to the eye of driver, calculates the maximum value of the integral projection of ocular and the width of integral domain Ratio, and judge whether driver is in a state of fatigue compared with given threshold T1；

Step 7, Image Acquisition or the different alarm sides of starting are returned to as a result, determining according to the tired comprehensive judgement of step 5,6 One kind of formula.

Compared with prior art, the present invention its remarkable advantage:

(1) present invention uses embedded system, small in size, easy to use；(2) reduce individual sex differernce to testing result Influence, improve fatigue judgement accuracy, have preferable practicability；(3) by combine people head and human eye two Notable feature carries out compound judgement, carries out recognition detection than single method, recognition accuracy is higher；(4) ocular image two Value processing adaptive approach used can preferably be partitioned into pupil and fringe region, not vulnerable to the influence of eyelash；(5) Differentiated by the state of mind to driver, is stopped not when starting warning device prompting driver is under fatigue state Breath, can effectively reduce traffic accident, provide a strong guarantee for the security of the lives and property of the people.

Present invention is further described in detail with reference to the accompanying drawing.

Detailed description of the invention

Fig. 1 is fatigue driving detection device structural schematic diagram.

Fig. 2 is connection schematic diagram inside arm processor equipment.

Fig. 3 is method for detecting fatigue driving flow diagram.

Fig. 4 is the vertical integral projection figure of the image of eye binary conversion treatment.

Fig. 5 is the horizontal integral projection figure of the image of eye binary conversion treatment.

Fig. 6 is PERCLOS measuring principle figure.

Fig. 7 is eyes image.

Fig. 8 is the image of the self-adaption binaryzation processing of eyes image.

Specific embodiment

In conjunction with Fig. 1, a kind of fatigue driving detection device, including for storing embedded system SD card, for acquire drive The USB camera of the person's of sailing direct picture, for equipment power supply data line, the warning device for prompting driver, for locating Manage the radiator fan of device heat dissipation；Further include: the arm processor equipment alarm different with progress determined for image procossing and fatigue The warning device of mode；The warning device includes that can be carried out not according to the Different Results of comprehensive descision driver fatigue state With the loudspeaker warning device and LED lamp alarm device of type of alarm alarm；

The arm processor equipment as shown in Figure 2 includes: power-switching circuit, usb circuit, crystal oscillating circuit, enables Signal circuit, SD card reading circuit；

Wherein, power supply interface conversion is realized in power-switching circuit part, first USB power supply circuit, passes through three poles of NXP type Pipe obtains 5V voltage, and centre one self-recovery fuse of concatenation makes device have the function of voltage protection, then USB power supply electricity The output on road connects voltage regulator circuit, and voltage regulator circuit is converted to three kinds of different burning voltages by voltage stabilizing chip；The work of ARM chip Determined with comprising image procossing and fatigue；Usb circuit is connected with LAN9512 chip, carries out data transmission, and receives USB and takes the photograph As the video information of head is incoming, information is passed in ARM chip processor by Video Decoder；Crystal oscillating circuit passes through crystal Oscillator generates clock signal, provides timing for processor；Enable signal circuit passes through 5V input voltage and field effect transistor Generate two kinds of signals, for provide SDRAM enable signal and ARM chip needed for RUN signal, SDRAM provides fortune for system Row space, while storing the compression image information of the 3-5 minute acquired recently；LED state indicator light is shown for equipment state； Linux system in SD card is read into equipment by SD card reading circuit, so that equipment is had running environment, and realize the reading of data Enter and stores；HDMI is that screen shows spare interface；The trigger signal of I/O interface offer warning device.

Video stream information is converted into digital information by conversion circuit and device by USB camera, and ARM chip is as main place Unit is managed, Face detection module is started, carries out Face detection and state analysis, treated, and image information is temporarily stored in In SDRAM, eye recognition module obtains face recognition module from SDRAM treated image information, carries out human eye state knowledge Not, then tired determination module according to Face detection module and the state outcome of human eye state identification module carries out tired judgement, It will finally determine that result is converted into electric signal and sends warning device to from I/O interface.

In conjunction with Fig. 3-5, the detection method of the fatigue driving detection device is comprised the following steps:

Step 1, camera is initialized, setting camera reads in the attribute value of picture, i.e., sets the image resolution ratio of reading It is set to 640 × 480；

Step 2, USB camera acquires image, and image information is conveyed to ARM embeded processor；

Step 3, image preprocessing, i.e. image down, gray processing processing；

As a kind of preferred scheme, image downscaling method specifically: by image down 1/2, the method for use is part Averaging method preferably retains original image information while reducing picture size, and the image after diminution can reduce operand, Improve real-time.

Step 4, existing face and human eye feature classifier in OpenCV machine vision library are loaded, training in advance is passed through Face characteristic classifier locating human face region obtains face key feature points with Harr feature detection mode, according to feature Point location human face region, and respectively obtain human face region upper left angle point and bottom right angular coordinate (x₁,y₁)、(x₂,y₂), calculate people Center point coordinate (the H of face area image_x,H_y), whereinAccording to (x₁,y₁)、(x₂,y₂) Two o'clock coordinate outlines human face region with rectangle frame；

Step 5, fatigue state is determined by face state: detects the face position of driver using Adaboost algorithm It sets, Image Acquisition is returned if face is not detected, if after collecting face, calculating human face region image center position (H_x, H_y) the image center position (I that arrives with initial acquisition_x,I_y) in longitudinal deviation value Diff, in unit period (10 seconds), system Count amount of images and total number of images amount that deviation value Diff is greater than given threshold T1 (initial to read in the 1/4 of picture altitude value), meter The vibration frequency of face center is calculated, that is, frequency of nodding judges whether driver is in a state of fatigue；

By face state determine fatigue state comprising the following steps:

5.1, after the center position for obtaining human face region, calculate the ordinate H of face center_yWith the ordinate of image I_yDifference Diff=| H_y-I_y|；

5.2, statistical unit period (10 seconds) interior difference Diff are greater than the amount of images n and image totalframes of given threshold T1 N；

5.3, the ratio of amount of images n and image totalframes N are calculated, ratio is compared with given threshold T, if ratio Threshold value T dry greatly, then judge that driver is in a state of fatigue；Threshold value T is sized to 0.68, by experiment gained.

Step 6, pass through human eye state determine fatigue state: using Adaboost algorithm detect driver eye there are positions It sets, Image Acquisition is returned if human eye is not detected, if after collecting human eye, carrying out feature extraction, meter to the eye of driver Calculate the maximum value of the integral projection of the image of ocular binary conversion treatment and the width ratio of integral domain, and and given threshold T2 compares, and if more than threshold value T2, then judges that the eyes of driver are closures, unit of account period (10 seconds) driver, which closes one's eyes, to scheme As the ratio of quantity and total amount of images judges that driver is in a state of fatigue if ratio is greater than given threshold T；Threshold value T2 Value is according to obtained by experiment, and value is 2.0 in the present invention.

Fatigue state is determined by human eye state: specific comprising step in detail below:

6.1 position human eye by human eye feature classifier in human face region；

Human eye area, position fixing process and people are positioned by human eye feature classifier trained in advance in human face region image Face zone location is identical, and the feature classifiers only loaded are different；Calculate the center point coordinate of human eye area image, calculating process With calculate as human face region image center coordinate, judge human eye central point to locating human face region rectangle frame lower boundary Whether the ratio of distance and rectangle frame height meets normal organ distribution proportion, if judging, error is larger, may detection To eyebrow, detection is re-started.

6.2 pairs of ocular images carry out gaussian filtering, removal noise, using adaptive binary conversion treatment；

For different threshold values, the image that binary conversion treatment obtains is different, and the present invention is carried out using adaptive threshold Image binaryzation processing, can preferably be partitioned into pupil region, i.e. setting minimum threshold low and max-thresholds high, image Threshold tau used in binary conversion treatment is 0.05 to be gradually incremented by as step-length using numerical values recited, to same piece image constantly two-value again Change processing, obtains the image of a series of binary conversion treatment, therefrom finds out the smallest image of non-zero connected domain quantity as to be checked Altimetric image.The processing of self-adaption binaryzation used by step 6.2, specific method are as follows:

6.2.1. minimum threshold low=0.1 (0.1 times of image minimum gradation value) is set, max-thresholds high=0.5 (0.5 times of image maximum gradation value), threshold tau incremental steps step=0.05；

6.2.2. by imgs=binary (I, τ) constantly to the processing of ocular image binaryzation；

6.2.3. after the processing for finding non-zero connected region minimum number in a series of image imgs of binary conversion treatments Image img1 and corresponding threshold tau；

6.2.4. opening operation is carried out to the image img1 of the binary conversion treatment of non-zero connected region minimum number and inner hole is filled out It fills；

6.2.5 the largest connected domain in the image img1 of binary conversion treatment is looked for, and removes other connected domains, is obtained final The image img of binary conversion treatment；

Wherein, imgs indicates that the image of binary conversion treatment, bianry indicate binary conversion treatment process, and I indicates ocular Image, τ are the binarization thresholds being gradually incremented by by step 6.2.1 from minimum threshold low；Imgs=binary (I, τ) is this hair The representation of bright image binaryzation processing.

6.3 calculate the integral projection of the image img of binary conversion treatment, and maximum value and the integral for calculating vertical integral projection are wide The ratio r ate2 of the ratio r ate1 of degree and the maximum value of horizontal integral projection and integral breadth；

Wherein, the formula that the calculating of vertical integral projection uses:

The formula that the calculating of horizontal integral projection uses:

As shown in Fig. 3, from S_v(x) the maximum value V of vertical integral projection is found out in_max, from S_h(x) horizontal integral is found out in The maximum value H of projection_max, calculate separately out the peak width V of vertical integral projection and horizontal integral projection_width、H_width, then It calculates:

Rate1=V_max/V_width；

Rate2=H_max/H_width；

Wherein, S_v(x) be image img after binary conversion treatment unit width corresponding to the sum of pixel value, S_h(x) it is The sum of pixel value corresponding to the unit height of the image img of binary conversion treatment, Y₁=1, Y₂Equal to the height of ocular image Degree, X₁=1, X₂Equal to the width of ocular image, I (x, y) is that (x, y) is corresponding in ocular binary conversion treatment image Pixel value, (x, y) are binary conversion treatment image coordinate value, and rate1, rate2 respectively indicate vertical integral projection maximum value and hang down The ratio of the ratio of direct integral view field width, horizontal integral projection maximum value and horizontal integral projection peak width；

6.4 combine rate1 and rate2, calculateJudge whether driver is in tired shape State；By the value of rate compared with given threshold T2, determine that the opening and closing state of human eye, the even value of rate are greater than T2, indicate When the closure degree of human eye reaches 80%, it is regarded as current eye closing completely；The size of threshold value T2 is by normal to driver The acquisition of eyes image data when driving acquires image when different driver's normal drivings in 10 minutes, according to above-mentioned step Suddenly a series of rate value is calculated, and obtains a final value T2 by way of averaging.

Wherein, rate indicates vertical and horizontal integral projection depth-width ratio example relationship integrated value,Be rate1 and The weight coefficient of rate2,WithValue take 0.4 and 0.6 respectively in the present invention, value size is by experiment gained；It is comprehensive Consider upright projection and floor projection, first is that prevent one direction integral projection parameter from determining that there is one-sidedness for fatigue, second is that It is more conducive to distinguishing eyes being to be closed or open.

6.5 carry out tired judgement using the P80 method of PERCLOS, in the unit period of statistics setting (10 seconds), eye Eyeball closing time accounts for the percentage of overall time, if ratio has been more than preset threshold value T, that is, has been regarded as current drivers In fatigue driving；

It is described further in conjunction with P80 method of the Fig. 6 to PERCLOS:

There are three types of standards in the application by PERCLOS: P70, P80 and EM, respectively indicating eyes closed degree is 70%, 80% With 50%.Experiments have shown that P80 standard effect is best, therefore, the present invention judges degree of fatigue using the criterion of P80.t1 At the time of initial time when for people's emmetropia state, i.e. human eye opening degree are 80%；T2 is human eye in closing course, people At the time of eye opening degree is 20%；T3 is human eye after being closed completely again in opening process, and human eye opening degree reaches 20% At the time of；T4 is that human eye completes primary blink process, at the time of when being restored to normal open configuration；

After acquirement t1, t2, t3, t4, the value f of PERCLOS is calculated:

F is the percentage for the eyes closed time accounting for set period of time；

Statistics is in unit period (10 seconds), the number of image frames and image totalframes of eyes closed, the value of the PERCLOS F is equal to:

In the present invention, if the value of f is greater than T, determine that driver is in a state of fatigue.

Step 7, according to step 5,6 tired comprehensive judgement as a result, if it is determined that fatigue then return to Image Acquisition, if it is determined that It is tired then start warning device.

By step 5 and step 6 comprehensive judgement as a result, starting corresponding type of alarm:

When step 5, judgement driver is in a state of fatigue, then starts the alarm of LED lamp alarm device；

When step 6, judgement driver is in a state of fatigue, then starts the alarm of loudspeaker warning device；

When step 5, step 6 determine that driver is in a state of fatigue, then start LED lamp alarm device and loudspeaker report simultaneously Alarm device alarm.

Fig. 7,8 are the effect picture of the image of ocular image and self-adaption binaryzation processing, binary conversion treatment side respectively Method is to find optimal threshold from a series of threshold values, can make the quantity of the non-zero connected domain of the image of eye binary conversion treatment At least, inner hole filling is then carried out, finds largest connected domain, which is exactly the part to be partitioned into.Human eye pupil Bore region, which can be preferably divided, to be come out, and is not influenced by eyelash, the figure of finally obtained binary conversion treatment It is good as playing the role of for subsequent calculating integral projection.

Fatigue driving detection device of the invention, design is simple, small in size, and installation is easy on automobile.In use, Image collecting device is preferably mounted on windshield, the left anterior-superior part of position of driver or the roof of front upper right On, instrument panel center can also be placed the image acquisition device in, the positive image of driver can be captured through steering wheel.Image It is warning device power supply with vehicle-mounted USB charger before processing and warning device can be fixed on windshield.

Claims

1. A fatigue driving detection device, comprising an SD card for storing an embedded system, a USB camera for collecting the front image of the driver, a data cable for powering the device, an alarm device for prompting the driver, A cooling fan for dissipating heat from a processor; it is characterized in that it also includes: an ARM processor device for image processing and fatigue determination, and an alarm device for performing different alarm modes; As a result, the horn alarm device and the LED light alarm device for alarming in different alarm modes;

The ARM processor device includes a face positioning module, a human eye state identification module, and a fatigue determination module; after the face positioning module receives the picture information, the Adaboost algorithm is used to locate the face and identify the face state through the coordinate difference; the human eye state The recognition module obtains the image of the face area in the face localization module, uses the Adaboost algorithm to locate the human eye area, and recognizes the human eye state by using the integral projection of the eye adaptive binarization processing image; the fatigue determination module is based on the face state and human The eye state is fatigued, and then the judgment result is converted into an electrical signal, which is sent to the alarm device through the I/O interface;

The face location module, the human eye state recognition module, and the fatigue determination module are as follows:

Face localization module: Use the trained face feature classifier to locate the face area, and calculate the vertical deviation between the position of the face center point and the image center point, according to the deviation range and the set time. Fluctuation frequency to judge whether the driver is in a state of fatigue;

Human eye state recognition module: After locating the face area, according to the distribution ratio of human facial features, the 3/5 part of the human face is regarded as the area of interest, and the trained human eye classifier is used to locate the human eye area; Perform feature extraction on the eyes, calculate the ratio of the maximum vertical and horizontal integral projections of the binarized image of the eye area to the width of the integral area, and combine the two ratios to comprehensively determine the current state of the human eye, that is, the human eye. Eye closure, and determine whether the driver's current mental state is fatigued through the set judgment criteria;

The human eye state recognition module realizes to judge whether the driver is in a fatigue state through the following steps:

4.1 Use the Adaboost algorithm to detect the presence of the driver's eyes, and locate the human eyes in the face area through the human eye feature classifier;

4.2 Perform Gaussian filtering on the eye area image, remove noise, and use adaptive binarization processing;

4.3 Calculate the integral projection of the binarized image img of the eye, calculate the ratio rate1 of the maximum value of the vertical integral projection to the integral width and the ratio rate2 of the maximum value of the horizontal integral projection to the integral width;

Among them, the formula used for the calculation of the vertical integral projection:

The formula used for the calculation of the horizontal integral projection:

Find the maximum values V _max and H _max of the vertical integral projection and the horizontal integral projection respectively, and calculate the area widths V _width and H _width of the vertical integral projection and the horizontal integral projection respectively, and then calculate:

rate1=V _max /V _width ;

rate2=H _max /H _width ;

Among them, S _v (x) is the sum of the pixel values corresponding to the unit width of the binarized image img, and _Sh (x) is the sum of the pixel values corresponding to the unit height of the binarized image img , Y ₁ =1, Y ₂ is equal to the height of the eye area image, X ₁ =1, X ₂ is equal to the width of the eye area image, I(x, y) represents (x, y) in the eye area binarized image y) the corresponding pixel value, (x, y) is the coordinate value of the binarized image, rate1, rate2 respectively represent the proportional relationship between the height and width of the integral projection;

4.4 Combine rate1 and rate2 to calculate Determine whether the driver is in a state of fatigue; compare the value of rate with the set threshold T2 to determine the open and closed state of the human eye, that is, if the value of rate is greater than the set threshold T2, it means that the closed degree of the human eye reaches 80%. It is considered that the eyes have been completely closed at present; where rate represents the comprehensive value of the vertical and horizontal integral projection height-width ratio, is the weight coefficient of rate1 and rate2;

4.5 The P80 method of PERCLOS is used to determine the fatigue. The percentage of the eye closure time in the total time in the set unit period is calculated. If the ratio exceeds the preset threshold T, it is considered that the current driver is already in fatigue driving; Calculate the value f of PERCLOS:

In unit time, count the number of image frames with eyes closed and the total number of image frames, the value f of PERCLOS is equivalent to:

f is the percentage of eye closure time in the set time period; if the value of f is greater than the set threshold T, it is determined that the driver is in a fatigued state;

Fatigue judgment module: According to the judgment results of the face state and the human eye state, the final judgment of the driver's mental state is carried out.

2. a kind of fatigue driving detection device as claimed in claim 1, is characterized in that, described ARM processor equipment comprises: power conversion circuit, USB interface circuit, crystal oscillator circuit, enable signal circuit, SD card reading circuit;

Power conversion circuit, first of all, the USB power supply circuit realizes the power supply interface conversion, obtains 5V voltage through the NXP type triode, and connects a self-recovery fuse in series, and then the output of the USB power supply circuit is connected to the voltage regulator circuit, which is converted into a voltage regulator chip. Three different stable voltages; the role of the ARM chip includes image processing and fatigue determination; the USB interface circuit is connected to the LAN9512 chip for data transmission, receiving video information from the USB camera, and passing the information to the ARM chip through the video decoder In the processor; the crystal oscillator circuit generates the clock signal through the crystal oscillator to provide timing for the processor; the enable signal circuit generates two signals through the 5V input voltage and the field effect transistor, which are used to provide the enable signal of the SDRAM and the requirements of the ARM chip RUN signal, SDRAM provides running space for the system, and stores the compressed image information collected recently for 3-5 minutes; LED status indicator is used for device status display; SD card reading circuit reads the Linux system in SD card into In the equipment, the equipment has an operating environment, and realizes the reading and storage of data; HDMI is the reserved interface for screen display; I/O interface provides the trigger signal of the alarm device;

The USB camera converts the video stream information into digital information through the conversion circuit and device. The ARM chip is used as the main processing unit to start the face positioning module to perform face positioning and status analysis. The processed image information is temporarily stored in SDRAM. The state recognition module obtains the image information processed by the face positioning module from the SDRAM, and recognizes the state of the human eye. Then the fatigue determination module performs fatigue determination according to the state results of the face positioning module and the human eye state recognition module, and finally converts the determination result. The electrical signal is transmitted from the I/O interface to the alarm device.

3. a kind of fatigue driving detection device as claimed in claim 1, is characterized in that, described face location module realizes to judge whether driver is in fatigued state by following steps:

3.1 Use the Adaboost algorithm to detect the driver's face position, and obtain the longitudinal difference Diff between the center point position (H _x ,H _y ) of the face area and the initially collected image center point position (I _x , I _y ) ;

3.2 After obtaining the center point position of the face area, calculate the difference between the ordinate _Hy of the face center point and the ordinate I _y of the image Diff=|H _y -I _y |;

3.3 Count the number of images n and the total number of image frames N whose difference Diff is greater than the set threshold T1 in the statistical unit period;

3.4 Calculate the ratio of the number of images n to the total number of image frames N, and compare the ratio with the set threshold T3. If the ratio is greater than the set threshold T3, it is determined that the driver is in a fatigued state.

4. a kind of fatigue driving detection device as claimed in claim 1 is characterized in that, described fatigue determination module carries out final determination to the mental state of driver by following comprehensive result:

According to the comprehensive judgment results of the face positioning module and the human eye state identification module, three alarm modes are carried out: when the face positioning module determines that the driver is in a fatigued state, the LED light alarm device will be activated; when the human eye state identification module determines that the driver is in a fatigue state If the driver is in a fatigued state, the horn alarm device will be activated to give an alarm; when both the face positioning module and the human eye state recognition module determine that the driver is in a fatigued state, the LED light alarm device and the horn alarm device will be activated at the same time.

5. A fatigue driving detection method realized based on the fatigue driving detection device according to claim 1, is characterized in that, comprises the steps:

Step 1, initialize the camera, set the attribute value of the image read by the camera;

Step 2, the USB camera captures the image, and transmits the image information to the ARM processor device;

Step 3, image preprocessing, that is, image reduction and grayscale processing;

Step 4: Load the existing face and eye feature classifiers in the OpenCV machine vision library, and locate the face area through the pre-trained face feature classifier, that is, use the Haar feature detection method to obtain the key feature points of the human face. The feature points locate the face area, and obtain the coordinates (x ₁ , y ₁ ) and (x ₂ , y ₂ ) of the upper left corner and the lower right corner of the face area respectively, and calculate the center point coordinates (H _x , y 2 ) of the face area image. H _y ), where According to the coordinates of (x ₁ , y ₁ ) and (x ₂ , y ₂ ), the face area is framed by a rectangular frame;

Step 5, determine the fatigue state by the state of the face: use the Adaboost algorithm to detect the position of the driver's face, calculate the longitudinal deviation between the position of the face center point and the image center point, and compare the deviation value with the set threshold T1 and determine it. The fluctuation frequency of the center point of the face in time, that is, the frequency of nodding, to judge whether the driver is in a state of fatigue;

Step 6, determine the fatigue state by the state of the human eye: use the Adaboost algorithm to detect the position of the driver's eye, perform feature extraction on the driver's eye, and calculate the ratio of the maximum value of the integral projection of the eye area to the width of the integral area, And compare it with the set threshold T2 to determine whether the driver is in a fatigued state; it specifically includes the following steps:

6.1 Use the Adaboost algorithm to detect the presence of the driver's eyes, and locate the eyes in the face area through the human eye feature classifier;

6.2 Perform Gaussian filtering on the eye area image, remove noise, and use adaptive binarization processing;

6.3 Calculate the integral projection of the binarized image img, calculate the ratio of the maximum value of the vertical integral projection to the integral width rate1 and the ratio of the maximum value of the horizontal integral projection to the integral width rate2;

The formula used for the calculation of the horizontal integral projection:

rate1=V _max /V _width ;

rate2=H _max /H _width ;

Among them, S _v (x) is the sum of the pixel values corresponding to the unit width of the binarized image img, and _Sh (x) is the sum of the pixel values corresponding to the unit height of the binarized image img, Y ₁ =1, Y ₂ is equal to the height of the eye area image, X ₁ =1, X ₂ is equal to the width of the eye area image, I(x, y) is (x, y) in the binarized image of the eye area ) corresponding pixel value, (x, y) is the coordinate value of the binarized image, rate1, rate2 represent the ratio of the maximum vertical integral projection to the width of the vertical integral projection area, the maximum horizontal integral projection and the width of the horizontal integral projection area, respectively ratio;

6.4 Combining rate1 and rate2, calculate Determine whether the driver is in a state of fatigue; compare the value of rate with the set threshold T2 to determine the open and closed state of the human eye, that is, if the value of rate is greater than the set threshold T2, it means that the closed degree of the human eye reaches 80%. It is deemed that the eyes are currently completely closed;

Among them, rate represents the integrated value of the vertical and horizontal integral projection height-width ratio relationship, is the weight coefficient of rate1 and rate2;

6.5 The P80 method of PERCLOS is used to determine fatigue. The percentage of eye closure time in the total time in the set unit period is calculated. If the ratio exceeds the preset threshold T, it is deemed that the current driver is already in fatigue driving; Calculate the value f of PERCLOS:

In unit time, count the number of image frames with eyes closed and the total number of image frames per unit time, the value f of PERCLOS is equivalent to:

Step 7: According to the comprehensive determination results of fatigue in steps 5 and 6, it is determined to return to image acquisition or to activate one of different alarm modes.

6. fatigue driving detection method as claimed in claim 5, is characterized in that, determining fatigue state by face state described in step 5 specifically comprises the following steps:

5.1 Use the Adaboost algorithm to detect the driver's face position, and obtain the longitudinal difference Diff between the center point position (H _x ,H _y ) of the face area and the initially collected image center point position (I _x , I _y ) ;

5.2 After obtaining the center point position of the face area, calculate the difference between the ordinate _Hy of the face center point and the ordinate I _y of the image Diff=|H _y -I _y |;

5.3 Count the number of images n and the total number of frames N of images whose difference Diff is greater than the set threshold T1 in the unit period;

5.4 Calculate the ratio of the number of images n to the total number of image frames N, and compare the ratio with the set threshold T3. If the ratio is greater than the set threshold T3, it is determined that the driver is in a fatigued state.

7. The fatigue driving detection method according to claim 6, wherein the binarization process described in step 6.2, the specific method is:

6.2.1. Set the minimum threshold low=0.1, the maximum threshold high=0.5, and the threshold τ increment step=0.05;

6.2.2. Continuously binarize the eye area image through imgs=binary(I,τ);

6.2.3. From a series of binarized image imgs, find the binarized image img1 with the least number of non-zero connected regions and its corresponding threshold τ;

6.2.4. Perform the opening operation on the binarized image img1 with the least number of non-zero connected regions and fill the inner holes;

6.2.5 Find the largest connected domain in the binarized image img1, and remove other connected domains to obtain the final binarized image img;

Among them, imgs represents the binarized image, bianry represents the binarization process, I represents the eye area image, and τ is the binarization threshold gradually increasing from the minimum threshold low according to step 6.2.1; imgs=binary(I ,τ) is the representation form of image binarization processing in the present invention.

8. The fatigue detection method as claimed in claim 5, wherein the determination method is specifically: comprehensively judged by step 5 and step 6, it is characterized in that, the different alarm modes described in step 7 are: when step 5 determines that the driver is fatigued When the driver is in a fatigued state in step 6, the horn alarm device is activated; when both steps 5 and 6 determine that the driver is in a fatigued state, the LED light alarm device and the horn are activated at the same time. Alarm device alarms.