CN111754729A - Fatigue driving prompting device and prompting method - Google Patents

Abstract

The invention discloses a fatigue driving prompting device, comprising: a fatigue state detection component, a fatigue reminder interaction component and a communication component. The fatigue state detection component collects and processes the driver's image, and outputs driving characteristic information according to the driver's image. The fatigue reminder interaction component is connected to the fatigue state detection component, and the fatigue reminder interaction component generates fatigue level information according to the driving feature information. The communication component connects the fatigue reminder interaction component and the vehicle feedback component, the communication component sends the fatigue level information generated by the fatigue reminder interaction component to the vehicle feedback component, and the vehicle feedback component provides corresponding feedback to the driver according to the fatigue level information. Vehicle feedback components are various onboard components that are native to the vehicle. The invention also discloses a fatigue driving prompting method. According to the real-time state of the driver, the present invention can provide various sensory stimulations including hearing, vision, smell and touch to relieve fatigue.

Description

Fatigue driving prompting device and prompting method

technical field

The invention relates to the technical field of automobile intelligentization, and more particularly, to a technology for judging and prompting driver fatigue driving behavior.

Background technique

Fatigue driving is one of the major hidden dangers of traffic safety in the process of car use. When the driver is in a state of fatigue, the perception of the surrounding environment, the ability to judge and the ability to control the vehicle are reduced to varying degrees, and traffic accidents are very likely to occur. At present, the in-vehicle fatigue driving detection system mainly uses two methods: one is to judge whether the driver is fatigued or not by analyzing the driving behavior of the driver such as turning the steering wheel and stepping on the brake. This method is greatly affected by the driver's driving habits, and there is a false alarm rate. higher problem. The other uses image processing technology to judge fatigue driving based on a single detection amount such as the driver's eye state or head drooping state. Since the use of a single detection amount is greatly disturbed by the driver's habitual actions and accidental actions, it is also There is a problem that the judgment accuracy is low.

After the driver is found to be driving fatigued, most of the reminder interaction methods for fatigued driving currently use a single reminder method such as timed voice broadcast and steering wheel vibration, which has the problem of poor real-time and adaptability. In the state of driver fatigue driving, as the number of prompts increases, the human body is easy to form immunity to a single stimulus, and the prompt effect is limited. When it is detected that the driver is driving fatigued, the most correct approach is to stop and rest in time to relieve fatigue. However, in the actual driving process, it is often impossible to find a service area, a rest area, or inconvenient parking, etc., and it is impossible to stop and rest in time. In addition, in a state of fatigue, the driver is required to go to the service area to stop and rest, which will increase the risk of driving.

SUMMARY OF THE INVENTION

The present invention aims to propose a new fatigue driving detection and prompting technology.

According to an embodiment of the present invention, a fatigue driving prompting device is provided, which includes: a fatigue state detection component, a fatigue reminder interaction component, and a communication component. The fatigue state detection component collects and processes the driver's image, and outputs driving characteristic information according to the driver's image. The fatigue reminder interaction component is connected to the fatigue state detection component, and the fatigue reminder interaction component generates fatigue level information according to the driving feature information. The communication component connects the fatigue reminder interaction component and the vehicle feedback component, the communication component sends the fatigue level information generated by the fatigue reminder interaction component to the vehicle feedback component, and the vehicle feedback component provides corresponding feedback to the driver according to the fatigue level information. Vehicle feedback components are various onboard components that are native to the vehicle.

According to an embodiment of the present invention, the fatigue state detection component is installed in the cockpit, and the fatigue state detection component obtains the driver's face image, extracts the eye feature image and the mouth feature image from the facial image, and extracts the eye feature image and the mouth feature image from the face image. The driving feature information is output from the partial feature image.

According to an embodiment of the present invention, the fatigue state detection component includes: an image acquisition device and an image processing device. The image capture device captures images of the driver. The image processing device is connected to the image acquisition device, and the image processing device obtains the driver's face image from the driver's image collected by the image acquisition device, extracts the eye feature image and the mouth feature image from the facial image, and extracts the eye feature image and the mouth feature image according to the eye feature. The image and the mouth feature image output driving feature information.

According to an embodiment of the present invention, the image acquisition device is an interior camera, the interior camera is aimed at the driver's seat area, the interior camera is an infrared camera with an active infrared light source, and the field of view of the interior camera covers the normal driving state. In all active areas of the driver's head, the resolution of the camera in the interior meets the requirements for the identification of the driver's eye movements and mouth movements.

According to an embodiment of the present invention, the image processing device includes: a face detection module, an eye image module, a mouth image module, and an image analysis module. The face detection module locates the position of the face image from the driver's image and obtains the driver's face image. The eye image module locates and extracts the eye image from the face image. The mouth image module locates and extracts the mouth image from the face image. The image analysis module is connected to the eye image module and the mouth image module, and the image analysis module outputs driving feature information according to the eye feature image and the mouth feature image.

According to an embodiment of the present invention, the driving characteristic information includes:

The eye opening is greater than 20% of the maximum opening, which is the eye opening signal;

The eye opening is less than or equal to 20% of the maximum opening, which is the closed eye signal;

The continuous eye-opening signal, eye-closing signal and eye-opening signal constitute the blinking signal;

Eyes closed signal lasts for 3 seconds or more, it is a doze signal;

If the mouth signal cannot be located for 3 seconds or more, it is a drowsiness signal;

The mouth opening is greater than the mouth width, which is a yawn signal.

According to an embodiment of the present invention, the fatigue reminder interaction component generates fatigue level information according to the driving feature information, the fatigue reminder interaction component counts the number of driving feature information per unit time, and generates four types of fatigue level information: normal, mild fatigue, moderate fatigue Fatigue and severe fatigue.

According to an embodiment of the present invention,

If the doze signal is greater than or equal to 1 time in a 1-minute period, it is severe fatigue;

Moderate fatigue occurs when the blink signal is greater than or equal to 30 times in a 1-minute period but there is no drowsiness signal, or the yawn signal is greater than or equal to 4 times in a 1-minute period but there is no drowsiness signal;

If the blink signal is greater than or equal to 24 times and less than 30 times but there is no drowsiness signal within a 1-minute period, or the yawn signal is greater than or equal to 2 times and less than 4 times but there is no drowsiness signal within a 1-minute period, it is mild fatigue;

The rest are normal.

According to an embodiment of the present invention, a vehicle feedback assembly includes an instrument panel, an air conditioning controller, and an infotainment device. The instrument cluster provides visual and audible feedback cues to the driver. The air conditioning controller provides somatosensory feedback prompts to the driver. Infotainment equipment provides mental and informational feedback cues to the driver.

According to an embodiment of the present invention, the vehicle feedback assembly further includes: a fragrance device and a seat massage controller. Fragrance devices provide drivers with olfactory feedback cues. Seat massage controllers provide haptic feedback cues to the driver.

According to an embodiment of the present invention,

Under mild fatigue, the vehicle feedback component performs low-intensity prompts, and the low-intensity prompts are performed by at least one of the instrument panel and the air conditioner controller: the instrument panel provides moderate visual and auditory feedback prompts, and the air conditioner controller reduces the temperature and adds new energy. Air volume for somatosensory feedback;

Under moderate fatigue, the vehicle feedback component performs a medium-intensity prompt, and the medium-intensity prompt is performed by at least one of the instrument panel, the air conditioner controller, the infotainment device, and the fragrance device: the instrument panel provides rapid visual and auditory feedback prompts, air conditioning The controller lowers the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route in the rest area, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts;

Under severe fatigue, the vehicle feedback component performs high-intensity prompts, and the high-intensity prompts are performed by at least one of the instrument panel, air conditioning controller, infotainment device, fragrance device, and seat massage controller: the instrument panel provides a quick visual and auditory feedback prompts, the air conditioning controller reduces the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route of the rest area, and the aroma equipment releases refreshing fragrance to provide olfactory feedback. Prompt, the seat massage controller turns on seat massage to provide tactile feedback to the driver.

According to an embodiment of the present invention, the communication component is an in-vehicle gateway.

According to an embodiment of the present invention, a method for prompting fatigue driving is provided, including:

In the fatigue detection step, the image of the driver is collected and processed, and the driving feature information is output according to the image of the driver;

Fatigue reminder step, generating fatigue level information based on driving characteristic information;

In the feedback prompting step, the fatigue level information is sent to the vehicle feedback component, and the vehicle feedback component provides corresponding feedback prompts to the driver according to the fatigue level information.

According to an embodiment of the present invention, in the fatigue detection step, a face image of the driver is obtained, an eye feature image and a mouth feature image are extracted from the face image, and driving feature information is output according to the eye feature image and the mouth feature image.

According to an embodiment of the present invention, the driving characteristic information includes:

The eye opening is greater than 20% of the maximum opening, which is the eye opening signal;

The eye opening is less than or equal to 20% of the maximum opening, which is the closed eye signal;

The continuous eye-opening signal, eye-closing signal and eye-opening signal constitute the blinking signal;

Eyes closed signal lasts for 3 seconds or more, it is a doze signal;

If the mouth signal cannot be located for 3 seconds or more, it is a drowsiness signal;

The mouth opening is greater than the mouth width, which is a yawn signal.

According to an embodiment of the present invention, in the fatigue reminder step, the fatigue level information is generated according to the driving feature information, the fatigue reminder interactive component counts the number of driving feature information per unit time, and generates four types of fatigue level information: normal, mild fatigue, Moderate and severe fatigue.

According to an embodiment of the present invention,

If the doze signal is greater than or equal to 1 time in a 1-minute period, it is severe fatigue;

Moderate fatigue occurs when the blink signal is greater than or equal to 30 times in a 1-minute period but there is no drowsiness signal, or the yawn signal is greater than or equal to 4 times in a 1-minute period but there is no drowsiness signal;

If the blink signal is greater than or equal to 24 times and less than 30 times but there is no drowsiness signal within a 1-minute period, or the yawn signal is greater than or equal to 2 times and less than 4 times but there is no drowsiness signal within a 1-minute period, it is mild fatigue;

The rest are normal.

According to an embodiment of the present invention,

Under mild fatigue, the vehicle feedback component performs low-intensity prompts, and the low-intensity prompts are performed by at least one of the instrument panel and the air conditioner controller: the instrument panel provides moderate visual and auditory feedback prompts, and the air conditioner controller reduces the temperature and adds new energy. Air volume for somatosensory feedback;

Under moderate fatigue, the vehicle feedback component performs a medium-intensity prompt, and the medium-intensity prompt is performed by at least one of the instrument panel, the air conditioner controller, the infotainment device, and the fragrance device: the instrument panel provides rapid visual and auditory feedback prompts, air conditioning The controller lowers the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route in the rest area, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts;

Under severe fatigue, the vehicle feedback component performs high-intensity prompts, and the high-intensity prompts are performed by at least one of the instrument panel, air conditioning controller, infotainment device, fragrance device, and seat massage controller: the instrument panel provides a quick visual and auditory feedback prompts, the air conditioning controller reduces the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route of the rest area, and the aroma equipment releases refreshing fragrance to provide olfactory feedback. Prompt, the seat massage controller turns on seat massage to provide tactile feedback to the driver.

The fatigue driving prompting device and prompting method of the present invention can provide a variety of sensory stimuli including hearing, vision, smell and touch according to the real-time state of the driver, and reduce the impact of fatigued driving from two aspects: relieving fatigue and helping the driver to stop and rest in time , effectively protect the driver's safe driving, avoid traffic accidents caused by fatigue driving, and endanger the life and property safety of the driver and others.

Description of drawings

FIG. 1 discloses a structural block diagram of a fatigue driving prompting device according to an embodiment of the present invention.

FIG. 2 discloses a structural block diagram of a fatigue state detection component in a fatigue driving prompting device according to an embodiment of the present invention.

FIG. 3 discloses a structural block diagram of an image processing device in a fatigue driving reminder device according to an embodiment of the present invention.

FIG. 4 discloses the process of determining the fatigue level in the fatigue driving prompting device according to an embodiment of the present invention.

FIG. 5 discloses the working process of the fatigue driving prompting device according to an embodiment of the present invention.

FIG. 6 discloses a flowchart of a method for prompting fatigue driving according to an embodiment of the present invention.

Detailed ways

FIG. 1 discloses a structural block diagram of a fatigue driving prompting device according to an embodiment of the present invention. Referring to FIG. 1 , the device for prompting fatigue driving includes: a fatigue state detection component 100 , a fatigue reminder interaction component 200 and a communication component 300 . The fatigue state detection component 100 collects and processes the image of the driver, and outputs driving characteristic information according to the image of the driver. The fatigue reminder interaction component 200 is connected to the fatigue state detection component 100, and the fatigue reminder interaction component 200 generates fatigue level information according to the driving feature information. The communication component 300 connects the fatigue reminder interaction component 200 and the vehicle feedback component, the communication component 300 sends the fatigue level information generated by the fatigue reminder interaction component to the vehicle feedback component, and the vehicle feedback component provides corresponding feedback prompts to the driver according to the fatigue level information.

The fatigue state detection component 100 is installed in the cockpit, and the fatigue state detection component obtains the driver's facial image, extracts the eye feature image and the mouth feature image from the facial image, and outputs the driving feature information according to the eye feature image and the mouth feature image. . By collecting the data of the eye feature image and the mouth feature image, the information to detect whether the driver is fatigued driving is more comprehensive, and the high probability of misjudgment caused by the single sampling information is reduced. FIG. 2 discloses a structural block diagram of a fatigue state detection component in a fatigue driving prompting device according to an embodiment of the present invention. In the illustrated embodiment, the fatigue state detection assembly 100 includes: an image acquisition device 110 and an image processing device 120 .

The image capturing device 110 captures an image of the driver. In one embodiment, the image capture device 110 is an interior camera, the interior camera is aimed at the driver's seat area, and the interior camera is an infrared camera with an active infrared light source. The field of view of the interior camera covers the entire active area of the driver's head in normal driving conditions to ensure that the driver's eye and mouth characteristic images can be collected under normal driving conditions. The resolution of the interior camera can meet the recognition requirements of the driver's eye movements and mouth movements. Due to the small magnitude of eye movements and mouth movements, it is necessary to ensure that the interior camera has a high enough resolution to clearly identify eye movements and mouth movements. In an implementation example, the interior camera is installed in front of the driver, and can be optionally placed on the dashboard behind the steering wheel, on the left A-pillar of the driver, or on the front windshield frame, etc. The camera's lens is aimed at the driver's seat Area. The interior camera is an infrared camera and an active infrared light source. The active infrared light source can make image acquisition not affected by external lighting conditions. The horizontal field of view of the interior camera is greater than 60°, and the vertical field of view is greater than 35°, which can ensure that all the active areas of the driver's head are within the camera's field of view during normal driving. The driver's face image can be collected. The resolution of the interior camera is 1280(H) x 720(V), at which eye movements and mouth movements can be clearly identified. The imager of the interior camera uses a global shutter.

The image processing device 120 is connected to the image acquisition device 110 . The image processing device 120 obtains the driver's face image from the image of the driver collected by the image acquisition device 110, extracts the eye feature image and the mouth feature image from the face image, and outputs the image according to the eye feature image and the mouth feature image. Driving characteristic information. FIG. 3 discloses a structural block diagram of an image processing device in a fatigue driving reminder device according to an embodiment of the present invention. In the embodiment shown in FIG. 3 , the image processing device 120 includes: a face detection module 121 , an eye image module 122 , a mouth image module 123 and an image analysis module 124 . The face detection module 121 locates the position of the face image from the driver's image and acquires the driver's face image. The eye image module 122 locates and extracts the eye image from the facial image. The mouth image module 123 locates and extracts the mouth image from the facial image. The image analysis module 124 is connected to the eye image module 122 and the mouth image module 123, and the image analysis module 124 outputs driving characteristic information according to the eye characteristic image and the mouth characteristic image. In one embodiment, the driving feature information mainly includes the following three actions of the driver: blinking, yawning, and dozing. Specifically, the driving feature information output according to the eye feature image and the mouth feature image includes:

The eye opening is greater than 20% of the maximum opening, which is the eye opening signal.

When the eye opening is less than or equal to 20% of the maximum opening, it is a closed eye signal.

The continuous eye-opening signal, eye-closing signal and eye-opening signal constitute the blink signal.

Eyes closed signal lasts for 3 seconds or more, it is a doze signal.

If the mouth signal cannot be located for 3 seconds or more, it is a drowsiness signal. When the driver is drowsy, he usually appears in a head-down position, and in the head-down position, the mouth is blocked and the mouth signal cannot be located. Therefore, the continuous inability to locate the mouth signal will also be used as a doze signal.

The mouth opening is greater than the mouth width, which is a yawn signal.

The fatigue reminder interaction component 200 generates fatigue level information according to the driving characteristic information. The fatigue reminder interaction component counts the number of driving characteristic information per unit time, and generates four types of fatigue level information: normal, mild fatigue, moderate fatigue and severe fatigue. In one embodiment, the fatigue reminder interaction component counts the times of three types of driving characteristic information of blinking, yawning, and dozing in a 1-minute period to determine the fatigue level. In one embodiment, the judgment criteria used are as follows:

If the doze signal is greater than or equal to 1 time in a 1-minute period, it is severe fatigue;

Moderate fatigue occurs when the blink signal is greater than or equal to 30 times in a 1-minute period but there is no drowsiness signal, or the yawn signal is greater than or equal to 4 times in a 1-minute period but there is no drowsiness signal;

If the blink signal is greater than or equal to 24 times and less than 30 times but there is no drowsiness signal within a 1-minute period, or the yawn signal is greater than or equal to 2 times and less than 4 times but there is no drowsiness signal within a 1-minute period, it is mild fatigue;

The rest are normal.

FIG. 4 discloses the process of determining the fatigue level in the fatigue driving prompting device according to an embodiment of the present invention. In the embodiment shown in FIG. 4 , according to the quantity of driving characteristic information corresponding to the fatigue level, the judgment is made in the order of severe fatigue, moderate fatigue, mild fatigue and normal at one time. Referring to Figure 4, after collecting the characteristic images of the driver's eyes and mouth, first determine whether the number of dozes in 1 minute is greater than or equal to 1 time, if so, it is determined as a severe fatigue driving, if not, go to the next step. The second part judges that the blink signal is greater than or equal to 30 times in a 1-minute period (no drowsiness signal), or the yawn signal is greater than or equal to 4 times in a 1-minute period (no drowsiness signal). judge. The third step judges that the blink signal is greater than or equal to 24 times and less than 30 times in a 1-minute period (no drowsiness signal), or the yawn signal is greater than or equal to 2 times and less than 4 times in a 1-minute period (no drowsiness signal), if so, it is mild Fatigue driving, if not, go to the next step. None of the above conditions are met, and it is judged as a normal state.

In one embodiment, the communication component 300 is an in-vehicle gateway. The fatigue level information generated by the fatigue reminder interaction component 200 is sent to the vehicle feedback component through the vehicle-mounted gateway, and the vehicle feedback component provides corresponding feedback prompts to the driver according to the fatigue level information. In one embodiment, the vehicle feedback assembly includes a dashboard 310 , an air conditioning controller 320 and an infotainment device 330 .

Dashboard 310 provides visual and audible feedback cues to the driver. The instrument panel 310 provides visual and auditory stimulation with light tube combinations and alarm sounds, and the intensity of the prompts can be adjusted according to different fatigue levels. When it is detected that the driver is in a slightly fatigued driving state, the fatigue driving warning light on the instrument panel flashes slowly, and a gentle warning sound is issued to remind the driver that the driver is in a fatigued state. If it is detected that the driver is in a state of moderate or severe fatigue driving, the fatigue driving warning light on the instrument panel flashes rapidly, and a rapid alarm tone sounds. By changing the urgency of the prompt sound, the stimulation of the driver's auditory senses is strengthened, the warning effect is enhanced, and the driver is reminded to stop and rest in time.

The air-conditioning controller 320 provides a somatosensory feedback prompt to the driver. The driver stays in a closed space for a long time, which is easy to cause hypoxia and drowsiness, resulting in fatigue driving. Under different fatigue levels, the air conditioning controller 320 can open the external circulation mode, increase the fresh air volume, update the air in the car, relieve the fatigue and sleepiness caused by insufficient oxygen supply, and at the same time, reduce the temperature of the air outlet to provide the driver with somatosensory stimulation , to help drivers get rid of fatigue.

The infotainment device 330 provides mental and information feedback prompts to the driver. The infotainment device 330 can give the driver a mental boost and trigger the navigation module to provide real-time parking and rest suggestions. When it is detected that the driver is in a state of moderate or severe fatigue driving, it will play refreshing music to cheer the driver up, and display the nearest service area or rest area, provide a navigation route, and suggest the driver to take a rest nearby. Suddenly turning on the music that helps to refresh your mind and switching navigation information may cause the driver to be distracted, so this function is only turned on when driving with moderate or severe fatigue.

In another embodiment, for high-end models with more abundant configurations, the vehicle feedback component further includes: a fragrance device 340 and a seat massage controller 350 .

Fragrance device 340 provides olfactory feedback cues to the driver. When low-intensity somatosensory, visual, and auditory stimuli cannot achieve the ideal prompting effect, and the driver is detected to be in a moderately fatigued driving state. For models equipped with in-vehicle fragrance equipment, the fragrance equipment 340 releases a refreshing fragrance, provides olfactory stimulation for the driver, regulates the driver's spirit and emotion, temporarily relieves driving fatigue, and buys time for parking and rest.

The seat massage controller 350 provides haptic feedback cues to the driver. When driving for a long time, the driver's posture is fixed and the movements are restricted, which can easily lead to poor blood circulation, causing muscle stiffness, physical fatigue and other uncomfortable symptoms. When it is detected that the driver is in a heavily fatigued driving state, for models with seat massage function, the seat massage controller 350 starts seat massage, and directly stimulates the driver's somatosensory senses by massaging some muscles of the driver's lower back, cervical vertebra and other muscles. Promote human blood circulation and effectively improve fatigue symptoms caused by long-term driving. However, suddenly starting the seat massage may affect the driver's normal driving, so this function is only triggered when the low-intensity and medium-intensity reminders are not effective.

FIG. 5 discloses the working process of the fatigue driving prompting device according to an embodiment of the present invention. The working process of the fatigue state detection component 100 and the fatigue reminder interaction component 200 is shown: severe fatigue driving corresponds to high-intensity reminder, moderate fatigue driving corresponds to medium-intensity reminder, mild fatigue driving corresponds to low-intensity reminder, and no reminder in normal state. in particular,

Under mild fatigue, the vehicle feedback component performs low-intensity prompts, and the low-intensity prompts are performed by at least one of the instrument panel 310 and the air conditioning controller 320: the instrument panel provides moderate visual and auditory feedback prompts, and the air conditioning controller reduces the temperature, Increase the fresh air volume for somatosensory feedback prompts. The instrument panel 310 and the air conditioner controller 320 may select only one of the components for prompting, or both may simultaneously prompt for prompting.

Under moderate fatigue, the vehicle feedback component performs a medium-intensity prompt, and the medium-intensity prompt is performed by at least one of the instrument panel 310, the air conditioning controller 320, the infotainment device 330, and the fragrance device 340: wherein the instrument panel provides rapid visual and auditory perception Feedback prompts, the air-conditioning controller reduces the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route of the rest area, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts. The instrument panel 310, the air conditioning controller 320, the infotainment device 330 and the fragrance device 340 may select only one of the components for prompting, or may select a combination of multiple components for prompting. Usually for moderate fatigue, it is necessary to use 2-3 component combinations for prompting.

Under severe fatigue, the vehicle feedback component performs high-intensity prompts, and the high-intensity prompts are executed by at least one of the instrument panel 310, the air conditioning controller 320, the infotainment device 330, the fragrance device 340, and the seat massage controller 350: where the meter The panel provides urgent visual and auditory feedback prompts, the air conditioning controller reduces the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides mental feedback prompts for playing music, and provides information feedback prompts for indicating the navigation route of the rest area, and the fragrance equipment releases refreshing. The fragrance provides olfactory feedback prompts, and the seat massage controller turns on seat massage to provide tactile feedback prompts to the driver. The instrument panel 310, the air conditioning controller 320, the infotainment device 330, the fragrance device 340 and the seat massage controller 350 may select only one of the components for prompting, or may select a combination of multiple components for prompting. Usually, for severe fatigue, it is necessary to use 4-5 component combinations for prompting.

The present invention also discloses a method for prompting fatigue driving, and FIG. 6 discloses a flowchart of the method for prompting fatigue driving according to an embodiment of the present invention. Referring to Figure 6, the fatigue driving prompting method includes the following steps:

In the fatigue detection step 410, the image of the driver is collected and processed, and driving characteristic information is output according to the image of the driver. In the fatigue detection step, a face image of the driver is obtained, an eye feature image and a mouth feature image are extracted from the face image, and driving feature information is output according to the eye feature image and the mouth feature image. The driving feature information includes: the eye opening degree is greater than 20% of the maximum opening degree, which is the eye opening signal; the eye opening degree is less than or equal to 20% of the maximum opening degree, which is the eye closing signal; continuous eye opening signal, eye signal and open eye signal The eye signal constitutes a blink signal; the eye closing signal lasts for 3 seconds or more, it is a doze signal; the mouth signal that cannot be located for 3 seconds or more is a doze signal; the mouth opening is greater than the mouth width, it is a yawn signal. The fatigue detection step 410 is performed by the fatigue state detection module, and the specific implementation details are consistent with the foregoing contents, and are not repeated here.

Fatigue reminding step 420 , generating fatigue level information according to the driving characteristic information. In the fatigue reminder step 420, fatigue level information is generated according to the driving characteristic information, and the fatigue reminder interaction component counts the number of driving characteristic information per unit time, and generates four types of fatigue level information: normal, mild fatigue, moderate fatigue and severe fatigue. Among them: the doze signal is greater than or equal to 1 time in a 1-minute period, indicating severe fatigue; the blinking signal is greater than or equal to 30 times in a 1-minute period but there is no dozing signal, or the yawning signal is greater than or equal to 4 times but no dozing signal in a 1-minute period, it is medium Severe fatigue; blinking signals greater than or equal to 24 times and less than 30 times but no drowsiness signal in a 1-minute period, or yawning signals greater than or equal to 2 times and less than 4 times but no drowsiness signal in a 1-minute period, it is mild fatigue; other conditions are: normal. The fatigue reminder step 420 is performed by the fatigue reminder interaction module, and the specific implementation details are consistent with the foregoing content, and are not repeated here.

In the feedback prompting step 430, the fatigue level information is sent to the vehicle feedback component, and the vehicle feedback component provides corresponding feedback prompts to the driver according to the fatigue level information. Under mild fatigue, the vehicle feedback component performs low-intensity prompts, and the low-intensity prompts are performed by at least one of the instrument panel and the air conditioner controller: the instrument panel provides moderate visual and auditory feedback prompts, and the air conditioner controller reduces the temperature and adds new energy. Air volume for somatosensory feedback prompts. Under moderate fatigue, the vehicle feedback component performs a medium-intensity prompt, and the medium-intensity prompt is performed by at least one of the instrument panel, the air conditioner controller, the infotainment device, and the fragrance device: the instrument panel provides rapid visual and auditory feedback prompts, air conditioning The controller lowers the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route in the rest area, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts. Under severe fatigue, the vehicle feedback component performs high-intensity prompts, and the high-intensity prompts are performed by at least one of the instrument panel, air conditioning controller, infotainment device, fragrance device, and seat massage controller: the instrument panel provides a quick visual and auditory feedback prompts, the air conditioning controller reduces the temperature and increases the fresh air volume for somatosensory feedback prompts, the infotainment equipment provides spiritual feedback prompts for playing music, provides information feedback prompts for indicating the navigation route of the rest area, and the aroma equipment releases refreshing fragrance to provide olfactory feedback. Prompt, the seat massage controller turns on seat massage to provide tactile feedback to the driver. The feedback prompting step 430 is performed by the communication component, the instrument panel, the air conditioner controller, the infotainment device, the fragrance device, and the seat massage controller, and the specific implementation details are the same as those described above, and will not be repeated here.

The fatigue driving prompting device and prompting method of the present invention can provide a variety of sensory stimuli including hearing, vision, smell and touch according to the real-time state of the driver, and reduce the impact of fatigued driving from two aspects: relieving fatigue and helping the driver to stop and rest in time , effectively protect the driver's safe driving, avoid traffic accidents caused by fatigue driving, and endanger the life and property safety of the driver and others.

It should also be noted that the above-listed embodiments are only specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and the similar changes or deformations made subsequently can be directly derived from the contents disclosed in the present invention or can be easily thought of by those skilled in the art, and all belong to the protection scope of the present invention. . The above-mentioned embodiments are provided for those skilled in the art to realize or use the present invention. Those skilled in the art can make various modifications or changes to the above-mentioned embodiments without departing from the inventive concept of the present invention. The protection scope of the present invention is not limited by the above-mentioned embodiments, but should be the maximum scope conforming to the innovative features mentioned in the claims.

Claims (18)

1. A fatigue driving prompting device, comprising:

the fatigue state detection component acquires and processes the image of the driver and outputs driving characteristic information according to the image of the driver;

the fatigue reminding interactive component is connected to the fatigue state detection component and generates fatigue grade information according to the driving characteristic information;

the communication component is connected with the fatigue reminding interaction component and the vehicle feedback component, the communication component sends fatigue grade information generated by the fatigue reminding interaction component to the vehicle feedback component, and the vehicle feedback component carries out corresponding feedback prompt on a driver according to the fatigue grade information.

2. The fatigue driving prompting device according to claim 1, wherein the fatigue state detecting component is mounted in the cabin, the fatigue state detecting component acquires a face image of the driver, extracts an eye feature image and a mouth feature image from the face image, and outputs the driving feature information based on the eye feature image and the mouth feature image.

3. The fatigue driving prompting device according to claim 2, wherein the fatigue state detecting component includes:

the image acquisition equipment acquires an image of the driver;

and an image processing device connected to the image capturing device, the image processing device acquiring a face image of the driver from the image of the driver captured by the image capturing device, extracting an eye feature image and a mouth feature image from the face image, and outputting driving feature information based on the eye feature image and the mouth feature image.

4. The fatigue driving prompting device according to claim 3, wherein the image acquisition device is an interior camera, the interior camera is directed at the driver seat area, the interior camera is an infrared camera with an active infrared light source, the field angle of the interior camera covers the whole active area of the driver's head in a normal driving state, and the resolution of the interior camera meets the identification requirements of the driver's eye action and mouth action.

5. The fatigue driving prompting apparatus according to claim 3, wherein the image processing device includes:

the face detection module is used for positioning the position of the face image from the image of the driver and acquiring the face image of the driver;

the eye image module is used for positioning and extracting an eye image from the face image;

a mouth image module that locates and extracts a mouth image from the face image;

and the image analysis module is connected to the eye image module and the mouth image module and outputs driving characteristic information according to the eye characteristic image and the mouth characteristic image.

6. The fatigue driving presentation device according to claim 5, wherein the driving characteristic information includes:

the eye opening degree is greater than 20% of the maximum opening degree and is an eye opening signal;

the eye opening degree is less than or equal to 20% of the maximum opening degree, and the eye opening degree is an eye closing signal;

the continuous eye opening signal, the eye closing signal and the eye opening signal form a blink signal;

the eye-closing signal lasts for 3 seconds or more, and is a drowsiness signal;

the mouth signal can not be positioned for 3 seconds or more, and the signal is a drowsy signal;

the opening degree of the mouth is larger than the width of the mouth, and the mouth is a Hardgree signal.

7. The fatigue driving prompting device according to claim 6, wherein the fatigue prompting interacting component generates fatigue grade information according to the driving characteristic information, the fatigue prompting interacting component counts the number of times of the driving characteristic information in unit time and generates four kinds of fatigue grade information: normal, mild, moderate and severe fatigue.

8. The fatigue driving prompting device according to claim 7,

the drowsiness signal is more than or equal to 1 time within a period of 1 minute, which is severe fatigue;

the blink signal is more than or equal to 30 times but no drowsiness signal within a period of 1 minute, or the yawning signal is more than or equal to 4 times but no drowsiness signal within a period of 1 minute, and the fatigue is moderate;

the blink signal is more than or equal to 24 times and less than 30 times in a 1-minute period without the drowsiness signal, or the yawning signal is more than or equal to 2 times and less than 4 times in a 1-minute period without the drowsiness signal, and the fatigue is light;

the rest is normal.

9. The fatigue driving prompting device according to claim 7, wherein the vehicle feedback component includes:

the instrument panel provides visual and auditory feedback prompts for a driver;

the air conditioner controller provides somatosensory feedback prompts for a driver;

an infotainment device provides mental and informational feedback cues to a driver.

10. The fatigue driving prompting device according to claim 9, wherein the vehicle feedback assembly further includes:

the fragrance device provides olfactory feedback prompts for a driver;

a seat massage controller that provides tactile feedback cues to the driver.

11. The fatigue driving prompting device according to claim 10,

under light fatigue, the vehicle feedback assembly carries out low-intensity prompt, and the low-intensity prompt is executed by at least one of an instrument panel and an air conditioner controller: the instrument panel provides mild visual and auditory feedback prompts, and the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts;

under moderate fatigue, the vehicle feedback component performs medium intensity prompt, and the medium intensity prompt is executed by at least one of an instrument panel, an air conditioner controller, an infotainment device and a fragrance device: the instrument panel provides prompt visual and auditory feedback prompts, the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts, the information entertainment equipment provides mental feedback prompts for playing music and information feedback prompts for indicating a rest area navigation route, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts;

under severe fatigue, the vehicle feedback assembly performs high-intensity prompting, which is performed by at least one of an instrument panel, an air conditioning controller, an infotainment device, a fragrance device, and a seat massage controller: the instrument panel provides prompt visual and auditory feedback prompts, the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts, the information entertainment equipment provides mental feedback prompts for playing music and information feedback prompts for indicating a rest area navigation route, the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts, and the seat massage controller starts seat massage to provide tactile feedback prompts for a driver.

12. The fatigue driving prompting device of claim 1, wherein the communication component is an in-vehicle gateway.

13. A fatigue driving prompting method, comprising:

a fatigue detection step, namely acquiring and processing an image of a driver, and outputting driving characteristic information according to the image of the driver;

a fatigue reminding step, namely generating fatigue grade information according to the driving characteristic information;

and a feedback prompting step, namely sending the fatigue grade information to a vehicle feedback component, and carrying out corresponding feedback prompting on the driver by the vehicle feedback component according to the fatigue grade information.

14. The fatigue driving presentation method according to claim 13, wherein the fatigue detection step acquires a face image of the driver, extracts an eye feature image and a mouth feature image from the face image, and outputs the driving feature information based on the eye feature image and the mouth feature image.

15. The fatigue driving presentation device according to claim 14, wherein the driving characteristic information includes:

the eye opening degree is greater than 20% of the maximum opening degree and is an eye opening signal;

the eye opening degree is less than or equal to 20% of the maximum opening degree, and the eye opening degree is an eye closing signal;

the continuous eye opening signal, the eye closing signal and the eye opening signal form a blink signal;

the eye-closing signal lasts for 3 seconds or more, and is a drowsiness signal;

the mouth signal can not be positioned for 3 seconds or more, and the signal is a drowsy signal;

the opening degree of the mouth is larger than the width of the mouth, and the mouth is a Hardgree signal.

16. The fatigue driving prompting device according to claim 15, wherein in the fatigue prompting step, fatigue grade information is generated according to the driving characteristic information, the fatigue prompting interactive component counts the number of times of the driving characteristic information in a unit time, and generates four kinds of fatigue grade information: normal, mild, moderate and severe fatigue.

17. The fatigue driving prompting device according to claim 16,

the drowsiness signal is more than or equal to 1 time within a period of 1 minute, which is severe fatigue;

the blink signal is more than or equal to 30 times but no drowsiness signal within a period of 1 minute, or the yawning signal is more than or equal to 4 times but no drowsiness signal within a period of 1 minute, and the fatigue is moderate;

the blink signal is more than or equal to 24 times and less than 30 times in a 1-minute period without the drowsiness signal, or the yawning signal is more than or equal to 2 times and less than 4 times in a 1-minute period without the drowsiness signal, and the fatigue is light;

the rest is normal.

18. The fatigue driving prompting device according to claim 16,

under light fatigue, the vehicle feedback assembly carries out low-intensity prompt, and the low-intensity prompt is executed by at least one of an instrument panel and an air conditioner controller: the instrument panel provides mild visual and auditory feedback prompts, and the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts;

under moderate fatigue, the vehicle feedback component performs medium intensity prompt, and the medium intensity prompt is executed by at least one of an instrument panel, an air conditioner controller, an infotainment device and a fragrance device: the instrument panel provides prompt visual and auditory feedback prompts, the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts, the information entertainment equipment provides mental feedback prompts for playing music and information feedback prompts for indicating a rest area navigation route, and the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts;

under severe fatigue, the vehicle feedback assembly performs high-intensity prompting, which is performed by at least one of an instrument panel, an air conditioning controller, an infotainment device, a fragrance device, and a seat massage controller: the instrument panel provides prompt visual and auditory feedback prompts, the air conditioner controller reduces the temperature and increases the fresh air volume to perform somatosensory feedback prompts, the information entertainment equipment provides mental feedback prompts for playing music and information feedback prompts for indicating a rest area navigation route, the fragrance equipment releases refreshing fragrance to provide olfactory feedback prompts, and the seat massage controller starts seat massage to provide tactile feedback prompts for a driver.

Images