CN110544361A - monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system - Google Patents
monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system Download PDFInfo
- Publication number
- CN110544361A CN110544361A CN201910825530.2A CN201910825530A CN110544361A CN 110544361 A CN110544361 A CN 110544361A CN 201910825530 A CN201910825530 A CN 201910825530A CN 110544361 A CN110544361 A CN 110544361A
- Authority
- CN
- China
- Prior art keywords
- driver
- monitoring
- fatigue
- frame
- video frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 86
- 201000010099 disease Diseases 0.000 title claims abstract description 28
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 28
- 238000012937 correction Methods 0.000 title claims abstract description 17
- 230000003993 interaction Effects 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims description 35
- 230000002452 interceptive effect Effects 0.000 claims description 27
- 238000001514 detection method Methods 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 26
- 206010048232 Yawning Diseases 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 14
- 230000009191 jumping Effects 0.000 claims description 13
- 230000002159 abnormal effect Effects 0.000 claims description 10
- 239000002699 waste material Substances 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 8
- 230000001815 facial effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000012790 confirmation Methods 0.000 claims description 6
- 210000003128 head Anatomy 0.000 claims description 6
- 241001282135 Poromitra oscitans Species 0.000 claims description 5
- 206010053156 Musculoskeletal discomfort Diseases 0.000 claims description 3
- 230000004397 blinking Effects 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 claims description 3
- 210000000887 face Anatomy 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000003139 buffering effect Effects 0.000 claims description 2
- 230000009471 action Effects 0.000 description 6
- 206010039203 Road traffic accident Diseases 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000006378 damage Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 206010008111 Cerebral haemorrhage Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 206010015037 epilepsy Diseases 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/0202—Child monitoring systems using a transmitter-receiver system carried by the parent and the child
- G08B21/0205—Specific application combined with child monitoring using a transmitter-receiver system
- G08B21/0208—Combination with audio or video communication, e.g. combination with "baby phone" function
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/0202—Child monitoring systems using a transmitter-receiver system carried by the parent and the child
- G08B21/0205—Specific application combined with child monitoring using a transmitter-receiver system
- G08B21/0211—Combination with medical sensor, e.g. for measuring heart rate, temperature
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/0202—Child monitoring systems using a transmitter-receiver system carried by the parent and the child
- G08B21/0225—Monitoring making use of different thresholds, e.g. for different alarm levels
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/0202—Child monitoring systems using a transmitter-receiver system carried by the parent and the child
- G08B21/0269—System arrangements wherein the object is to detect the exact location of child or item using a navigation satellite system, e.g. GPS
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/0202—Child monitoring systems using a transmitter-receiver system carried by the parent and the child
- G08B21/028—Communication between parent and child units via remote transmission means, e.g. satellite network
- G08B21/0283—Communication between parent and child units via remote transmission means, e.g. satellite network via a telephone network, e.g. cellular GSM
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
- G08B25/01—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium
- G08B25/10—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium using wireless transmission systems
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Child & Adolescent Psychology (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Engineering & Computer Science (AREA)
- Heart & Thoracic Surgery (AREA)
- Cardiology (AREA)
- Radar, Positioning & Navigation (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Traffic Control Systems (AREA)
- Emergency Alarm Devices (AREA)
Abstract
the invention discloses a monitoring and warning system with a sudden disease monitoring and warning function and a fatigue driving monitoring and warning correction system, which comprises an identity recognition module, a disease monitoring and warning system, a fatigue driving warning system, an interaction module and a positioning device, wherein the identity recognition module is used for recognizing the identity of a user; the identity recognition system is used for recognizing the identity of a driver and starting the system; the disease monitoring and alarming system and the fatigue driving early warning system are respectively used for monitoring the physical condition and the fatigue driving condition of a driver, informing the driver, and giving an alarm and avoiding danger urgently according to the situation; the interaction module is used for the system to interact with the driver so as to avoid misjudgment of the system on the physical condition and the fatigue condition of the driver; the positioning device is used for positioning the positions of the vehicles so as to position the positions of the vehicles and facilitate rescue. The system simultaneously monitors sudden diseases and fatigue driving conditions of the driver, and has important significance for reducing road traffic safety accidents.
Description
Technical Field
the invention belongs to the field of driving auxiliary devices for assisting the driving of a driver, and particularly relates to a monitoring and warning system with a sudden disease monitoring and warning function and a fatigue driving monitoring and early warning correction system.
background
Road traffic injuries are fatal disasters, and about 120 million people worldwide are invisibly deprived of life each year. According to the data of the world health organization, the traffic accidents are ranked ninth among the causes of human death and morbidity. In addition, according to statistical data, three main causes of traffic accidents are shown: fatigue driving, speeding and drunk driving. The detection technology of overspeed driving is mature, and with the enhancement of supervision, the overspeed driving has a descending trend. The drunk driving harm is great, but the number of death people caused by drunk driving is greatly reduced in China due to the enhancement of the punishment on the drunk driving. Therefore, fatigue driving is still an important reason threatening road traffic safety at present, and becomes one of the most important hidden dangers of traffic safety at present.
If early warning could be given before fatigue driving has not produced a serious hazard, the number of traffic accidents would be greatly reduced. The fatigue driving detection mainly comprises a contact type method and a non-contact type method, wherein the contact type method is mainly used for detecting three indexes of electroencephalogram, electrocardio and myoelectricity of a driver, and the non-contact type method is mainly used for detecting action parameters of the driver and a vehicle through a camera, and the action parameters comprise facial features of the driver, a vehicle driving route, rotation of a steering wheel and the like. The contact method is not suitable for being placed in a cab due to a large detection system, and meanwhile, the contact with a driver is needed, so that the interference on the driver is great, the application of the contact detection method is limited, and the non-contact method becomes a main direction for researching fatigue detection.
In recent years, many researchers have conducted intensive research on non-contact fatigue detection methods, and the obtained results mainly focus on face positioning, eye positioning and tracking, fatigue-related features and theoretical judgment methods thereof, but the recognition rate of fatigue is not ideal at present.
Although the existing fatigue driving detection method is not perfect, fatigue driving can be supervised by a driver by strictly controlling driving time, and the occurrence of a part of traffic accidents cannot be avoided in advance, for example, sudden diseases such as heart disease, cerebral hemorrhage and epilepsy of the driver occur, and the sudden diseases have no symptoms, are sudden and are extremely dangerous, so that the driver loses the capability of controlling the vehicle in a very short time, and great safety risks are caused to the driver and other road vehicles.
therefore, the research on the system which is accurate, real-time and feasible and has the functions of monitoring and alarming sudden diseases and monitoring, early warning and correcting fatigue driving is great trend, the system has important significance for reducing road traffic safety accidents, when the driver drives in fatigue, a prompt is given to remind the driver of being in a dangerous fatigue driving state, and after the driving fatigue state occurs, the driver needs to have a rest in time to adjust the state, so that traffic accidents are avoided.
Disclosure of Invention
in view of this, the present application aims to provide a monitoring and early warning correction system with a sudden disease monitoring and warning function and fatigue driving monitoring and early warning function. In order to achieve the above object, the present application provides the following technical solutions.
the system has a sudden disease monitoring and alarming function and a fatigue driving monitoring and early warning correction system, and comprises an identity recognition module, a disease monitoring and alarming system, a fatigue driving early warning system, an interaction module and a positioning device; wherein
The identity recognition module is used for recognizing the identity of a driver and starting the system;
The disease monitoring and alarming system is used for monitoring the physical condition of the driver, informing the driver, and alarming and avoiding danger emergently according to the physical condition of the driver;
the fatigue driving early warning system is used for monitoring whether a driver is in fatigue driving or not, and alarming and avoiding danger urgently according to the fatigue state of the driver;
the interaction module is used for the system to interact with the driver so as to avoid misjudgment of the system on the physical condition and the fatigue condition of the driver;
The positioning device is used for positioning the positions of the vehicles so as to position the positions of the vehicles and facilitate rescue.
preferably, the identity recognition module is a fingerprint unlocking device, an infrared face recognition camera or a mobile terminal.
Preferably, the disease monitoring and alarming system comprises a sensor, a heart rate control module, a wireless transmission module and a mobile terminal; wherein,
the sensor comprises two groups of heart rate variability monitoring chips, the monitoring chips are symmetrically attached to the outer side wall of the steering wheel respectively and are in a strip shape, and the length of each group of monitoring chips is not less than 1/4 of the peripheral length of the steering wheel;
The heart rate control module is connected with a vehicle-mounted computer, and the vehicle-mounted computer is connected with the interaction module.
preferably, the fatigue driving early warning system comprises a camera, a buffer and a processor; wherein
the camera is arranged on an instrument panel in front of a driver and used for acquiring a facial image of the driver;
The buffer is used for buffering a process file in the fatigue driving judgment process;
The processor is used for carrying out data processing on the facial image acquired by the camera to obtain the driving fatigue of the driver.
Preferably, the disease monitoring and alarming method based on the system comprises the following steps:
S1.1, a driver holds a steering wheel in a hand to drive in a vehicle, and the sensor monitors the electrocardiogram of the driver;
s1.2, the mobile terminal receives and displays the electrocardiogram data, so that a driver can conveniently check the electrocardiogram data of the driver at any time; if the heart rate of the driver is abnormal, the vehicle-mounted computer wakes up the interactive module, if the driver cannot interact with the interactive module or the driver directly feeds back discomfort of the body of the driver through the interactive module, the vehicle-mounted computer controls the vehicle to carry out danger avoiding treatment, meanwhile, the heart rate control module contacts an emergency contact person through the mobile terminal and sends a short message to the emergency contact person, the short message comprises the positioning and the current heart rate of the driver, and meanwhile, the mobile terminal calls the emergency contact person to remind the emergency contact person.
Preferably, the fatigue driving early warning method based on the system comprises the following steps:
s2.1, the camera collects video frames and marks time labels on the video frames;
S2.2, the processor judges whether the video frame is a first frame or not according to the time tag on the video frame;
s2.3, if the video frame is the first frame, skipping to S2.4;
if the video frame is not the first frame, determining small areas around the mouth and eye areas of the previous frame of video as the feature extraction subareas of the video frame according to the face detection result of the previous frame of video cached in the cache, and then jumping to the step S2.6;
S2.4, carrying out face detection on the video frame to obtain a face area
If the face area is a normal proportion area, the face area is a feature extraction area, and then the step S2.5 is skipped;
if the face area is an abnormal proportion area, the driver is indicated to be lowering the head, the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
s2.5, carrying out special diagnosis extraction in the feature extraction area;
s2.6, mouth monitoring and eye monitoring are carried out;
If the mouth and the eyes are monitored in the first frame, jumping to the step S2.8;
if the mouth and eyes are not monitored in the first frame, the driver is indicated to turn around the head, which means that the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
If the mouth and the eye are not obtained by monitoring in the feature extraction subarea in the non-first frame, jumping to the step S2.7;
S2.7, carrying out face detection on the two feature extraction subareas
If the two feature extraction subareas detect faces, the driver is indicated to be twisting, the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
If the face is not detected in the eye feature extraction subarea, it is indicated that the driver may have fallen down because of dozing, the fatigue level is marked as four, and the step S2.9 is skipped;
S2.8 monitoring whether yawning occurs and whether frequent blinking occurs
if the driver yawns, the driver does not blink frequently, the fatigue level is marked as second, and the step S2.9 is skipped;
if the driver yawns and blinks frequently, marking the fatigue level as three, and skipping to the step S2.9;
if the driver does not yawn and does not blink, skipping to the step S2.1;
If the driver does not yawn but blinks frequently, temporarily marking the fatigue level as one, and skipping to the step S2.9;
S2.9 starting the interactive module to interact with the driver
For the fatigue level one, the interaction module interacts with the driver, for example: the method comprises the steps that whether a driver is tired or not is determined through communication with the driver in a voice mode, if the driver does not feed back fatigue, the step S2.1 is skipped, and if the driver confirms that the driver is tired, an interaction module reminds the driver to stop for a rest in time;
aiming at the fatigue grade II, the interactive module performs interactive confirmation with the driver, if the driver is confirmed to be tired, the driver is reminded to stop for rest, and the driver is asked whether to open music or open a window, a skylight and the like;
aiming at the third fatigue level, the interactive module and the driver carry out interactive confirmation, if the driver has feedback, the driver indicates that the driver still has intelligence, the vehicle window is directly opened, and the emergency stop lamp is turned on; if the driver has no feedback, the driver is deeply tired, and the fatigue grade is improved to grade four;
and aiming at the fatigue grade four, the system directly controls the vehicle to decelerate, turns on an emergency stop lamp and turns on a vehicle window.
preferably, the interaction comprises voice interaction and gesture interaction; and the interactive content and mode can be set individually by the driver.
Preferably, in the step S2.4, in the process of detecting the face, correction processing is first performed on the image pixels, specifically:
Assuming that the pixel value of a certain pixel Pi of a given image is, after color correction, the pixel value of the pixel Pi is, which is:
preferably, the basis for judging whether the face region is a normal proportion region in the step S2.4 is as follows: setting a proportion threshold value of the length of the face in the proportion of the length of the photo, wherein if the proportion of the length of the face area in the length of the video frame is higher than the proportion threshold value, the face area is a normal proportion area; and if the proportion of the length of the face region in the length of the video frame is less than a proportion threshold, the face region is an abnormal proportion region.
preferably, before step S2.5 is performed, the feature extraction region needs to be divided into an upper half and a lower half, where the upper half is used for monitoring the eye, and the lower half is used for monitoring the mouth, so as to ensure that the mouth monitoring and the eye monitoring are performed simultaneously.
The advantages and effects of the present application are as follows.
the interaction module is arranged, interaction between a driver and the system is achieved through the interaction module, and the accuracy of system alarming is guaranteed.
the sensors in the prior art are arranged on a driver body next to the skin, and the sensors of the disease monitoring and alarming system 102 are arranged on a steering wheel, so that the driving comfort and monitoring convenience of the driver are improved.
In the face detection process, the full-face detection is only carried out on the first frame of video, so that the data processing efficiency is greatly improved;
the method and the device monitor the mouth and eyes of the driver at the same time, comprehensively judge the fatigue condition of the driver through two factors, and improve the monitoring accuracy;
The system realizes the classification of the fatigue condition of the driver, carries out corresponding emergency strategies aiming at different fatigue conditions, avoids one-time cutting, and has high humanization degree.
In the process of face recognition, the image is subjected to color correction, brightness information is removed from the pixels RGB, the ground color of the image is reserved, and the correctness of face detection is guaranteed.
the foregoing description is only an overview of the technical solutions of the present application, so that the technical means of the present application can be more clearly understood and the present application can be implemented according to the content of the description, and in order to make the above and other objects, features and advantages of the present application more clearly understood, the following detailed description is made with reference to the preferred embodiments of the present application and the accompanying drawings.
the above and other objects, advantages and features of the present application will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
FIG. 1 is a block diagram of the present application with a sudden illness monitoring and warning function and a fatigue driving monitoring and early warning correction system;
Wherein, 100-system; 101-an identity recognition module; 102-a disease monitoring alarm system; 103-fatigue driving early warning system; 104-an interaction module; 105-positioning means.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. In the following description, specific details such as specific configurations and components are provided only to help the embodiments of the present application be fully understood. Accordingly, it will be apparent to those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, descriptions of well-known functions and constructions are omitted in the embodiments for the sake of clarity and conciseness.
it should be appreciated that reference throughout this specification to "one embodiment" or "the embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrase "one embodiment" or "the present embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
It is further noted that, herein, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion.
Example 1
The system 100 has a sudden disease monitoring and alarming function and a fatigue driving monitoring and early warning correction function, and the system 100 comprises an identity recognition module 101, a disease monitoring and alarming system 102, a fatigue driving early warning system 103, an interaction module 104 and a positioning device 105.
wherein
the identity recognition module 101 is used for recognizing the identity of the driver and starting the system;
the disease monitoring and alarming system 102 is used for monitoring the physical condition of the driver, informing the driver, and alarming and avoiding danger emergently according to the physical condition of the driver;
The fatigue driving early warning system 103 is used for monitoring whether the driver is in fatigue driving or not, and giving an alarm and avoiding danger urgently according to the fatigue state of the driver;
the interaction module 104 is used for the system to interact with the driver so as to avoid misjudgment of the system on the physical condition and the fatigue condition of the driver, and to grade the physical condition and the fatigue condition of the driver so as to determine the next action of the system;
the positioning device 105 is used for positioning the position of the vehicle so as to position the position of the vehicle for rescue.
Further, the identity module 101 is a fingerprint unlocking device, the fingerprint unlocking device is arranged in the automobile central control, and the driver acquires the fingerprint and starts the system through the fingerprint unlocking device.
as an alternative, the identity recognition module 101 is an infrared face recognition camera, which is disposed on a rearview mirror of an automobile, and the camera recognizes a driver to realize system startup.
As an alternative, the identity module 101 is started by a mobile terminal such as a mobile phone, and the mobile terminal can also realize functions of transferring, monitoring signals, calculating, issuing instructions to the inside of the vehicle, sending calls and asking for help information to the outside, and the like.
example 2
on the basis of embodiment 1, the present embodiment further defines the specific structure and operation principle of the disease monitoring and alarm system 102, which are described in detail below.
The disease monitoring and alarming system 102 comprises a sensor, a heart rate control module, a wireless transmission module and a mobile terminal.
The sensor comprises two groups of Heart Rate Variability (HRV) monitoring chips, the monitoring chips are symmetrically attached to the outer side wall and the strip of the steering wheel respectively, and the length of each group of monitoring chips is not less than 1/4 of the peripheral length of the steering wheel.
preferably, the monitoring chip is further arranged on a neck headrest or a waist headrest or a bracelet to perform more comprehensive heart rate variability monitoring.
the monitoring chip is powered by the button battery, converts acquired analog signals into digital signals, and packages the digital signals and transmits the digital signals to the communication gateway through a private protocol in a wireless signal mode.
The heart rate control module comprises a communication gateway and a connection module.
the communication gateway is used for receiving the data packet sent by the sensor and marking the data packet by using a receiving time stamp and an identification number.
The connection module is connected to a vehicle-mounted computer of the vehicle itself, which is connected to the interaction module 104. If the heart rate control module monitors that the heart rate of the driver is abnormal, the vehicle-mounted computer wakes up the interactive module 104, and if the driver cannot interact with the interactive module 104 or the driver directly feeds back discomfort of the driver through the interactive module 104, the vehicle-mounted computer controls the vehicle to carry out danger avoiding treatment. Meanwhile, the heart rate control module contacts the emergency contact through the mobile terminal.
Preferably, the risk avoidance process includes, but is not limited to, self-braking, turning on an emergency stop light, or opening a window.
the wireless transmission module is a Bluetooth module and comprises a protocol converter, and the protocol converter converts the data packet from a private protocol to a Bluetooth protocol.
The wireless transmission module is connected with the communication gateway and supplies power to the communication gateway through the vehicle-mounted power supply.
the mobile terminal is used for receiving and displaying the electrocardiogram data, so that a driver can conveniently check the electrocardiogram data of the driver at any time. The mobile terminal is connected to a wireless transmission module, preferably a bluetooth connection.
If the heart rate control module monitors that the heart rate of the driver is abnormal and the interaction module 104 determines that the driver really has physical discomfort, the mobile terminal sends a short message to an emergency contact, wherein the short message comprises the positioning of the driver and the current heart rate, and meanwhile, the mobile terminal calls the emergency contact.
Example 3
on the basis of the embodiments 1-2, the present embodiment further defines the specific composition and the operation principle of the fatigue driving warning system 103, which are specifically as follows.
the fatigue driving early warning system 103 comprises a camera, a buffer and a processor.
Wherein
The camera is arranged on an instrument panel in front of a driver and used for acquiring a facial image of the driver.
the processor is used for carrying out data processing on the facial image acquired by the camera to obtain the driving fatigue of the driver.
The specific process of the system for carrying out fatigue driving early warning is as follows:
S1, the camera collects video frames and marks time labels on the video frames;
s2 the processor judges whether the video frame is the first frame according to the time label on the video frame;
s3, if the video frame is the first frame, jumping to S4;
if the video frame is not the first frame, determining small areas around the mouth and eye areas of the previous frame of video as the feature extraction sub-areas of the video frame according to the face detection result of the previous frame of video cached in the cache, and then jumping to the step S6;
S4, carrying out face detection on the video frame to obtain a face area
if the face area is a normal proportion area, the face area is a feature extraction area, and then the step S5 is skipped;
If the face area is an abnormal proportion area, it indicates that the driver is lowering the head, meaning that the video frame is a waste frame, then go to step S1, and delete the video frame by the processor;
S5, extracting special diagnosis in the feature extraction area;
S6 mouth and eye monitoring;
if the mouth and the eyes are monitored in the first frame, jumping to step S8;
If the mouth and eyes are not monitored in the first frame, the driver is turning the head, which means that the video frame is a waste frame, the process goes to step S1, and the processor deletes the video frame;
if the mouth and the eye are not monitored in the feature extraction sub-region in the non-first frame, jumping to step S7;
S7 face detection is carried out on the two feature extraction subareas
if the two feature extraction subareas detect faces, the driver is indicated to be twisting, which means that the video frame is a waste frame, the step S1 is skipped, and the processor deletes the video frame;
If the face is not detected in the eye feature extraction sub-area, it indicates that the driver may have fallen down because of dozing, and the fatigue level is marked as four, and the process goes to step S9;
S8 monitors for yawning and frequent blinking
if the driver yawns, blinks frequently, marks the driver as fatigue level two, and jumps to step S9;
If the driver yawns and blinks frequently, marking the fatigue level as three, and jumping to the step S9;
If the driver does not yawn and does not blink, jumping to step S1;
if the driver does not yawn but blinks frequently, temporarily marking the fatigue level as one, and jumping to the step S9;
S9 starting the interactive module 104 to interact with the driver
for fatigue level one, the interaction module 104 interacts with the driver, for example: the method comprises the steps that whether a driver is tired or not is determined through communication with the driver in a voice mode, if the driver does not feed back fatigue, the step goes to step S1, and if the driver confirms that the driver is tired, an interaction module 104 reminds the driver to stop for a rest in time;
Aiming at the fatigue level two, the interactive module 104 performs interactive confirmation with the driver, if the driver is confirmed to be tired, the driver is reminded to stop for rest, and the driver is asked whether to open music or open a window, a skylight and the like;
Aiming at the fatigue grade III, the interactive module 104 performs interactive confirmation with the driver, if the driver has feedback, the driver indicates that the driver still has intelligence, the vehicle window is directly opened, and the emergency stop lamp is turned on; if the driver has no feedback, the driver is deeply tired, and the fatigue grade is improved to grade four;
And aiming at the fatigue grade four, the system directly controls the vehicle to decelerate, turns on an emergency stop lamp and turns on a vehicle window.
preferably, the interaction includes but is not limited to voice interaction, gesture interaction; and the interactive content and mode can be set individually by the driver.
it is worth noting that in the prior art, there are many face detection technologies, the present application performs face detection based on image pixel information, and the existing face detection technology based on image pixel information directly adopts original pixels of an image to perform recognition so as to recognize skin color, and further obtain a face. Obviously, the color deviation of the color image due to the exposure is easily affected, that is, the brightness of the photo easily affects the correctness of the face detection, and based on this, in the step S4, the present embodiment first performs the correction processing on the image pixels in the face detection process, specifically:
Assuming that the pixel value of a certain pixel Pi of a given image is, after color correction, the pixel value of the pixel Pi is, which is:
The pixel value after color correction not only removes the brightness information from the pixel RGB, but also keeps the ground color thereof, thereby ensuring the correctness of face detection.
the basis for determining whether the face region is a normal proportion region in step S4 is as follows: setting a proportion threshold value of the length of the face in the proportion of the length of the photo, wherein if the proportion of the length of the face area in the length of the video frame is higher than the proportion threshold value, the face area is a normal proportion area; and if the proportion of the length of the face region in the length of the video frame is less than a proportion threshold, the face region is an abnormal proportion region.
In order to improve the work recognition efficiency, before the step S5 is executed, the feature extraction area needs to be divided into an upper half and a lower half, the upper half is used for monitoring the eyes, and the lower half is used for monitoring the mouth, so as to ensure that the mouth monitoring and the eye monitoring are performed simultaneously.
The step S8 of monitoring whether yawning is performed specifically includes obtaining an upper lip boundary line and a lower lip boundary line through mouth monitoring, where a distance between the two lines indicates a mouth opening size, and setting a mouth opening threshold value if the mouth opening size of N consecutive frames exceeds the mouth opening threshold value, which means that the driver has yawned, because yawning is a continuous, relatively slow large mouth opening process, and is different from temporary large mouth opening (e.g., speaking).
In step S8, it is monitored whether to blink frequently, specifically, the area of the black pixel region in the case of opening eyes is different from that in the case of closing eyes, and further, the blink rate may be determined by calculating the number of consecutive frames that the eyes remain closed, and if the blink rate exceeds a set blink threshold, it means that the driver blinks frequently.
The previous description of all disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. The system has a sudden disease monitoring and alarming function and a fatigue driving monitoring and early warning correction system, and is characterized by comprising an identity recognition module, a disease monitoring and alarming system, a fatigue driving early warning system, an interaction module and a positioning device; wherein
the identity recognition system is used for recognizing the identity of a driver and starting the system;
the disease monitoring and alarming system is used for monitoring the physical condition of the driver, informing the driver, and alarming and avoiding danger emergently according to the physical condition of the driver;
the fatigue driving early warning system is used for monitoring whether a driver is in fatigue driving or not, and alarming and avoiding danger urgently according to the fatigue state of the driver;
The interaction module is used for the system to interact with the driver so as to avoid misjudgment of the system on the physical condition and the fatigue condition of the driver;
the positioning device is used for positioning the positions of the vehicles so as to position the positions of the vehicles and facilitate rescue.
2. The system of claim 1, wherein the identification system is a fingerprint unlocker, an infrared face recognition camera, or a mobile terminal.
3. The system of claim 1, wherein the disease monitoring alarm system comprises a sensor, a heart rate control module, a wireless transmission module and a mobile terminal; wherein,
the sensor comprises two groups of heart rate variability monitoring chips, the monitoring chips are symmetrically attached to the outer side wall of the steering wheel respectively and are in a strip shape, and the length of each group of monitoring chips is not less than 1/4 of the peripheral length of the steering wheel;
The heart rate control module is connected with a vehicle-mounted computer, and the vehicle-mounted computer is connected with the interaction module.
4. The system of claim 1, wherein the driver fatigue warning system comprises a camera, a buffer, and a processor; wherein
The camera is arranged on an instrument panel in front of a driver and used for acquiring a facial image of the driver;
the buffer is used for buffering a process file in the fatigue driving judgment process;
the processor is used for carrying out data processing on the facial image acquired by the camera to obtain the driving fatigue of the driver.
5. disease monitoring and alarm method based on the system according to any one of claims 1 to 4, characterized in that the method is specifically:
S1.1, a driver holds a steering wheel in a hand to drive in a vehicle, and the sensor monitors the electrocardiogram of the driver;
s1.2, the mobile terminal receives and displays the electrocardiogram data, so that a driver can conveniently check the electrocardiogram data of the driver at any time; if the heart rate of the driver is abnormal, the vehicle-mounted computer wakes up the interactive module, if the driver cannot interact with the interactive module or the driver directly feeds back discomfort of the body of the driver through the interactive module, the vehicle-mounted computer controls the vehicle to carry out danger avoiding treatment, meanwhile, the heart rate control module contacts an emergency contact person through the mobile terminal and sends a short message to the emergency contact person, the short message comprises the positioning and the current heart rate of the driver, and meanwhile, the mobile terminal calls the emergency contact person to remind the emergency contact person.
6. The fatigue driving early warning method based on the system of any one of claims 1 to 4, wherein the method specifically comprises:
S2.1, the camera collects video frames and marks time labels on the video frames;
s2.2, the processor judges whether the video frame is a first frame or not according to the time tag on the video frame;
S2.3, if the video frame is the first frame, skipping to S2.4;
if the video frame is not the first frame, determining small areas around the mouth and eye areas of the previous frame of video as the feature extraction subareas of the video frame according to the face detection result of the previous frame of video cached in the cache, and then jumping to the step S2.6;
S2.4, carrying out face detection on the video frame to obtain a face area
if the face area is a normal proportion area, the face area is a feature extraction area, and then the step S2.5 is skipped;
if the face area is an abnormal proportion area, the driver is indicated to be lowering the head, the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
S2.5, carrying out special diagnosis extraction in the feature extraction area;
S2.6, mouth monitoring and eye monitoring are carried out;
if the mouth and the eyes are monitored in the first frame, jumping to the step S2.8;
if the mouth and eyes are not monitored in the first frame, the driver is indicated to turn around the head, which means that the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
if the mouth and the eye are not obtained by monitoring in the feature extraction subarea in the non-first frame, jumping to the step S2.7;
s2.7, carrying out face detection on the two feature extraction subareas
If the two feature extraction subareas detect faces, the driver is indicated to be twisting, the video frame is a waste frame, the step S2.1 is skipped, and the processor deletes the video frame;
if the face is not detected in the eye feature extraction subarea, it is indicated that the driver may have fallen down because of dozing, the fatigue level is marked as four, and the step S2.9 is skipped;
s2.8 monitoring whether yawning occurs and whether frequent blinking occurs
if the driver yawns, the driver does not blink frequently, the fatigue level is marked as second, and the step S2.9 is skipped;
if the driver yawns and blinks frequently, marking the fatigue level as three, and skipping to the step S2.9;
If the driver does not yawn and does not blink, skipping to the step S2.1;
if the driver does not yawn but blinks frequently, temporarily marking the fatigue level as one, and skipping to the step S2.9;
s2.9 starting the interactive module to interact with the driver
For the fatigue level one, the interaction module interacts with the driver, for example: the method comprises the steps that whether a driver is tired or not is determined through communication with the driver in a voice mode, if the driver does not feed back fatigue, the step S2.1 is skipped, and if the driver confirms that the driver is tired, an interaction module reminds the driver to stop for a rest in time;
Aiming at the fatigue grade II, the interactive module performs interactive confirmation with the driver, if the driver is confirmed to be tired, the driver is reminded to stop for rest, and the driver is asked whether to open music or open a window, a skylight and the like;
Aiming at the third fatigue level, the interactive module and the driver carry out interactive confirmation, if the driver has feedback, the driver indicates that the driver still has intelligence, the vehicle window is directly opened, and the emergency stop lamp is turned on; if the driver has no feedback, the driver is deeply tired, and the fatigue grade is improved to grade four;
and aiming at the fatigue grade four, the system directly controls the vehicle to decelerate, turns on an emergency stop lamp and turns on a vehicle window.
7. the method of claim 6, wherein the interaction comprises a voice interaction, a gesture interaction; and the interactive content and mode can be set individually by the driver.
8. the method according to claim 6, wherein in the step S2.4, in the face detection process, the correction processing is first performed on the image pixels, specifically:
Assuming that the pixel value of a certain pixel Pi of a given image is, after color correction, the pixel value of the pixel Pi is, which is:
；
；
。
9. the method of claim 6, wherein the step S2.4 of determining whether the face region is a normal proportion region is based on: setting a proportion threshold value of the length of the face in the proportion of the length of the photo, wherein if the proportion of the length of the face area in the length of the video frame is higher than the proportion threshold value, the face area is a normal proportion area; and if the proportion of the length of the face region in the length of the video frame is less than a proportion threshold, the face region is an abnormal proportion region.
10. The method according to claim 6, wherein step S2.5 is performed by first dividing the feature extraction area into an upper half and a lower half, wherein the upper half is used for monitoring the eyes and the lower half is used for monitoring the mouth, so as to ensure that the mouth monitoring and the eyes monitoring are performed simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910825530.2A CN110544361B (en) | 2019-09-03 | 2019-09-03 | Monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910825530.2A CN110544361B (en) | 2019-09-03 | 2019-09-03 | Monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110544361A true CN110544361A (en) | 2019-12-06 |
| CN110544361B CN110544361B (en) | 2021-09-03 |
Family
ID=68711044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910825530.2A Active CN110544361B (en) | 2019-09-03 | 2019-09-03 | Monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110544361B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113593184A (en) * | 2021-07-21 | 2021-11-02 | 深圳市传海科技有限公司 | Method for restraining fatigue driving and related device |
| CN113891049A (en) * | 2021-11-02 | 2022-01-04 | 湖北车安达信息科技有限公司 | Vehicle-mounted video monitoring data processing method and device based on big data |
| CN115137941A (en) * | 2022-06-21 | 2022-10-04 | 北京中科心研科技有限公司 | Emergency awakening method and system based on fatigue recognition model and electrical stimulation |
| CN115352450A (en) * | 2022-08-12 | 2022-11-18 | 重庆长安汽车股份有限公司 | Vehicle driving assistance method and device, vehicle and storage medium |
| CN120116949A (en) * | 2025-05-13 | 2025-06-10 | 上海朗晖慧科技术有限公司 | A vehicle fatigue monitoring device based on dynamic recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102211522A (en) * | 2011-04-29 | 2011-10-12 | 四川长虹电器股份有限公司 | Unsafe state early-warning device and method for drivers |
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN104504856A (en) * | 2014-12-30 | 2015-04-08 | 天津大学 | Fatigue driving detection method based on Kinect and face recognition |
| CN107170190A (en) * | 2017-05-31 | 2017-09-15 | 泉州创先力智能科技有限公司 | A kind of dangerous driving warning system |
| CN109886148A (en) * | 2019-01-29 | 2019-06-14 | 唐山明天科技有限公司 | A kind of driver's active warning system and device based on recognition of face |
-
2019
- 2019-09-03 CN CN201910825530.2A patent/CN110544361B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102211522A (en) * | 2011-04-29 | 2011-10-12 | 四川长虹电器股份有限公司 | Unsafe state early-warning device and method for drivers |
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN104504856A (en) * | 2014-12-30 | 2015-04-08 | 天津大学 | Fatigue driving detection method based on Kinect and face recognition |
| CN107170190A (en) * | 2017-05-31 | 2017-09-15 | 泉州创先力智能科技有限公司 | A kind of dangerous driving warning system |
| CN109886148A (en) * | 2019-01-29 | 2019-06-14 | 唐山明天科技有限公司 | A kind of driver's active warning system and device based on recognition of face |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113593184A (en) * | 2021-07-21 | 2021-11-02 | 深圳市传海科技有限公司 | Method for restraining fatigue driving and related device |
| CN113891049A (en) * | 2021-11-02 | 2022-01-04 | 湖北车安达信息科技有限公司 | Vehicle-mounted video monitoring data processing method and device based on big data |
| CN115137941A (en) * | 2022-06-21 | 2022-10-04 | 北京中科心研科技有限公司 | Emergency awakening method and system based on fatigue recognition model and electrical stimulation |
| CN115137941B (en) * | 2022-06-21 | 2024-05-03 | 北京中科心研科技有限公司 | Emergency awakening method and system based on fatigue recognition model and electric stimulation |
| CN115352450A (en) * | 2022-08-12 | 2022-11-18 | 重庆长安汽车股份有限公司 | Vehicle driving assistance method and device, vehicle and storage medium |
| CN120116949A (en) * | 2025-05-13 | 2025-06-10 | 上海朗晖慧科技术有限公司 | A vehicle fatigue monitoring device based on dynamic recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110544361B (en) | 2021-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110544361B (en) | Monitoring and warning system with sudden disease monitoring and warning functions and fatigue driving monitoring and warning correction system | |
| US10572746B2 (en) | Apparatus detecting driving incapability state of driver | |
| CN109410524B (en) | A driver fatigue monitoring system and monitoring method thereof | |
| CN106585624B (en) | Driver status monitoring method and device | |
| CN107640152B (en) | A reliability control device and method for a lane keeping assist system | |
| US20140121927A1 (en) | Vehicle emergency evacuation device | |
| US20150379362A1 (en) | Imaging device based occupant monitoring system supporting multiple functions | |
| CN106515706A (en) | Vehicle safety driving control method and vehicle safety driving control system | |
| CN114512030A (en) | Driving incapability state detection device for driver | |
| CN112896175B (en) | Driving behavior analysis system and method | |
| CN106530621A (en) | Safe driving method and apparatus based on smart wearable equipment | |
| JP2016045714A (en) | On-vehicle control device | |
| TW201924982A (en) | Method for vehicle control combined with physiological monitoring capable of interfering with the vehicle control and performing physiological monitoring to provide threshold values of different degrees for those physiological signals and corresponding warnings | |
| CN107719065B (en) | Dangerous driving prevention and intelligent driving environment adjusting method and system | |
| CN112932007A (en) | Intelligent helmet and control method thereof | |
| CN104376684A (en) | Monitoring system capable of preventing fatigue driving | |
| CN110525444A (en) | A kind of processing method and processing device for driver's body abnormality situation | |
| CN118289021A (en) | Driving safety system based on multi-source data fusion and intelligent control method | |
| JP2009166783A (en) | Symptom estimation device | |
| CN115366881B (en) | Dangerous driving intervention method, dangerous driving intervention equipment, dangerous driving intervention storage medium and dangerous driving intervention device | |
| CN207328380U (en) | Automobile accounts for seat, safety belt setting fastening identification long-distance monitorng device and system | |
| CN211427470U (en) | Fatigue driving reminding device | |
| CN209921399U (en) | Vehicle steering wheel with driving auxiliary device | |
| CN119279538A (en) | Intelligent monitoring and safety emergency system based on vehicle driver health status | |
| CN107169481A (en) | A kind of based reminding method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |