CN110301921A - System and method for carrying out drowsiness detection and intervention - Google Patents

Abstract

It provides a kind of for detecting the whether sleepy vehicle-mounted monitoring of the driver in vehicle and interfering system by the multiple physiological signals for monitoring driver.Vehicle mounted surveillance and interfering system include at least processor and device, the device is desirably integrated into safety belt or is attached on safety belt as discrete hardware equipment, which includes at least ECG sensor and/or pulse transducer, respiration transducer, acceleration transducer, filtering system and intervention module.The filtering system further includes for inhibiting noise and reducing one or more filters of motion artifacts.The processor is configured as the physiological signal that will test and is compared with the signal in the study module for being stored in vehicle-mounted monitoring and interfering system, with the sleepy state of determination.If it is determined that driver is in sleepy state, then caution signal is exported to alert driver.

Description

Systems and methods for sleepiness detection and intervention

technical field

The present disclosure relates generally to in-vehicle monitoring and intervention systems, and more particularly to systems and methods for detecting physiological signals of a driver using a seatbelt through adaptive motion artifact cancellation to determine drowsiness status and intervene.

Background technique

The design and manufacture of the vehicle is mature, and sound guidelines and standards have been established to ensure the safety and flawlessness of the vehicle. However, the drowsy or fatigued state of the driver is responsible for so many accidents and casualties that the strength of the vehicle cannot be avoided. To prevent vehicle accidents, more preemptive measures are needed to detect inattentive or drowsy driving early on.

According to the Sleep in America survey conducted by the National Sleep Foundation (NSF), approximately 60 percent of adult drivers admitted to driving when they felt drowsy in the past year, which could represent as many as 168 million U.S. drivers. experience. In 2014, the National Highway Traffic Safety Administration (NHTSA) identified 846 fatalities related to drowsy driving. This can be due to driver fatigue, the influence of drugs or alcohol, and other unexpected medical conditions such as heart attack, stroke, etc. These hazardous conditions are some of the leading causes of road accidents in the United States and other countries, posing significant risk and danger to drivers, other passengers, occupants of nearby vehicles, and pedestrians.

In view of the issues raised above, various monitoring measures have been used or proposed so far to determine the driver's attentiveness. A conventional method uses 'steering pattern' and 'steering torque' to analyze the mental state of the driver by detecting the steering pattern and lane keeping behavior. However, road geometry, weather conditions, and road conditions can affect steering angles and reduce system accuracy. Another method is an image-based method that captures the driver's head posture, facial expression or eye movement to determine whether the driver is awake or drowsy. However, accuracy can also be affected by artifacts such as the driver wearing sunglasses or the driver having an expressionless face.

In some other applications, heartbeat sensors are embedded in car seats to measure driver stress levels. Typically, car seats monitor the driver's heartbeat through multiple sensors on the backrest surface that detect the heart's electrical impulses. The system actively monitors heart rate and alerts the driver if the driver may fall asleep while driving. However, embedding sensors into car seats adds to the complexity of installation and repair. In most cases, such systems can only be embedded when a car is manufactured, and cannot be added to an existing car. The system's flexibility is also limited and may not be suitable for all types of vehicles.

Therefore, there is a need in the art for an on-board monitoring and intervention system that overcomes the shortcomings of prior art systems, provides accurate measurements of driver drowsiness, and responds quickly to intervene and alert the driver when the driver is drowsy .

Contents of the invention

Exemplary embodiments of the present disclosure provide a method and an in-vehicle detection and intervention system for determining a drowsiness state of a driver in a vehicle. The method includes: a detection process, which may include measuring an electrocardiogram (ECG) signal and/or a pulse signal, a respiration signal, and an acceleration signal; a filtering process for performing noise suppression and adaptive motion artifact removal; and extracting one or more heart rate variability (HRV) parameters from said filtered ECG signal and/or pulse signal, and analyzing said one or more HRV parameters, said filtered The amplitude of the breathing signal and the frequency of the filtered breathing signal are used to determine the drowsiness state of the driver.

According to a further aspect of the present disclosure, acceleration signals of the vehicle are measured by one or more three-axis accelerometers.

According to a further aspect of the present disclosure, in order to reduce motion artifacts on the ECG signal and/or the pulse signal and the respiration signal based on the acceleration signal, one or more adaptive filtering methods and a or a variety of digital filtering methods. The one or more adaptive filtering methods include using one or more adaptive filters, and the one or more digital filtering methods include using one or more finite impulse response (FIR) filters, infinite impulse response (IIR) filter or Kalman filter.

According to another aspect of the present disclosure, in order to extract one or more HRV parameters from the ECG signal and/or pulse signal to analyze and determine whether the driver is drowsy, the RR of the ECG signal and/or pulse signal Intervals perform power spectrum analysis. The one or more HRV parameters include one or more parameters selected from the group consisting of a high frequency (HF) index, a low frequency (LF) index, and a LF/HF ratio.

According to a further aspect of the present disclosure, the step of analyzing the one or more HRV parameters and the breathing signal using a predetermined drowsiness detection algorithm further includes the step of: by one or more processors based on the driver's one or more biometric parameters determining a probabilistic model and/or threshold for the LF/HF ratio; determining, by the one or more processors, characterizing the breathing signal based on the driver's one or more biometric parameters Probability model and/or thresholds for ;

And storing the probability model and/or threshold value of the LF/HF ratio, and the probability model and/or threshold value characterizing the respiratory signal by one or more memory elements in the training module. The predetermined drowsiness detection algorithm determines the LF/HF ratio status by comparing the LF/HF ratio to a probability model of the LF/HF ratio and/or a threshold; and by comparing the filtered breath The signal is compared to a set of intrinsic breathing data in the trained model to determine a breathing condition; thereby, a state of drowsiness of the driver is determined based on the LF/HF ratio condition and the breathing condition.

According to further aspects of the present disclosure, the driver's biometric parameters include one or more parameters selected from the group consisting of the driver's age, gender, body mass index (BMI), and ethnic group.

According to additional aspects of the present disclosure, an onboard detection and intervention system includes one or more processors and devices, wherein the devices include one or more ECG sensors and/or pulse sensors, at least one respiration sensor, and at least one filter. The apparatus also includes one or more three-axis accelerometers; and an intervention module, wherein the intervention module further includes a transmission module for sending an in-vehicle alert or a smartphone alert. The one or more processors are configured to perform a method of processing ECG signals, respiration signals, and acceleration signals to determine a drowsiness state of the driver.

According to a further aspect of the present disclosure, the one or more ECG sensors and/or pulse sensors are spaced a predetermined distance from each other along the harness.

According to a further aspect of the present disclosure, the one or more breathing sensors are placed on a seat belt to measure the breathing pattern of the driver. In some embodiments, the device further comprises a clip for attaching the device as a discrete hardware device to a seat belt of the vehicle. In some alternative embodiments, the device is integrated into the vehicle's seat belt using a plurality of sensors sewn onto the seat belt and a flexible PCB.

The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the detailed description below. Other features, structures, characteristics, and advantages of the present disclosure will be apparent from the description and drawings, and from the claims.

Description of drawings

In the drawings, like reference numbers indicate identical or functionally similar elements throughout the figures, and together with the following detailed description, the accompanying drawings are incorporated in and form a part of this specification, serve to illustrate various embodiments and Various principles and advantages are explained according to the present embodiment.

FIG. 1 is a block diagram schematically showing the overall structure of an on-board monitoring and intervention system according to some embodiments of the present disclosure.

FIG. 2 is a block diagram illustrating a filtering system according to some embodiments of the system disclosed in FIG. 1 .

FIG. 3 is a flowchart illustrating a method for detecting whether a driver in a vehicle is drowsy according to some embodiments of the present disclosure.

Figure 4 is a flowchart illustrating a method for extracting features from an ECG signal after noise filtering according to some embodiments of the system disclosed in Figure 1 .

5 is a top view of an in-vehicle monitoring and intervention device integrated into a seat belt, according to some embodiments of the present disclosure.

6 is a top view of an in-vehicle monitoring and intervention system as a discrete hardware device that can be attached to a seat belt, according to some embodiments of the present disclosure.

FIG. 7 is a side view of the apparatus of FIG. 6 .

8 is a graph showing the relationship between a probabilistic warning function and LF/HF ratio according to the present disclosure.

Skilled artisans will appreciate that elements in the figures, particularly conceptual views, are illustrated for simplicity and clarity and have not necessarily been drawn to scale.

Detailed ways

The following detailed description is merely exemplary in nature and is not intended to limit the disclosure or its application and/or uses. It should be appreciated that a large number of variations exist. The detailed description will enable those of ordinary skill in the art to practice the exemplary embodiments of the present disclosure without undue experimentation, and it should be understood that without departing from the scope of the present disclosure as set forth in the appended claims Various changes or modifications may be made in the function and arrangement of the methods and steps of operations described in the exemplary embodiments.

The present disclosure relates to an onboard monitoring and intervention system. The following terms are used in the specification and appended claims. The term "vehicle" as used herein includes, but is not limited to, automobiles, buses, trucks, trains, cable cars, ships, ferries, ships, airplanes, helicopters, and the like. Accordingly, "pilot" as used herein may include captains, pilots, and the like.

The term "electrocardiogram" or "ECG" as used herein refers to a procedure or device that detects the electrical activity of the heart using electrodes placed near the driver's heart, but preferably without direct contact with the driver's skin.

The term "pulse sensor" as used herein refers to a program or device that monitors the pulse using a program or device placed at a location capable of measuring the driver's pulse, which may be an electrode near the heart to detect the heart pulse, but preferably does not need to be associated with The driver's skin is in direct contact, or it could be a wearable device located on the wrist.

The term "heart rate variability" or "HRV" as used herein is the physiological phenomenon of temporal interval variation of the autonomic nervous system activity of the heart. By extracting the RR interval from the ECG signal and/or pulse signal and performing power spectrum analysis on it, the ECG signal and/or pulse signal can be divided into one or more HRV parameters, including high frequency (HF) index, low frequency ( LF) index and very low frequency (VLF) index. Unless otherwise stated or indicated, HF ranges from 0.15 Hz to 0.4 Hz in the RR interval, LF ranges from 0.04 Hz to 0.15 Hz in the RR interval, and VLF ranges from 0.003 Hz in the RR interval to 0.04Hz. The term "LF-HF ratio" as used herein means a measure of sympathovagal balance.

The term "microcontroller" or "MCU" as used herein includes central processing units, microprocessors, microcomputers, single-chip computers, cloud computing systems, integrated circuits, etc., and systems including the above.

The term "smartphone" as used herein includes any mobile device such as a mobile phone, tablet, phablet, smart watch or associated operating system capable of running programmed applications and communicating with the present on-board monitoring and intervention system (IOS, Android, etc.) for other portable devices.

As used herein, the term "application" is an abbreviation of the term "application software", which means a software program that can run on a smartphone and is designed to be used by itself, in conjunction with another software application, and/or as The addition of another software application to perform a specific task or function.

It should be understood that throughout the specification and claims herein, when it is described that an element is "coupled" or "connected" to another element, the element may be "directly coupled" or "directly connected" to the other element or via a third element. "Coupled" or "connected" to another element. In contrast, it will be understood that when an element is described as being "directly coupled" or "directly connected" to another element, there are no intervening elements present. The connections between elements may be physical, logical, electrical or any combination thereof.

Part A briefly introduces a method for detecting whether a driver in a vehicle is drowsy based on multiple physiological signals of the driver and the overall structure of the on-board monitoring and intervention system. Part B presents the filtering system for suppressing noise and reducing motion artifacts. Part C further explains how to determine the drowsiness state of the driver. Part D explains the structure of the equipment used to perform on-board monitoring and intervention.

A. Overall structure of the on-board surveillance and intervention system

Broadly, the present disclosure provides an on-board monitoring and intervention system for determining the drowsiness state of a driver in a vehicle, the system including one or more ECG sensors 210 and/or pulse sensors, one or more respiration sensors 220 , a three-axis accelerometer 230 , a filtering system 300 including one or more filters, a feature extraction module 410 , a drowsiness detection module 420 , a training module 430 and an intervention module 510 . The term "sensor" is used to refer generically and collectively to ECG sensor 210 and/or pulse sensor, respiration sensor 220 and acceleration sensor 230 . The term "filter" is used to refer generically and collectively to signal filters 311,312, adaptive filters 331,332 and finite impulse response (FIR) filters 341,342. In some embodiments, filtering system 300 includes one or more filters for suppressing noise and removing motion artifacts of vehicle, driver, or both motion.

When a person is driving a vehicle, it is important that the person fasten the seat belt 510 . The seat belt 510 is designed to reduce the impact on the driver when the vehicle collides or brakes suddenly. Therefore, seat belt 510 can prevent death or injury in an accident. Since the seat belt 510 is the only object that is in direct contact with the driver's body at all times, it can serve a more preventive function than its usual life-saving purpose in the event of an accident. Therefore, the present disclosure provides a way to determine the driver's mental state and monitor the driver's mental state before any danger occurs by measuring the driver's heartbeat 110 (S210) and the driver's breathing pattern 120 (S220) using the sensors on the seat belt 510. methods for personnel to intervene or warn. The system is designed to monitor heart rate and sound an alert if the driver may fall asleep while driving. Furthermore, motion sensors (eg, three-axis accelerometer 230 ) are also integrated into seat belt 510 to measure vehicle motion 130 ( S230 ) to significantly reduce any inaccuracies caused by motion artifacts.

Referring now to FIG. 1 , a block diagram illustrating the general architecture of an on-board monitoring and intervention system according to some embodiments of the present disclosure is shown. Sensors in the system detect a number of physiological signals of the driver in the vehicle. In this disclosure, the physiological signals represent the driver's heartbeat 110 and breathing pattern 120 . Accordingly, the on-board monitoring and intervention system may determine the driver's mental state based on measurements of heartbeat 110 (and/or pulse) and breathing pattern 120 using a predetermined drowsiness detection algorithm. Although the ECG signal is used as one of the measurement signals in this embodiment, the measurement of the pulse signal may also be used to determine the mental state of the driver.

One or more ECG sensors and/or pulse sensors have been developed, including a non-contact electrocardiogram (ECG) sensor 210 that monitors the heart through the use of non-contact sensing electrodes that are Do not directly contact the driver's skin. In some embodiments, the ECG sensor and/or pulse sensor comprises an electrode sensor or an optical sensor. In some embodiments of the present disclosure, two or more ECG sensors 210 are placed on the seat belt 510 to measure the driver's heartbeat 110 (S210) to obtain the continuous and periodic Measurements; two or more pulse sensors can also be placed on the seat belt 510 or at the driver's wrist as a wearable device to measure the driver's pulse, obtaining continuous and periodic measurements of the pulse signal . One or more sensors are placed at predetermined intervals at various locations near the driver's heart to improve the quality of the measurements collected. In order to improve the QRS complex of the acquired ECG signal h(t), the ECG sensors are arranged at a distance of at least 10 cm from each other along the safety belt.

The breath sensor 220 is capable of measuring the driver's inhalation and exhalation. It is possible but impractical to use nasal and oral sensors to measure airflow or volume for the purpose of generally monitoring the driver's physiological signals. In the present disclosure, one or more respiration sensors 220 are placed on the harness to capture body motion during inhalation and exhalation. Each sensor may be an abdominal breathing motion tracker placed in an area close to the driver's chest or abdomen so that the driver's breathing pattern 120 may be continuously monitored at a constant sampling rate. In one embodiment, the constant sampling rate is 128 samples per second. This provides the respiration signal r(t) for further analysis. Respiratory characteristics include the waveform, amplitude, frequency, inspiratory and expiratory slope, etc. of the respiratory signal.

The three-axis accelerometer 230 measures the vehicle motion 130 and tracks the acceleration signal a(t) to improve the accuracy of the acquired ECG signal h(t) and the acquired respiration signal r(t). It can also be used for pulse signals in the presence of pulse signals. This can significantly reduce motion artifacts that can be produced by the vehicle's motion or speed changes. In certain embodiments, the present disclosure uses other motion detection devices including 3-axis gyro sensors, angular position sensors, digital angle sensors, 1-axis accelerometers, 2-axis accelerometers, 4-axis accelerometers, 5-axis accelerometers, axis accelerometer, 6-axis accelerometer, etc.), or other vehicle monitoring systems (including vehicle speed monitoring systems, vehicle speedometers, devices using the Global Positioning System (GPS), etc.), to collect acceleration signals a(t).

In order to realize accurate measurement of ECG signal h(t) and respiratory signal r(t), noise filtering S310 is indispensable. Pulse signal noise can also be filtered in the presence of a pulse signal. The present disclosure utilizes filtering system 300 to suppress noise and perform adaptive motion artifact removal. Filtering system 300 includes one or more filters selected from the group consisting of signal filters 311,312, adaptive filters 331,332 and FIR filters 341,342. In some embodiments, the filtering system 300 and the filters therein may be discrete components, or be composed of a microcontroller unit (MCU), a custom integrated circuit, a field programmable gate array (FPGA), other semiconductor devices, or any of the foregoing. Appropriate combination to achieve.

As shown in FIG. 2 , a block diagram illustrating a filtering system 300 is depicted. A signal filter 311 is used to perform a first stage noise filtering on the input ECG signal h(t), and another signal filter 312 is used to perform a second stage noise filtering on the input respiration signal r(t). The two signal filters 311, 312 can be implemented by using 50/60 Hz notch filters, band-stop filters or band-pass filters to select the frequency range to be extracted. Signal filters 311, 312 are capable of rejecting other noise signals or harmonics of higher or lower frequencies. The center frequency can be adjusted or adjusted according to the actual situation in the corresponding application, and the bandwidth can be adjusted or adjusted. In some embodiments, the signal filter 311 for the ECG signal has a spread of 0.5 Hz to 40 Hz in order to obtain h(t). The signal filter 312 for the respiration signal has a spread of 0.1 Hz or 10 Hz in order to obtain r(t). Other frequency ranges may be used for the signal filters 311, 312 without departing from the scope or spirit of the invention.

Since heart rate variability (HRV) is particularly sensitive to artifacts, artifacts can lead to significant errors in determining a driver's drowsiness state. It is important to eliminate unwanted elements related to vehicle motion in the ECG. Similarly, the same filtering system used for noise removal can also be used for the respiration signal to improve signal quality. In the present disclosure, the combination of adaptive filter 331 and FIR filter 341 is used to significantly reduce motion artifacts or other electrophysiological signals on the ECG signal h(t). It can also be used for pulse signals in the presence of pulse signals. The acceleration signal a(t) is related to motion artifacts and is used to compensate for the motion of the vehicle. Similarly, the combination of adaptive filter 332 and FIR filter 342 is used to significantly reduce motion artifacts or other electrophysiological signals on the respiration signal r(t). Send the filtered ECG signal eh(t) and the filtered respiratory signal er(t) to the feature extraction module 410 and the drowsiness detection module 420, and use the filtered ECG signal at the feature extraction module 410 and the drowsiness detection module 420 eh(t) and the filtered respiration signal er(t) to extract one or more HRV parameters and determine the drowsiness state of the driver.

Referring now to FIG. 1 again, the filtered ECG signal eh(t) is sent to and processed by the feature extraction module 410 to extract the RR interval from the filtered ECG signal (S411) and calculate the RR Power spectral analysis is performed to extract one or more HRV parameters from the filtered ECG signal (S410), preferably in both time and frequency domains. It can also be used for pulse signals in the presence of pulse signals. Specifically, the HRV parameters include one or more parameters selected from the group consisting of a high frequency (HF) index (S412), a low frequency (LF) index (S413), and a very low frequency (VLF) index. In some embodiments, other parameters and indices can be extracted (S414), for example, the standard deviation (SDNN) of all normal sinus (NN) interval indices, the square root of the sum of mean square deviations of adjacent NN intervals (RMSSD ), the standard deviation of the difference between adjacent NN intervals (SDSD), the number of heartbeats with a difference between adjacent NN intervals greater than 50ms in the entire record (NN50), the percentage of NN50 in the total number of NN intervals (PNN50) or any suitable combination of the foregoing. The drowsiness detection module 420 uses these HRV parameters and the filtered respiration signal er(t) for subsequent mental state determinations.

Mental state determination S420 refers to identifying a driver's consciousness or drowsiness state by analyzing a plurality of physiological signals including HRV parameters and respiration signals using a predetermined drowsiness detection algorithm. Methods for determining a driver's drowsiness state are discussed in Section C of this disclosure.

In certain embodiments, the on-board monitoring and intervention system may include a training module 430 for storing and tracking trends in measured driver-specific physiological signals. Training module 430 includes one or more memory elements. The memory element stores one or more threshold values of the HRV parameter, the threshold value of the magnitude of the respiration signal, and the threshold value of the frequency of the driver's respiration signal in an array of memory cells. In some embodiments, the memory unit may be a device-readable storage medium such as a non-transitory storage device. The memory unit may be, for example, a digital memory, a magnetic storage medium, an optically readable digital storage medium, a semiconductor device, or any suitable combination of the foregoing. More specific examples of storage devices include the following: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disc ROM (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. The one or more processors calculate thresholds for one or more HRV parameters, thresholds for the magnitude of the driver's breathing signal, and driver The frequency threshold of the respiration signal.

The intervention module 510 receives a signal from the drowsiness detection module 420 when it is determined that the driver in the vehicle is drowsy. Intervention by sending an alert signal S510, preferably to a dashboard in the vehicle 610 or to a connected smartphone 620 via Bluetooth or other wireless communication technology, can warn the driver of the danger and attempt to Wake the driver from a drowsy state.

In some embodiments, the filtered ECG signal eh(t) (in other embodiments, may be a filtered pulse signal) and the filtered respiration signal er(t) are digitized and transmitted by the transmission module to the smartphone to extract HRV parameters (S410), determine drowsiness status (S420) and send an alarm signal for intervention (S510). The application program in the smartphone is designed to receive the signal sent by the seat belt or the discrete hardware device through the feature extraction module 410 , drowsiness detection module 420 , training module 430 and intervention module 510 within the application program. The app can determine the state of drowsiness and issue an in-vehicle alert 610 or a smartphone alert 620 through the transmission module in the intervention module 510 (S510).

In some alternative embodiments, feature extraction module 410, drowsiness detection module 420, training module 430, and intervention module 510 may be integrated and included in a microcontroller unit (MCU), custom integrated circuit, digital signal processor (DSP) , field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable I/O devices, other semiconductor devices, or any suitable combination of the foregoing. The device may determine the state of drowsiness and issue an in-vehicle alert 610 or a smartphone alert 620 by a transmission module in the intervention module 510 (S510).

B. Filtering system for suppressing noise and reducing motion artifacts

FIG. 2 shows the structure of a filtering system 300 . The purpose of the filtering system 300 is to suppress noise and remove motion artifacts due to motion of the vehicle, driver, or both. By looking closely at motion artifacts caused by vehicle motion, we can assume that the driver's motion can trigger the three-axis accelerometer 230, giving equation (1):

y(t)=w _x (k) a _x (t)+w _y (k) a _y (t)+w _z (k) a _z (t) (2)

According to equation (2), y(t) is the signal output from the FIR filters 341, 342, and is denoted as yh(t) when the filtered signal is an ECG signal, and when the filtered signal is a respiratory signal , denoting it as yr(t). In other embodiments, the filtered pulse signal can be used to perform the following calculations. Since the acceleration signal a(t) is related to motion artifacts, we can derive weights [w _x (k)w _y (k)w _z (k)] for heartbeat and respiration separately, where w _- (k) is 1 ×M matrix, so:

H(t)-yh(t)=eh(t) (3)

R(t)-yr(t)=er(t) (4)

Both eh(t) and er(t) are relatively clean ECG and respiration signals.

Adaptive filters 331, 332 as used herein may be implemented by least mean square (LMS) adaptive filters, recursive least squares (RLS) adaptive filters, or gradient adaptive Lagrid-Gallery (GALL) filters .

B1.LMS adaptive filter

By using an LMS adaptive filter, the difference between the desired signal and the actual measured signal is used to determine the optimal filter coefficients. In order to obtain a clean ECG signal, we must minimize the cost function J(t) using equation (5):

By applying stochastic gradient descent, we can get EQN(6):

by using with Proportional quantities continue from w _m to w _m+1 , we can get equation (7):

w _m+1 ＝w _m +μe(t)a(tk) (7)

In the formula:

μ is an arbitrary value from about 0.1 to 0.0001;

m is the index referring to the filter element; and

J represents the cost function for the amount of difference between the desired signal and y.

B2.RLS adaptive filter

An alternative approach using an RLS adaptive filter can provide a similar effect by recursively finding the coefficients that minimize the weighted linear least squares cost function associated with the ECG signal h(t) and the respiration signal r(t), consider the ECG signal Both h(t) and the respiration signal R(t) are determined.

R ^-1 (t)＝λ ^-1 [R ^-1 (t-1)-k(t)a ^T (t)R ^-1 (t-1)] (11)

Therefore, the filter coefficients can be derived as:

b(t)=b(t-1)+k(t)ε(t) (12)

In the formula:

b(t) is the filter coefficient;

λ is the forgetting factor;

a(t) is the input noise signal;

ε(t) is the error filtered signal.

B3. GALL filter

For the case of using the GALL filter, the structure of the conventional GALL filter shown in FIG. 4 of the reference [1] and disclosed in Table I of the reference [2] can be applied. The GALL filter can effectively reduce the motion artifact component of the respiratory signal and improve the signal quality.

In one embodiment, a GALL filter is used to filter the ECG signal.

C. Determining the driver's drowsiness status

In order to determine the drowsiness state of the driver, the feature extraction module 410 is programmed to extract the RR interval from the filtered ECG signal eh(t) (S411), and perform time domain analysis and frequency domain analysis to extract one or more HRV parameter (S410). In a typical ECG signal, there are different patterns that carry useful information for identifying the driver's mental state, including P waves, QRS complexes, T waves, and U waves. In order to obtain accurate measurements, pattern recognition of the ECG signal is particularly important. The feature extraction module 410 may first obtain the RR interval by analyzing the ECG signal (or pulse signal) using a nonlinear method, wherein the RR interval is calculated based on the time interval calculation of two consecutive R peaks of the ECG signal (or pulse signal) .

By performing time-domain analysis, the feature extraction module 410 can obtain one or more HRV parameters or other indicators (S414), including SDNN, RMSSD, SDSD, NN50 and PNN50. SDNN is the standard deviation of the mean NN interval calculated over a short period of time (typically 5 minutes). RMSSD is the square root of the mean of the squared differences between adjacent NNs. SD is the standard deviation of the difference between adjacent NNs. NN50 is the number of consecutive NN pairs whose difference is greater than 50ms. PNN50 is the percentage of NN50 in the total NN

By performing frequency-domain analysis on the RR intervals, the feature extraction module 410 can obtain a count of the number of NN intervals at each frequency band (usually including HF, LF, and VLF), so that the normalized high frequency ( nHF), normalized low frequency (nLF), and the ratio of LF to HF (LF/HF):

nHF=HF/(TP-VLF)*100 (13)

nLF＝LF/(TP-VLF)*100 (14)

HF%＝100*HF/(LF+HF) (16)

LF/HF and HF% are the main factors in determining the drowsiness state of the driver because they can change significantly when the driver goes from awake to sleep cycle. If the driver did not get enough sleep, eg less than 4 hours of sleep in the previous night, the corresponding HF% value for the driver may be significantly higher compared to the HF% for adequate sleep. Therefore, multiple thresholds for identifying whether a driver has adequate sleep can be inferred using the LF and HF metrics. In certain embodiments, HF% is used specifically to classify whether a person is getting enough sleep.

However, all HRV metrics can vary significantly for different drivers. Factors including age, gender, body mass ratio (BMI), and driver ethnic group can affect all HRV metrics. Given the wide variation of all HRV indices, a classification for individual factors was employed to determine specific thresholds for each driver based on the training data.

Advantageously, the present disclosure utilizes predetermined test set distributions to determine a normal distribution for each HRV parameter and breathing parameter for each driver. Since it was found that the LF/HF of male drivers is significantly higher than that of female drivers, and the age of the driver has an inverse relationship with LF/HF. Therefore, classification can be obtained by characterizing the changes. The resulting probability model and/or thresholds may provide a range of typical situations for a particular set of drivers and assume that individual drivers in the set follow a normal distribution. Additionally, with the determined probability models and/or thresholds, the training module 430 can fine-tune the distribution to reflect individual biometrics to further improve accuracy.

In some embodiments, the training module 430 stores patterns of various biological parameters, such as HF index, LF index, LF/HF, and other breathing parameters in different drowsy states, where LF/HF is the key to determine driver drowsiness. The most important parameter of the state. The training module identifies the state of drowsiness to which the driver belongs and correlates the biometric parameters, in particular the initially recorded HRV index. The drowsiness state of the driver will be used to activate a warning function based on the probability of the drowsy state, ie a probabilistic warning function.

The drowsiness detection module 420 analyzes the driver's drowsiness state from the training module 430 in relation to the measured HRV parameters and the measured breathing parameters. The LF/HF condition is determined by comparing the measured LF/HF with the LF/HF in the training dataset as shown in Figure 8. When the probabilistic warning function is 1, the driver is expected to be drowsy and it is necessary to warn the driver. In one embodiment, there is a predetermined threshold, such as 0.7, so that when the probabilistic warning function is equal to or greater than this threshold, the driver will be warned. In this approach, LF/HF is used to indicate how likely it is that the driver is in a drowsy phase when compared to the data in the training module 430 . Whether to warn is determined according to the probabilistic warning function obtained from the training module 430 . On the other hand, the respiratory condition is determined based on the correlation r _xy of the respiratory signal and the sleepy respiratory signal, and the respiratory condition is calculated to infer a probabilistic warning function. The probabilistic warning function curve consists of the parameters shown in equation (17):

In the formula:

x is the intrinsic respiration signal dataset;

y is the newly recorded respiration data.

The parameters of the obtained respiratory signal are compared with the data in the training module 430 to obtain a probabilistic warning function to determine whether to warn.

In some embodiments, the drowsiness detection module will use a machine learning algorithm to judge whether the driver is in a drowsy state according to the input parameters. The input parameters of the machine learning model are the above-mentioned physiological parameters of the driver and parameters acquired through the training module in the data set. Machine learning models include supervised learning models or algorithms such as deep learning.

To improve the performance of drowsiness detection, big data analysis can be employed to collect data from users with similar biometric parameters, including HRV index and respiratory index, through the network by a mobile application. Other personal information, including age, gender, BMI, eating habits, sleeping habits, medication intake, and workload of the day can also be parameters in machine learning algorithms. The system, combined with the above-mentioned machine learning algorithm, can more accurately estimate the driver's drowsiness state.

D. Structure of equipment used to perform on-board detection and intervention

5 to 7 illustrate different views and structures of an apparatus for performing on-board monitoring and intervention according to embodiments of the disclosure. FIG. 5 is an exemplary system integrated into a harness 510 , while FIGS. 6 and 7 illustrate separate hardware devices that may be attached to the harness 510 .

The onboard monitoring and intervention system may include one or more processors and devices including one or more ECG sensors 210, one or more respiration sensors 220, accelerometer 230, MCU 520, battery 530, universal serial bus (USB) port 540 , clip 550 and flex cable 560 . MCU 520 may further include other circuitry for performing noise or motion artifact filtering and wireless communication. The device may also include one or more pulse sensors, or one or more ECG sensors may be replaced by a pulse sensor.

FIG. 5 shows certain embodiments of a device integrated into a seat belt 510 . There are two ECG sensors 210 , one of which is connected to the MCU 520 by a flexible cable 560 . The flexible cable 560 allows great flexibility in adjusting the position of the ECG sensor 210 so that the system can be adapted to people of different body sizes. Another ECG sensor 210, respiration sensor 220, and accelerometer 230 may be integrated into the harness 510 and connected to the MCU 520 with wires, e-textiles, or other conductive fabric (eg, copper nylon fabric). This has the advantage that the harness remains flexible while the cables do not protrude. The device is powered by battery 530 . The pulse sensor can be integrated into the seat belt 510 and connected to the MCU 520 by a flexible cable, or placed on the driver's wrist and connected to the MCU 520 by wire or wirelessly.

Alternatively, as shown in FIGS. 6 and 7 , the device may be a discrete hardware device that may be connected to the harness 510 . The ECG sensors 210 are placed diagonally so that the distance between them can be maximized. A clip 550 is attached at the bottom of the device so that the device can be securely attached to the seat belt 510 and freely moved along the seat belt 510 closer to the driver's heart. And this setting can also be adjusted according to the body shape and needs of different people. In both configurations, the device is also equipped with a USB port 540 for charging the battery 530 or transferring data for recording. USB port 540 may be a micro USB port, USB-C port, mini USB port, or other type of port connector.

In some embodiments, the MCU 520 includes a feature extraction module 410 , a drowsiness detection module 420 , a training module 430 and an intervention module 510 . The physiological signals are processed by the MCU 520 to determine the drowsiness state of the driver. If the driver is determined to be drowsy, an in-vehicle alert 610 or a smartphone alert 620 is issued by the transmission module to alert the driver.

In some alternative embodiments, MCU 520 includes only the transport module. The filtered physiological signals are digitized and sent to a smartphone for further processing. The device and the smartphone may be connected via any type of connection or network, including local area network (LAN), wide area network (WAN), or via other devices, for example, via the Internet using an Internet Service Provider (ISP), via other wireless connections (eg, near field communication) or via a hardwired connection (eg, USB connection). In some alternative embodiments, a smartphone may be used as an intermediary device, which may further transmit the filtered physiological signals received from the device to processors of other devices without processing.

In some embodiments of the present disclosure, circuits in the system may be at least partially implemented by software programs, transistors, logic gates, analog circuit blocks, semiconductor devices, other electronic devices, or a combination of any of the foregoing circuit structures. Since some circuits may be implemented in software, actual connections and structures may vary depending on how the software is programmed.

It will be apparent to those skilled in the art that various modifications and changes can be made in the structure of this invention without departing from the scope or spirit of the invention. In view of the foregoing description, it is intended that the present invention cover such modifications and variations of the present invention if they come within the scope of the following claims and their equivalents.

E. Cited References

The following documents are cited in this patent application. References [1]-[2] are incorporated herein by reference.

[1] Zhengbo Zhang et al., "Adaptive motion artefact reduction in respiration and ECG signals for wearable healthcare monitoring systems", Medical & biological engineering & computing, 52: 1019-1030, 2014.

[2] Zoran Fejzoet et al., "Adaptive Laguerre-lattice Filters", IEEE Transactions on Signal Processing, Vol. l45, No. 12, December 1997.

Claims (19)

1. a kind of method for determining the sleepy state of the driver in vehicle, which is characterized in that the described method includes:

Detection process, comprising:

By one or more ECG sensors and/or pulse transducer measure the driver electrocardiogram (ECG) signal and/ Or pulse signal；

Breath signal is measured based on the breathing pattern of the driver by one or more respiration transducers；And

Measure the acceleration signal of the vehicle, the driver or both；

For executing the filtering of adaptive motion artifact elimination, comprising: reduce the ECG letter based on the acceleration signal Number and/or pulse signal and the breath signal on motion artifacts, to obtain filtered ECG signal and/or arteries and veins respectively It fights signal and filtered breath signal；And

The determination process executed by machine learning algorithm, comprising:

One or more heart rate variability (HRV) parameters are extracted from the filtered ECG signal and/or pulse signal；With And

One or more of HRV parameters and the breath signal are analyzed using scheduled drowsiness detection algorithm to drive described in determination The sleepy state for the person of sailing.

2. the method according to claim 1, wherein measuring the vehicle by one or more three axis accelerometers Acceleration signal.

3. the method according to claim 1, wherein the filtering further includes by one or more signals Filter is filtered to inhibit noise the ECG signal and/or pulse signal and the breath signal.

4. the method according to claim 1, wherein based on the acceleration signal reduce the ECG signal and/ Or the step of motion artifacts on pulse signal and the breath signal, is the following steps are included: use one or more adaptive filters Wave method and one or more digital filtering methods are filtered the motion artifacts of the vehicle, the driver or both.

5. according to the method described in claim 4, it is characterized in that, one or more adaptive filter methods include using One or more sef-adapting filters.

6. the method according to claim 1, wherein from the filtered ECG signal and/or pulse signal The step of extracting one or more HRV parameter includes: between extraction RR in the filtered ECG signal and/or pulse signal Phase；And power spectrumanalysis is carried out to the RR interphase.

7. the method according to claim 1, wherein one or more of HRV parameters include from by high frequency (HF) the one or more parameters selected in the group of index, low frequency (LF) index and LF/HF ratio composition.

8. the method according to claim 1, wherein described use scheduled drowsiness detection algorithm analysis described one The step of a or multiple HRV parameters and the breath signal, is further comprising the steps of:

The LF/HF ratio is determined based on one or more biometric parameters of the driver by one or more processors Threshold value；

One or more biometric parameters by one or more of processors based on the driver determine the breathing The threshold value and/or probabilistic model of signal；And

By one or more memory elements in training module store the LF/HF threshold value and breath signal threshold value and/or Probabilistic model.

9. the method according to claim 1, wherein the scheduled drowsiness detection algorithm determines:

LF/HF ratio condition, wherein by by the threshold value and/or probabilistic model of the LF/HF ratio and the LF/HF ratio It is compared to determine the LF/HF ratio condition；And

Breath state, wherein by by the threshold value and/or probabilistic model of the filtered breath signal and the breath signal It is compared to obtain the breath state；

The sleepy state of the driver is determined based on the LF/HF ratio condition and the breath state as a result,.

10. according to the method described in claim 8, it is characterized in that, the biometric parameter of the driver includes from by institute State the one or more parameters selected in the group of age of driver, gender, body mass index (BMI) and racial group's composition.

11. a kind of vehicle-mounted detection and interfering system, at the one or more including the sleepy state for determining vehicle driver Manage device and device, which is characterized in that described device includes:

One or more ECG sensors and/or pulse transducer；

One or more respiration transducers；And

The one or more filters selected from the group being made of traffic filter, sef-adapting filter and FIR filter；

Wherein:

One or more of processors are configured as executing processing ECG signal and/or pulse signal, breath signal and acceleration The method of signal determines the sleepy state of driver with determination process according to claim 1.

12. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that described device further include one or Multiple three axis accelerometers；And intervention module.

13. vehicle-mounted detection according to claim 12 and interfering system, which is characterized in that the intervention module further includes using In the transmission module for sending interior warning or smart phone warning.

14. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that one or more of ECG are passed Sensor and/or pulse transducer are spaced each other preset distance along the safety belt.

15. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that by one or more of breathings Sensor is placed on safety belt, to measure the breathing pattern of the driver.

16. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that described device further includes for inciting somebody to action Described device is attached to the clip of the safety belt of the vehicle as discrete hardware device.

17. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that be integrated into described device described In the safety belt of vehicle.

18. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that using electric wire, electronic textile, One or more of processors are connected to one or more of ECG sensors by copper nylon fabric or other conductive fabrics And/or pulse transducer, one or more of respiration transducers and one or more of filters.

19. vehicle-mounted detection according to claim 11 and interfering system, which is characterized in that the pulse transducer is included in On the wearable device of driver's wrist.