A Portable Multi-Modal Cushion for Continuous Monitoring of a Driver’s Vital Signs
<p>Model of active electrode (altered from [<a href="#B17-sensors-23-04002" class="html-bibr">17</a>]).</p> "> Figure 2
<p>Principle of reflective PPG (inspired by [<a href="#B37-sensors-23-04002" class="html-bibr">37</a>]).</p> "> Figure 3
<p>Principle of MIM (altered from [<a href="#B19-sensors-23-04002" class="html-bibr">19</a>]).</p> "> Figure 4
<p>System overview. On the left side, a block diagram of the system is depicted. The orange highlighted block depicts the top side of the PCB (shown on the top right). The blue highlighted block depicts the bottom side of the PCB. On the right side, the cushion with the controller box is depicted.</p> "> Figure 5
<p>Simulator and protocol.</p> "> Figure 6
<p>Workflow for processing of signals. Please note that the peak detection for the cECG signals is performed with the Pan–Tompkins algorithm and the algorithm of Brüser et al. is used to extract the HR of the SCG.</p> "> Figure 7
<p>Signals with good quality. The reference peaks of the conductive ECG and impedance pneumography are shown as dashed red or blue lines, respectively.</p> "> Figure 8
<p>Signals with bad quality. The reference peaks of the conductive ECG and impedance pneumography are shown as dashed red or blue lines, respectively.</p> "> Figure 9
<p>Example of unreliable respiratory reference. The respiratory signals extracted from rPPG<math display="inline"><semantics> <msub> <mrow/> <mn>1</mn> </msub> </semantics></math> and rPPG<math display="inline"><semantics> <msub> <mrow/> <mn>3</mn> </msub> </semantics></math> are shown for comparison.</p> "> Figure 10
<p>AUC of HR for different stages for cECG and reflective PPG. The bar shows the median across all participants and the lines show the standard deviation.</p> "> Figure 11
<p>AUC of RR for different stages for cECG, MIM and reflective PPG. The bar shows the median across all participants and the lines show the standard deviation.</p> "> Figure 12
<p>AUC of RR for different stages for SCG. The bar shows the median across all participants and the lines show the standard deviation.</p> ">
Abstract
:1. Introduction
- An integrated, portable device with dedicated hardware for measuring HR and RR unobtrusively.
- A detailed technological description of the integration with respect to all components.
- A publicly available dataset of the recorded sensor data in a real-world scenario without constraints on clothes and movement. The data should enable researchers to verify the presented results, test their own sensor fusion algorithms and contribute to the training of machine learning models. Furthermore, the data should contribute to the data-sharing paradigm.
- An analysis of the vital signs showing high coverage and high quality for HR and fair quality for RR with respect to each modality independently. The analysis should give new insights into problems and opportunities for each modality with respect to different driving situations.
2. Materials and Methods
2.1. Capacitive Electrocardiography
2.2. Reflective Photoplethysmography
2.3. Magnetic Induction Measurement
2.4. Seismocardiography
3. System Setup
3.1. 4xU Sensors
3.2. Controller Box
4. Experimental Evaluation
- Driving without talking to simulate a single driver.
- Controlled movements, which could be expected during driving, i.e., head torsion left/right, body rotation left/right, adjusting the position on the seat, leaning forward. After each movement, a pause of around 10 was made.
- Driving while talking to the study staff to simulate a passenger.
- Sitting in the seat without driving or talking to obtain a clean signal for reference.
4.1. Preprocessing
4.2. Extraction of Vital Signs
5. Results
5.1. Qualitative Results
5.2. Quantitative Results
5.2.1. cECG
5.2.2. Reflective PPG
5.2.3. MIM
5.2.4. SCG
6. Discussion
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
AUC | Area under Curve |
BRV | Breathing rate variability |
cECG | Capacitive electrocardiography |
DRL | Driven right leg |
ECG | Electrocardiography |
HR | Heart rate |
HRV | Heart rate variability |
IBI | Inter-beat interval |
IR | Infrared |
IRI | Inter-breath interval |
I2C | Inter-Integrated Circuit |
LED | Light-emitting diode |
MDPI | Multidisciplinary Digital Publishing Institute |
MIM | Magnetic induction measurement |
PCB | Printed circuit board |
PD | Photodiode |
PPG | Photoplethysmography |
rPPG | Reflective photoplethysmography |
RR | Respiratory rate |
RTC | Real-time clock |
SCG | Seismocardiography |
SD | Standard deviation |
SNR | Signal-to-noise ratio |
SPI | Serial Peripheral Interface |
Microcontroller |
References
- Tefft, B.C. Acute sleep deprivation and culpable motor vehicle crash involvement. Sleep 2018, 41, zsy144. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Gonçalves, M.; Amici, R.; Lucas, R.; Åkerstedt, T.; Cirignotta, F.; Horne, J.; Léger, D.; McNicholas, W.T.; Partinen, M.; Téran-Santos, J.; et al. Sleepiness at the wheel across Europe: A survey of 19 countries. J. Sleep Res. 2015, 24, 242–253. [Google Scholar] [CrossRef]
- Merlhiot, G.; Bueno, M. How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review. Accid. Anal. Prev. 2021, 170, 106536. [Google Scholar] [CrossRef] [PubMed]
- On-Road Automated Driving (ORAD) committee. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE Int. 2018, 4970, 1–5. [Google Scholar] [CrossRef]
- Roth, G.A.; Abate, D.; Abate, K.H.; Abay, S.M.; Abbafati, C.; Abbasi, N.; Abbastabar, H.; Abd-Allah, F.; Abdela, J.; Abdelalim, A.; et al. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018, 392, 1736–1788. [Google Scholar] [CrossRef] [Green Version]
- Kleiger, R.E.; Stein, P.K.; Bosner, M.S.; Rottman, J.N. Time domain measurements of heart rate variability. Cardiol. Clin. 1992, 10, 487–498. [Google Scholar] [CrossRef]
- Baek, H.J.; Chung, G.S.; Kim, K.K.; Kim, J.S.; Park, K.S. Photoplethysmogram measurement without direct skin-to-sensor contact using an adaptive light source intensity control. IEEE Trans. Inf. Technol. Biomed. 2009, 13, 1085–1088. [Google Scholar] [CrossRef]
- Shusterman, V.; Barnea, O. Sympathetic nervous system activity in stress and biofeedback relaxation. Monitoring SNS activity with the photoplethysmographic-wave envelope and temperature-variability signals. IEEE Eng. Med. Biol. Mag. 2005, 24, 52–57. [Google Scholar] [CrossRef]
- Hjortskov, N.; Rissén, D.; Blangsted, A.K.; Fallentin, N.; Lundberg, U.; Søgaard, K. The effect of mental stress on heart rate variability and blood pressure during computer work. Eur. J. Appl. Physiol. 2004, 92, 84–89. [Google Scholar] [CrossRef]
- Pattinson, K.T.S. Opioids and the control of respiration. Br. J. Anaesth. 2008, 100, 747–758. [Google Scholar] [CrossRef] [Green Version]
- Liu, H.; Allen, J.; Zheng, D.; Chen, F. Recent development of respiratory rate measurement technologies. Physiol. Meas. 2019, 40, 07TR01. [Google Scholar] [CrossRef] [Green Version]
- Garrido, D.; Assioun, J.J.; Keshishyan, A.; Sanchez-Gonzalez, M.A.; Goubran, B. Respiratory Rate Variability as a Prognostic Factor in Hospitalized Patients Transferred to the Intensive Care Unit. Cureus 2018, 10, e2100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kundinger, T.; Sofra, N.; Riener, A. Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection. Sensors 2020, 20, 1029. [Google Scholar] [CrossRef] [Green Version]
- Zhang, Q.; Wu, Q.; Zhou, Y.; Wu, X.; Ou, Y.; Zhou, H. Webcam-based, non-contact, real-time measurement for the physiological parameters of drivers. Measurement 2017, 100, 311–321. [Google Scholar] [CrossRef]
- Manullang, M.C.T.; Lin, Y.H.; Lai, S.J.; Chou, N.K. Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review. Sensors 2021, 21, 7777. [Google Scholar] [CrossRef] [PubMed]
- Kim, D.K. Wireless Vital Signal Tracking for Drivers Using Micro-Doppler Seatback Radar. In Proceedings of the 2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS), Paris, France, 26–28 February 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Leicht, L.; Eilebrecht, B.; Weyer, S.; Wartzek, T.; Leonhardt, S. Active humidification for capacitive-resistive ECG-systems. Biomed. Eng. 2014, 59, 758–909. [Google Scholar] [CrossRef] [Green Version]
- Leonhardt, S.; Leicht, L.; Teichmann, D. Unobtrusive Vital Sign Monitoring in Automotive Environments-A Review. Sensors 2018, 18, 3080. [Google Scholar] [CrossRef] [Green Version]
- Walter, M.; Eilebrecht, B.; Wartzek, T.; Leonhardt, S. The smart car seat: Personalized monitoring of vital signs in automotive applications. Pers. Ubiquitous Comput. 2011, 15, 707–715. [Google Scholar] [CrossRef]
- Sahayadhas, A.; Sundaraj, K.; Murugappan, M. Detecting driver drowsiness based on sensors: A review. Sensors 2012, 12, 16937–16953. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Giannakakis, G.; Grigoriadis, D.; Giannakaki, K.; Simantiraki, O.; Roniotis, A.; Tsiknakis, M. Review on psychological stress detection using biosignals. IEEE Trans. Affect. Comput. 2019, 13, 440–460. [Google Scholar] [CrossRef]
- Mathissen, M.; Hennes, N.; Faller, F.; Leonhardt, S.; Teichmann, D. Investigation of Three Potential Stress Inducement Tasks During On-Road Driving. IEEE Trans. Intell. Transp. Syst. 2021, 23, 4823–4832. [Google Scholar] [CrossRef]
- Baek, H.J.; Lee, H.B.; Kim, J.S.; Choi, J.M.; Kim, K.K.; Park, K.S. Nonintrusive biological signal monitoring in a car to evaluate a driver’s stress and health state. Telemed. J. e-Health 2009, 15, 182–189. [Google Scholar] [CrossRef] [PubMed]
- Warnecke, J.M.; Boeker, N.; Spicher, N.; Wang, J.; Flormann, M.; Deserno, T.M. Sensor Fusion for Robust Heartbeat Detection during Driving. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Guadalajara, Mexico, 1–5 November 2021; pp. 447–450. [Google Scholar] [CrossRef]
- Leicht, L.; Walter, M.; Mathissen, M.; Antink, C.H.; Teichmann, D.; Leonhardt, S. Unobtrusive Measurement of Physiological Features Under Simulated and Real Driving Conditions. IEEE Trans. Intell. Transp. Syst. 2022, 23, 4767–4777. [Google Scholar] [CrossRef]
- Wang, J.; Warnecke, J.M.; Haghi, M.; Deserno, T.M. Unobtrusive Health Monitoring in Private Spaces: The Smart Vehicle. Sensors 2020, 20, 2442. [Google Scholar] [CrossRef] [PubMed]
- Warnecke, J.M.; Wang, J.; Cakir, T.; Spicher, N.; Ganapathy, N.; Deserno, T.M. Registered report protocol: Developing an artifact index for capacitive electrocardiography signals acquired with an armchair. PLoS ONE 2021, 16, e0254780. [Google Scholar] [CrossRef]
- Hoog Antink, C.; Leonhardt, S.; Schulz, F.; Walter, M. MuSeSe—A multisensor armchair for unobtrusive vital sign estimation and motion artifact analysis. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Republic of Korea, 11–15 July 2017; pp. 857–860. [Google Scholar] [CrossRef]
- Hong, S.; Heo, J.; Park, K.S. Signal Quality Index Based on Template Cross-Correlation in Multimodal Biosignal Chair for Smart Healthcare. Sensors 2021, 21, 7564. [Google Scholar] [CrossRef]
- Yu, X.; Neu, W.; Vetter, P.; Bollheimer, L.C.; Leonhardt, S.; Teichmann, D.; Antink, C.H. A Multi-Modal Sensor for a Bed-Integrated Unobtrusive Vital Signs Sensing Array. IEEE Trans. Biomed. Circuits Syst. 2019, 13, 529–539. [Google Scholar] [CrossRef]
- Davies, A.; Scott, A. Starting to Read ECGs; Springer London: London, UK, 2015. [Google Scholar] [CrossRef]
- Richardson, P.C. The insulated electrode: A pasteless electrocardiographic technique. In 20th Annual Conference on Engineering in Medicine and Biology; Wellesley Press: Wellesley, MA, USA, 1967; Volume 9, pp. 15–17. [Google Scholar]
- Winter, B.B.; Webster, J.G. Driven-right-leg circuit design. IEEE Trans. Biomed. Eng. 1983, 30, 62–66. [Google Scholar] [CrossRef]
- Keun Kim, K.; Kyu Lim, Y.; Suk Park, K. Common mode noise cancellation for electrically non-contact ECG measurement system on a chair. In Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 17–18 January 2005; pp. 5881–5883. [Google Scholar] [CrossRef]
- Wong, M.Y.M.; Pickwell-MacPherson, E.; Zhang, Y.T. Contactless and continuous monitoring of heart rate based on photoplethysmography on a mattress. Physiol. Meas. 2010, 31, 1065–1074. [Google Scholar] [CrossRef]
- Teichmann, D.; Teichmann, M.; Weitz, P.; Wolfart, S.; Leonhardt, S.; Walter, M. SensInDenT-Noncontact Sensors Integrated Into Dental Treatment Units. IEEE Trans. Biomed. Circuits Syst. 2017, 11, 225–233. [Google Scholar] [CrossRef]
- Wang, W.; den Brinker, A.C.; Stuijk, S.; de Haan, G. Algorithmic Principles of Remote PPG. IEEE Trans. Biomed. Eng. 2017, 64, 1479–1491. [Google Scholar] [CrossRef] [Green Version]
- Vas, R. Electronic Device for Physiological Kinetic Measurements and Detection of Extraneous Bodies. IEEE Transactions Biomed. Eng. 1967, 14, 2–6. [Google Scholar] [CrossRef]
- Inan, O.T.; Migeotte, P.F.; Park, K.S.; Etemadi, M.; Tavakolian, K.; Casanella, R.; Zanetti, J.; Tank, J.; Funtova, I.; Prisk, G.K.; et al. Ballistocardiography and seismocardiography: A review of recent advances. IEEE J. Biomed. Health Inform. 2015, 19, 1414–1427. [Google Scholar] [CrossRef] [Green Version]
- Yu, X.; Neu, W.; Vetter, P.; Bollheimer, C.; Leonhardt, S.; Teichmann, D. Inter-modal and Intra-modal interference in a Multi-Modal Sensor for Non-contact Monitoring of Vital Signs in Patient Beds. In Proceedings of the 11th International Conference of Bioelectromagnetism, Aachen, Germany, 23–25 May 2018. [Google Scholar] [CrossRef]
- Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; Koltun, V. CARLA: An Open Urban Driving Simulator. arXiv 2017, arXiv:1711.03938. [Google Scholar]
- Pan, J.; Tompkins, W.J. A real-time QRS detection algorithm. IEEE Trans. Biomed. Eng. 1985, 32, 230–236. [Google Scholar] [CrossRef]
- Brüser, C.; Winter, S.; Leonhardt, S. Robust inter-beat interval estimation in cardiac vibration signals. Physiol. Meas. 2013, 34, 123–138. [Google Scholar] [CrossRef]
- Rapczynski, M.; Werner, P.; Saxen, F.; Al-Hamadi, A. How the Region of Interest Impacts Contact Free Heart Rate Estimation Algorithms. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 2027–2031. [Google Scholar] [CrossRef]
- Ernst, H.; Scherpf, M.; Malberg, H.; Schmidt, M. Optimal color channel combination across skin tones for remote heart rate measurement in camera-based photoplethysmography. Biomed. Signal Process. Control. 2021, 68, 102644. [Google Scholar] [CrossRef]
- Heiberger, R.M.; Holland, B. Statistical Analysis and Data Display; Springer: New York, NY, USA, 2015. [Google Scholar] [CrossRef]
- Plesinger, F.; Jurco, J.; Halamek, J.; Jurak, P. SignalPlant: An open signal processing software platform. Physiol. Meas. 2016, 37, N38–N48. [Google Scholar] [CrossRef] [PubMed]
- Birrenkott, D.A.; Pimentel, M.A.F.; Watkinson, P.J.; Clifton, D.A. A Robust Fusion Model for Estimating Respiratory Rate from Photoplethysmography and Electrocardiography. IEEE Trans. Biomed. Eng. 2018, 65, 2033–2041. [Google Scholar] [CrossRef] [PubMed]
- Finnegan, E.; Davidson, S.; Harford, M.; Jorge, J.; Watkinson, P.; Young, D.; Tarassenko, L.; Villarroel, M. Pulse arrival time as a surrogate of blood pressure. Sci. Rep. 2021, 11, 22767. [Google Scholar] [CrossRef]
Participant | Age [Years] | Gender | Weight [kg] | Height [cm] | # Layers of Clothes | Material |
---|---|---|---|---|---|---|
1 | 24 | male | 70 | 174 | 1 | cotton, polyester |
2 | 22 | male | 70 | 186 | 1 | cotton |
3 | 21 | male | 92 | 203 | 1 | cotton |
4 | 20 | male | 80 | 181 | 1 | cotton |
5 | 19 | male | 60 | 165 | 2 | cotton |
6 | 24 | female | 61 | 165 | 1 | cotton |
7 | 25 | male | 75 | 184 | 1 | cotton, polyester |
8 | 22 | male | 93 | 187 | 1 | cotton |
9 | 27 | male | 80 | 180 | 1 | cotton |
10 | 26 | female | 66 | 170 | 1 | cotton |
11 | 23 | male | 71 | 187 | 1 | cotton |
12 | 23 | male | 80 | 183 | 1 | cotton |
13 | 28 | male | 82 | 173 | 1 | cotton, polyester, spandex |
14 | 23 | male | 78 | 174 | 1 | cotton |
15 | 25 | male | 93 | 180 | 1 | cotton |
16 | 29 | male | 77 | 189 | 1 | cotton |
17 | 22 | male | 71 | 180 | 1 | cotton |
18 | 25 | male | 65 | 178 | 1 | cotton, polyester |
19 | 30 | male | 80 | 174 | 1 | cotton |
20 | 60 | male | 65 | 172 | 2 | cotton |
Participant | ||||||||
---|---|---|---|---|---|---|---|---|
1 | 0.852 | 0.893 | 0.517 | 0.883 | 0.695 | 0.538 | 0.614 | 0.666 |
2 | 0.023 | 0.104 | 0.025 | 0.374 | 0.649 | 0.383 | 0.766 | 0.736 |
3 | 0.959 | 0.954 | 0.958 | 0.935 | 0.597 | 0.384 | 0.374 | 0.395 |
4 | 0.754 | 0.749 | 0.345 | 0.724 | 0.875 | 0.879 | 0.92 | 0.832 |
5 | 0.928 | 0.938 | 0.826 | 0.945 | 0.855 | 0.721 | 0.84 | 0.837 |
6 | 0.3 | 0.135 | 0.21 | 0.705 | 0.186 | 0.758 | 0.391 | 0.712 |
7 | 0.793 | 0.781 | 0.692 | 0.83 | 0.784 | 0.713 | 0.538 | 0.689 |
8 | 0.74 | 0.737 | 0.71 | 0.682 | 0.737 | 0.711 | 0.729 | 0.767 |
9 | 0.872 | 0.789 | 0.882 | 0.761 | 0.853 | 0.889 | 0.866 | 0.954 |
10 | 0.037 | 0.018 | 0.021 | 0.061 | 0.334 | 0.39 | 0.821 | 0.705 |
11 | 0.829 | 0.876 | 0.733 | 0.794 | 0.568 | 0.608 | 0.736 | 0.814 |
12 | 0.901 | 0.858 | 0.26 | 0.855 | 0.741 | 0.599 | 0.712 | 0.739 |
13 | 0.048 | 0.165 | 0.027 | 0.248 | 0.868 | 0.921 | 0.779 | 0.837 |
14 | 0.918 | 0.572 | 0.339 | 0.556 | 0.868 | 0.868 | 0.788 | 0.862 |
15 | 0.793 | 0.805 | 0.779 | 0.811 | 0.573 | 0.436 | 0.391 | 0.59 |
16 | 0.852 | 0.917 | 0.331 | 0.878 | 0.614 | 0.471 | 0.707 | 0.672 |
17 | 0.893 | 0.912 | 0.69 | 0.852 | 0.876 | 0.8 | 0.918 | 0.932 |
18 | 0.568 | 0.741 | 0.264 | 0.694 | 0.173 | 0.17 | 0.174 | 0.17 |
19 | 0.821 | 0.512 | 0.163 | 0.382 | 0.884 | 0.858 | 0.876 | 0.93 |
20 | 0.266 | 0.219 | 0.148 | 0.371 | 0.501 | 0.269 | 0.74 | 0.866 |
median | 0.8072 | 0.7650 | 0.3420 | 0.7424 | 0.7157 | 0.6590 | 0.7382 | 0.7530 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Linschmann, O.; Uguz, D.U.; Romanski, B.; Baarlink, I.; Gunaratne, P.; Leonhardt, S.; Walter, M.; Lueken, M. A Portable Multi-Modal Cushion for Continuous Monitoring of a Driver’s Vital Signs. Sensors 2023, 23, 4002. https://doi.org/10.3390/s23084002
Linschmann O, Uguz DU, Romanski B, Baarlink I, Gunaratne P, Leonhardt S, Walter M, Lueken M. A Portable Multi-Modal Cushion for Continuous Monitoring of a Driver’s Vital Signs. Sensors. 2023; 23(8):4002. https://doi.org/10.3390/s23084002
Chicago/Turabian StyleLinschmann, Onno, Durmus Umutcan Uguz, Bianca Romanski, Immo Baarlink, Pujitha Gunaratne, Steffen Leonhardt, Marian Walter, and Markus Lueken. 2023. "A Portable Multi-Modal Cushion for Continuous Monitoring of a Driver’s Vital Signs" Sensors 23, no. 8: 4002. https://doi.org/10.3390/s23084002