Real Time Apnoea Monitoring of Children Using the Microsoft Kinect Sensor: A Pilot Study
<p>Microsoft Kinect v2 sensor.</p> "> Figure 2
<p>System overview. The proposed system includes a Kinect sensor connected to a laptop via Microsoft Kinect adapter; a real-time system based on motion magnification and motion detection; and software built-in Kinect library, including body index and skeleton tracking.</p> "> Figure 3
<p>Skeletal joints provided in the Kinect code library. The Region of interest is the yellow pentagon defined by the 5 points.</p> "> Figure 4
<p>Red, green & blue image, depth map, thermal, skeleton tracking, and body index from the Kinect v2 sensor.</p> "> Figure 5
<p>A five minutes simulation for healthy subject where the subject was asked to hold his breath twice during the measurement for the first scenario (without a blanket).</p> "> Figure 6
<p>A five minutes simulation for healthy subject where the subject was asked to hold his breath twice during the measurement for the first scenario (with a blanket).</p> "> Figure 7
<p>A five minutes simulation for healthy subject where the subject was asked to hold his breath twice during the measurement for the second scenario (without a blanket).</p> "> Figure 8
<p>A five minutes simulation for healthy subject where the subject was asked to hold his breath twice during the measurement for the second scenario (with a blanket).</p> "> Figure 9
<p>(<b>a</b>) Correlation plot; (<b>b</b>) Bland-Altman plot of the difference between measured data and reference data (first scenario without a blanket).</p> "> Figure 10
<p>(<b>a</b>) Correlation plot; (<b>b</b>) Bland-Altman plot of the difference between measured data and reference data (first scenario with a blanket).</p> "> Figure 11
<p>(<b>a</b>) Correlation plot; (<b>b</b>) Bland-Altman plot of the difference between measured data and reference data (second scenario without a blanket).</p> "> Figure 12
<p>(<b>a</b>) Correlation plot; (<b>b</b>) Bland-Altman plot of the difference between measured data and reference data (second scenario with a blanket).</p> ">
Abstract
:1. Introduction
2. Related Works
3. Kinect Sensor
4. Materials and Methods
4.1. Experimental Setup
4.2. System Design/Overview
4.3. Data Analysis
4.3.1. Modified Motion Magnification System
4.3.2. Motion Detection Based on Frame Differencing
4.4. Respiratory Calculations
5. Results
6. Conclusions
Acknowledgments
Author Contributions
Conflicts of interest
References
- Pediatrics, A.A.O. Apnea, sudden infant death syndrome, and home monitoring. Pediatrics 2003, 111, 914–917. [Google Scholar]
- Strehle, E.M.; Gray, W.K.; Gopisetti, S.; Richardson, J.; McGuire, J.; Malone, S. Can home monitoring reduce mortality in infants at increased risk of sudden infant death syndrome? A systematic review. Acta Paediatr. 2012, 101, 8–13. [Google Scholar] [CrossRef] [PubMed]
- Red Nose. National Scientific Advisory Group (NSAG). Information Statement: Home Monitoring; National SIDS Council of Australia: Melbourne, Australia, 2016; Available online: https://rednose.com.au/downloads/Home_Monitoring-Safe_Sleeping-Information_Statement.pdf (accessed on 25 December 2016).
- Bennett, A.D. Home apnea monitoring for infants. Adv. Nurse Practitioners. 2002, 10, 47–54. [Google Scholar]
- Sheppard, I.; Morris, L.; Blackstock, D. Medication Safety Alerts. Can. J. Hosp. Pharm. 2004, 57, 176–179. [Google Scholar]
- Fu, L.Y.; Moon, R.Y. Apparent life-threatening events (ALTEs) and the role of home monitors. Pediatr. Rev. 2007, 28, 203–208. [Google Scholar] [CrossRef] [PubMed]
- Beelke, M.; Angeli, S.; Del Sette, M.; Gandolfo, C.; Cabano, M.E.; Canovaro, P.; Nobili, L.; Ferrillo, F. Prevalence of patent foramen ovale in subjects with obstructive sleep apnea: A transcranial Doppler ultrasound study. Sleep Med. 2003, 4, 219–223. [Google Scholar] [CrossRef]
- Marchionni, P.; Scalise, L.; Ercoli, I.; Tomasini, E. An optical measurement method for the simultaneous assessment of respiration and heart rates in preterm infants. Rev. Sci. Instrum. 2013, 84, 121705. [Google Scholar] [CrossRef] [PubMed]
- Min, S.D.; Yoon, D.J.; Yoon, S.W.; Yun, Y.H.; Lee, M. A study on a non-contacting respiration signal monitoring system using Doppler ultrasound. Med. Biol. Eng. Comput. 2007, 45, 1113–1119. [Google Scholar] [CrossRef] [PubMed]
- Uenoyama, M.; Matsui, T.; Yamada, K.; Suzuki, S.; Takase, B.; Suzuki, S.; Ishihara, M.; Kawakami, M. Non-contact respiratory monitoring system using a ceiling-attached microwave antenna. Med. Biol. Eng. Comput. 2006, 44, 835–840. [Google Scholar] [CrossRef] [PubMed]
- Droitcour, A.; Lubecke, V.; Lin, J.; Boric-Lubecke, O. A microwave radio for Doppler radar sensing of vital signs. In Proceedings of the 2001 IEEE MTT-S International Microwave Symposium Digest, 20–24 May 2001; Volume 1, pp. 175–178.
- Abbas, A.K.; Heiman, K.; Orlikowsky, T.; Leonhardt, S. Non-contact respiratory monitoring based on real-time IR-thermography. In Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Munich, Germany, 7–12 September 2009; pp. 1306–1309.
- Yang, Y.; Karmakar, N.; Zhu, X. A portable wireless monitoring system for sleep apnoea diagnosis based on active RFID technology. In Proceedings of the Microwave Conference Proceedings (APMC), Melbourne, Australia, 5–8 December 2011; pp. 187–190.
- Yang, Y.; Zhu, X.; Ma, K.; Simorangkir, R.B.; Karmakar, N.C.; Esselle, K.P. Development of Wireless Transducer for Real-Time Remote Patient Monitoring. IEEE Sens. J. 2016, 16, 4669–4670. [Google Scholar] [CrossRef]
- Jones, M.H.; Goubran, R.; Knoefel, F. Reliable respiratory rate estimation from a bed pressure array. In Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA, 30 August–3 September 2006; pp. 6410–6413.
- Mahdavi, H.; Rosell Ferrer, F.J. Electromagnetic coupling simulagions for a magnetic induction sensor for sleep monitoring. In Proceedings of the XXII Congreso Anual de la Sociedad Española de Ingeniería Biomédica, Barcelona, Spain, 26–28 November 2014; pp. 1–4.
- Scalise, L.; De Leo, A.; Primiani, V.M.; Russo, P.; Shahu, D.; Cerri, G. Non-contact monitoring of the respiration activity by electromagnetic sensing. In Proceedings of the 2011 IEEE International Workshop on Medical Measurements and Applications Proceedings (MeMeA), Bari, Italy, 30–31 May 2011; pp. 418–422.
- Seeton, R.; Adler, A. Sensitivity of a single coil electromagnetic sensor for non-contact monitoring of breathing. In Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vancouver, BC, Canada, 20–25 August 2008; pp. 518–521.
- Fei, J.; Pavlidis, I. Analysis of breathing air flow patterns in thermal imaging. In Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA, 30 August–3 September 2006; pp. 946–952.
- Murthy, R.; Pavlidis, I.; Tsiamyrtzis, P. Touchless monitoring of breathing function. In Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Francisco, CA, USA, 1–5 September 2004; Volume 1, pp. 1196–1199.
- Frigola, M.; Amat, J.; Pagès, J. Vision Based Respiratory Monitoring System. In Proceedings of the 10th Mediterranean Conference on Control and Automation (MED 2002), Lisbon, Portugal, 9–13 July 2002; pp. 9–12.
- Nakajima, K.; Matsumoto, Y.; Tamura, T. Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed. Physiol. Meas. 2001, 22, 21–28. [Google Scholar] [CrossRef]
- Nakajima, K.; Osa, A.; Miike, H. A method for measuring respiration and physical activity in bed by optical flow analysis. In Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA, 30 October–2 November 1997; Volume 5, pp. 2054–2057.
- Al-Naji, A.; Chahl, J. Remote respiratory monitoring system based on developing motion magnification technique. Biomed. Signal Process. Control. 2016, 29, 1–10. [Google Scholar] [CrossRef]
- Christian, S.; Jochen, P.; Joachim, H. Time-of-flight sensor for respiratory motion gating. Med. Phys. 2008, 35, 3090–3093. [Google Scholar]
- Falie, D.; David, L.; Ichim, M. Statistical algorithm for detection and screening sleep apnea. In Proceedings of the International Symposium on Signals, Circuits and Systems (ISSCS 2009), Iasi, Romania, 9–10 July 2009; pp. 1–4.
- Blanik, N.; Abbas, A.K.; Venema, B.; Blazek, V.; Leonhardt, S. Hybrid optical imaging technology for long-term remote monitoring of skin perfusion and temperature behavior. J. Biomed. Opt. 2014, 19. [Google Scholar] [CrossRef] [PubMed]
- Bousefsaf, F.; Maaoui, C.; Pruski, A. Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate. Biomed. Signal Process. Control. 2013, 8, 568–574. [Google Scholar] [CrossRef]
- Nilsson, L.; Johansson, A.; Kalman, S. Monitoring of respiratory rate in postoperative care using a new photoplethysmographic technique. J. Clin. Monit. Comput. 2000, 16, 309–315. [Google Scholar] [CrossRef] [PubMed]
- Verkruysse, W.; Svaasand, L.O.; Nelson, J.S. Remote plethysmographic imaging using ambient light. Opt. Express 2008, 16, 21434–21445. [Google Scholar] [CrossRef] [PubMed]
- Xia, J.; Siochi, R.A. A real-time respiratory motion monitoring system using KINECT: Proof of concept. Med. Phys. 2012, 39, 2682–2685. [Google Scholar] [CrossRef] [PubMed]
- Yang, C.; Cheung, G.; Chan, K.L.; Stankovic, V. Sleep monitoring via depth video compression & analysis. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Chengdu, China, 14–18 July 2014; pp. 1–6.
- Yu, M.C.; Wu, H.; Liou, J.L.; Lee, M.S.; Hung, Y.-P. Breath and Position Monitoring during Sleeping with a Depth Camera. In Proceedings of the International Conference on Health Informatics, Algarve, Portugal, 1–4 February 2012; pp. 12–22.
- Aoki, H.; Nakamura, H.; Fumoto, K.; Nakahara, K.; Teraoka, M. Basic study on non-contact respiration measurement during exercise tolerance test by using kinect sensor. In Proceedings of the 2015 IEEE/SICE International Symposium on System Integration (SII), Nagoya, Japan, 11–13 December 2015; pp. 217–222.
- Centonze, F.; Schatz, M.; Prochazka, A.; Kuchynka, J.; Vysata, O.; Cejnar, P.; Valis, M. Feature extraction using MS Kinect and data fusion in analysis of sleep disorders. In Proceedings of the 2015 International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), Prague, Czech Republic, 29–30 October 2015; pp. 1–5.
- Ortmüller, J.; Gauer, T.; Wilms, M.; Handels, H.; Werner, R. Respiratory surface motion measurement by Microsoft Kinect. Curr. Dir. Biomed. Eng. 2015, 1, 270–273. [Google Scholar] [CrossRef]
- Tanaka, M. Application of depth sensor for breathing rate counting. In Proceedings of the 2015 10th Asian Control Conference (ASCC), Kota Kinabalu, Malaysia, 31 May–3 June 2015; pp. 1–5.
- Harte, J.M.; Golby, C.K.; Acosta, J.; Nash, E.F.; Kiraci, E.; Williams, M.A.; Arvanitis, T.N.; Naidu, B. Chest wall motion analysis in healthy volunteers and adults with cystic fibrosis using a novel Kinect-based motion tracking system. Med. Biol. Eng. Comput. 2016, 54, 1631–1640. [Google Scholar] [CrossRef] [PubMed]
- Kumagai, S.; Uemura, R.; Ishibashi, T.; Nakabayashi, S.; Arai, N.; Kobayashi, T.; Kotoku, J.I. Markerless Respiratory Motion Tracking Using Single Depth Camera. Open J. Med. Imaging 2016, 6, 20–31. [Google Scholar] [CrossRef]
- Tahavori, F.; Adams, E.; Dabbs, M.; Aldridge, L.; Liversidge, N.; Donovan, E.; Jordan, T.; Evans, P.; Wells, K. Combining marker-less patient setup and respiratory motion monitoring using low cost 3D camera technology. In SPIE Medical Imaging: International Society for Optics and Photonics; SPIE: Orlando, FL, USA, 2015; Volume 9415, pp. 1–7. [Google Scholar]
- Lee, J.; Hong, M.; Ryu, S. Sleep monitoring system using kinect sensor. Int. J. Distrib. Sens. Netw. 2015, 2015, 1–9. [Google Scholar] [CrossRef]
- Samir, M.; Golkar, E.; Rahni, A.A.A. Comparison between the KinectTM V1 and KinectTM V2 for respiratory motion tracking. In Proceedings of the 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, 19–21 October 2015; pp. 150–155.
- Kim, C.; Yun, S.; Jung, S.-W.; Won, C.S. Color and depth image correspondence for Kinect v2. In Advanced Multimedia and Ubiquitous Engineering; Springer: New York, NY, USA, 2015; pp. 111–116. [Google Scholar]
- Yang, L.; Zhang, L.; Dong, H.; Alelaiwi, A.; El Saddik, A. Evaluating and improving the depth accuracy of Kinect for Windows v2. Sens. J. 2015, 15, 4275–4285. [Google Scholar] [CrossRef]
- Sarbolandi, H.; Lefloch, D.; Kolb, A. Kinect range sensing: Structured-light versus Time-of-Flight Kinect. Comput. Vis. Image Underst. 2015, 139, 1–20. [Google Scholar] [CrossRef]
- Mutto, C.D.; Zanuttigh, P.; Cortelazzo, G.M. Time-of-Flight Cameras and Microsoft Kinect (TM); Springer Publishing Company, Incorporated: New York, NY, USA, 2012. [Google Scholar]
- The Primesensor Reference Design. Available online: http://www.primesensor.com (accessed on 25 December 2016).
- González-Audícana, M.; Saleta, J.L.; Catalán, R.G.; García, R. Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans. Geosci. Remote Sens. 2004, 42, 1291–1299. [Google Scholar] [CrossRef]
- Tsai, D.-M.; Chiang, C.-H. Rotation-invariant pattern matching using wavelet decomposition. Pattern Recognit. Lett. 2002, 23, 191–201. [Google Scholar] [CrossRef]
- Patidar, P.; Gupta, M.; Srivastava, S.; Nagawat, A.K. Image de-noising by various filters for different noise. Int. J. Comput. Appl. 2010, 9, 45–50. [Google Scholar] [CrossRef]
- Wadhwa, N.; Rubinstein, M.; Durand, F.; Freeman, W.T. Phase-based video motion processing. ACM Trans. Graph. 2013, 32. [Google Scholar] [CrossRef] [Green Version]
- Wu, H.-Y.; Rubinstein, M.; Shih, E.; Guttag, J.V.; Durand, F.; Freeman, W.T. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 2012, 31, 65. [Google Scholar] [CrossRef]
- Al-Naji, A.; Chahl, J. Non-contact heart activity measurement system based on video imaging analysis. Int. J. Pattern Recognit. Artif. Intell. 2017, 31, 1–21. [Google Scholar] [CrossRef]
- Madhukar, B.; Narendra, R. Lanczos resampling for the digital processing of remotely sensed images. In Proceedings of the International Conference on VLSI, Communication, Advanced Devices, Signals & Systems and Networking; Springer: New York, NY, USA, 2013; pp. 403–411. [Google Scholar]
- Martínez-Martín, E.; del Pobil, Á.P. Motion detection in static backgrounds. In Robust Motion Detection in Real-Life Scenarios; Springer: New York, NY, USA, 2012; pp. 5–42. [Google Scholar]
- Hossain, M.S.; Hossain, M.E.; Khalid, M.S.; Haque, M.A. A Belief Rule-Based (BRB) Decision Support System for Assessing Clinical Asthma Suspicion. In Proceedings of the Scandinavian Conference on Health Informatics, Grimstad, Norway, 21–22 August 2014; Linköping University Electronic Press, 2014; pp. 83–89. [Google Scholar]
- Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Int. J. Nurs. Stud. 2010, 47, 931–936. [Google Scholar] [CrossRef]
Features | Kinect v1 | Kinect v2 |
---|---|---|
Depth sensor type | Structured light | Time of Flight (ToF) |
Red, Green & Blue (RGB) camera resolution | 640 × 480, 30 fps | 1920 × 1080, 30 fps |
Infrared (IR) camera resolution | 320 × 240, 30 fps | 512 × 424, 30 fps |
Field of view of RGB image | 62° × 48.6° | 84.1° × 53.8° |
Field of view of depth image | 57° × 43° | 70° × 60° |
Operative measuring range | 0.8 m–4 m (Default); 0.4 m–3.5 m (Near) | 0.5 m–4.5 m |
Skeleton joints defined | 20 joints | 25 joints |
Maximum skeletal tracking | 2 | 6 |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Al-Naji, A.; Gibson, K.; Lee, S.-H.; Chahl, J. Real Time Apnoea Monitoring of Children Using the Microsoft Kinect Sensor: A Pilot Study. Sensors 2017, 17, 286. https://doi.org/10.3390/s17020286
Al-Naji A, Gibson K, Lee S-H, Chahl J. Real Time Apnoea Monitoring of Children Using the Microsoft Kinect Sensor: A Pilot Study. Sensors. 2017; 17(2):286. https://doi.org/10.3390/s17020286
Chicago/Turabian StyleAl-Naji, Ali, Kim Gibson, Sang-Heon Lee, and Javaan Chahl. 2017. "Real Time Apnoea Monitoring of Children Using the Microsoft Kinect Sensor: A Pilot Study" Sensors 17, no. 2: 286. https://doi.org/10.3390/s17020286