Posture Recognition Using Ensemble Deep Models under Various Home Environments
"> Figure 1
<p>Structure of VGGNet. FC, fully connected.</p> "> Figure 2
<p>Structure of ResNet.</p> "> Figure 3
<p>Dense connectivity.</p> "> Figure 4
<p>Structure of DenseNet.</p> "> Figure 5
<p>Stem layer of InceptionResNetV2. BN, batch normalization; ReLU, rectified linear unit.</p> "> Figure 6
<p>Structure of Xception.</p> "> Figure 7
<p>Astra camera for capturing posture images. RBG, red, blue, green; USB, Universal Serial Bus.</p> "> Figure 8
<p>Graphical user interface (GUI) for constructing the posture database.</p> "> Figure 9
<p>Environment for capturing posture images using Astra.</p> "> Figure 10
<p>Examples of posture images captured from Astra camera in various home environments: (<b>a</b>) standing normal; (<b>b</b>) standing bent; (<b>c</b>) sitting sofa; (<b>d</b>) sitting chair; (<b>e</b>) sitting floor; (<b>f</b>) sitting squat; (<b>g</b>) lying face; (<b>h</b>) lying back; (<b>i</b>) lying side; (<b>j</b>) lying crouched.</p> "> Figure 10 Cont.
<p>Examples of posture images captured from Astra camera in various home environments: (<b>a</b>) standing normal; (<b>b</b>) standing bent; (<b>c</b>) sitting sofa; (<b>d</b>) sitting chair; (<b>e</b>) sitting floor; (<b>f</b>) sitting squat; (<b>g</b>) lying face; (<b>h</b>) lying back; (<b>i</b>) lying side; (<b>j</b>) lying crouched.</p> "> Figure 11
<p>Examples of five types of original and preprocessed images: (<b>a</b>) original image; (<b>b</b>) type 0; (<b>c</b>) type 1; (<b>d</b>) type 2; (<b>e</b>) type 4; (<b>f</b>) type 5.</p> "> Figure 12
<p>Ensemble deep models using the preprocessed images with different convolutional neural network (CNN) models.</p> "> Figure 13
<p>Various ensemble deep models by input types and the pre-trained CNNs.</p> "> Figure 14
<p>Examples of four categories with 10 postures.</p> "> Figure 15
<p>Data configuration for training, validation, and testing data set.</p> "> Figure 16
<p>Structure of the simple CNN.</p> "> Figure 17
<p>Confusion matrix for VGG19 with preprocessing of type 2 without data augmentation.</p> "> Figure 18
<p>Confusion matrix for VGG19 with preprocessing of type 2.</p> "> Figure 19
<p>Training process for InceptionResNetV2 with preprocessing of type 2: (<b>a</b>) accuracy; (<b>b</b>) loss.</p> "> Figure 20
<p>Activations for InceptionResNetV2 with preprocessing of type 2, which exhibited the best performance for the posture data of 11 classes without data augmentation under nontransfer learning: (<b>a</b>) activations of first convolution; (<b>b</b>) activations of last convolution.</p> "> Figure 21
<p>Activations of the last ReLU for InceptionResNetV2 with preprocessing of type 2, which exhibited the best performance for the posture data of 11 classes without data augmentation under nontransfer learning.</p> "> Figure 22
<p>Comparison of the average total accuracies for the different types of preprocessing under transfer learning.</p> "> Figure 23
<p>Comparison of the average total accuracies for the different models under transfer learning.</p> "> Figure 24
<p>Performance comparison by various ensemble deep models.</p> ">
Abstract
:1. Introduction
2. Deep Models of CNN
2.1. VGGNet
2.2. ResNet
2.3. DenseNet
2.4. InceptionResNetV2
2.5. Xception
3. Database and Experiment Method
3.1. Construction of Database for Posture Recognition
3.2. Preprocessing Types
3.3. Posture Recognition Using Ensemble CNN Models
4. Experimental Results
4.1. Database
4.2. Experimental Results
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Park, J.H.; Song, K.H.; Kim, Y.S. A kidnapping detection using human pose estimation in intelligent video surveillance systems. J. Korea Soc. Comput. Inf. 2018, 23, 9–16. [Google Scholar]
- Qiang, B.; Zhang, S.; Zhan, Y.; Xie, W.; Zhao, T. Improved convolutional pose machines for human pose estimation using image sensor data. Sensors 2019, 19, 718. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Huang, Z.; Liu, Y.; Fang, Y.; Horn, B.K.P. Video-Based Fall Detection for Seniors with Human Pose Estimation. In Proceedings of the 4th International Conference on Universal Village, Boston, MA, USA, 21–24 October 2018; pp. 1–4. [Google Scholar]
- Chan, K.C.; Koh, C.K.; Lee, C.S.G. An automatic design of factors in a human-pose estimation system using neural networks. IEEE Trans. Syst. Man Cybern. Syst. 2016, 46, 875–887. [Google Scholar] [CrossRef]
- Veges, M.; Varga, V.; Lorincz, A. 3D human pose estimation with siamese equivariant embedding. Neurocomputing 2019, 339, 194–201. [Google Scholar] [CrossRef]
- Stommel, M.; Beetz, M.; Wu, W. Model-free detection, encoding, retrieval, and visualization of human poses from Kinect data. IEEE Asme Trans. Mechatron. 2015, 20, 865–875. [Google Scholar] [CrossRef]
- Shum, H.P.H.; Ho, E.S.L.; Jiang, Y.; Takagi, S. Real-time posture reconstruction for Microsoft Kinect. IEEE Trans. Cybern. 2013, 43, 1357–1369. [Google Scholar] [CrossRef]
- Lee, J.; Joo, H.; Lee, J.; Chee, Y. Automatic classi?cation of squat posture using inertial sensors: Deep learning approach. Sensors 2020, 20, 361. [Google Scholar] [CrossRef] [Green Version]
- Chowdhury, I.R.; Saha, J.; Chowdhury, C. Detailed Activity Recognition with Smartphones. In Proceedings of the Fifth International Conference on Emerging Applications of Information Technology, Kolkata, India, 12–13 January 2018; pp. 1–4. [Google Scholar]
- Wu, Z.; Zhang, J.; Chen, K.; Fu, C. Yoga posture recognition and quantitative evaluation with wearable sensors based on two-stage classifer and prior bayesian network. Sensors 2019, 19, 5129. [Google Scholar] [CrossRef] [Green Version]
- Idris, M.I.; Zabidi, A.; Yassun, I.M.; Ali, M.S.A.M. Human Posture Recognition Using Android Smartphone and Artificial Neural Network. In Proceedings of the IEEE Control and System Gradate Research Colloquium, Shah Alam, Malaysia, 10–11 August 2015; pp. 120–124. [Google Scholar]
- Pak, M.S.; Kim, S.H. A Review of Deep Learning in Image Recognition. In Proceedings of the International Conference on Computer Applications and Information Processing Technology, Kuta Bali, Indonesia, 8–10 August 2017; pp. 1–3. [Google Scholar]
- Diao, W.; Sun, X.; Zheng, X.; Dou, F.; Wang, H.; Fu, K. Efficient saliency-based object detection in remote sensing images using deep belief networks. IEEE Geosci. Remote Sens. Lett. 2016, 13, 137–141. [Google Scholar] [CrossRef]
- Sasaki, H.; Horiuchi, T.; Kato, S. A Study on Vision-Based Mobile Robot Learning by Deep Q-Network. In Proceedings of the Annual Conference of Society of Instrument Control Engineers, Kanazawa, Japan, 19–22 September 2017; pp. 799–804. [Google Scholar]
- Chang, C.H. Deep and shallow architecture of multilayer neural networks. IEEE Neural Netw. Learn. Syst. 2015, 26, 2477–2486. [Google Scholar] [CrossRef]
- Mou, L.; Ghamisi, P.; Zhu, X.X. Deep recurrent neural networks for hyperspectral image classification. IEEE Geosci. Remote Sens. 2017, 55, 3639–3655. [Google Scholar] [CrossRef] [Green Version]
- Callet, P.L.; Viard-Gaudin, C.; Barba, D. A convolutional neural network approach for objective video quality assessment. IEEE Neural Netw. 2006, 17, 1316–1327. [Google Scholar] [CrossRef] [PubMed]
- Hou, J.C.; Wang, S.S.; Lai, Y.H.; Tsao, Y.; Chang, H.W.; Wang, H.M. Audio-visual speech enhancement using multimodal deep convolutional neural networks. IEEE Emerg. Top. Comput. Intell. 2018, 2, 117–128. [Google Scholar] [CrossRef] [Green Version]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2015, arXiv:1409.1556v6. Available online: https://arxiv.org/abs/1409.1556 (accessed on 5 December 2019).
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Huang, G.; Liu, Z.; Maaten, L.V.D.; Weinberger, K.Q. Densely connected convolutional networks. arXiv 2018, arXiv:1608.06993v5. Available online: https://arxiv.org/abs/1608.06993 (accessed on 5 December 2019).
- Szegedy, C.; Ioffe, S.; Vanhoucke, V. Inception-v4, Inception-ResNet and the impact of residual connections on learning. arXiv 2016, arXiv:1602.07261v2. Available online: https://arxiv.org/abs/1602.07261 (accessed on 5 December 2019).
- Chollet, F. Xception: Deep learning with depthwise separable convolutions. arXiv 2017, arXiv:1610.02357v3. Available online: https://arxiv.org/abs/1610.02357 (accessed on 8 December 2019).
- Shao, L.; Zhu, F.; Li, X. Transfer learning for visual categorization: A survey. IEEE Neural Netw. Learn. Syst. 2015, 26, 1019–1034. [Google Scholar] [CrossRef]
- Tompson, J.; Jain, A.; LeCun, Y.; Bregler, C. Joint training of a convolutional network and a graphical model for human pose estimation. arXiv 2014, arXiv:1406.2984v2. Available online: https://arxiv.org/abs/1406.2984 (accessed on 8 December 2019).
- Carreira, J.; Agrawal, P.; Fragkiadaki, K.; Malik, J. Human pose estimation with iterative error feedback. arXiv 2016, arXiv:1507.06550v3. Available online: https://arxiv.org/abs/1507.06550 (accessed on 8 December 2019).
- Pishchulin, L.; Insafutdinov, E.; Tang, S.; Andres, B.; Andriluka, M.; Gehler, P.; Schiele, B. DeepCut: Joint subset partition and labeling for multi person pose estimation. arXiv 2016, arXiv:1511.06645v2. Available online: https://arxiv.org/abs/1511.06645 (accessed on 8 December 2019).
- Girshick, R. Fast R-CNN. arXiv 2015, arXiv:1504.08083v2. Available online: https://arxiv.org/abs/1504.08083 (accessed on 8 December 2019).
- Insafutdinov, E.; Pishchulin, L.; Andres, B.; Andriluka, M.; Schiele, B. DeeperCut: A deeper, stronger, and faster multi-person pose estimation model. Adv. Concepts Intell. Vis. Syst. 2016, 9910, 34–50. [Google Scholar]
- Georgakopoulos, S.V.; Kottari, K.; Delibasis, K.; Plagianakos, V.P.; Maglogiannis, I. Pose recognition using convolutional neural networks on omni-directional images. Neurocomputing 2018, 280, 23–31. [Google Scholar] [CrossRef]
- Liu, Y.; Xu, Y.; Li, S.B. 2-D Human Pose Estimation from Images Based on Deep Learning: A Review. In Proceedings of the 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference, Xi’an, China, 25–27 May 2018; pp. 462–465. [Google Scholar]
- Lee, J.N.; Byeon, Y.H.; Kwak, K.C. Design of ensemble stacked auto-encoder for classification of horse gaits with MEMS inertial sensor technology. Micromachines 2018, 9, 411. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Maguolo, G.; Nanni, L.; Ghidoni, S. Ensemble of convolutional neural networks trained with different activation features. arXiv 2019, arXiv:1905.02473v4. Available online: https://arxiv.org/abs/1905.02473 (accessed on 10 December 2019).
- Kim, M.G.; Pan, S.B. Deep learning based on 1-D ensemble networks using ECG for real-time user recognition. IEEE Trans. Ind. Inform. 2019, 15, 5656–5663. [Google Scholar] [CrossRef]
- Kahlouche, S.; Ouadah, N.; Belhocine, M.; Boukandoura, M. Human Pose Recognition and Tacking Using RGB-D Camera. In Proceedings of the 8th International Conference on Modelling, Identification and Control, Algiers, Algeria, 15–17 November 2016; pp. 520–525. [Google Scholar]
- Na, Y.J.; Wang, C.W.; Jung, H.Y.; Ho, J.G.; Choi, Y.K.; Min, S.D. Real-Time Sleep Posture Recognition Algorithm Using Kinect System. In Proceedings of the Korean Institute of Electrical Engineers Conference on Biomedical System, Hoengseong, Korea, 15–17 February 2016; pp. 27–30. [Google Scholar]
- Kim, S.C.; Cha, J.H. Posture Recognition and Spatial Cognition with Hybrid Sensor. In Proceedings of the Korean Society of Precision Engineering Conference, Jeju, Korea, 29–31 May 2013; pp. 971–972. [Google Scholar]
- Giancola, S.; Valenti, M.; Sala, R. A survey on 3D cameras: Metrological comparison of time-of-flight, structured-light and active stereoscopy technologies. In Springer Briefs in Computer Science; Zdonik, S., Shekhar, S., Wu, X., Jain, L.C., Padua, D., Shen, X.S., Furht, B., Subrahmanian, V.S., Hebert, M., Ikeuchi, K., et al., Eds.; Springer Nature: Cham, Switzerland, 2018; pp. 29–39. ISBN 9783319917603. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An incremental improvement. arXiv 2018, arXiv:1804.02767v1. Available online: https://arxiv.org/abs/1804.02767 (accessed on 10 December 2019).
- Choi, H.S.; Lee, B.H.; Yoon, S.R. Biometric authentication using noisy electrocardiograms acquired by mobile sensors. IEEE Access 2016, 4, 1266–1273. [Google Scholar] [CrossRef]
- RMSprop Optimization Algorithm for Gradient Descent with Neural Networks. Available online: https://insidebigdata.com/2017/09/24/rmsprop-optimization-algorithm-gradient-descent-neural-networks/ (accessed on 16 May 2019).
- Zeiler, M.D. Adadelta: An adaptive learning rate method. arXiv 2012, arXiv:1212.5701v1. Available online: https://arxiv.org/abs/1212.5701 (accessed on 13 December 2019).
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2017, arXiv:1412.6980v9. Available online: https://arxiv.org/abs/1412.6980 (accessed on 13 December 2019).
|
|
|
|
|
|
|
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 | |
---|---|---|---|---|---|---|
SimpleCNN | Training | 91.50 | 70.82 | 81.12 | 85.42 | 68.87 |
Validation | 75.65 | 58.70 | 63.74 | 57.98 | 59.34 | |
Testing | 45.80 | 61.03 | 67.18 | 60.85 | 63.48 | |
VGG16 | Training | 89.40 | 88.44 | 91.72 | 85.87 | 85.76 |
Validation | 74.50 | 74.62 | 76.92 | 72.65 | 74.59 | |
Testing | 63.44 | 70.12 | 76.64 | 70.78 | 75.31 | |
VGG19 | Training | 90.09 | 91.17 | 89.56 | 83.06 | 86.18 |
Validation | 74.51 | 74.22 | 77.18 | 73.12 | 74.93 | |
Testing | 64.87 | 69.82 | 78.35 | 71.46 | 74.94 | |
ResNet50 | Training | 60.03 | 71.56 | 60.67 | 67.48 | 68.05 |
Validation | 58.44 | 69.10 | 59.92 | 65.61 | 65.64 | |
Testing | 55.60 | 66.83 | 57.92 | 65.55 | 61.81 | |
DenseNet201 | Train | 62.17 | 70.52 | 62.53 | 70.65 | 59.44 |
Validation | 60.68 | 67.56 | 61.27 | 68.44 | 58.34 | |
Test | 54.51 | 64.16 | 55.88 | 66.51 | 50.68 | |
InceptionResNetV2 | Train | 44.98 | 55.63 | 55.10 | 59.68 | 56.01 |
Validation | 43.86 | 53.67 | 54.71 | 59.39 | 55.90 | |
Test | 40.28 | 49.35 | 53.20 | 59.61 | 52.48 | |
Xception | Train | 43.56 | 57.12 | 51.18 | 57.21 | 56.52 |
Validation | 42.34 | 54.73 | 49.62 | 55.98 | 54.15 | |
Test | 34.74 | 50.03 | 46.09 | 54.66 | 52.32 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
SimpleCNN | 25.36 | 48.16 | 53.88 | 51.63 | 51.54 |
VGG16 | 48.95 | 62.63 | 67.34 | 63.65 | 66.79 |
VGG19 | 51.18 | 61.94 | 69.89 | 64.71 | 66.95 |
ResNet50 | 53.12 | 66.30 | 62.76 | 60.36 | 64.90 |
DenseNet201 | 55.01 | 62.59 | 61.03 | 61.37 | 57.10 |
InceptionResNetV2 | 43.79 | 56.24 | 55.60 | 54.68 | 58.26 |
Xception | 36.16 | 52.27 | 53.69 | 50.03 | 55.27 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
Raw | 56.29 | 74.40 | 79.53 | 72.92 | 78.46 |
VGG16 | 75.52 | 79.82 | 88.05 | 83.36 | 85.84 |
VGG19 | 76.23 | 80.67 | 88.86 | 83.67 | 86.47 |
ResNet50 | 74.91 | 84.35 | 78.73 | 82.39 | 80.43 |
DenseNet201 | 70.87 | 79.73 | 77.08 | 81.73 | 73.60 |
InceptionResNetV2 | 57.96 | 68.76 | 72.99 | 74.71 | 74.08 |
Xception | 52.08 | 66.72 | 66.19 | 67.19 | 69.41 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
Raw | 46.40 | 69.94 | 74.51 | 70.31 | 74.99 |
VGG16 | 69.71 | 77.73 | 85.25 | 82.17 | 83.32 |
VGG19 | 70.71 | 78.64 | 86.34 | 82.63 | 84.51 |
ResNet50 | 78.25 | 88.50 | 86.99 | 83.46 | 87.08 |
DenseNet201 | 75.36 | 82.73 | 85.59 | 82.44 | 83.35 |
InceptionResNetV2 | 64.67 | 77.96 | 79.46 | 75.49 | 83.11 |
Xception | 57.41 | 72.38 | 76.00 | 67.58 | 75.63 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 | |
---|---|---|---|---|---|---|
SimpleCNN | Training | 69.89 | 66.35 | 65.54 | 62.11 | 67.34 |
Validation | 62.92 | 59.47 | 63.02 | 43.42 | 61.97 | |
Testing | 42.70 | 52.63 | 64.35 | 50.00 | 60.15 | |
VGG16 | Training | 82.07 | 82.77 | 86.78 | 81.40 | 82.16 |
Validation | 73.73 | 73.02 | 76.40 | 72.37 | 74.29 | |
Testing | 61.62 | 67.35 | 76.05 | 68.84 | 72.01 | |
VGG19 | Training | 82.69 | 83.98 | 85.57 | 82.80 | 83.27 |
Validation | 73.13 | 73.21 | 76.74 | 72.64 | 74.14 | |
Testing | 62.86 | 67.39 | 76.54 | 69.71 | 73.67 | |
ResNet50 | Training | 58.33 | 67.89 | 57.76 | 66.10 | 62.82 |
Validation | 56.74 | 64.85 | 55.84 | 63.07 | 61.36 | |
Testing | 53.43 | 60.67 | 49.63 | 63.04 | 54.15 | |
DenseNet201 | Training | 68.33 | 71.84 | 73.12 | 67.96 | 67.00 |
Validation | 66.35 | 17.27 | 70.03 | 18.34 | 63.61 | |
Testing | 57.04 | 66.41 | 64.25 | 61.08 | 54.24 | |
InceptionResNetV2 | Training | 50.22 | 59.04 | 58.68 | 58.99 | 60.40 |
Validation | 48.10 | 15.05 | 56.78 | 15.15 | 58.42 | |
Testing | 44.58 | 52.55 | 52.91 | 56.69 | 57.27 | |
Xception | Training | 44.31 | 57.97 | 54.08 | 55.36 | 57.00 |
Validation | 42.85 | 15.09 | 09.51 | 14.30 | 55.22 | |
Testing | 35.15 | 52.54 | 45.63 | 53.53 | 50.94 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
SimpleCNN | 24.20 | 38.35 | 49.95 | 40.17 | 46.41 |
VGG16 | 46.99 | 61.19 | 66.78 | 61.65 | 63.88 |
VGG19 | 48.94 | 60.45 | 67.47 | 62.40 | 65.12 |
ResNet50 | 48.85 | 64.00 | 58.69 | 58.22 | 59.42 |
DenseNet201 | 54.66 | 63.75 | 64.24 | 59.67 | 60.11 |
InceptionResNetV2 | 46.22 | 55.96 | 53.21 | 53.93 | 59.68 |
Xception | 38.02 | 54.36 | 50.43 | 47.67 | 53.82 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
Raw | 51.43 | 60.43 | 78.71 | 61.96 | 73.37 |
VGG16 | 72.37 | 78.44 | 87.52 | 82.19 | 83.76 |
VGG19 | 74.44 | 78.19 | 87.92 | 82.49 | 84.83 |
ResNet50 | 69.54 | 78.70 | 73.47 | 78.49 | 75.94 |
DenseNet201 | 71.00 | 80.70 | 81.17 | 79.79 | 75.83 |
InceptionResNetV2 | 61.67 | 71.71 | 73.74 | 72.34 | 77.13 |
Xception | 52.50 | 68.65 | 66.38 | 66.55 | 68.93 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 |
---|---|---|---|---|---|
Raw | 42.31 | 53.72 | 73.55 | 58.97 | 68.33 |
VGG16 | 66.13 | 77.50 | 84.75 | 81.15 | 81.78 |
VGG19 | 68.83 | 76.71 | 85.24 | 81.22 | 82.46 |
ResNet50 | 70.85 | 85.37 | 85.05 | 79.46 | 84.69 |
DenseNet201 | 73.14 | 82.72 | 85.44 | 83.68 | 84.92 |
InceptionResNetV2 | 67.04 | 78.69 | 79.21 | 74.61 | 83.64 |
Xception | 58.73 | 73.89 | 73.76 | 66.28 | 75.34 |
Single Model (%) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 | |
---|---|---|---|---|---|---|
VGG19 | Training | 91.67 | 90.99 | 91.20 | 89.88 | 87.37 |
Validation | 90.42 | 90.19 | 89.76 | 88.79 | 86.75 | |
Testing | 88.40 | 87.76 | 90.22 | 88.29 | 87.06 | |
ResNet50 | Training | 95.47 | 96.37 | 96.16 | 95.60 | 95.62 |
Validation | 92.70 | 92.30 | 93.54 | 92.04 | 92.54 | |
Testing | 89.99 | 88.77 | 91.78 | 88.43 | 90.48 | |
ResNet101 | Training | 96.81 | 96.02 | 96.11 | 95.16 | 95.58 |
Validation | 93.86 | 92.44 | 92.90 | 91.97 | 91.94 | |
Testing | 90.93 | 88.04 | 91.29 | 88.32 | 89.17 | |
InceptionResNetV2 | Training | 95.15 | 95.25 | 95.65 | 92.80 | 94.31 |
Validation | 93.78 | 91.89 | 94.01 | 92.04 | 91.90 | |
Testing | 92.54 | 88.98 | 93.32 | 89.50 | 91.09 |
Single Model (hh:mm) | Type 0 | Type 1 | Type 2 | Type 3 | Type 4 | |
---|---|---|---|---|---|---|
VGG19 | Training | 26:35 | 28:38 | 27:04 | 27:37 | 28:10 |
ResNet50 | Training | 10:39 | 12:22 | 12:09 | 08:02 | 08:45 |
ResNet101 | Training | 25:46 | 27:05 | 27:30 | 26:09 | 27:01 |
InceptionResNetV2 | Training | 53:23 | 57:54 | 59:09 | 58:25 | 61:44 |
Ensemble System | Majority | Average | |
---|---|---|---|
EV19TNet | Testing | 92.77 | 93.42 |
ER50TNet | Testing | 93.70 | 94.12 |
ER101TNet | Testing | 93.94 | 94.51 |
EIR2TNet | Testing | 94.76 | 95.34 |
Ensemble System | Majority | Average | |
---|---|---|---|
ET0MNet | Testing | 93.08 | 94.07 |
ET1MNet | Testing | 91.53 | 92.77 |
ET2MNet | Testing | 93.94 | 94.58 |
ET3MNet | Testing | 91.30 | 92.31 |
ET4MNet | Testing | 92.86 | 93.67 |
Ensemble System (hh:mm) | Majority | |
---|---|---|
EV19TNet | Training | 138:04 |
ER50TNet | Training | 51:57 |
ER101TNet | Training | 133:31 |
EIR2TNet | Training | 290:35 |
ET0MNet | Training | 116:23 |
ET1MNet | Training | 125:59 |
ET2MNet | Training | 125:52 |
ET3MNet | Training | 120:13 |
ET4MNet | Training | 125:40 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Byeon, Y.-H.; Lee, J.-Y.; Kim, D.-H.; Kwak, K.-C. Posture Recognition Using Ensemble Deep Models under Various Home Environments. Appl. Sci. 2020, 10, 1287. https://doi.org/10.3390/app10041287
Byeon Y-H, Lee J-Y, Kim D-H, Kwak K-C. Posture Recognition Using Ensemble Deep Models under Various Home Environments. Applied Sciences. 2020; 10(4):1287. https://doi.org/10.3390/app10041287
Chicago/Turabian StyleByeon, Yeong-Hyeon, Jae-Yeon Lee, Do-Hyung Kim, and Keun-Chang Kwak. 2020. "Posture Recognition Using Ensemble Deep Models under Various Home Environments" Applied Sciences 10, no. 4: 1287. https://doi.org/10.3390/app10041287