Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG
"> Figure 1
<p>Hand gestures performed by each subject in this study the neutral or rest position are shown in (<b>A).</b> The gestures are: (<b>B</b>) hand open, (<b>C</b>) hand close, (<b>D</b>) pronation (forearm), (<b>E</b>) supination (forearm), (<b>F</b>) extension (wrist), (<b>G</b>) flexion (wrist), (<b>H</b>) side grip (<b>I),</b> fine grip (<b>J),</b> pointer and (<b>K</b>) agree.</p> "> Figure 2
<p>Architecture of the convolutional neural network.</p> "> Figure 3
<p>Mean classification error averaged for all subjects for each learning rate across different training iterations (lower is better).</p> "> Figure 4
<p>Performance comparison of the network at different learning rates for classification of individual gestures. Values closer to circumference indicate better performance and values closer to origin represent poor performance.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Subjects
2.2. Data Acquisition
2.3. Experiment Protocol
2.4. Convolutional Neural Network (CNN)
2.5. Parametric Optimization
2.6. Hyper-Parameters
2.7. Analysis
3. Results
3.1. Learning Rate vs. Epochs
3.2. Subject-Wise Average Performance
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Scheme, E.; Englehart, K. Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. J. Rehabil. Res. Dev. 2011, 48, 643–659. [Google Scholar] [CrossRef] [PubMed]
- Phinyomark, A.; Phukpattaranont, P.; Limsakul, C. A review of control methods for electric power wheelchairs based on electromyography signals with special emphasis on pattern recognition. Iete Tech. Rev. 2011, 28, 316–326. [Google Scholar] [CrossRef]
- Saponas, T.S.; Tan, D.S.; Morris, D.; Balakrishnan, R.; Turner, J.; Landay, J.A. Enabling always-available input with muscle-computer interfaces. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, Victoria, BC, Canada, 4 October 2009; pp. 167–176. [Google Scholar]
- Yousefi, J.; Hamilton-Wright, A. Characterizing EMG data using machine-learning tools. Comput. Biol. Med. 2014, 51, 1–13. [Google Scholar] [CrossRef] [PubMed]
- Geethanjali, P. Myoelectric control of prosthetic hands: State-of-the-art review. Med Devices 2016, 9, 247. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Roche, A.D.; Rehbaum, H.; Farina, D.; Aszmann, O.C. Prosthetic myoelectric control strategies: A clinical perspective. Curr. Surg. Rep. 2014, 2, 44. [Google Scholar] [CrossRef]
- Waris, A.; Niazi, I.K.; Jamil, M.; Englehart, K.; Jensen, W.; Kamavuako, E.N. Multiday evaluation of techniques for EMG based classification of hand motions. IEEE J. Biomed. Health Inform. 2018, 23, 1526–1534. [Google Scholar] [CrossRef] [PubMed]
- Qiu, J.; Wu, Q.; Ding, G.; Xu, Y.; Feng, S. A survey of machine learning for big data processing. Eurasip J. Adv. Signal Process. 2016, 2016, 67. [Google Scholar] [CrossRef] [Green Version]
- Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 2015, 61, 85–117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Park, K.-H.; Lee, S.-W. Movement intention decoding based on deep learning for multiuser myoelectric interfaces. In Proceedings of the 2016 4th International Winter Conference on Brain-Computer Interface (BCI), Gangwon-do, Korea, 22–24 February 2016; pp. 1–2. [Google Scholar]
- Atzori, M.; Cognolato, M.; Müller, H. Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands. Front. Neurorobotics 2016, 10, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Cote-Allard, U.; Fall, C.L.; Campeau-Lecours, A.; Gosselin, C.; Laviolette, F.; Gosselin, B. Transfer learning for sEMG hand gestures recognition using convolutional neural networks. In Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada, 5–8 October 2017; pp. 1663–1668. [Google Scholar]
- Zia ur Rehman, M.; Waris, A.; Gilani, S.; Jochumsen, M.; Niazi, I.; Jamil, M.; Farina, D.; Kamavuako, E. Multiday EMG-based classification of hand motions with deep learning techniques. Sensors 2018, 18, 2497. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Waris, A.; Niazi, I.K.; Jamil, M.; Gilani, O.; Englehart, K.; Jensen, W.; Shafique, M.; Kamavuako, E.N. The effect of time on EMG classification of hand motions in able-bodied and transradial amputees. J. Electromyogr. Kinesiol. 2018, 40, 72–80. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zhai, X.; Jelfs, B.; Chan, R.H.; Tin, C. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front. Neurosci. 2017, 11, 379. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chen, L.; Fu, J.; Wu, Y.; Li, H.; Zheng, B. Hand Gesture Recognition Using Compact CNN Via Surface Electromyography Signals. Sensors 2020, 20, 672. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Huang, D.; Chen, B. Surface EMG Decoding for Hand Gestures Based on Spectrogram and CNN-LSTM. Proceedings of 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI), Xi’an, China, 21–22 September 2019; pp. 123–126. [Google Scholar]
- Tsinganos, P.; Cornelis, B.; Cornelis, J.; Jansen, B.; Skodras, A. Deep Learning in EMG-based Gesture Recognition. In Proceedings of the PhyCS, Seville, Spain, 19–21 September 2018; pp. 107–114. [Google Scholar]
- Pinzón-Arenas, J.O.; Jiménez-Moreno, R.; Herrera-Benavides, J.E. Convolutional Neural Network for Hand Gesture Recognition using 8 different EMG Signals. In Proceedings of the 2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA), Bucaramanga, Colombia, 24–26 April 2019; pp. 1–5. [Google Scholar]
- Bergstra, J.; Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 2012, 13, 281–305. [Google Scholar]
- Ameri, A.; Akhaee, M.A.; Scheme, E.; Englehart, K. Real-time, simultaneous myoelectric control using a convolutional neural network. PLoS ONE 2018, 13, e0203835. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ortiz-Catalan, M.; Brånemark, R.; Håkansson, B. BioPatRec: A modular research platform for the control of artificial limbs based on pattern recognition algorithms. Source Code Biol. Med. 2013, 8, 11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Smith, L.N. Cyclical learning rates for training neural networks. In Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 24–31 March 2017; pp. 464–472. [Google Scholar]
- Smith, S.L.; Le, Q.V. A bayesian perspective on generalization and stochastic gradient descent. arXiv 2017, arXiv:1710.06451. [Google Scholar]
- Bengio, Y. Practical recommendations for gradient-based training of deep architectures. In Neural Networks: Tricks of the Trade; Springer: Berlin/Heidelberg, Germany, 2012; pp. 437–478. [Google Scholar]
- Glorot, X.; Bordes, A.; Bengio, Y. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Lauderdale, FL, USA, 11–13 April 2011; pp. 315–323. [Google Scholar]
- Cho, K.; Raiko, T.; Ihler, A.T. Enhanced gradient and adaptive learning rate for training restricted Boltzmann machines. In Proceedings of the 28th international conference on machine learning (ICML-11), Washington, DC, USA, 28 June–2 July 2011; pp. 105–112. [Google Scholar]
- Bordes, A.; Bottou, L.; Gallinari, P. SGD-QN: Careful quasi-Newton stochastic gradient descent. J. Mach. Learn. Res. 2009, 10, 1737–1754. [Google Scholar]
- Englehart, K.; Hudgins, B. A robust, real-time control scheme for multifunction myoelectric control. Ieee Trans. Bio-Med Eng. 2003, 50, 848–854. [Google Scholar] [CrossRef] [PubMed]
- He, K.; Sun, J. Convolutional neural networks at constrained time cost. In Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, 7–12 June 2015; pp. 5353–5360. [Google Scholar]
Learning Rate | Epochs | ||||
---|---|---|---|---|---|
20 | 40 | 60 | 80 | 100 | |
0.00001 | 50.9% | 40.1% | 33.4% | 29.5% | 28.8% |
0.0001 | 20.7% | 14.1% | 10.4% | 10.2% | 10% |
0.001 | 14.1% | 11.3% | 9.8% | 8.3% | 8% |
0.01 | 31.2% | 25% | 23.4% | 20.7% | 23% |
0.1 | 68.8% | 66.1% | 66% | 63.4% | 58.8% |
Subjects | Learning Rate Average Classification Accuracy (%) | ||||
---|---|---|---|---|---|
0.00001 | 0.0001 | 0.001 | 0.01 | 0.1 | |
Subject 1 | 52.9 | 63 | 87.3 | 86.1 | 13.3 |
Subject 2 | 57.6 | 89.2 | 88.4 | 67.1 | 31.9 |
Subject 3 | 68.8 | 93.5 | 92.3 | 79.3 | 24.3 |
Subject 4 | 70.5 | 90 | 92.6 | 82.3 | 30.6 |
Subject 5 | 74.6 | 92.6 | 93.5 | 86.8 | 45.5 |
Subject 6 | 68.4 | 90 | 94.2 | 75.8 | 40 |
Subject 7 | 57.8 | 89.6 | 88.6 | 67.2 | 31.6 |
Subject 8 | 63.1 | 85.4 | 89.4 | 77.7 | 39 |
Subject 9 | 56 | 88.3 | 90.5 | 73.6 | 24.4 |
Subject 10 | 61 | 85.4 | 88.2 | 67.7 | 31.5 |
Subject 11 | 64 | 83.3 | 87.3 | 77.3 | 42 |
Subject 12 | 66.4 | 89.4 | 90 | 76.6 | 42.5 |
Subject 13 | 61.7 | 83.3 | 88.4 | 81.3 | 42.3 |
Subject 14 | 59 | 85.8 | 87.4 | 53.8 | 31.6 |
Subject 15 | 70.5 | 90.4 | 87.5 | 69.8 | 32.6 |
Subject 16 | 66.8 | 84.4 | 86.7 | 75.1 | 44.5 |
Subject 17 | 63.7 | 87.9 | 87.8 | 73.4 | 45.9 |
Subject 18 | 74.6 | 92.9 | 93.1 | 86.7 | 45.4 |
Learning Rate | Epochs | ||||
---|---|---|---|---|---|
20 | 40 | 60 | 80 | 100 | |
0.00001 | 49.1% | 59.9% | 66.6% | 70.5% | 71.2% |
0.0001 | 79.3% | 85.9% | 89.6% | 89.8% | 90% |
0.001 | 85.9% | 88.7% | 90.2% | 91.7% | 92% |
0.01 | 68.8% | 75% | 76.6% | 79.3% | 77% |
0.1 | 31.2% | 33.9% | 34% | 36.6% | 41.2% |
Learning Rate | Mean Classification Error |
---|---|
0.00001 | |
0.0001 | |
0.001 | |
0.01 | |
0.1 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Asif, A.R.; Waris, A.; Gilani, S.O.; Jamil, M.; Ashraf, H.; Shafique, M.; Niazi, I.K. Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG. Sensors 2020, 20, 1642. https://doi.org/10.3390/s20061642
Asif AR, Waris A, Gilani SO, Jamil M, Ashraf H, Shafique M, Niazi IK. Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG. Sensors. 2020; 20(6):1642. https://doi.org/10.3390/s20061642
Chicago/Turabian StyleAsif, Ali Raza, Asim Waris, Syed Omer Gilani, Mohsin Jamil, Hassan Ashraf, Muhammad Shafique, and Imran Khan Niazi. 2020. "Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG" Sensors 20, no. 6: 1642. https://doi.org/10.3390/s20061642