Comparing Sampling Strategies for Tackling Imbalanced Data in Human Activity Recognition
<p>Overview of the process used for splitting, oversampling, and evaluating the data.</p> "> Figure 2
<p>Activity distribution of the Opportunity dataset.</p> "> Figure 3
<p>Activity distribution of the PAMAP2 dataset.</p> "> Figure 4
<p>Activity distribution of the ADL dataset.</p> "> Figure 5
<p>Opportunity minority classes. Comparing the impact of DBM, NDBM, and CBM on activity recognition performance, using MLP for the most underrepresented activities <span class="html-italic">Open_Fridge</span>, <span class="html-italic">Open_Drawer3</span>, and <span class="html-italic">Close_Drawer3</span>. The reported means of <span class="html-italic">F</span>1 scores are obtained from 30 repetitions. The <span class="html-italic">F</span>1 score is in %.</p> "> Figure 6
<p>ADL minority classes. Comparing the impact of DBM, NDBM, and CBM on activity recognition performance, using MLP for the most underrepresented activities (<span class="html-italic">Going Up/Downstairs</span> (<span class="html-italic">GUDS</span>), <span class="html-italic">Standing Up, Walking and Going Up/Downstairs</span> (<span class="html-italic">SWGUDS</span>), and <span class="html-italic">Walking and Talking with Someone</span> (<span class="html-italic">WATWS</span>)). The reported means of <span class="html-italic">F</span>1 scores are obtained from 30 repetitions. The <span class="html-italic">F</span>1 score is in %.</p> "> Figure 7
<p>PAMAP2 minority classes. Comparing the impact of DBM, NDBM, and CBM on activity recognition performance, using MLP for the most underrepresented activities (ascending stairs, descending stairs, rope jumping, and running). The reported means of <span class="html-italic">F</span>1 scores are obtained from 30 repetitions. The <span class="html-italic">F</span>1 score is in %.</p> "> Figure 8
<p>Comparing run times in seconds of the proposed DBM and CBM for all training datasets. The number of samples in the training sets for the ADL, Opportunity, and PAMAP2 datasets were 11,776, 1569, and 6450, respectively.</p> "> Figure A1
<p><span class="html-italic">F</span>1 score of baseline (SVM), the proposed method, and the sampling methods for the Opportunity, PAMAP2, and ADL datasets. The reported means of <span class="html-italic">F</span>1 scores were obtained from 30 repetitions.</p> "> Figure A2
<p><span class="html-italic">F</span>1 score of baseline (RF), the proposed method, and the sampling methods for the Opportunity, PAMAP2, and ADL datasets. The reported means of <span class="html-italic">F</span>1 scores were obtained from 30 repetitions.</p> "> Figure A3
<p><span class="html-italic">F</span>1 score of baseline (LR), the proposed method, and the sampling methods for the Opportunity, PAMAP2 and ADL datasets. The reported means of <span class="html-italic">F</span>1 scores were obtained from 30 repetitions.</p> "> Figure A4
<p><span class="html-italic">F</span>1 score of baseline (KNN), the proposed method, and the sampling methods for the Opportunity, PAMAP2, and ADL datasets. The reported means of <span class="html-italic">F</span>1 scores were obtained from 30 repetitions.</p> ">
Abstract
:1. Introduction
- We evaluate six sampling methods (SMOTE, Random_SMOTE, SMOTE_Tomeklinks, MSMOTE, CBSO, and ProWSyn) as solutions to the class imbalance problem across three commonly used datasets.
- We introduce three novel hybrid sampling approaches and show how these build on and improve upon their constituent methods. These are (1) DBM, a distance-based method that combines SMOTE and Random_SMOTE, (2) NDBM, a noise detection-based method that combines SMOTE_Tomeklinks and MSMOTE, and (3) CBM, a cluster-based method that combines CBSO and ProWSyn.
- We compare how useful the sampling methods are to improve the learning from imbalanced human activity data using both shallow and deep machine learning algorithms. Specifically, we test KNN, Logistic regression (LR), Random Forest (RF) and Support Vector Machine (SVM), and a Multilayer perceptron (MLP) [18,19]. We show that the sampling methods are only useful to improve the performance of the MLP compared to the other classifiers for imbalanced human activity data.
2. Related Work
3. Sampling Methods
3.1. Distance-Based
3.2. Noise Detection-Based
3.3. Cluster-Based
3.4. Proposed Hybrid Methods
4. Datasets
4.1. Opportunity
4.2. PAMAP2
4.3. ADL
5. Data Analysis
5.1. Data Preprocessing
5.2. Parameters Setting
5.3. Evaluation Method
6. Results
6.1. Distance-Based Method (DBM)
6.2. Noise Detection-Based Method (NDBM)
6.3. Cluster-Based Method (CBM)
6.4. Comparing the Performance of the Proposed Sampling Approaches DBM, NDBM, and CBM
6.5. Results for Minority Activities
6.6. Run Times for DBM, NDBM, and CBM
6.7. Statistical Analysis
6.7.1. ANOVA on PAMAP2
6.7.2. Friedman Test on ADL and Opportunity
7. Discussion and Future Work
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Appendix A
Algorithms | Parameters | ADL | Opportunity | PAMAP2 |
---|---|---|---|---|
SVM | gamma | 0.1 | 0.1 | 0.1 |
C | 20 | 20 | 20 | |
kernel | rbf | rbf | rbf | |
max_iter | −1 | −1 | −1 | |
decision_function_shape | ovr | ovr | ovr | |
LR | multi_class | multinomial | multinomial | multinomial |
solver | newton-cg | sag | sag | |
max_iter | 250 | 250 | 250 | |
C | 2 | 2 | 2 | |
penalty | L2 | L2 | L2 | |
KNN | n_neighbors | 3 | 5 | 3 |
algorithm | auto | auto | auto |
Appendix B
Data | Classifier | F1 Score |
---|---|---|
ADL | KNN | 85.63 (±0.043) |
LR | 84.51 (±0.026) | |
MLP | 87.2 (±0.047) | |
RF | 82.76 (±0.037) | |
SVM | 90.76 (±0.037) | |
Opportunity | KNN | 31.36 (±0.052) |
LR | 26.03 (±0.012) | |
MLP | 28.85 (±0.017) | |
RF | 33.15 (±0.032) | |
SVM | 34.04 (±0.012) | |
PAMAP2 | KNN | 69.44 (±0.033) |
LR | 64.81 (±0.094) | |
MLP | 71.85 (±0.081) | |
RF | 71.72 (±0.057) | |
SVM | 75.18 (±0.06) |
Appendix C
References
- Ordóñez, F.J.; Roggen, D. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors 2016, 16, 115. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chen, K.; Zhang, D.; Yao, L.; Guo, B.; Yu, Z.; Liu, Y. Deep Learning for Sensor-Based Human Activity Recognition: Overview, Challenges, and Opportunities. ACM Comput. Surv. 2021, 54, 1–40. [Google Scholar] [CrossRef]
- Bao, L.; Intille, S.S. Activity Recognition from User-Annotated Acceleration Data BT-Pervasive Computing; Springer: Berlin/Heidelberg, Germany, 2004; pp. 1–17. [Google Scholar]
- Wang, J.; Chen, Y.; Gu, Y.; Xiao, Y.; Pan, H. SensoryGANs: An Effective Generative Adversarial Framework for Sensor-based Human Activity Recognition. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–8. [Google Scholar]
- Ward, J.A.; Lukowicz, P.; Troster, G.; Starner, T.E. Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers. IEEE Trans. Pattern Anal. 2006, 28, 1553–1567. [Google Scholar] [CrossRef] [Green Version]
- Van Laerhoven, K.; Gellersen, H.W. Spine versus porcupine: A study in distributed wearable activity recognition. In Proceedings of the Eighth International Symposium on Wearable Computers, Arlington, VA, USA, 31 October–3 November 2004; Volume 1, pp. 142–149. [Google Scholar] [CrossRef] [Green Version]
- Catellier, D.J.; Hannan, P.J.; Murray, D.M.; Addy, C.L.; Conway, T.L.; Yang, S.; Rice, J.C. Imputation of missing data when measuring physical activity by accelerometry. Med. Sci. Sport. Exerc. 2005, 37, S555–S562. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Saeed, A.; Ozcelebi, T.; Lukkien, J. Synthesizing and Reconstructing Missing Sensory Modalities in Behavioral Context Recognition. Sensors 2018, 18, 2967. [Google Scholar] [CrossRef] [Green Version]
- Ramamurthy, S.R.; Roy, N. Recent trends in machine learning for human activity recognition—A survey. In Interdisciplinary Reviews: Data Mining and Knowledge Discovery; Wiley Online Library: Hoboken, NJ, USA, 2018; Volume 8, pp. 1–11. [Google Scholar]
- Lago, P.; Matsuki, M.; Inoue, S. Achieving Single-Sensor Complex Activity Recognition from Multi-Sensor Training Data. arXiv 2020, arXiv:2002.11284. [Google Scholar]
- Anliker, U.; Ward, J.; Lukowicz, P.; Troster, G.; Dolveck, F.; Baer, M.; Keita, F.; Schenker, E.; Catarsi, F.; Coluccini, L.; et al. AMON: A wearable multiparameter medical monitoring and alert system. IEEE Trans. Inf. Technol. Biomed. 2004, 8, 415–427. [Google Scholar] [CrossRef] [Green Version]
- Bulling, A.; Blanke, U.; Schiele, B. A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 2014, 46, 1–33. [Google Scholar] [CrossRef]
- Kwon, H.; Tong, C.; Haresamudram, H.; Gao, Y.; Abowd, G.D.; Lane, N.D.; Plötz, T. IMUTube: Automatic Extraction of Virtual on-Body Accelerometry from Video for Human Activity Recognition. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies; ACM: New York, NY, USA, 2020; Volume 4. [Google Scholar] [CrossRef]
- Kaur, H.; Pannu, H.S.; Malhi, A.K. A systematic review on imbalanced data challenges in machine learning: Applications and solutions. ACM Comput. Surv. 2019, 52, 4. [Google Scholar] [CrossRef] [Green Version]
- Nweke, H.F.; Teh, Y.W.; Mujtaba, G.; Alo, U.R.; Al-garadi, M.A. Multi-sensor fusion based on multiple classifier systems for human activity identification. Human-Centric Comput. Inf. 2019, 9, 1. [Google Scholar] [CrossRef] [Green Version]
- He, H.; Garcia, E.A. Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 2009, 21, 1263–1284. [Google Scholar]
- Guo, H.; Li, Y.; Shang, J.; Gu, M.; Huang, Y.; Gong, B. Learning from class-imbalanced data: Review of methods and applications. Expert Syst. Appl. 2017, 73, 220–239. [Google Scholar]
- Cook, D.J.; Krishnan, N.C. Activity Learning: Discovering, Recognizing and Predicting Human Behavior from Sensor Data; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
- Yang, J.; Nguyen, M.N.; San, P.P.; Li, X.L.; Krishnaswamy, S. Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 25–31 July 2015. [Google Scholar]
- Chen, Y.; Shen, C. Performance Analysis of Smartphone-Sensor Behavior for Human Activity Recognition. IEEE Access 2017, 5, 3095–3110. [Google Scholar] [CrossRef]
- Nweke, H.F.; Teh, Y.W.; Al-garadi, M.A.; Alo, U.R. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Syst. Appl. 2018, 105, 233–261. [Google Scholar] [CrossRef]
- Ni, Q.; Fan, Z.; Zhang, L.; Nugent, C.D.; Clel, I.; Zhang, Y.; Zhou, N. Leveraging wearable sensors for human daily activity recognition with stacked denoising autoencoders. Sensors 2020, 20, 5114. [Google Scholar] [CrossRef]
- Vandewiele, G.; Dehaene, I.; Kovács, G.; Sterckx, L.; Janssens, O.; Ongenae, F.; De Backere, F.; De Turck, F.; Roelens, K.; Decruyenaere, J.; et al. Overly optimistic prediction results on imbalanced data: A case study of flaws and benefits when applying over-sampling. Artif. Intell. Med. 2021, 111, 101987. [Google Scholar] [CrossRef]
- Sousa Lima, W.; Souto, E.; El-Khatib, K.; Jalali, R.; Gama, J. Human Activity Recognition Using Inertial Sensors in a Smartphone: An Overview. Sensors 2019, 19, 3213. [Google Scholar] [CrossRef] [Green Version]
- Akbari, A.; Jafari, R. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In Proceedings of the IPSN 2019-18th International Conference on Information Processing in Sensor Networks, Montreal, QC, Canada, 16–18 April 2019; pp. 85–96. [Google Scholar]
- Dietterich, T.G. Ensemble methods in machine learning. In Proceedings of the International Workshop on Multiple Classifier Systems, Cagliari, Italy, 21–23 June 2000; pp. 1–15. [Google Scholar]
- Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
- Dong, Y.; Wang, X. A new over-sampling approach: Random-SMOTE for learning from imbalanced data sets. Lect. Notes Comput. Sci. 2011, 7091, 343–352. [Google Scholar]
- Zhu, X.; Wu, X. Class Noise vs. Attribute Noise: A Quantitative Study. Artif. Intell. Rev. 2004, 22, 177–210. [Google Scholar] [CrossRef]
- He, H.; Ma, Y. Imbalanced Learning: Foundations, Algorithms, and Applications, 1st ed.; Wiley-IEEE Press: Hoboken, NJ, USA, 2013. [Google Scholar]
- Frenay, B.; Verleysen, M. Classification in the Presence of Label Noise: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 2014, 25, 845–869. [Google Scholar] [CrossRef] [PubMed]
- Batista, G.E.A.P.A.; Prati, R.C.; Monard, M.C. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 2004, 6, 20–29. [Google Scholar] [CrossRef]
- Hu, S.; Liang, Y.; Ma, L.; He, Y. MSMOTE: Improving classification performance when training data is imbalanced. In Proceedings of the 2nd International Workshop on Computer Science and Engineering: WCSE 2009, Qingdao, China, 28–30 October 2009. [Google Scholar]
- Kubat, M.; Matwin, S. Addressing the Curse of Imbalanced Training Sets: One-Sided Selection. In Proceedings of the Fourteenth International Conference on Machine Learning, Nashville, TN, USA, 8–12 July 1997; pp. 179–186. [Google Scholar]
- Saez, J.A.; Luengo, J.; Stefanowski, J.; Herrera, F. SMOTE-IPF: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering. Inf. Sci. 2015, 291, 184–203. [Google Scholar] [CrossRef]
- Hoens, T.R.; Chawla, N.V. Imbalanced datasets: From sampling to classifiers. In Foundations, Algorithms, and Applications: In Imbalanced Learning; Wiley: Hoboken, NJ, USA, 2013; pp. 43–59. [Google Scholar] [CrossRef]
- Kovács, G. smote-variants: A Python Implementation of 85 Minority Oversampling Techniques. Neurocomputing 2019, 366, 352–354. [Google Scholar] [CrossRef]
- Barua, S.; Islam, M.; Murase, K. ProWSyn: Proximity Weighted Synthetic Oversampling Technique for Imbalanced Data Set Learning. Adv. Knowl. Discov. Data Min. 2013, 7819, 317–328. [Google Scholar]
- Chavarriaga, R.; Sagha, H.; Calatroni, A.; Digumarti, S.T.; Tröster, G.; Millán, J.D.; Roggen, D. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognit. Lett. 2013, 34, 2033–2042. [Google Scholar] [CrossRef] [Green Version]
- Reiss, A.; Stricker, D. Creating and benchmarking a new dataset for physical activity monitoring. In Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, Crete, Greece, 6–8 June 2012. [Google Scholar]
- Casale, P.; Pujol, O.; Radeva, P. Personalization and user verification in wearable systems using biometric walking patterns. Pers. Ubiquitous Comput. 2012, 16, 563–580. [Google Scholar] [CrossRef]
- Roggen, D.; Calatroni, A.; Rossi, M.; Holleczek, T.; Förster, K.; Tröster, G.; Lukowicz, P.; Bannach, D.; Pirkl, G.; Ferscha, A.; et al. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 2010 Seventh International Conference on Networked Sensing Systems (INSS), Kassel, Germany, 15–18 June 2010; pp. 233–240. [Google Scholar]
- Shoaib, M.; Bosch, S.; Incel, O.D.; Scholten, H.; Havinga, P.J.M. Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors 2016, 16, 426. [Google Scholar] [CrossRef]
- Tang, Q.; Vidrine, D.J.; Crowder, E.; Intille, S.S. Automated detection of puffing and smoking with wrist accelerometers. In Proceedings of the PervasiveHealth 2014: 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany, 20–23 May 2014; pp. 80–87. [Google Scholar]
- Lara, O.D.; Labrador, M.A. A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 2013, 15, 1192–1209. [Google Scholar] [CrossRef]
- Shoaib, M.; Bosch, S.; Scholten, H.; Havinga, P.J.M.; Incel, O.D. Towards detection of bad habits by fusing smartphone and smartwatch sensors. In Proceedings of the 2015 IEEE International Conference on Pervasive Computing and Communication Workshops, St. Louis, MO, USA, 23–27 March 2015; pp. 591–596. [Google Scholar]
- Pedregosa, F.; Varoquaux, G. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- Sokolova, M.; Lapalme, G. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 2009, 45, 427–437. [Google Scholar] [CrossRef]
- Pirttikangas, S.; Fujinami, K.; Nakajima, T. Feature Selection and Activity Recognition from Wearable Sensors. In Proceedings of the Third International Conference on Ubiquitous Computing Systems UCS’06, Seoul, Korea, 11–13 October 2006; Springer: Berlin/Heidelberg, Germany, 2006; pp. 516–527. [Google Scholar] [CrossRef]
- Ghorbani, R.; Ghousi, R. Comparing Different Resampling Methods in Predicting Students’ Performance Using Machine Learning Techniques. IEEE Access 2020, 8, 67899–67911. [Google Scholar] [CrossRef]
- Shahi, A.; Deng, J.D.; Woodford, B.J. A streaming ensemble classifier with multi-class imbalance learning for activity recognition. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017; pp. 3983–3990. [Google Scholar]
- Jantschi, L.; Bolboaca, S.D. Computation of Probability Associated with Anderson-Darling Statistic. Mathematics 2018, 6, 88. [Google Scholar] [CrossRef] [Green Version]
- Demšar, J. Statistical Comparisons of Classifiers over Multiple Data Sets. J. Mach. Learn. Res. 2006, 7, 1–30. [Google Scholar]
- Fisher, R.A. Statistical Methods and Scientific Inference; Hafner Publishing Co.: Oxford, UK, 1956; p. viii 175. [Google Scholar]
- Kaur, A.; Kaur, I. An empirical evaluation of classification algorithms for fault prediction in open source projects. J. King Saud-Univ.-Comput. Inf. Sci. 2018, 30, 2–17. [Google Scholar] [CrossRef] [Green Version]
Dataset | Number of Subjects | Sample Rate | Window Size (s) | Sensor Position | Number of Sensors Used |
---|---|---|---|---|---|
Opportunity | 4 | 32 | 2 | Right Arm | 1 accelerometer |
PAMAP2 | 8 | 100 | 3 | Dominant Wrist | 1 accelerometer |
ADL | 15 | 52 | 10 | Chest | 1 accelerometer |
Feature | Description |
---|---|
Mean | It provides the average value of sensor data within a segment |
Standard deviation | It describes how much sensor data are spread around the mean |
Minimum | The minimum value of sensor data within a segment |
Maximum | The maximum value of sensor data within a segment |
Median | It finds the middle number of a sample within a segment |
Range | The difference between the maximum and the minimum of sensor data within a segment |
Hidden Layers | Activation Function | Optimizer | Loss Function | Learning Rate | Regularization | Epochs |
---|---|---|---|---|---|---|
100 | Relu | Adam | Cross-entropy | 0.001 | L2 penalty | 200 |
Data | Method | F1 Score | Recall | Precision |
---|---|---|---|---|
ADL | Baseline | 87.2 (±0.047) | 87.03 | 89.02 |
SMOTE | 92.24 (±0.069) | 91.44 | 94.21 | |
Random_SMOTE | 91.07 (±0.086) | 90.31 | 93.22 | |
DBM | 92.59 (±0.081) | 91.9 | 94.26 | |
Opportunity | Baseline | 28.85 (±0.017) | 34.1 | 29.57 |
SMOTE | 42.95 (±0.043) | 42.45 | 45.73 | |
Random_SMOTE | 42.74 (±0.04) | 42.19 | 45.75 | |
DBM | 48.49 (±0.052) | 48.18 | 50.63 | |
PAMAP2 | Baseline | 71.85 (±0.081) | 72.73 | 75.49 |
SMOTE | 74.73 (±0.055) | 74.93 | 77.69 | |
Random_SMOTE | 74.59 (±0.055) | 74.64 | 77.83 | |
DBM | 80.15 (±0.046) | 80.23 | 81.93 |
Data | Method | F1 Score | Recall | Precision |
---|---|---|---|---|
ADL | Baseline | 87.2 (±0.047) | 87.03 | 89.02 |
SMOTE_TomekLinks | 91.41 (±0.071) | 90.52 | 93.56 | |
MSMOTE | 90.7 (±0.067) | 89.65 | 92.66 | |
NDBM | 92.7 (±0.065) | 91.69 | 94.77 | |
Opportunity | Baseline | 28.85 (±0.017) | 34.1 | 29.57 |
SMOTE_TomekLinks | 42.89 (±0.039) | 43.15 | 45.34 | |
MSMOTE | 39.71 (±0.074) | 39.58 | 42.07 | |
NDBM | 46.95 (±0.067) | 46.97 | 48.86 | |
PAMAP2 | Baseline | 71.85 (±0.081) | 72.73 | 75.49 |
SMOTE_TomekLinks | 74.24 (±0.054) | 74.51 | 77.13 | |
MSMOTE | 73.73 (±0.059) | 73.78 | 77.03 | |
NDBM | 79.43 (±0.054) | 79.46 | 81.35 |
Data | Method | F1 Score | Recall | Precision |
---|---|---|---|---|
ADL | Baseline | 87.2 (±0.047) | 87.03 | 89.02 |
CBSO | 91.16 (±0.09) | 90.22 | 93.66 | |
ProWSyn | 91.56 (±0.091) | 90.98 | 93.7 | |
CBM | 92.96 (0.087) | 91.93 | 95.29 | |
Opportunity | Baseline | 28.85 (±0.017) | 34.1 | 29.57 |
CBSO | 42.92 (±0.023) | 42.96 | 45.12 | |
ProWSyn | 42.78 (±0.055) | 43.47 | 44.99 | |
CBM | 48.87 (±0.045) | 48.82 | 50.67 | |
PAMAP2 | Baseline | 71.85 (±0.081) | 72.73 | 75.49 |
CBSO | 75.69 (±0.042) | 75.43 | 78.19 | |
ProWSyn | 74.42 (±0.054) | 74.4 | 77.5 | |
CBM | 80.98 (±0.051) | 80.9 | 82.54 |
Data | Method | F1 Score | Recall | Precision |
---|---|---|---|---|
ADL | Baseline | 87.2 (±0.047) | 87.03 | 89.02 |
DBM | 92.59 (±0.081) | 91.9 | 94.26 | |
NDBM | 92.7 (±0.065) | 91.69 | 94.77 | |
CBM | 92.96 (±0.087) | 91.93 | 95.29 | |
Opportunity | Baseline | 28.85 (±0.017) | 34.1 | 29.57 |
DBM | 48.49 (±0.052) | 48.18 | 50.63 | |
NDBM | 46.95 (±0.067) | 46.97 | 48.86 | |
CBM | 48.87 (±0.045) | 48.82 | 50.67 | |
PAMAP2 | Baseline | 71.85 (±0.081) | 72.73 | 75.49 |
DBM | 80.15 (±0.046) | 80.23 | 81.93 | |
NDBM | 79.43 (±0.054) | 79.46 | 81.35 | |
CBM | 80.98 (±0.051) | 80.9 | 82.54 |
Data | Mean | Standard Deviation | Sample Size | p-Value |
---|---|---|---|---|
ADL | 0.8840 | 0.0399 | 45 | 0.0007 |
Opportunity | 0.3773 | 0.0548 | 45 | 0.0000 |
PAMAP2 | 0.7272 | 0.0406 | 45 | 0.0680 |
Data | Degrees of Freedom | Sum of Squares | Mean Square | F Value | p-Value |
---|---|---|---|---|---|
PAMAP2 | 8 | 0.0067 | 0.0008 | 0.4602 | 0.8757 |
Data | Degrees of Freedom | Chi-Square | p-Value |
---|---|---|---|
ADL | 8 | 21.8133 | 0.0053 |
Opportunity | 8 | 24.2133 | 0.0021 |
Classifier | CBSO | NDBM | CBM | DBM | MSMOTE | Pro-WSyn | Random_SMOTE | SMOTE_TomekLinks | SMOTE |
---|---|---|---|---|---|---|---|---|---|
KNN | 1 | 7 | 9 | 4 | 5 | 8 | 2 | 6 | 3 |
LR | 1 | 8 | 3 | 9 | 2 | 5 | 6 | 7 | 4 |
MLP | 3 | 8 | 9 | 7 | 1 | 5 | 2 | 4 | 6 |
RF | 1 | 6 | 9 | 4 | 7 | 8 | 3 | 5 | 2 |
SVM | 1 | 8 | 7 | 9 | 2 | 3 | 6 | 4 | 5 |
Sum of ranks | 7 | 37 | 37 | 33 | 17 | 29 | 19 | 26 | 20 |
Classifier | CBSO | NDBM | CBM | DBM | MSMOTE | Pro-WSyn | Random_SMOTE | SMOTE_TomekLinks | SMOTE |
---|---|---|---|---|---|---|---|---|---|
KNN | 5 | 6 | 9 | 7 | 1 | 4 | 8 | 3 | 2 |
LR | 5 | 9 | 7 | 8 | 1 | 2 | 6 | 4 | 3 |
MLP | 5 | 7 | 9 | 8 | 1 | 3 | 2 | 4 | 6 |
RF | 4 | 5 | 8 | 3 | 1 | 9 | 7 | 6 | 2 |
SVM | 2 | 7 | 8 | 9 | 1 | 4 | 3 | 5 | 6 |
Sum of ranks | 21 | 34 | 41 | 35 | 5 | 22 | 26 | 22 | 19 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alharbi, F.; Ouarbya, L.; Ward, J.A. Comparing Sampling Strategies for Tackling Imbalanced Data in Human Activity Recognition. Sensors 2022, 22, 1373. https://doi.org/10.3390/s22041373
Alharbi F, Ouarbya L, Ward JA. Comparing Sampling Strategies for Tackling Imbalanced Data in Human Activity Recognition. Sensors. 2022; 22(4):1373. https://doi.org/10.3390/s22041373
Chicago/Turabian StyleAlharbi, Fayez, Lahcen Ouarbya, and Jamie A Ward. 2022. "Comparing Sampling Strategies for Tackling Imbalanced Data in Human Activity Recognition" Sensors 22, no. 4: 1373. https://doi.org/10.3390/s22041373