Wearables, Social Networking and Veracity : The Building Blocks of a Verified Exercise Application Chiung Ching Ho Mehdi Sharifi Faculty of Computing and Informatics Multimedia University Cyberjaya, Malaysia ccho@mmu.edumy Faculty of Computing and Informatics Multimedia University Cyberjaya, Malaysia omidthekaiser@gmail.com Abstract—Research and development of exercise recognition applications have predominantly focused on motion related exercise, with not much emphasis on weight lifting exercise. At the same time, while such applications supports the posting of completed exercise session on social network, the veracity of the post is entirely determined by the user of the application. In this paper, we present the building blocks for a weight lifting application. It recognizes and counts the number of repetitions of a weight lifting exercise, and subsequently posts it on the user's behalf, thus ensuring the veracity of the post. Our empirical results demonstrate the potential of such an application. Keywords-wearables, weight lifting exercise recognition, social media integration, machine learning I. INTRODUCTION The number of wearable devices for health and fitness continues to grow and is projected to reach 169.5 million by 2017 [1] . These devices are primarily are equipped with sensors such as accelerometers, gyroscopes, heart rate monitors, and galvanometers. Such devices are used to record health and fitness related metrics. Metrics measured are often heavily related to motions made during walking or running, and includes number of steps, heart-rate, distance travelled, and speed of movement, calories expended, bloodpressure and quality of sleep. These metrics are then aggregated by health and fitness applications to record the quantity and effectiveness of a monitored exercise session. However, to date, there are very few applications which are focused on weight lifting exercises. The reports of the health and fitness application are then often shared on social media. The sharing of beneficial exercise and fitness sessions on social media not only brings about increases in self-esteem but has been reported to increase physical activity [2], and weight loss [3]. The extra motivation and support to persist with exercise and fitness sessions via social networking adds to the effectiveness of using wearable devices for the purpose of weight loss [4], and reduction of calorie consumption [5]. Although social network support is important, it is equally important that the information shared on social networks with regards to health and fitness session be trustable to encourage greater participation. The effectiveness of using wearable devices for increased health and fitness activities is not only improved through participation in social networking, but can be further improved through financial benefits. Weight loss can be increased if there are financial benefits to do so [6], [7], [8]. Financial rewards that encourages weight loss need not only be monetary in nature, as studies have shown that increased health insurance benefits as a reward can similarly promote health and fitness activity [9]. The latter approach has seen increasing traction as the reduction in insurance pay-outs for weight-related diseases [10], [11] are often much more than the financial incentives offered for weight loss. The convergence of wearable devices and social network posting of health and fitness workouts offers another mean of rewarding and motivating individuals who are working out. Traditionally, verified weight loss is the main proof of the success of health and fitness sessions. The mining of social media data [12] in future will lead to similar insights provided the veracity of the social media health and fitness post is established. In this paper, we present the anatomy of a verified exercise application – an application which will be able to recognize the type of exercise performed and number of repetitions per exercise and post the completed session on social media with minimal human intervention. The reduction of human input ensures the veracity of the information posted, and can be used by organizations to reward and motivate individuals towards increased health and fitness activity. This paper is organized in the following order : Section I contains the Introduction, while Section II contains review of related literature. Section III presents the building blocks of the application and its related methodology, and is followed by Section IV which shows the results achieved using the same methodology using a reference data set. Section V presents the conclusion of the paper as well as future research directions. II. LITERATURE REVIEW A. Wearable devices and social networking for health and fitness The idea of using wearable devices for health and fitness activity recognition has its roots in wearable devices being used to measure day-to-day action recognition, which includes sitting, standing, walking, jogging, running, traversing a staircase, and riding of bicycles, cars and lifts [13], [14]. Subsequently, action recognition using wearable devices saw use for sports recognition, with sporting activities such as throwing a football, rowing on an exercise machine and Nordic walking being recognized [14]. Wearable devices are also used to promote physical networking, whereby a user’s digital information is exchanged through a physical handshake [15]. In digital social networking, social network allows users of social network linked health and fitness app to motivate one another, through the sharing of personal achievements. Examples of social networks dedicated to health and fitness includes Fitrocracy, Extra Pounds, Daily Mile, Traineo, MBodyment,Spark People, Map My Fitness, Endomondo etc. and so forth. B. Weight lifting exercise recognition While there have been many efforts to recognize motion based exercises, there have not been as much work reported for recognition of weight lifting exercises using free weights. This research gap needs to be addressed as resistance training through free-weight weight lifting exercises are just as important as cardiovascular exercise typically experienced while jogging or running [16]. At the same time, it has been reported that an individual will tend to use their arm more than they use their legs in their daily living [17]. The placement of wearable devices for weight lifting exercise recognition is important as it affect the reading of sensors on the wearable devices, typically accelerometers and gyroscopes. Acceleration values are recorded by accelerometers across three axes; while the gyroscope measures rotation of the device (a positive value on the gyroscope reading infers a counter-clockwise movement). Recofit [18] placed the sensor on the right forearm, while [19] had the sensor worn on the hand (glove) and on the waist. myHealthAssistant [20] utilized sensors placed on the chest and waist to recognize exercises such as the bicep curl. C. Veracity of social network post The idea of veracity of a social network post can be loosely divided into a dichotomy which addresses the following:  Claim to identity  Truth of content Claim of identity is when the person who posts a social media post has their identity verified. This is normally done through a pairwise check of a username and password combination. Claim of identity is important as a first line of defense against attempts to “tweetjack”, where a social network account is used fraudulently to post content without the account owner’s consent. In light of recent attacks on username and passwords databases, a more sophisticated way of guaranteeing claim to identity might be to use secured biometric access , using modalities such as fingerprints [21] or multimodal biometrics [22]. Establishing the truth of content posted on social network is a difficult task. Current approaches tie the truth of content to the trustworthiness of the social network account. The social network is frequently modelled as a graph to infer trust levels. Behavioral and propagation trust, focusing on retweets was used as a measure of trust [23], while random walks over a graph was used as a trust indicator [24]. For group interaction over social networks, trust was measured as a factor of the propagation of their network [25]. A mid-way approach to affirm the veracity of a social network post is to implement claim of identity with automated content. Firstly, authenticity is proven, akin to Twitter’s Verified account status [26] or Facebook’s Verified page [27]. Subsequently, automated post is posted on the user’s behalf with minimal user input or interaction. Such an approach has been proven useful for the purpose of public service announcement. The template is used to format your paper and style the text. All margins, column widths, line spaces, and text fonts are prescribed; please do not alter them. You may note peculiarities. For example, the head margin in this template measures proportionately more than is customary. This measurement and others are deliberate, using specifications that anticipate your paper as one part of the entire proceedings, and not as an independent document. Please do not revise any of the current designations. D. Autocorrelation peaks Weight lifting exercises are periodic in nature, with distinct and repeated movements. In this respect, it is highly similar to highly periodic signals in other domains, such as musical pitch, or EKG readings. In those domains, autocorrelation has been used successfully to detect periodicity. The peaks of an auto-correlated signal from accelerometer have been used to indicate gait cycles [28], and have been used to count repetitions of exercises [29], especially if the signals were taken from one axis. III. APPLICATION BUILDING BLOCKS AND METHODOLOGY A. Application Overview The application has input from a Texas Instruments (TI) eZ430 Chronos Wireless Watch, which is equipped with sensors such as a three axis accelerometer and a pressure sensor. It is connected wirelessly to a USB emulator wireless for data logging and programming purposes [30]. Figure 1 shows a TI eZ430 Chronos Wireless Watch. Figure 1. Texas Instruments (TI) eZ430 Chronos Wireless Watch The eZ430 watch is worn on the wrist as the user performs weight lifting exercises. Signals from the three axis accelerometer is subsequently read through the USB emulator device and is stored offline for exercise recognition. Figure 2 shows the block diagram of the application. Figure 3. Methodology for weight lifting exercise recognition application Smoothing of the accelerometer signal is a necessary step to reduce noise caused by jittery movements during the weight lifting exercise, which by itself may be due to bad form during the execution of the weight lifting exercise. In our work, a Butterworth low-pass filter is applied for smoothing purposes for accelerometer signals from all three axes X, Y and Z. Equation (1) shows the equation for a Butterworth filter.  Figure 2. Block diagram of weight lifting exercise recognition application The weight lifting exercise application is composed of several modules. The accelerometer values from the eZ430 watch is stored offline in a database, where it will be preprocessed. Preprocessing will involve filtering, windowing and feature extraction as described in Section III B. Subsequently the preprocessed signal will undergo peak counting for the purpose of repetition counting, using an autocorrelation process. Features extracted will be used for the weight lifting exercise recognition task. B. Methodology for weight lifting exercise recognition The methodology used for weight lifting recognition is shown in Figure 3. H  j    1   1 w0  2n   Subsequently, a five second sliding window is applied to the smoothed signal. Feature extraction is performed on the accelerometer values. The features used for the purpose of weight lifting exercise recognition are statistical features, and it includes the following:  Mean  Max  Median  Min  Standard deviation  Variance The features are extracted from the three axes X, Y, and Z which results in a feature vector which has a length of 18. The extracted feature vectors will then be used for the purpose of weight lifting exercise recognition. Each feature vector will be labeled with the corresponding weight lifting exercise type. Upon classification, a weight lifting exercise session will be labeled with the highest scoring label, and the result is stored in a data store. Number of repetition per session is counted by applying an autocorrelation function, and subsequently the number of autocorrelation peaks will be counted. Each peak represents one repetition, and the result will be stored in a data store. The results of the weight lifting exercise recognition process and repetition process is then posted on a social network using the user's credentials to complete the process. IV. PRELIMINARY RESULTS AND DISCUSSION The methodology presented in Section III was evaluated on a reference data set collected by Velloso et al. [31]. In this dataset, six participants performed five Unilateral Dumbbell Bicep Curls of different varieties with 10 repetitions. The five variations are:  exactly according to the specification (Class A)  throwing the elbows to the front (Class B)  lifting the dumbbell only halfway (Class C)  lowering the dumbbell only halfway (Class D)  throwing the hips to the front (Class E) Sensors are placed on the dumbbell, and are also placed on the arm, forearm and belt. Each sensor can capture accelerometer, gyroscope, magnetometer, roll, pitch, and yaw values, however, we chose to use forearm accelerometer values only for the purpose of weight lifting exercise recognition. Velloso et al. focused on the quality of each weight lifting exercise session, while we focused on weight lifting exercise recognition. Forearm accelerometer values in the dataset were filtered using a Butterworth filter for smoothing purposes as part of the preprocessing process. Sliding windows of 200 samples with slide size of five samples were subsequently applied. The preprocessed signal then underwent feature extraction to extract an 18-length feature vector. The extracted feature vector was then used for classification purposes. Classification was performed using the WEKA data mining toolkit [32], using a 10-fold approach. The results of the classification experiments are shown in Table 1. TABLE I. Classifier Random Forest k-NN C 4.5 The choice of classifiers used, and their corresponding measures, builds on the work done by Velloso et al. Random Forest was chosen as a classifier to accommodate the inherent noise in sensor data. K-Nearest Neighbor (kNN) was chosen as a classifier as it is similarly nonparametric in nature, and has been used successfully for related accelerometer based classification. C 4.5 was chosen to contrast with Random Forest, to investigate the effects of using a random forest verses a single decision tree. As the experiment's aim was to classify multiple classes, an aggregated result is presented. False Positive Rates (FPR), Recall and Area under ROC Curve (AUC), and Precision for all classes of exercise are weighted proportionate to the number of their instances. This approach is a more accurate description of the result obtained as compared to simply averaging the results across classes. Table 2 shows the results for individual classes using the best performing classifier, Random Forest. TABLE II. INDIVIDUAL EXERCISE RECOGNITION RESULTS Exercise Class FPR Recall A B C D E 0.002 0.001 0.002 0.002 0.001 0.993 0.992 0.992 0.996 0.992 Precision reported in [31] 0.976 0.973 0.982 0.981 0.991 Precision 0.990 0.995 0.992 0.993 0.994 The results obtained thus far exceed the results reported by Velloso et al., who reported a weighted accuracy result of 0.982. Results for individual exercise classes in our work also exceeded those reported by Velloso et al. We also replicated the leave-one-subject-out test performed by Velloso et al. to test on whether the classifier will perform as well when subjected to new data. A user was randomly selected to be left out, and was used as the testing set. The results of this experiment are shown in Table 3. TABLE III. RESULTS OF LEAVE-ONE-SUBJECT-OUT EXPERIMENT Classifier Precision TPR FPR C.4.5 0.729 0.654 0.0083 RANDOM FOREST 0.631 0.615 0.0097 WEGHTED EXERCISE RECOGNITION RESULTS Weighted FPR 0.002 Weighted Recall 0.993 Weighted AUC 1.000 Weighted Precision 0.993 0.02 0.004 0.991 0.986 0.995 0.993 0.991 0.986 The results achieved using the proposed methodology is very encouraging, and it suggests the following:  As the data used was recorded from the sensor placed on the forearm, this suggest that similar outcomes can be expected using the TI eZ430 Chronos Wireless Watch which is worn on the     wrist during the recording of weight lifting exercises A minimal (from Velloso et al.'s initial 96 long feature vector to our 18 long feature vector) feature vector extracted from accelerometer data alone is sufficient for effective weight lifting exercise recognition It is important to smooth accelerometer data from sensors, and good results can be obtained by using the Butterworth filter The Random Forest classifier is a good choice for classifying 'noisy' sensor data Weight lifting exercise recognition requires sufficient training data, as 'new' data from a person who was not enrolled during the training phase resulted in significantly lower recognition rates. V. [5] [6] [7] [8] FUTURE WORK AND CONCLUSION This paper presents the building block and methodology for a verified exercise application. Such an application will meet the need for a trusted application which posts update on completed health and fitness sessions on social networks, with a focus on weight lifting exercises. The methodology used for exercise recognition has been proven viable through experiments performed on a reference data set. The second stage of the work is to perform data capture of users as they perform weight lifting exercises, and to implement repetition count through the use of autocorrelation peaks. Future works that may be attempted is to incorporate gyroscope data in addition to accelerometer data, and also to use more sophisticated statistical descriptors as part of the feature vector. [9] [10] [11] REFERENCES [1] [2] [3] [4] B. Dolan, “By 2017: 170M wearable wireless health and fitness devices | mobihealthnews,” Feb-2012. [Online]. Available: http://mobihealthnews.com/16415/by-2017-170mwearable-wireless-health-and-fitness-devices/. [Accessed: 16-Oct-2014]. D. N. Cavallo, D. F. Tate, A. V. Ries, J. D. Brown, R. F. DeVellis, and A. S. Ammerman, “A Social Media– Based Physical Activity Intervention,” Am. J. Prev. Med., vol. 43, no. 5, pp. 527–532, Nov. 2012. K. O. Hwang, A. J. Ottenbacher, A. P. Green, M. R. Cannon-Diehl, O. Richardson, E. V. Bernstam, and E. J. Thomas, “Social support in an Internet weight loss community,” Int. J. Med. Inf., vol. 79, no. 1, pp. 5–13, Jan. 2010. S. L. Shuger, V. W. Barry, X. Sui, A. McClain, G. A. Hand, S. Wilcox, R. A. Meriwether, J. W. Hardin, and S. N. Blair, “Electronic feedback in a diet- and physical activity-based lifestyle intervention for [12] [13] [14] [15] weight loss: a randomized controlled trial,” Int. J. Behav. Nutr. Phys. Act., vol. 8, no. 1, p. 41, 2011. T. Pels, C. Kao, and S. Goel, “FatBelt: Motivating Behavior Change Through Isomorphic Feedback,” in Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA, 2014, pp. 123–124. R. W. Jeffery, “Financial incentives and weight control,” Prev. Med., vol. 55, Supplement, pp. S61– S67, Nov. 2012. J. Cawley and J. A. Price, “A case study of a workplace wellness program that offers financial incentives for weight loss,” J. Health Econ., vol. 32, no. 5, pp. 794–803, Sep. 2013. J. T. Kullgren, A. B. Troxel, G. Loewenstein, D. A. Asch, L. A. Norton, L. Wesby, Y. Tao, J. Zhu, and K. G. Volpp, “Individual- Versus Group-Based Financial Incentives for Weight LossA Randomized, Controlled Trial,” Ann. Intern. Med., vol. 158, no. 7, pp. 505– 514, Apr. 2013. D. Arterburn, E. O. Westbrook, C. J. Wiese, E. J. Ludman, D. C. Grossman, P. A. Fishman, E. A. Finkelstein, R. W. Jeffery, and A. Drewnowski, “Insurance coverage and incentives for weight loss among adults with metabolic syndrome,” Obes. Silver Spring Md, vol. 16, no. 1, pp. 70–76, Jan. 2008. E. V. Lambert and T. L. Kolbe-Alexander, “Innovative strategies targeting obesity and noncommunicable diseases in South Africa: what can we learn from the private healthcare sector?,” Obes. Rev., vol. 14, pp. 141–149, Nov. 2013. D. N. Patel, C. Nossel, E. Alexander, and D. Yach, “Innovative Business Approaches for Incenting Health Promotion in Sub-Saharan Africa: Progress and Persisting Challenges,” Prog. Cardiovasc. Dis., vol. 56, no. 3, pp. 356–362, Nov. 2013. R. C. Mosley Jr, “Social media analytics: Data mining applied to insurance Twitter posts,” in Casualty Actuarial Society E-Forum, Winter 2012 Volume 2, 2012, p. 1. J. Lester, T. Choudhury, and G. Borriello, “A Practical Approach to Recognizing Physical Activities,” in Pervasive Computing, K. P. Fishkin, B. Schiele, P. Nixon, and A. Quigley, Eds. Springer Berlin Heidelberg, 2006, pp. 1–16. M. Ermes, J. Parkka, J. Mantyjarvi, and I. Korhonen, “Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions,” IEEE Trans. Inf. Technol. Biomed., vol. 12, no. 1, pp. 20–26, Jan. 2008. G. Kortuem and Z. Segall, “Wearable Communities: Augmenting Social Networks with Wearable Computers,” IEEE Pervasive Computing, vol. 2, no. 1, pp. 71–78, 2003. [16] M. L. Pollock, B. A. Franklin, G. J. Balady, B. L. Chaitman, J. L. Fleg, B. Fletcher, M. Limacher, I. L. Piña, R. A. Stein, M. Williams, and T. Bazzarre, “Resistance Exercise in Individuals With and Without Cardiovascular Disease Benefits, Rationale, Safety, and Prescription An Advisory From the Committee on Exercise, Rehabilitation, and Prevention, Council on Clinical Cardiology, American Heart Association,” Circulation, vol. 101, no. 7, pp. 828–833, Feb. 2000. [17] G. F. Fletcher, G. Balady, S. N. Blair, J. Blumenthal, C. Caspersen, B. Chaitman, S. Epstein, E. S. S. Froelicher, V. F. Froelicher, I. L. Pina, and M. L. Pollock, “Statement on Exercise: Benefits and Recommendations for Physical Activity Programs for All Americans A Statement for Health Professionals by the Committee on Exercise and Cardiac Rehabilitation of the Council on Clinical Cardiology, American Heart Association,” Circulation, vol. 94, no. 4, pp. 857–862, Aug. 1996. [18] D. Morris, T. S. Saponas, A. Guillory, and I. Kelner, “RecoFit: Using a Wearable Sensor to Find, Recognize, and Count Repetitive Exercises,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2014, pp. 3225–3234. [19] K.-H. Chang, M. Y. Chen, and J. Canny, “Tracking Free-weight Exercises,” in Proceedings of the 9th International Conference on Ubiquitous Computing, Berlin, Heidelberg, 2007, pp. 19–37. [20] C. Seeger, A. Buchmann, and K. Van Laerhoven, “myHealthAssistant: A Phone-based Body Sensor Network that Captures the Wearerś Exercises throughout the Day,” in The 6th International Conference on Body Area Networks (BodyNets), Beijing, China, 2011. [21] C. C. Ho and C. Eswaran, “Consodilation of fingerprint databases: A Malaysian case study,” in In the Proceedings of the 2011 11th International Conference on Hybrid Intelligent Systems (HIS), 2011, pp. 455–462. [22] C. C. Ho and C. Eswaran, “Multimodal Biometric For Secured Role Based Access Control In E-Commerce System,” in The Proceedings of the 2009 International Conference on e-Commerce, eAdministration, e-Society, and e-Education., Singapore, 2009. [23] S. Adali, R. Escriva, M. K. Goldberg, M. Hayvanovych, M. Magdon-Ismail, B. K. Szymanski, W. A. Wallace, and G. Williams, “Measuring behavioral trust in social networks,” in 2010 IEEE International Conference on Intelligence and Security Informatics (ISI), 2010, pp. 150–152. [24] A. Mohaisen, N. Hopper, and Y. Kim, “Keep your friends close: Incorporating trust into social networkbased Sybil defenses,” in 2011 Proceedings IEEE INFOCOM, 2011, pp. 1943–1951. [25] S. Al-Oufi, H.-N. Kim, and A. El Saddik, “A group trust metric for identifying people of trust in online social networks,” Expert Syst. Appl., vol. 39, no. 18, pp. 13173–13181, Dec. 2012. [26] Twitter, “Twitter Help Center | FAQs about verified accounts,” 2014. [Online]. Available: https://support.twitter.com/articles/119135-faqsabout-verified-accounts#. [Accessed: 21-Oct-2014]. [27] Facebook Inc., “What’s a verified profile or Page?,” 2014. [Online]. Available: https://www.facebook.com/help/196050490547892. [Accessed: 21-Oct-2014]. [28] C. C. Ho, C. Eswaran, K.-W. Ng, and J.-Y. Leow, “An Unobtrusive Android Person Verification Using Accelerometer Based Gait,” in Proceedings of the 10th International Conference on Advances in Mobile Computing & Multimedia (MoMM2012), Bali, 2012, pp. 271–274. [29] M. Muehlbauer, G. Bahle, and P. Lukowicz, “What Can an Arm Holster Worn Smart Phone Do for Activity Recognition?,” in 2011 15th Annual International Symposium on Wearable Computers (ISWC), 2011, pp. 79–82. [30] Texas Instruments, “eZ430-Chronos-915 - Chronos: Wireless development tool in a watch,” 2014. [Online]. Available: https://estore.ti.com/eZ430Chronos-915-Chronos-Wireless-development-tool-ina-watch-P1736.aspx. [Accessed: 22-Oct-2014]. [31] E. Velloso, A. Bulling, H. Gellersen, W. Ugulino, and H. Fuks, “Qualitative Activity Recognition of Weight Lifting Exercises,” in Proceedings of the 4th Augmented Human International Conference, New York, NY, USA, 2013, pp. 116–123. [32] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software,” ACM SIGKDD Explor. Newsl., vol. 11, no. 1, p. 10, Nov. 2009.