Smartphone-Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods
<p>Number of searches of “mood app” during the last 10 years. Values expressed in percentage relative to the total amount of searches on that topic. Source: Google Trends [<a href="#B17-sensors-19-03430" class="html-bibr">17</a>].</p> "> Figure 2
<p>Architecture of the monitoring platform.</p> "> Figure 3
<p>Russell’s circumplex model of mood [<a href="#B14-sensors-19-03430" class="html-bibr">14</a>].</p> "> Figure 4
<p>Screenshots of the ESM questions for assessing valence and arousal.</p> "> Figure 5
<p>ESM Management Interface.</p> "> Figure 6
<p>Percentage of questionnaires answered (blue), expired (orange) and actively dismissed (grey) for each participant during the entire study.</p> "> Figure 7
<p>Overall percentage of questionnaires answered (blue), expired (orange) and actively dismissed (grey) per interval of daily hours.</p> "> Figure 8
<p>Overall response rate registered per day of study. The red dashed vertical line splits the graphic in the two weeks of the study.</p> "> Figure 9
<p>Completion times of the questionnaires per day of study for the valence question. Times over 300 s have not been considered.</p> "> Figure 10
<p>Completion times of the questionnaires per day of study for the arousal question. Times over 300 s have not been considered.</p> "> Figure 11
<p>Time elapsed from the reception of the notification to the participant’s response per interval of daily hours.</p> "> Figure 12
<p>System Usability Scale (SUS) score obtained from each participant. The horizontal lines represent the mean SUS score of the system and the threshold value that indicates a good usability.</p> ">
Abstract
:1. Introduction
2. Related Work
3. Proposed Monitoring Platform
3.1. Platform Architecture
3.2. Ecological Momentary Assessment of Affective States
3.3. Flexible ESM Questionnaire Management
3.4. Automated Monitoring of Context
4. Evaluation
4.1. Participants
4.2. Methods
4.2.1. Procedure
4.2.2. Validity Indicators
4.2.3. Usability Evaluation
- (Q1) I think that I would like to use this system frequently.
- (Q2) I found the system unnecessarily complex.
- (Q3) I thought the system was easy to use.
- (Q4) I think that I would need the support of a technical person to be able to use this system.
- (Q5) I found the various functions in this system were well integrated.
- (Q6) I thought there was too much inconsistency in this system.
- (Q7) I would imagine that most people would learn to use this system very quickly.
- (Q8) I found the system very cumbersome to use.
- (Q9) I felt very confident using the system.
- (Q10) I needed to learn a lot of things before I could get going with this system.
4.3. Results
4.3.1. Response Rate
4.3.2. Completion Time
4.3.3. Elapsed Time from Notification Arrival to Response
4.3.4. Usability Assessment
- For odd-numbered items, the score is computed subtracting 1 to the user response.
- For even-numbered items, the score is computed subtracting the user response to 5.
- All the scores obtained—now ranging from 0 to 4, are added and multiplied by 2.5 to obtain the overall SUS score.
5. Discussion
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Ekkekakis, P. Affect, Mood, and Emotion. In Measurement in Sport and Exercise Psychology; Tenenbaum, G., Eklund, R.C., Kamata, A., Eds.; Human Kinetics: Champaign, IL, USA, 2012; pp. 321–332. [Google Scholar]
- Davidson, R. On emotion, mood, and related affective construct. In The Nature of Emotion: Fundamental Questions; Ekman, P., Davidson, R., Eds.; Oxford University Press: New York, NY, USA, 1994; pp. 51–55. [Google Scholar]
- Naragon-Gainey, K. Affective models of depression and anxiety: Extension to within-person processes in daily life. J. Affect. Disord. 2019, 243, 241–248. [Google Scholar] [CrossRef] [PubMed]
- Ortiz, A.; Bradler, K.; Garnham, J.; Slaney, C.; McLean, S.; Alda, M. Nonlinear dynamics of mood regulation in unaffected first-degree relatives of bipolar disorder patients. J. Affect. Disord. 2019, 243, 274–279. [Google Scholar] [CrossRef]
- Parkinson, B. Emotions are social. Br. J. Psychol. 1996, 87, 663–683. [Google Scholar] [CrossRef] [PubMed]
- Martinent, G.; Gareau, A.; Lienhart, N.; Nicaise, V.; Guillet-Descas, E. Emotion profiles and their motivational antecedents among adolescent athletes in intensive training settings. Psychol. Sport Exerc. 2018, 35, 198–206. [Google Scholar] [CrossRef]
- Huu Son, H. Toward a Computational Model of Mood. Procedia Comput. Sci. 2017, 110, 199–206. [Google Scholar] [CrossRef]
- Katerndahl, D.; Ferrer, R.; Best, R.; Wang, C.P. Dynamic Patterns in Mood Among Newly Diagnosed Patients With Major Depressive Episode or Panic Disorder and Normal Controls. Primary Care Companion J. Clin. Psychiatry 2007, 9, 183–187. [Google Scholar] [CrossRef] [Green Version]
- Kanning, M.; Schlicht, W. Be Active and Become Happy: An Ecological Momentary Assessment of Physical Activity and Mood. J. Sport Exerc. Psychol. 2010, 32, 253–261. [Google Scholar] [CrossRef] [PubMed]
- Vales-Alonso, J.; Lopez-Matencio, P.; Gonzalez-Castano, F.J.; Navarro-Hellin, H.; Banos-Guirao, P.J.; Perez-Martinez, F.J.; Martinez-Alvarez, R.P.; González-Jimenez, D.; Gil-Castineira, F.; Duro-Fernandez, R. Ambient Intelligence Systems for Personalized Sport Training. Sensors 2010, 10, 2359–2385. [Google Scholar] [CrossRef] [PubMed]
- Gravenhorst, F.; Muaremi, A.; Bardram, J.; Grunerbl, A.; Mayora, O.; Wurzer, G.; Frost, M.; Osmani, V.; Arnrich, B.; Lukowicz, P.; et al. Mobile phones as medical devices in mental disorder treatment: An overview. Pers. Ubiquitous Comput. 2015, 19, 335–353. [Google Scholar] [CrossRef]
- Csikszentmihalyi, M.; Larson, R. Validity and Reliability of the Experience-Sampling Method. In Flow and the Foundations of Positive Psychology; Springer: Dordrecht, The Netherlands, 2014; pp. 35–54. [Google Scholar] [CrossRef]
- Berkel, N.V.; Ferreira, D.; Kostakos, V. The Experience Sampling Method on Mobile Devices. ACM Comput. Surv. 2018, 50, 1–40. [Google Scholar] [CrossRef]
- Bailon, C.; Damas, M.; Pomares, H.; Sanabria, D.; Perakakis, P.; Goicoechea, C.; Banos, O. Intelligent Monitoring of Affective Factors Underlying Sport Performance by Means of Wearable and Mobile Technology. Proceedings 2018, 2, 1202. [Google Scholar] [CrossRef]
- Google. Consumer Barometer Study 2017—The Year of the Mobile Majority. Available online: https://www.thinkwithgoogle.com/intl/en-gb/advertising-channels/mobile/consumer-barometer-study-2017-year-mobile-majority/ (accessed on 23 May 2019).
- Yang, Y.S.; Ryu, G.W.; Choi, M. Methodological Strategies for Ecological Momentary Assessment to Evaluate Mood and Stress in Adult Patients Using Mobile Phones: Systematic Review. JMIR Mhealth Uhealth 2019, 7, e11215. [Google Scholar] [CrossRef] [PubMed]
- Google. Google Trends. Available online: www.google.com/trends (accessed on 22 May 2019).
- Courvoisier, D.S.; Eid, M.; Lischetzke, T.; Schreiber, W.H. Psychometric properties of a computerized mobile phone method for assessing mood in daily life. Emotion 2010, 10, 115–124. [Google Scholar] [CrossRef] [PubMed]
- Bolger, N.; Davis, A.; Rafaeli, E. Diary Methods: Capturing Life as it is Lived. Annu. Rev. Psychol. 2003, 54, 579–616. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rickard, N.; Arjmand, H.A.; Bakker, D.; Seabrook, E. Development of a Mobile Phone App to Support Self-Monitoring of Emotional Well-Being: A Mental Health Digital Innovation. JMIR Ment. Health 2016, 3, e49. [Google Scholar] [CrossRef] [PubMed]
- LiKamWa, R.; Liu, Y.; Lane, N.D.; Zhong, L. MoodScope: Building a Mood Sensor from Smartphone Usage Patterns. In Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Services, Taipei, Taiwan, 25–28 June 2013; pp. 25–28. [Google Scholar] [CrossRef]
- Ma, Y.; Xu, B.; Bai, Y.; Sun, G.; Zhu, R. Daily Mood Assessment Based on Mobile Phone Sensing. In Proceedings of the 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks, London, UK, 9–12 May 2012; pp. 142–147. [Google Scholar] [CrossRef]
- Asselbergs, J.; Ruwaard, J.; Ejdys, M.; Schrader, N.; Sijbrandij, M.; Riper, H. Mobile Phone-Based Unobtrusive Ecological Momentary Assessment of Day-to-Day Mood: An Explorative Study. J. Med. Internet Res. 2016, 18, e72. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Burns, M.N.; Begale, M.; Duffecy, J.; Gergle, D.; Karr, C.J.; Giangrande, E.; Mohr, D.C. Harnessing Context Sensing to Develop a Mobile Intervention for Depression. J. Med. Internet Res. 2011, 13, e55. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kanning, M.K.; Ebner-Priemer, U.W.; Schlicht, W.M. How to investigate within-subject associations between physical activity and momentary affective states in everyday life: A position statement based on a literature overview. Front. Psychol. 2013, 4, 187. [Google Scholar] [CrossRef]
- Ferreira, D.; Kostakos, V.; Dey, A.K. AWARE: Mobile context instrumentation framework. Front. ICT 2015, 2, 6. [Google Scholar] [CrossRef]
- AWARE. Open-Source Context-Instrumentation Framework for Everyone. Available online: https://awareframework.com/ (accessed on 25 May 2019).
- SQLite. SQLite Engine. 2017. Available online: http://www.sqlite.org/ (accessed on 25 May 2019).
- Watson, D.; Tellegen, A. Toward a consensual structure of mood. Psychol. Bull. 1985, 98, 219–235. [Google Scholar] [CrossRef]
- Watson, D.; Clark, L.A.; Tellegen, A. Development and validation of brief measures of positive and negative affect: The PANAS scales. J. Personal. Soc. Psychol. 1988, 54, 1063–1070. [Google Scholar] [CrossRef]
- Ekman, P. Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life; Times Books/Henry Holt: New York, NY, USA, 2003. [Google Scholar]
- Russell, J.A. A circumplex model of affect. J. Personal. Soc. Psychol. 1980, 39, 1161–1178. [Google Scholar] [CrossRef]
- Wohlfahrt-Laymann, J.; Hermens, H.; Villalonga, C.; Vollenbroek-Hutten, M.; Banos, O. MobileCogniTracker—A mobile experience sampling tool for tracking cognitive behaviour. J. Ambient. Intell. Humaniz. Comput. 2018, 1–18. [Google Scholar] [CrossRef]
- Meschtscherjakov, A.; Weiss, A.; Scherndl, T. Utilizing Emoticons on Mobile Devices within ESM studies to Measure Emotions in the Field. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’09), Bonn, Germany, 15–18 September 2009. [Google Scholar]
- Schimmack, U. Affect Measurement in Experience Sampling Research. J. Happiness Stud. 2003, 4, 79–106. [Google Scholar] [CrossRef]
- Attfield, S.; Kazai, G.; Lalmas, M.; Piwowarski, B. Towards a science of user engagement. In Proceedings of the Workshop on User Modelling for Web Applications, Hong Kong, China, 9–12 February 2011; pp. 1–8. [Google Scholar]
- Huang, M.H. Designing website attributes to induce experiential encounters. Comput. Hum. Behav. 2003, 19, 425–442. [Google Scholar] [CrossRef]
- Wenemark, M.; Persson, A.; Brage, H.N.; Svensson, T.; Kristenson, M. Applying Motivation Theory to Achieve Increased Response Rates, Respondent Satisfaction and Data Quality. J. Off. Stat. 2011, 27, 393–414. [Google Scholar]
- Gehlbach, H.; Barge, S. Anchoring and Adjusting in Questionnaire Responses. Basic Appl. Soc. Psychol. 2012, 34, 417–433. [Google Scholar] [CrossRef] [Green Version]
- Heron, K.E.; Smyth, J.M. Ecological Momentary Interventions: Incorporating Mobile Technology Into Psychosocial and Health Behavior Treatments. Br. J. Health Psychol. 2010, 15, 1–39. [Google Scholar] [CrossRef]
- Naughton, F.; Riaz, M.; Sutton, S. Response Parameters for SMS Text Message Assessments Among Pregnant and General Smokers Participating in SMS Cessation Trials. Nicotine Tob. Res. 2016, 18, 1210–1214. [Google Scholar] [CrossRef]
- Watson, N.; Wooden, M. Re-engaging with survey non-respondents: Evidence from three household panels. J. R. Stat. Soc. Ser. A (Stat. Soc.) 2014, 177, 499–522. [Google Scholar] [CrossRef]
- Ronacher, A. Serialization Framework for Java. 2010. Available online: http://flask.pocoo.org/ (accessed on 2 July 2019).
- Bailon, C. AWARE Plugin: Flexible ESM. 2019. Available online: https://github.com/carlos-bailon/com.aware.plugin.esm.flexible (accessed on 8 July 2019).
- Simple-XML. Serialization Framework for Java. 2013. Available online: http://simple.sourceforge.net/ (accessed on 25 May 2019).
- Dey, A.K. Understanding and Using Context. Pers. Ubiquitous Comput. 2001, 5, 4–7. [Google Scholar] [CrossRef]
- Mohr, D.C.; Zhang, M.; Schueller, S.M. Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. Annu. Rev. Clin. Psychol. 2017, 13, 23–47. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Banos, O.; Villalonga, C.; Bang, J.H.; Hur, T.H.; Kang, D.; Park, S.B.; Hyunh-The, T.; Vui, L.B.; Amin, M.B.; Razzaq, M.A.; et al. Human Behavior Analysis by means of Multimodal Context Mining. Sensors 2016, 16, 1264. [Google Scholar] [CrossRef] [PubMed]
- Van Berkel, N.; Goncalves, J.; Koval, P.; Hosio, S.; Dingler, T.; Ferreira, D.; Kostakos, V. Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; p. 12. [Google Scholar]
- Stone, A.A.; Kessler, R.C.; Haythomthwatte, J.A. Measuring Daily Events and Experiences: Decisions for the Researcher. J. Personal. 1991, 59, 575–607. [Google Scholar] [CrossRef] [Green Version]
- Verhagen, S.J.W.; Hasmi, L.; Drukker, M.; van Os, J.; Delespaul, P.A.E.G. Use of the experience sampling method in the context of clinical trials: Table 1. Evid. Based Ment. Health 2016, 19, 86–89. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Mcabe, K.O.; Mack, L.; Fleeson, W. A Guide for Data Cleaning in Experience Sampling Studies; Guilford Press: New York, NY, USA, 2012; pp. 321–338. [Google Scholar]
- Brooke, J. SUS-A quick and dirty usability scale. In Usability Evaluation in Industry; Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L., Eds.; Taylor&Francis: Abingdon, UK, 1996; pp. 189–194. [Google Scholar]
- Sauro, J. A Practical Guide to the System Usability Scale; Createspace Independent Publishing Platform: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
- Bangor, A.; Kortum, P.T.; Miller, J.T. An empirical evaluation of the system usability scale. Hum. Comput. Interact. 2008, 24, 574–594. [Google Scholar] [CrossRef]
- Van Berkel, N.; Goncalves, J.; Lovén, L.; Ferreira, D.; Hosio, S.; Kostakos, V. Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. Int. J. Hum.-Comp. Stud. 2019, 125, 118–128. [Google Scholar] [CrossRef]
Property | Description | Type |
---|---|---|
ESM_Type | Response format of the ESM question | string |
Title | Title of the ESM question | string |
Instructions | Instructions to answer the ESM question | string |
SubmitText | Text of the button to submit the answer | string |
Canceltext | Text of the button to cancel the question | string |
ExpirationThreshold | Time available for completing the question before it is removed (in seconds) | int |
NotificationTimeout | Time that the notification remains in the notification bar (in seconds) | int |
ScaleStartRandom *† | Enable the initialization of the slider in a random position | boolean |
ScaleStartRandomValues *† | Enable the initialization of the slider in a random position among a range of values | int |
ScaleStart † | Fixed initial position for the slider | int |
ScaleStep † | Value increment of each step of the slider | int |
ScaleMin † | Minimum value of the scale | int |
ScaleMinLabel † | Label displayed for the minimum value of the scale | string |
ScaleMax † | Maximum value of the scale | int |
ScaleMaxLabel † | Label displayed for the maximum value of the scale | string |
LeftImageUrl *† | URL of the image displayed on the minimum value of the scale | string |
RightImageUrl *† | URL of the image displayed on the maximum value of the scale | string |
ScheduleTime | Time (HH:MM) on which the ESM is scheduled | datetime |
ScheduleRandom | Enable the random scheduling of the ESM within an interval of time | boolean |
ScheduleRandomAmount ‡ | Amount of times that the ESM is scheduled in the interval | int |
ScheduleRandomInterval ‡ | Minimum time left between random ESMs | int |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bailon, C.; Damas, M.; Pomares, H.; Sanabria, D.; Perakakis, P.; Goicoechea, C.; Banos, O. Smartphone-Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods. Sensors 2019, 19, 3430. https://doi.org/10.3390/s19153430
Bailon C, Damas M, Pomares H, Sanabria D, Perakakis P, Goicoechea C, Banos O. Smartphone-Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods. Sensors. 2019; 19(15):3430. https://doi.org/10.3390/s19153430
Chicago/Turabian StyleBailon, Carlos, Miguel Damas, Hector Pomares, Daniel Sanabria, Pandelis Perakakis, Carmen Goicoechea, and Oresti Banos. 2019. "Smartphone-Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods" Sensors 19, no. 15: 3430. https://doi.org/10.3390/s19153430