[go: up one dir, main page]
Skip to main content

Advertisement

SpringerLink
Log in

Culture as a Sensor? A Novel Perspective on Human Activity Recognition

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Human Activity Recognition (HAR) systems are devoted to identifying, amidst the sensory stream provided by one or more sensors located so that they can monitor the actions of a person, portions related to the execution of a number of a-priori defined activities of interest. Improving the performance of systems for Human Activity Recognition is a long-standing research goal: solutions include more accurate sensors, more sophisticated algorithms for the extraction and analysis of relevant information from the sensory data, and the enhancement of the sensory analysis with general or person-specific knowledge about the execution of the activities of interest. Following the latter trend, in this article we propose the association and enhancement of the sensory data analysis with cultural information, that can be seen as an estimate of person-specific information, relieved of the burden of a long/complex setup phase. We propose a culture-aware Human Activity Recognition system which associates the recognition response provided by a state-of-the-art, culture-unaware HAR system with culture-specific information about where and when activities are most likely performed in different cultures, encoded in an ontology. The merging of the cultural information with the culture-unaware responses is done by a Bayesian Network, whose probabilistic approach allows for avoiding stereotypical representations. Experiments performed offline and online, using images acquired by a mobile robot in an apartment, show that the culture-aware HAR system consistently outperforms the culture-unaware HAR system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/.

  2. The OWL-2 terminology for describing ontologies [38] defines classes, properties and individuals. However, we prefer the term instance to individual because the latter is commonly used as a synonym of person, which might lead to confusion in this article.

  3. http://caressesrobot.org/en/2018/03/08/caresses-scenarios-and-guidelines-available/.

  4. https://en.wikipedia.org/wiki/Afternoon.

  5. https://cloud.google.com/vision/.

  6. Publicly available on Github at: https://github.com/TingChiaChiang/Culturally-Competent-Human-Activity-Recognition-with-a-Pepper-Robot.

  7. http://doc.aldebaran.com/2-4/naoqi/motion/index.html.

References

  1. Agarwal P, Verma R, Mallik A (2016) Ontology based disease diagnosis system with probabilistic inference. In: 2016 1st India international conference on information processing (IICIP). IEEE, pp 1–5

  2. Aggarwal JK, Xia L (2014) Human activity recognition from 3D data: a review. Pattern Recognit Lett 48:70–80

    Article  Google Scholar 

  3. Bakar U, Ghayvat H, Hasanm S, Mukhopadhyay S (2016) Activity and anomaly detection in smart home: a survey. In: Next Generation Sensors and Systems. Springer, pp 191–220

  4. Banerjee T, Keller JM, Popescu M, Skubic M (2015) Recognizing complex instrumental activities of daily living using scene information and fuzzy logic. Comput Vis Image Underst 140:68–82. https://doi.org/10.1016/j.cviu.2015.04.005

    Article  Google Scholar 

  5. Bruno B, Mastrogiovanni F, Sgorbissa A, Vernazza T, Zaccaria R (2013) Analysis of human behavior recognition algorithms based on acceleration data. ICRA 2013:2293–2299

    Google Scholar 

  6. Bruno B, Mastrogiovanni F, Sgorbissa A (2014) A public domain dataset for ADL recognition using wrist-placed accelerometers. In: 23rd IEEE international symposium on robot and human interactive communication (IEEE RO-MAN 2014)

  7. Bruno B, Mastrogiovanni F, Pecora F, Sgorbissa A, Saffiotti A (2017) A framework for culture-aware robots based on fuzzy logic. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–6

  8. Bruno B, Recchiuto CT, Papadopoulos I, Saffiotti A, Koulouglioti C, Menicatti R, Mastrogiovanni F, Zaccaria R, Sgorbissa A (2019) Knowledge representation for culturally competent personal robots: requirements, design principles, implementation, and assessment. Int J Soc Robot. https://doi.org/10.1007/s12369-019-00519-w

    Article  Google Scholar 

  9. Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv (CSUR) 46(3):33

    Article  Google Scholar 

  10. Carvalho RN, Laskey KB, Costa PC (2017) PR-OWL—a language for defining probabilistic ontologies. Int J Approx Reason 91:56–79

    Article  MathSciNet  Google Scholar 

  11. Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24(6):961–974

    Article  Google Scholar 

  12. Cook DJ, Crandall AS, Thomas BL, Krishnan NC (2013) CASAS: a smart home in a box. Computer 46(7):62–69

    Article  Google Scholar 

  13. Coppola C, Krajńík T, Duckett T, Bellotto N (2016) Learning temporal context for activity recognition. Front Artif Intell Appl 285:107–115

    Google Scholar 

  14. Crispim-Junior CF, Buso V, Avgerinakis K, Meditskos G, Briassouli A, Benois-Pineau J, Kompatsiaris I, Bremond F (2016) Semantic event fusion of different visual modality concepts for activity recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1598–1611. https://doi.org/10.1109/TPAMI.2016.2537323

    Article  Google Scholar 

  15. Faria DR, Vieira M, Premebida C, Nunes U (2015) Probabilistic human daily activity recognition towards robot-assisted living. In: 2015 24th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 582–587

  16. Fjellstrm C (2004) Mealtime and meal patterns from a cultural perspective. Scand J Nutr 48(4):161–164. https://doi.org/10.1080/11026480410000986

    Article  Google Scholar 

  17. Froehlich JE, Larson E, Campbell T, Haggerty C, Fogarty J, Patel SN (2009) Hydrosense: infrastructure-mediated single-point sensing of whole-home water activity. In: Proceedings of the 11th international conference on Ubiquitous computing. ACM, pp 235–244

  18. Gayathri K, Easwarakumar K, Elias S (2017) Probabilistic ontology based activity recognition in smart homes using Markov Logic Network. Knowl Based Syst 121:173–184. https://doi.org/10.1016/j.knosys.2017.01.025

    Article  Google Scholar 

  19. Guarino N, et al (1998) Formal ontology and information systems. In: Proceedings of FOIS, pp 81–97

  20. Guptill AE, Copelton DA, Lucal B (2017) Food and society: principles and paradoxes. Wiley, Hoboken

    Google Scholar 

  21. Hofstede G, Hofstede GJ, Minkov M (1991) Cultures and organizations: software of the mind, vol 2. Citeseer

  22. Katz S, Chinn A, Cordrey L (1959) Multidisciplinary studies of illness in aged persons: a new classification of functional status in activities of daily living. J Chronic Dis 9(1):55–62

    Article  Google Scholar 

  23. Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. IEEE Pervasive Comput IEEE Comput Soc IEEE Commun Soc 9(1):48

    Article  Google Scholar 

  24. Latfi F, Lefebvre B, Descheneaux C (2007) Ontology-based management of the telehealth smart home, dedicated to elderly in loss of cognitive autonomy. In: OWLED, vol 258

  25. Law M (1993) Evaluating activities of daily living: directions for the future. Am J Occup Ther 47:233–237

    Article  Google Scholar 

  26. Lawton M, Brody E (1969) Assessment of older people: self-maintaining and instrumental activities of daily living. The Gerontologist 9:179–186

    Article  Google Scholar 

  27. Lugrin B, Frommel J, André E (2015) Modeling and evaluating a bayesian network of culture-dependent behaviors. Cult Comput 2015:33–40

    Google Scholar 

  28. Menicatti R, Bruno B, Sgorbissa A (2017) Modelling the influence of cultural information on vision-based human home activity recognition. In: 2017 14th International conference on ubiquitous robots and ambient intelligence (URAI), pp 32–38

  29. Okeyo G, Chen L, Wang H, Sterritt R (2014) Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob Comput 10(PART B):155–172. https://doi.org/10.1016/j.pmcj.2012.11.004

    Article  Google Scholar 

  30. Onofri L, Soda P, Pechenizkiy M, Iannello G (2016) A survey on using domain and contextual knowledge for human activity recognition in video streams. Expert Syst Appl 63:97–111. https://doi.org/10.1016/j.eswa.2016.06.011

    Article  Google Scholar 

  31. Papadopoulos I (2006) Transcultural health and social care: development of culturally competent practitioners. Elsevier, Amsterdam

    Google Scholar 

  32. Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28(6):976–990

    Article  Google Scholar 

  33. Rehm M, Bee N, Endrass B, Wissner M, André E (2007) Too close for comfort? Adapting to the user’s cultural background. HCM 2007:85–94

    Article  Google Scholar 

  34. Scalmato A, Sgorbissa A, Zaccaria R (2013) Describing and recognizing patterns of events in smart environments with description logic. IEEE Trans Cybern 43(6):1882–1897. https://doi.org/10.1109/TSMCB.2012.2234739

    Article  Google Scholar 

  35. Shoaib M, Bosch S, Incel O, Scholten H, Havinga P (2015) A survey of online activity recognition using mobile phones. Sensors 15(1):2059–2085

    Article  Google Scholar 

  36. Soo-Hoo F (2016) How women around the world get clean. https://www.refinery29.com/en-us/2016/01/101925/cultural-differences-women-showering

  37. Trovato G, Ham JR, Hashimoto K, Ishii H, Takanishi A (2015) Investigating the effect of relative cultural distance on the acceptance of robots. ICSR 2016:664–673

    Google Scholar 

  38. W3C Owl Working Group and others (2009) OWL 2 web ontology language document overview

  39. Weiss GM, Timko JL, Gallagher CM, Yoneda K, Schreiber AJ (2016) Smartwatch-based activity recognition: a machine learning approach. In: 2016 IEEE-EMBS international conference on biomedical and health informatics (BHI). IEEE, pp 426–429

  40. Ye J, Stevenson G, Dobson S (2015) KCAR: a knowledge-driven approach for concurrent activity recognition. Pervasive Mob Comput 19(2):47–70. https://doi.org/10.1016/j.pmcj.2014.02.003

    Article  Google Scholar 

Download references

Funding

This work has been partially supported by the European Commission Horizon2020 Research and Innovation Programme under Grant Agreement No. 737858 (CARESSES), and by the Erasmus+ programme under Grant Agreement No. 2014-2616/001-001 (EMARO+).

Author information

Authors and Affiliations

  1. Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Via Opera Pia 13, 16145, Genoa, Italy

    Ting-Chia Chiang, Barbara Bruno, Roberto Menicatti, Carmine Tommaso Recchiuto & Antonio Sgorbissa

Authors
  1. Ting-Chia Chiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara Bruno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roberto Menicatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carmine Tommaso Recchiuto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio Sgorbissa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barbara Bruno.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chiang, TC., Bruno, B., Menicatti, R. et al. Culture as a Sensor? A Novel Perspective on Human Activity Recognition. Int J of Soc Robotics 11, 797–814 (2019). https://doi.org/10.1007/s12369-019-00590-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-019-00590-3

Keywords

Search

Navigation