University of Plymouth PEARL https://pearl.plymouth.ac.uk Faculty of Science and Engineering School of Engineering, Computing and Mathematics 2019-03 Personalization in Long-Term Human-Robot Interaction Irfan, B http://hdl.handle.net/10026.1/13844 10.1109/hri.2019.8673076 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) IEEE All content in PEARL is protected by copyright law. Author manuscripts are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author. This is the author’s manuscript that was accepted on November 22, 2018. The final version of this work is published by IEEE in 2019 14th ACM/IEEE International Conference on Human Robot Interaction (HRI), available at DOI: 10.1109/HRI.2019.8673076. This work is made available online in accordance with the publisher’s policies. Please refer to any applicable terms of use of the publisher. Personalization in Long-Term Human-Robot Interaction Bahar Irfan Aditi Ramachandran Samuel Spaulding Dylan F. Glas Centre for Robotics and Neural Systems Social Robotics Lab Personal Robots Group Futurewei Technologies University of Plymouth, UK Yale University, USA MIT Media Lab, USA Huawei, USA bahar.irfan@plymouth.ac.uk aditi.ramachandran@yale.edu samuelsp@media.mit.edu dylan.f.glas@gmail.com Iolanda Leite Kheng Lee Koay Department of Robotics, Perception and Learning Royal Institute of Technology, Sweden iolanda@kth.se Adaptive Systems Research Group University of Hertfordshire, UK k.l.koay@herts.ac.uk Abstract—For practical reasons, most human-robot interaction (HRI) studies focus on short-term interactions between humans and robots. However, such studies do not capture the difficulty of sustaining engagement and interaction quality across long-term interactions. Many real-world robot applications will require repeated interactions and relationship-building over the long term, and personalization and adaptation to users will be necessary to maintain user engagement and to build rapport and trust between the user and the robot. This full-day workshop brings together perspectives from a variety of research areas, including companion robots, elderly care, and educational robots, in order to provide a forum for sharing and discussing innovations, experiences, works-in-progress, and best practices which address the challenges of personalization in long-term HRI. Index Terms—Personalization; Long-Term Interaction; Human-Robot Interaction; Adaptation; Long-Term Memory; User Modeling; User Recognition with other researchers about the problems they have encountered during their studies and their respective solutions. II. BACKGROUND Recent advances on the automatic perception of user actions and affective states [4], [5] are extending the possibilities for adaptation and personalization in HRI [6]. There has been an increasing interest in studying robots that can adapt to the affective states [7], [8] or engagement level [9] of users, as well as to other user preferences like proxemics [10]. Adaptive and personalized interactions are particularly relevant when robots are expected to interact with the same user for extended periods, as is the case of service robots [11] in domains like assisted living [12] and collaborative manufacturing. I. I NTRODUCTION III. TARGET AUDIENCE AND T OPICS Long-term human-robot interaction (HRI) is essential in areas such as companion robots [1], rehabilitation, and education. However, interactions based on fixed collections of behaviors can become repetitive over time, causing user engagement to decrease after the novelty effect wears off. Personalization can help improve user engagement in longterm interactions, by adapting to the user’s personality, preferences, needs [2], or by recalling shared memories with the user [3]. Moreover, personalizing the interaction can facilitate establishing rapport and trust between the user and the robot. However, long-term HRI studies require substantial resources, may not always be technically feasible especially if the robots are deployed “in the wild”, and often do not provide generalizable results due to the variability of subject needs, making it challenging for researchers to publish results. This workshop focuses on studies on adaptivity to users, context, environment, and tasks in long-term interactions in a variety of fields (e.g. companion robots, collaborative tasks, education, rehabilitation, elderly care). We intend to create a medium for researchers to share their work in progress, to introduce their preliminary results, and to share and discuss We encourage researchers and students from HRI, robotics, cognitive science, rehabilitation, and educational backgrounds to contribute. We invite short papers of 2-4 pages, including works-in-progress containing preliminary results, technical reports, case studies, surveys, and state-of-the-art research of personal robots and long-term studies in any of these fields. The accepted papers will be published on the workshop website as well as in arXiv. The workshop welcomes contributions across a wide range of topics including, but not limited to: • Personalization in HRI for companion robots, collaborative tasks, education, rehabilitation, elderly care • Adaptation algorithms for long-term interactions • User modeling • Long-term memory (episodic, semantic, associative) • User recognition • Long-term HRI studies • Conversational agents in long-term interactions • Engagement in long-term HRI • Evaluation in long-term HRI • Challenges/Guidelines for field studies in long-term HRI • • Design and methodologies for repeated HRI Autonomy in long-term interaction IV. W ORKSHOP OVERVIEW The aim of this full-day workshop is to provide a forum for researchers to share ideas and discuss recent research methodologies for personalization in long-term HRI. The workshop will consist of: • Keynotes: Invited researchers Takayuki Kanda (Kyoto University, Japan), Hae Won Park (MIT Media Lab, USA) and Ognjen Rudovic (MIT Media Lab, USA) will present their experiences and perspectives on the topic. • Full talks: The authors of the accepted full-length (3-4 page) research papers will give 12-minute presentations followed by 3-minute question sessions. • Short talks: The authors of the accepted short length (2 pages) research papers will give 2-3 minute introductions to their papers followed by 3-minute question sessions. • Interactive activities (brainstorming): In groups, workshop attendees will brainstorm to identify problems that can arise in long-term HRI and come up with solutions using available technologies. Brainstorming sessions will consist of two parts: 1) Identify problems: Groups will come up with the main issues that can arise in long-term interaction. 2) Find a solution: The problems will be grouped by topic, and 3 topics will be chosen. Each group will be asked to find a solution to an assigned topic using a commercially-available robot and currentlyavailable technology. At the end of the session, each group will give a 1-minute pitch of their solution. V. O RGANIZERS Bahar Irfan is an Early-Stage Researcher and a PhD candidate at the Centre for Robotics and Neural Systems, University of Plymouth and AI Lab, SoftBank Robotics Europe, France, in the joint Marie Skłodowska-Curie ITN project APRIL. Her work focuses on multi-modal person recognition and personalization in long-term HRI for conversational agents. She is also working in a joint project on socially assistive robotics with Colombian School of Engineering Julio Garavito jointly funded by the Royal Academy of Engineering. Aditi Ramachandran is a sixth year PhD candidate in the Social Robotics Lab at Yale University. Her research focuses on personalized social robots in education. Samuel Spaulding is a PhD student in the Personal Robots Group at the MIT Media Lab. His thesis research is focusing on building robots that can learn personalized cognitive and affective models of users over repeated interactions across different tasks. Dylan F. Glas is a Senior Robotics Software Architect at Futurewei Technologies. His research interests include interaction design frameworks, autonomous social behavior, and learning by imitation for social robots. Iolanda Leite is an Assistant Professor at the School of Computer Science and Electrical Engineering at KTH. Her research interests are in the areas of HRI and Artificial Intelligence. She aims to develop autonomous socially intelligent robots that can assist people over long periods of time. Kheng Lee Koay joined the University of Hertfordshire as a Senior Research Fellow with the Adaptive Systems Research Group in 2003, and has been a Senior Lecturer since 2016. His research expertise includes Mobile Robotics, Robotic Home Companions, HRI and Human-Robot Proxemics. In particular, his research focuses on human centred socially acceptable human-robot interactions, and experimental design and evaluation methodologies. ACKNOWLEDGMENT This work is partially funded by the EU H2020 Marie Skłodowska-Curie Actions ITN project APRIL (grant 674868). R EFERENCES [1] F. Amirabdollahian, R. o. d. Akker, S. Bedaf, R. Bormann, H. Draper, V. Evers, J. G. Prez, G. J. Gelderblom, C. G. Ruiz, D. Hewson, N. Hu, K. L. Koay, B. Kröse, H. Lehmann, P. Marti, H. Michel, H. PrevotHuille, U. Reiser, J. Saunders, T. Sorell, J. Stienstra, D. Syrdal, M. Walters, and K. Dautenhahn, “Assistive technology design and development for acceptable robotics companions for ageing years,” Paladyn, Journal of Behavioral Robotics, vol. 4, no. 2, pp. 94–112, 2013. [2] J. Saunders, D. S. Syrdal, K. L. Koay, N. Burke, and K. Dautenhahn, “Teach Me-Show Me – End-user personalization of a smart home and companion robot,” IEEE Transactions on Human-Machine Systems, vol. 46, no. 1, pp. 27–40, Feb 2016. [3] J. Saez-Pons, D. S. Syrdal, and K. Dautenhahn, “What has happened today? memory visualisation of a robot companion to assist users memory,” Journal of Assistive Technologies, vol. 9, no. 4, pp. 207–218, 2015. [Online]. Available: https://doi.org/10.1108/JAT-02-2015-0004 [4] Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, “A survey of affect recognition methods: Audio, visual, and spontaneous expressions,” IEEE transactions on pattern analysis and machine intelligence, vol. 31, no. 1, pp. 39–58, 2009. [5] C. A. Corneanu, M. O. Simón, J. F. Cohn, and S. E. Guerrero, “Survey on rgb, 3d, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications,” IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 8, pp. 1548–1568, 2016. [6] M. Ahmad, O. Mubin, and J. Orlando, “A systematic review of adaptivity in human-robot interaction,” Multimodal Technologies and Interaction, vol. 1, no. 3, p. 14, 2017. [7] I. Leite, A. Pereira, G. Castellano, S. Mascarenhas, C. Martinho, and A. Paiva, “Modelling empathy in social robotic companions,” in International Conference on User Modeling, Adaptation, and Personalization. Springer, 2011, pp. 135–147. [8] G. Gordon, S. Spaulding, J. K. Westlund, J. J. Lee, L. Plummer, M. Martinez, M. Das, and C. Breazeal, “Affective personalization of a social robot tutor for children’s second language skills.” in AAAI, 2016, pp. 3951–3957. [9] D. Szafir and B. Mutlu, “Pay attention!: designing adaptive agents that monitor and improve user engagement,” in Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2012, pp. 11–20. [10] R. Mead and M. J. Mataric, “Toward robot adaptation of human speech and gesture parameters in a unified framework of proxemics and multimodal communication,” in 2015 IEEE International Conference on Robotics and Automation Workshop on Machine Learning for Social Robots, ser. ICRA’15, 2015. [11] M. K. Lee and J. Forlizzi, “Designing adaptive robotic services,” Proc. of IASDR09, 2009. [12] A. Tapus, C. Ţăpuş, and M. J. Matarić, “Userrobot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy,” Intelligent Service Robotics, vol. 1, no. 2, p. 169, 2008.