Papers by Matthias Rauterberg
Personal and Ubiquitous Computing, Apr 5, 2012
Lifelogging tools aim to precisely capture daily experiences of people from the first-person pers... more Lifelogging tools aim to precisely capture daily experiences of people from the first-person perspective. Although there have been numerous lifelogging tools developed for users to record the external environment around them, the internal part of experience characterized by emotions seems to be neglected in the lifelogging field. However, the internal experiences of people are important and, therefore, lifelogging tools should be able to capture not only the environmental data, but also emotional experiences, thereby providing a more complete archive of past events. Moreover, there are implicit emotions that cannot be consciously experienced, but still influence human behaviors and memories. It has been proven that conscious emotions can be recognized from physiological signals of the human body. This fact may be used to enhance life-logs with information about unconscious emotions, which otherwise would remain hidden. On the other hand, it is not clear if unconscious emotions can be recognized from physiological signals and differentiated from conscious emotions. Therefore, an experiment was designed to elicit emotions (both conscious and unconscious) with visual and auditory stimuli and to record cardiovascular responses of 34 participants. The experimental results showed that heart rate responses to the presentation of the stimuli are unique for every category of the emotional stimuli and allow differentiation between various emotional experiences of the participants.
In this paper, we describe our ongoing work towards understanding how interactive comics could af... more In this paper, we describe our ongoing work towards understanding how interactive comics could affect time perception. An interactive comic story, The Dreaming Wine, has been created based on our current understanding of interactive comics, time perception and the relation between both. As a starting point for our research, we conducted an experiment on whether the amount of panels would affect the time perception of the comic reader. A list of principles was applied in splitting comic panels. The findings of the experiment are expected to provide a better understanding of how panels in interactive comics have an effect on the reader's time perception.
Frontiers in sustainable food systems, May 10, 2021
Lecture Notes in Computer Science, 2018
At present, the most effective way to deal with the demographic shift of elders is encouraging th... more At present, the most effective way to deal with the demographic shift of elders is encouraging them to live in nursing homes for more effective health care. However, such move dramatically increases their risk of social isolation. A contextual inquiry in a local nursing home revealed that most elderly people have few connections with the outside environment because of their decreased mobility and have difficulty establishing relationships with fellow residents. In addition, they show great interests in sharing stories. Based on the above findings, we present Interactive Gallery, a system consisting of a cluster of specially designed camera kits and a gallery-like interactive installation utilized with the metaphor of gallery and postcards. This system aims to make the elders feel more connected to the outside environment, stimulate them to reminisce, and further facilitate their sharing of stories with fellow residents and citizens from local communities.
I would like to thank my co-promotor, dr. Jun Hu. He has been always supportive and encouraging d... more I would like to thank my co-promotor, dr. Jun Hu. He has been always supportive and encouraging during the entire project. I would also like to thank for his patient work of proofreading and improving this thesis. Without his support, this thesis would have been a distant dream. I would like to thank my second promotor, prof.dr.ir. Loe Feijs. His extraordinary theoretical expertises in formal methods helped to improve the theoretical soundness of this project. I would also like to thank for his quick and careful proofreading of this thesis. I would like to thank prof.dr.ir. A. Nijholt, dr. P. Dabnichki, prof.dr. P.M.E. De Bra and dr. L.P.J.J. Noldus for agreeing to serve as my doctoral committee and for their insightful and helpful comments. I would like to thank my colleagues at Designed Intelligence group. Cheefai Tan built the flight simulator which made the user experiments of this PhD project possible. Rene Ahn translated the summary of this thesis into "samenvatting". Christoph Bartneck helped me with the questionaires used in user experiments. Ben Salem guided me as my daily supervisor for the first year of my PhD project. Razvan Cristescu helped me on the formal methods. Wei Chen helped me on the topic of fast Fourier transformation. I would like to thank my colleagues within the SEAT project consortium, prof. Ferri Aliabadi, Frazer McKimm, Jose Gisbert, Tomas Vyhlidal, Bert Arnrich, Esteve Farres, Peter Dabnichki, and those who are not mentioned here, for the interesting discussions and invaluable comments on my research. vii viii Finally, and most importantly, I would like to thank my family, my wife Shudong, for her love and sacrifice to the family; my lovely daughter Yuchen Liu, for the happiness and hope she brings to the family; my parents Xingli Liu and Chunyue Yang, for their unselfish love and supports all these years; my younger brother Jun Liu and sister-in-law Rui Zhao, for their help and support.
We are exploring a new application of virtual and augmented reality for a novel direction in huma... more We are exploring a new application of virtual and augmented reality for a novel direction in human-computer interaction named 'cultural computing', which aims to provide a new medium for cultural translation and subconscious metamorphosis. In this application both virtual and robotic agents are employed as an interactive dialogue figure. The main objective of this project is to create an interactive installation named ALICE that encourages people in Western culture to reflect on themselves, based on the narrative of 'Alice's Adventures in Wonderland' which address issues such as logic, rationality, and self.
In the Human Computer Interaction (HCI) field, it has been a long tradition of concern of accessi... more In the Human Computer Interaction (HCI) field, it has been a long tradition of concern of accessing computer systems by people with visual impairments. It is important to develop high quality user interfaces, accessible, usable, and desirable for these people. In this paper, we first report a preliminary background review about HCI design for people with visual disability. The review of the problems in social interaction the blind people may encounter, are also presented. Further, we narrow down our research scope and focus on gaze and eye contact, which have important social meaning in face-to-face communication. We then identify our research objective that is to design gaze simulation for people with visual impairments. Finally, we report the prototypes of our research project and the progress made so far.
Driven by the maturing of Internet and the development of the Internet of Things, people are beco... more Driven by the maturing of Internet and the development of the Internet of Things, people are becoming connected not only via the smart personal devices, but also via the increasingly responsive and connected environments, which brings up new opportunities to involve potential user group like elderly people into this network through tangible interaction and embedded systems. This paper presents the design and implementation of a participatory view-sharing system which consists of a group of specially designed camera kits and a gallery-like interactive installation, aiming to increase the social connection between elderly people living in a care home and the local communities around. The camera kits are designed to provide openness for local people to share sceneries via public participation, while the interactive installation presents the continuous real-time changes of the shared outside views, and triggers further communication between sharers and receivers through a “postcard-send...
IEEE Access, 2020
Gaze behaviors contain rich information regarding a person's emotions and engagements. Reciprocal... more Gaze behaviors contain rich information regarding a person's emotions and engagements. Reciprocal eye contact can invoke feelings of liking between two strangers. But blind people cannot perceive and establish the eye contact with sighted counterparts, causing their feelings of social isolation and low confidence in conversations. Thus, our research purpose is to let blind people perceive and react gaze behaviors in social interactions. A Social Glasses system has been implemented iteratively to deliver the multisensory feedback channels of the ''eye contact'', integrating both visual and tactile feedback. Specifically, the system consists of a Social Glasses device and a tactile wristband, which are worn by a blind person. The Social Glasses simulates the natural gaze for the blind person, aiming at establishing the ''eye contact'' between blind and sighted people. The tactile wristband enables the blind person to perceive the corresponding tactile feedback when an ''eye contact'' happens. To test the system, we conducted a user experiment with 40 participants, including 10 blind-sighted pairs (N = 20) and 10 blindfolded-sighted pairs (N = 20), to see how it could help increase the communication quality between blind and sighted people, as well as to suggest implications for its design. Our main findings demonstrated that both the simulated gaze and the tactile feedback were significantly effective to enhance the communication quality in blind-sighted conversations. Overall, we contribute (1) empirical research findings on how a Social Glasses system enhances the communication quality in blind-sighted conversations; and (2) design principles to inform future assistive wearable device for augmenting social interactions.
Handbook of Experimental Pharmacology, 2019
Journal of Ambient Intelligence and Smart Environments, 2018
In the past decades, we have witnessed a proliferation of interactive public displays for adverti... more In the past decades, we have witnessed a proliferation of interactive public displays for advertisement, entertainment and exhibition. We believe they also have great potential in the public spaces of caring environments if supported by related knowledge of design and research. This study explores how to design and evaluate interactive public displays in caring environments with a case study. In this paper, we describe the design process of OutLook, which is part of an initial participatory system specially designed for nursing homes to explore the possibilities of connecting people. It aims to enhance nursing home residents' social wellbeing through a "look-outside" and a "postcard-sending" metaphor. A field trial was performed to assess the effects of OutLook on nursing home residents' social behavior and feelings of connectedness. Key design factors for the effects and lessons learned were proposed as regard to design concept, design ideation, form of design, content, interfaces, interactions and field trial.
Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, 2016
Gaze and eye contact are frequently in social occasions used among sighted people. Gaze is consid... more Gaze and eye contact are frequently in social occasions used among sighted people. Gaze is considered as a predictor of attention and engagement between interlocutors in conversations. However, gaze signals from the sighted are not accessible for the blind person in face-to-face communication. In this paper, we present functional work-in-progress prototype, E-Gaze glasses, an assistive device based on an eye tracking system. E-Gaze simulates natural gaze for blind people, especially establishing the "eye contact" between blind and sighted people to enhance their engagement in face-to-face conversations. The gaze behavior is designed based on a turn-taking model, which interprets the corresponding relationship between the conclusive gaze behavior and the interlocutors' conversation flow.
Lecture Notes in Computer Science, 2008
In order to reduce both the psychological and physical stress in air travel, sensors are integrat... more In order to reduce both the psychological and physical stress in air travel, sensors are integrated in airplane seats to detect the gestures as input for in-flight entertainment systems. The content provided by the entertainment systems helps to reduce the psychological stress, and the gesture recognition is used as input for the content and hence stimulates people to move, which as a result would reduce the physical stress as well.
Lecture Notes in Computer Science, 2015
This tutorial aims to introduce the benefits of applying a service-oriented architecture (SOA) ap... more This tutorial aims to introduce the benefits of applying a service-oriented architecture (SOA) approach to serious games developers. For that end, we propose a hands-on session in which we will provide information on state-of-the-art services for serious games and guide developers in rethinking one of their existing games or game ideas using our SOA framework for serious games.
2009 2nd IEEE International Conference on Computer Science and Information Technology, 2009
In this paper, we present a software architecture support for biofeedback in-flight music systems... more In this paper, we present a software architecture support for biofeedback in-flight music systems to promote stress free air travels. Once the passenger sits in a seat of a flight, his/her bio signals are acquired via non intrusive sensors embedded in the seat and then are modeled into stress states. If the passenger is in a stress state, the system recommends a personalized stress reduction music playlist to the passenger to transfer him/her from the current stress state to the target comfort state; if the passenger is not in a stress state, the system recommends a personalized non stress induction music playlist to keep him/her at the comfort state. If the passenger does not accept the recommendation, he/she can browse the inflight music system and select preferred music himself/herself.
Lecture Notes in Computer Science, 2015
Serious games offer the potential to not only entertain and educate, but can also operate as a di... more Serious games offer the potential to not only entertain and educate, but can also operate as a diagnostic tool. While designing a game with the goal of a diagnostic tool, we faced many challenges. In this paper, we share our experiences in dealing with these challenges in the iterations of designing, implementing, and evaluating such a tool.
Computers & Education, 2015
There are currently a number of models, frameworks and methodologies for serious games analysis a... more There are currently a number of models, frameworks and methodologies for serious games analysis and design that provide useful interpretations of the possibilities and limitations offered by serious games. However, these tools focus mostly on high-level aspects and requirements and do not help understand how such high-level requirements can be concretely satisfied. In this paper, we present a conceptual model, called Activity Theory-based Model of Serious Games (ATMSG), that aims to fill this gap. ATMSG supports a systematic and detailed representation of educational serious games, depicting the ways that game elements are connected to each other throughout the game, and how these elements contribute to the achievement of the desired pedagogical goals. Three evaluation studies indicate that ATMSG helped participants, particularly those with gaming experience, identify and understand the roles of each component in the game and recognize the game's educational objectives.
International Journal of Industrial Ergonomics, 1995
The current state of traditional software development is surveyed and essential problems are inve... more The current state of traditional software development is surveyed and essential problems are investigated on the basis of empirical data and theoretical considerations. The concept of optimisation cycle is proposed as a solution for simultaneous engineering. The relationships of several different kinds of local optimisation cycles to the specifications, the communications, and the optimisation problem are integrated into a concept of participatory software development. Software development without integrated work-and task-organisational development is sub-optimal. User participation and prototyping significantly decrease cost and time overruns. Process moderation methods (e.g. workshops) are time effective and lead to best results for the analysis of requirements. Relevance to industry There is a growing realisation that the construction, implementation and performance of modern software products crucially depends on getting the social and organisational issues right. The user-oriented approach outlined in the paper explicitly aims to present a process model that incorporates method and techniques for the requirements analysis. The practical problems of this iterative cyclic process model are discussed and recommendations are made to guide industrial managers, engineers and human factor specialists involved in the process of simultaneous production.
International Journal of Human-Computer Studies, 2007
The cooperation between designers, engineers and scientists in the human-computer interaction (HC... more The cooperation between designers, engineers and scientists in the human-computer interaction (HCI) community is often difficult, and can only be explained by investigating the different paradigms by which they operate. This study proposes a paradigm model for designers, engineers and scientists, using three barriers to separate the professions. We then report on an empirical study that attempted to validate the understand/transform world barrier in the paradigm model using an online questionnaire. We conclude that the used 'Attitude About Reality' scale was unsuitable for measuring this barrier, whereas information about the educational background of the participants was a good predictor for the self-reported profession (designer, engineer or scientist). Interestingly, among the three professions, engineers appear to be the cohesive element, since they often have dual backgrounds, whereas very few participants had dual science/design backgrounds. Engineers could, therefore, build a bridge between designers and scientists, and through their integrative role, could guide the HCI community to realizing its full potential.
International Journal of Arts and Technology, 2008
In the paradigm of cultural computing, different cultures need different approaches to address th... more In the paradigm of cultural computing, different cultures need different approaches to address the cultural determinants that strongly influences our way of thinking, feeling and worldview in general. For the western culture, our answer to this need is an artistic and interactive installation (ALICE) based on the narrative 'Alice's Adventures in Wonderland'. To address the western culture characteristics highlighted in the narrative, six stages were selected and implemented as an interactive experience. From start to end, the user undergoes an immersive environment that integrates embodied and virtual agents, real and nature mimicking, and both virtual and augmented reality. Every stage challenges the hardware and software design to provide the intended experience, which at the overall system level yet have to be seamlessly integrated. A distributed and multi-layered architecture is designed to accommodate this need. After several pilot tests, the installation is ready as a cultural computing platform for the experiments that address the western cultural determinants.
Uploads
Papers by Matthias Rauterberg