- Vogt-Kölln-Str. 30
Di Fu
University of Hamburg, Dept. of Informatics, Faculty Member
- Chinese Academy of Sciences, Institute of Psychology, Graduate Studentadd
- I got my Ph.D. in Cognitive Neuroscience in 2020. I am now working as a postdoctoral researcher studying crossmodal a... moreI got my Ph.D. in Cognitive Neuroscience in 2020. I am now working as a postdoctoral researcher studying crossmodal attention and human-robot interaction at the department of Informatics, University of Hamburg.edit
Robot facial expressions and gaze are important factors for enhancing human-robot interaction (HRI), but their effects on human collaboration and perception are not well understood, for instance, in collaborative game scenarios. In this... more
Robot facial expressions and gaze are important factors for enhancing human-robot interaction (HRI), but their effects on human collaboration and perception are not well understood, for instance, in collaborative game scenarios. In this study, we designed a collaborative triadic HRI game scenario where two participants worked together to insert objects into a shape sorter. One participant assumed the role of a guide. The guide instructed the other participant, who played the role of an actor, to place occluded objects into the sorter. A humanoid robot issued instructions, observed the interaction, and displayed social cues to elicit changes in the two participants' behavior. We measured human collaboration as a function of task completion time and the participants' perceptions of the robot by rating its behavior as intelligent or random. Participants also evaluated the robot by filling out the Godspeed questionnaire. We found that human collaboration was higher when the robot displayed a happy facial expression at the beginning of the game compared to a neutral facial expression. We also found that participants perceived the robot as more intelligent when it displayed a positive facial expression at the end of the game. The robot's behavior was also perceived as intelligent when directing its gaze toward the guide at the beginning of the interaction, not the actor. These findings provide insights into how robot facial expressions and gaze influence human behavior and perception in collaboration.
Research Interests:
Increasing anthropomorphic robot behavioral design could affect trust and cooperation positively. However, studies have shown contradicting results and suggest a task-dependent relationship between robots that display emotions and trust.... more
Increasing anthropomorphic robot behavioral design could affect trust and cooperation positively. However, studies have shown contradicting results and suggest a task-dependent relationship between robots that display emotions and trust. Therefore, this study analyzes the effect of robots that display human-like emotions on trust, cooperation, and participants' emotions. In the between-group study, participants play the coin entrustment game with an emotional and a non-emotional robot. The results show that the robot that displays emotions induces more anxiety than the neutral robot. Accordingly, the participants trust the emotional robot less and are less likely to cooperate. Furthermore, the perceived intelligence of a robot increases trust, while a desire to outcompete the robot can reduce trust and cooperation. Thus, the design of robots expressing emotions should be task dependent to avoid adverse effects that reduce trust and cooperation.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests: Psychology, Computer Science, Gaze, ROBOT, Human Robot Interaction, and 4 moreAvatar, Crossmodal, ICub, and arXiv
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal... more
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for computational intelligent agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdi...
Research Interests:
Spoken language is one of the most efficientways to instruct robots about performing domestic tasks. However, the state of the environment has to be considered to plan and execute actions successfully. We propose a system that learns to... more
Spoken language is one of the most efficientways to instruct robots about performing domestic tasks. However, the state of the environment has to be considered to plan and execute actions successfully. We propose a system that learns to recognise the user’s intention and map it to a goal. A reinforcement learning (RL) system then generates a sequence of actions toward this goal considering the state of the environment. A novel contribution in this paper is the use of symbolic representations for both input and output of a neural Deep Q-network (DQN), which enables it to be used in a hybrid system. To show the effectiveness of our approach, the Tell-Me-Dave corpus is used to train an intention detection model and in a second step an RL agent generates the sequences of actions towards the detected objective, represented by a set of state predicates. We show that the system can successfully recognise command sequences fromthis corpus aswell as train the deep- RL network with symbolic i...
Research Interests:
Research Interests:
Research Interests:
To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging... more
To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention responses. In particular, human performance was better under the congruent audiovisual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audiovisual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment. While the human performance was overall superior, our trained model demonstrated that it could replicate attention responses similar to humans.
Research Interests:
Research Interests:
Sensory and emotional experiences such as pain and empathy are relevant to mental and physical health. The current drive for automated pain recognition is motivated by a growing number of healthcare requirements and demands for social... more
Sensory and emotional experiences such as pain and empathy are relevant to mental and physical health. The current drive for automated pain recognition is motivated by a growing number of healthcare requirements and demands for social interaction make it increasingly essential. Despite being a trending area, they have not been explored in great detail. Over the past decades, behavioral science and neuroscience have uncovered mechanisms that explain the manifestations of pain. Recently, also artificial intelligence research has allowed empathic machine learning methods to be approachable. Generally, the purpose of this paper is to review the current developments for computational pain recognition and artificial empathy implementation. Our discussion covers the following topics: How can AI recognize pain from unimodality and multimodality? Is it necessary for AI to be empathic? How can we create an AI agent with proactive and reactive empathy? This article explores the challenges and ...
Research Interests:
Recent studies have revealed the key importance of modelling personality in robots to improve interaction quality by empowering them with social-intelligence capabilities. Most research relies on verbal and non-verbal features related to... more
Recent studies have revealed the key importance of modelling personality in robots to improve interaction quality by empowering them with social-intelligence capabilities. Most research relies on verbal and non-verbal features related to personality traits that are highly context-dependent. Hence, analysing how humans behave in a given context is crucial to evaluate which of those social cues are effective. For this purpose, we designed an assistive memory game, in which participants were asked to play the game obtaining support from an introvert or extroverted helper, whether from a human or robot. In this context, we aim to (i) explore whether selective verbal and non-verbal social cues related to personality can be modelled in a robot, (ii) evaluate the efficiency of a statistical decision-making algorithm employed by the robot to provide adaptive assistance, and (iii) assess the validity of the similarity attraction principle. Specifically, we conducted two user studies. In the ...
Research Interests:
Research Interests:
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal... more
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for intelligent computational agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal... more
Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for intelligent computational agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Human decision-making behaviors in social contexts are largely driven by fairness considerations. The dual-process model suggests that in addition to cognitive processes, emotion contributes to economic decision-making. Although humor, as... more
Human decision-making behaviors in social contexts are largely driven by fairness
considerations. The dual-process model suggests that in addition to cognitive
processes, emotion contributes to economic decision-making. Although humor, as
an effective emotional regulation strategy to induce positive emotion, may influence
an individual’s emotional state and decision-making behavior, previous studies have
not examined how humor modulates fairness-related responses in the gain and loss
contexts simultaneously. This study uses the Ultimatum Game (UG) in gain and loss
contexts to explore this issue. The results show, in the gain context, viewing humorous
pictures compared to humorless pictures increased acceptance rates and this effect
was moderated by the offer size. However, we did not find the same effect in the loss
context. These findings indicate that humor’s affection for fairness considerations may
depend on the context and provide insight into the finite power of humor in human
sociality, cooperation and norm compliance.
considerations. The dual-process model suggests that in addition to cognitive
processes, emotion contributes to economic decision-making. Although humor, as
an effective emotional regulation strategy to induce positive emotion, may influence
an individual’s emotional state and decision-making behavior, previous studies have
not examined how humor modulates fairness-related responses in the gain and loss
contexts simultaneously. This study uses the Ultimatum Game (UG) in gain and loss
contexts to explore this issue. The results show, in the gain context, viewing humorous
pictures compared to humorless pictures increased acceptance rates and this effect
was moderated by the offer size. However, we did not find the same effect in the loss
context. These findings indicate that humor’s affection for fairness considerations may
depend on the context and provide insight into the finite power of humor in human
sociality, cooperation and norm compliance.