NPTEL
Theory of Emotion
Dr. Jainendra Shukla
jainendra@iiitd.ac.in
Computer Science and Engineering| Human-Centered Design
Agenda
1. Emotion Psychology
2. Emotion Models
NPTEL
3. Limitations of Traditional Affective Computing
4. Specificity of the emotions
5. Emotion and Brain Asymmetry
Emotion Psychology
1. What emotions themselves are?
2. How emotions are elicited or generated?
NPTEL
Emotion: A Definition
Emotion is a complex set of interactions among subjective and objective
factors, mediated by neural/hormonal systems, which can
NPTEL
(a) give rise to affective experiences such as feelings of arousal,
pleasure/displeasure;
(b) generate cognitive processes such as emotionally relevant perceptual
effects, appraisals, labeling processes;
(c) activate widespread physiological adjustments to the arousing
conditions; and
(d) lead to behavior that is often, but not always, expressive, goal, directed,
and adaptive.
- (P. Kleinginna and A. Kleinginna, 1981)
Emotions generation
● Do we run from a bear because we are afraid or are we afraid because
we run?
NPTEL
● (James, 1894) proposed that we are afraid because we run.
● Emotions are often accompanied by bodily responses (racing heart, tight
stomach, sweaty palms, tense muscles, and so on).
● Fear is not directly caused by sense perceptions but by certain thoughts
to which these perceptions may give rise (Worcester, 1893)
● The physiological responses return to the brain in the form of bodily
sensations, and the unique pattern of sensory feedback gives each
emotion its unique quality.
○ A well-armed hunter might feel joy | An ordinary man might feel curiosity
Bidirectional Projections
● Brain Impacts on the body via visceral efferent pathways
● Body impacts on the brain through afferent feedback
NPTEL
○ For Example: Laughter Yoga
■ Laughter is possible without humor or cognitive thought
● Voluntary contraction of facial muscles contributes to emotional
experience. (Strack et. al, 1988)
● Participants who hold a pencil with their lips, forcing their face to
prevent or inhibit a smile, rate cartoons as less amusing than
participants who hold pencil in their teeth, mimicking a smile (Strack et.
al, 1988 , Levenson et. al, 1990).
Emotion Models
⚫ A distinction can be made between perceived and felt (induced)
emotions (Gabrielsson, 2002)
NPTEL
⚫ Perceived emotion is the emotion recognized in the stimuli.
⚫ Induced emotion is the emotion experienced by the listener.
⚫ Representation of emotions:
⚫ Discrete or categorical model and
⚫ Dimensional models
Categorical Model
● In emotion recognition using classification methods, discrete emotion
models account for the largest parts of studies carried out.
NPTEL
● Closest to what we use in our everyday lives when we use a single word
to describe an affective state.
○ Darwin’s evolutionary view of emotions (Darwin, 1872).
● Six basic emotions happiness, anger, disgust, sadness, anxiety, and
surprise from universally recognized facial expressions. (P. Ekman and
W. Friesen, 1971)
○ Contempt as a seventh pan-cultural facial expression.
Categorical Model
● (+) From a computational point of view, these models are
easy to implement
NPTEL
● (-) It is difficult to model relations between the discrete
states.
● (-) Inconsistency is a source of criticism of these models,
since it is not agreed upon which emotions are basic and
which are not.
Dimensional models
● A 3D numerical vector
denotes the location of an
NPTEL
emotion within this space.
● VAD/PAD Model (Russell
and Mehrabian, 1977)
○ Pleasure-Displeasure Scale:
Pleasantness
○ Arousal-Non Arousal Scale:
Intensity/Energy
○ Dominance-Submissiveness:
Controlling and dominant
nature of the emotion Source:
https://artplusmarketing.com/how-to-create-viral-content-
with-studies-in-neuroscience-and-past-data-5e69c417adbe
PAD model
● This model has gained attention in the field of emotion
recognition lately, both in studies applying regression
NPTEL
methods and studies that discretize dimensions into few
areas.
● (+) Overcome the limitation of relating affective states to each
other by providing a distance between them, i.e. they allow
for computationally interpretable relations between
emotional states.
● Often, the third dimension is omitted, resulting in the VA
space: Circumplex model.
Circumplex Model
NPTEL
Source: Seemo: Liu, Z., Xu, A., Guo, Y., Mahmud, J. U., Liu, H., & Akkiraju, R. (2018, April). Seemo: A computational approach to
see emotions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-12).
Advantage of D in PAD model
● Fear can be characterized by
○ Valence: Negative
Arousal: Low/High
NPTEL
○
● Anger can be characterized by
○ Valence: Negative
○ Arousal: Low/High
● (-) We can not differentiate emotions that are overlapping!
● After including the D dimension:
● Fear
○ Valence: Negative,Arousal: Low/High , Dominance: Submissive
● Anger
○ Valence: Negative,Arousal: Low/High , Dominance: Control
● (+) Emotions that are overlapping in VA space, become
distinguishable in VAD space.
Problems in traditional AC
1. Human emotions include not only the emergency emotion stimulated by
intense instantaneous stimuli, but also the process emotion stimulated
NPTEL
by the accumulation of continued weak stimuli over a period of time.
2. Emergency emotions are quick and low-precision responses. The
computational complexity of traditional precision-oriented affective
computing is too high to handle emergency emotions.
3. Human emotions are personalized and the emotions of different human
individuals excited by the same stimulus can be different.
Specificity of the emotions
● Although basic emotions are characterized by specific facial expressions
(Ekman & Friesen, 1975), a single set of facial actions can become
NPTEL
different emotional expressions in different contexts (Barrett, 2012):
Fear and Anger
Source: https://www.paulekman.com/universal-emotions/
Fear
● Characterized by eyebrows raised and drawn together, wide open eyes,
tense lowered eyelids and stretched lips. (Ekman & Friesen, 1975)
●
NPTEL
Associated with activation within frontoparietal brain regions
(Tettamanti et. al, 2012) and a broad pattern of sympathetic activation
(Harrison et. al, 2013), including reduced heart rate variability (HRV).
● Also, associated with more numerous skin conductance responses and
larger electromyographic corrugator activity than is anger (Stemmler et
al., 2001)
Anger
● Characterized by lowered eyebrows drawn together, tense lowered
eyelids and pressed lips. (Ekman & Friesen, 1975)
NPTEL
● Left frontal PFC is activated (Harmon-Jones et al., 2010)
○ Literature is contradictory (Wacker et. al, 2008)
● No change in HRV (Rainville et. al., 2011)
● Anger may elicit either an anger-mirroring or a reciprocating fear
response (Harrison et al. 2013)
○ Psychophysiological responses will be different.
Disgust
● Characterized by raised upper lip, wrinkled nose bridge and raised
cheeks. (Ekman & Friesen, 1975)
NPTEL
● Differential skin conductance responses may depend on whether the
emotion is elicited by core-disgust inducing stimuli (e.g. Pictures of dirty
toilets) or body-boundary violating stimuli (e.g. mutilation scenes)
○ Unchanged or decreased skin-conductance for core-disgust inducing stimuli,
e.g. dirty toilets (Harrison et. al., 2013)
○ Increased skin-conductance for body-boundary violating stimuli, e.g.
mutilation scenes (Bradley et. al, 2001)
Sadness
● Characterized by raised inner eyebrows, lowered lip corners. (Ekman &
Friesen, 1975)
NPTEL
● Associated with increased blood flow in ventral regions (Mayberg et. al.,
1999).
● Crying related sadness:
○ Increased Heart rate
○ No change in HRV
○ Increased skin conductance
● Non Crying sadness:
○ Reduction in heart rate
○ Reduced HRV
○ Reduced skin conductance
○ Increased respiration
Happiness
● Characterized by tensed lower eyelids, raised cheeks and raised lip
corners. (Ekman & Friesen, 1975)
NPTEL
● The intensity of smiling in photographs has been found to predict
longevity (Abel and Kruger, 2010)
○ Individuals with no smiles: ranged from 72.9 years
○ Partial smiles: 75.0 years
○ Duchenne smiles: 79.9 years
● Happiness is associated with activation in medial prefrontal and
temporo parietal cortices (Tettamanti et al., 2012).
● A body of work further highlights the role of the left PFC in positive
affects.
● Happiness is associated with decreased HRV, amusement and joy are
associated with increases (Kreibig, 2010).
Emotion and Asymmetry
NPTEL
●Higher engagement of the left – relative to the right frontal brain – is related to
positive feelings and higher engagement. (Coan et al., 2003).
● Exception Alert! Anger = Left Brain Bias (Harmon-Jones et. al., 1998)
Source: https://imotions.com/blog/learning/best-practice/frontal-asymmetry-101-get-insights-motivation-emotions-eeg/
Putting Emotions into Design
● How to create designs that evoke emotions which result in positive user
experiences?
NPTEL
● Designers primarily focus on users’ needs in their interactions.
● But similar focus on their responses is required, which are naturally
emotional.
● Users have sophisticated thought processes going on most of the time.
● If you can elicit strong emotions in your users – you can use those
emotions to either create loyalty or to drive a customer to take action.
● Emotional design of a product or service affects its success.
● Designers aim to reach users on three cognitive levels—visceral,
behavioral and reflective—so users develop only positive associations
(sometimes including negative emotions) with products, services, etc.
Three Levels of Design
NPTEL
Source: Emotional Design: Why We Love (or Hate) Everyday Things by Don Norman
Putting Emotions into Design
● The Visceral Design: Affective Computing can help you to understand
the user’s feelings in order to improve the first impression.
NPTEL
● First time you bought an iPhone
● The Behavioral Design: Affective Computing can help you to
understand the emotions the users would feel during the tasks; both
successful and failed attempts. It can enable a thoughtful and emotional
flow process within the design.
● Keyboard typing on MacBook
● The Reflective Design: Affective Computing can enable better
understanding of post-usage emotional experience and enable an
emotional bond.
● Recall experience of a roller-coaster ride
References
1. P. Kleinginna and A. Kleinginna, “A categorized list of motivation
definitions, with a suggestion for a consensual definition,” Motivation
NPTEL
and Emotion, vol. 5, no. 4, pp. 263–291, 1981.
2. K. Scherer, “What are emotions? And how can they be measured?”
Social Science Information, vol. 44, no. 4, pp. 695–729, 2005.
3. R. Lazarus, Emotion & Adaption. Oxford University Press, 1991.
4. P. Ekman and W. Friesen, “Constants across cultures in the face and
emotion.,” Journal of Personality and Social Psychology, vol. 17, pp.
124–129, 1971.
5. J. Russell and A.Mehrabian, “Evidence for a three-factor theory of
emotions,” Journal of Research in Personality, vol. 11, pp. 273–94, 1977.
6. A.Moors, P. Ellsworth, K. Scherer, and N. Frijda, “Appraisal theories of
emotion: State of the art and future development,” Emotion Review, vol.
5, no. 2, pp. 119–124, 2013.
References
1. K. Scherer, “Emotions are emergent processes: they require a dynamic
computational architecture.,” Phil. Trans. of the Royal Society, Series B,
NPTEL
vol. 364, pp. 3459–74, 2009.
2. S.Marsella, J. Gratch, and P. Petta, “Computational models of emotion,”
in A Blueprint for an Affectively Competent Agent, K. Scherer, T.
Bänziger, and E. Roesch, Eds.,Oxford University Press, 2010, pp. 21–46.
3. R. Reisenzein, “Pleasure-arousal theory and the intensity of emotions.,”
Journal of Personality and Social Psychology, vol. 67, no. 3, pp.
525–539, 1994.
4. Gabrielsson, A. (2001). Emotion perceived and emotion felt: Same or
different?. Musicae scientiae, 5(1_suppl), 123-147.
References
1. Coan JA, Allen JJB. Frontal EEG asymmetry and behavioral activation
and inhibition systems. Psychophysiology 40: 106-114
NPTEL
2. Harmon-Jones E, Allen JJB. Anger and frontal brain activity: EEG
asymmetry consistent with approach motivation despite negative
affective valence. Journal of Personality and Social Psychology.
1998;74:1310–1316.
3. Strack et. al., 1988, Inhibiting and facilitating conditions of the human
smile: a non obtrusive test of the facial feedback hypothesis.
References
1. Levenson et. al., 1990, Voluntary facial action generates emotion
specific autonomic nervous system activity.
NPTEL
2. Harmon Jones et. al., 2010, The role of asymmetric frontal cortical
activity in emotion related phenomena: A review and update.
3. Murphy et. al., 2003, Functional neuroanatomy of emotions: A meta
analysis.
4. Costafread et. al., 2008, Predictors of amygdala activation during the
processing of emotional stimuli: A meta anlaysis of 385 PET and fMRI
studies.
