Chujun Lin

People express emotions via a variety of behaviors, including facial muscle movements, body poses and gestures, vocal prosody, and speech. To understand how people experience and perceive emotion, it is crucial to quantify and model these behaviors. However, existing methods are insufficient to address this need. Manually annotating behavior is very time-consuming, making it infeasible to do at scale. Moreover, common linear models cannot fully capture the complex, nonlinear, and interactive affective processes embodied by these behaviors. In this methodology review, we describe how deep learning addresses these challenges and thereby promises to advance naturalistic affective science. First, deep learning provides accessible and efficient tools to annotate dynamic, complex, multi-modal behaviors. These automated annotation tools can scale up behavioral quantification to a degree impossible with human coders, enabling many new, more naturalistic approaches to affective science. Seco...

People readily attribute many traits to faces: some look beautiful, some competent, some aggressive. These snap judgments have important consequences in real life, ranging from success in political elections to decisions in courtroom sentencing. Modern psychological theories argue that the hundreds of different words people use to describe others from their faces are well captured by only two or three dimensions, such as valence and dominance, a highly influential framework that has been the basis for numerous studies in social and developmental psychology, social neuroscience, and in engineering applications. However, all prior work has used only a small number of words (12 to 18) to derive underlying dimensions, limiting conclusions to date. Here we employed deep neural networks to select a comprehensive set of 100 words that are representative of the trait words people use to describe faces, and to select a set of 100 faces. In two large-scale, preregistered studies we asked part...

People tend to attribute positive qualities to other individuals who look physically attractive--the beauty-is-good stereotype in person perception. Here, we investigate how people might over-generalize this stereotype to nonhuman objects, in particular, graphs that present various information in scientific papers.

People tend to attribute positive qualities to other individuals who look physically attractive--the beauty-is-good stereotype in person perception. Here, we investigate how people might over-generalize this stereotype to nonhuman objects, in particular, graphs that present various information in news media.

People tend to attribute positive qualities to other individuals who look physically attractive: the beauty-is-good stereotype in person perception. Here, we investigate how people might over-generalize this stereotype to nonhuman objects, in particular, graphical data visualizations. We generated graphs in 6 most popular formats used by news media (bar plot, line plot, map, scatter plot, pie, heat map) regarding 6 different popular news topics (politics, sports, science and technology, economic and business, health, and nature). We digitally manipulated the graph in a specific format of a particular topic in four experimental conditions: looking more or less beautiful x being more or less misleading. We will investigate, whether making a graph look more (or less) beautiful makes people trust the graph more (or less), regardless of whether the graph is misleading.

This project is conducted by the COVID-DYNAMIC Team (http://conte.caltech.edu/content/covid-dynamic-investigator-team) The COVID-19 pandemic is confronting people with significant changes in affectively-laden media exposure as well as in social behavior. To quantify longitudinal effects that could inform public health policy, we will collect anonymized data over the internet on subject's implicit and explicit emotional biases, and characterize how these might relate to social, economic, moral and political judgments and decision-making. We will sample approximately 1000 English-speaking adult participants in the US over the internet anonymously, recruited through Prolific.co. We will also sample an extant registry of approximately 100 community-dwelling adults in Los Angeles that are part of the Caltech Conte Center on the Neurobiology of Social Decision-Making (http://conte.caltech.edu/content/covid-dynamic). We plan to test participants longitudinally for approximately one hou...

This thesis is motivated by the fascinating question of how people make inferences about others from their faces. How do we infer somebody's intent or personality merely from looking at them? I studied this question by investigating how people make trait attributions in two specific contexts -- political election (Chapter 2) and political corruption (Chapter 3) -- as well as how people make a large variety of trait attributions from faces in general (Chapter 4). I employed novel methods to representatively sample the words used to rate faces, and to select the facial stimuli themselves (e.g., using artificial neural networks), to test the reproducibility and generalizability of my results (e.g., pre-registration, generalization across participants from different cultures), and to elucidate the underlying mechanisms (e.g., mediation modeling, digital manipulation of facial stimuli). The results demonstrated that trait attributions from politician's faces were associated with ...

Processing faces is difficult for individuals with autism spectrum disorder (ASD). However, it remains unclear whether individuals with ASD are capable of making high-level social trait judgments from faces. Here, we comprehensively address this question using naturalistic face images and representatively sampled traits. Despite of intact underlying dimensions, people with ASD showed atypical judgments and reduced specificity within each trait. Deep neural networks revealed that these group differences were driven by discrepant judgments for certain types of faces and differential attention to certain features within a face. Our results were replicated in well-characterized in-lab participants and partially generalized to posed neutral faces (a preregistered study). Finally, atypical social trait judgments from faces in ASD were associated with socio-emotional experience during social interactions. Together, our results provide new insights into the computational bases and behaviora...

Scientists, policymakers, and the public increasingly rely on data visualizations – such as COVID tracking charts, weather forecast maps, and political polling graphs – to inform important decisions. The aesthetic decisions of graph-makers may produce graphs of varying visual appeal, independent of data quality. Here we tested whether the beauty of a graph influences how much people trust it. Across three studies, we sampled graphs from social media, news reports, and scientific publications, and consistently found that graph beauty predicted trust. In a fourth study, we manipulated both the graph beauty and misleadingness. We found that beauty, but not actual misleadingness, causally affected trust. These findings reveal a source of bias in the interpretation of quantitative data and indicate the importance of promoting data literacy in education.

Autism spectrum disorder (ASD) is characterized by difficulties in social processes, interactions, and communication. Yet, the neurocognitive bases underlying these difficulties are unclear. Here, we triangulated the trans-diagnostic approach to personality, social trait judgments of faces, and neurophysiology to investigate (1) the relative position of autistic traits in a comprehensive social-affective personality space and (2) the distinct associations between the social-affective personality dimensions and social trait judgment from faces in individuals with ASD and neurotypical individuals. We collected personality and facial judgment data from a large sample of online participants (N = 89 self-identified ASD; N = 308 neurotypical controls). Factor analysis with 33 sub-scales of 10 social-affective personality questionnaires identified a 4-dimensional personality space. This analysis revealed that ASD and control participants did not differ significantly along the personality d...

People readily attribute many traits to faces: some look beautiful, some competent, some aggressive. Modern psychological theories argue that the hundreds of different words people use to describe others from their faces are well captured by only two or three dimensions, such as valence and dominance, a highly influential framework that has been the basis for numerous studies across social and developmental psychology, social neuroscience, and engineering applications. However, all prior work has used only a small number of words (12 to 18) to derive underlying dimensions, limiting conclusions to date. Here we employed deep neural networks to select a comprehensive set of 100 words that are representative of the trait words people use to describe faces, and to select a set of 100 faces. In two large-scale, preregistered studies we asked participants to rate the 100 faces on the 100 words (obtaining 2,850,000 ratings from 1,710 participants), and discovered a novel set of four psycho...

People understand other individuals by considering both their momentary and enduring characteristics: their mental states and their traits. Mental states and traits are both independently useful for making sense of others’ behavior. However, people might gain even more insight by linking these two forms of social information together. Here we investigated this potential link between how perceivers attribute mental states and traits to other people. Prior research indicates that mental state and trait attributions are correlated. Here, we tested whether this correlation generalized to more naturalistic contexts, including forming impressions about unfamiliar target people based on videos (Study 1) and making judgments about personally familiar people such as family and friends (Study 2). In both studies, we found that trait attributions were correlated with the frequency distribution of mental state attributions. Having corroborated this association, we next sought to understand the ...

People readily (but often inaccurately) attribute traits to others based on faces. While the details of attributions depend on the language available to describe social traits, psychological theories argue that two or three dimensions (such as valence and dominance) summarize social trait attributions from faces. However, prior work has used only a small number of trait words (12 to 18), limiting conclusions to date. In two large-scale, preregistered studies we ask participants to rate 100 faces (obtained from existing face stimuli sets), using a list of 100 English trait words that we derived using deep neural network analysis of words that have been used by other participants in prior studies to describe faces. In study 1 we find that these attributions are best described by four psychological dimensions, which we interpret as “warmth”, “competence”, “femininity”, and “youth”. In study 2 we partially reproduce these four dimensions using the same stimuli among additional participa...

People spontaneously infer other people’s psychology from faces, encompassing inferences of their affective states, cognitive states, and stable traits such as personality. These judgments are known to be often invalid, but nonetheless bias many social decisions. Their importance and ubiquity have made them popular targets for automated prediction using deep convolutional neural networks (DCNNs). Here we investigated the applicability of this approach: how well does it generalize, and what biases does it introduce? We compared three distinct sets of features (from a face identification DCNN, an object recognition DCNN, and using facial geometry), and tested their prediction across multiple out-of-sample datasets. Across judgments and datasets, features from both pre-trained DCNNs provided better predictions than did facial geometry. However, predictions using object recognition DCNN features were not robust to superficial cues (e.g., color, hair style). Importantly, predictions usin...

People instantaneously evaluate faces with significant agreement on evaluations of social traits. However, the neural basis for such rapid spontaneous face evaluation remains largely unknown. Here, we recorded from 490 neurons in the amygdala and hippocampus in 5 neurosurgical patients and show that amygdala and hippocampal neurons encode a social trait space. We further investigated the temporal evolution and modulation on the social trait representation, and we employed encoding and decoding models to reveal the critical social traits for the trait space. We also recorded from another 259 neurons and replicated our findings using different social traits. Lastly, the neuronal social trait space may have a behavioral consequence likely involved in the abnormal processing of social information in autism. Together, our results suggest that there exists a neuronal population code for a comprehensive social trait space in the human amygdala and hippocampus that underlie spontaneous firs...

Neurons in the human medial temporal lobe (MTL) that are selective for the identity of specific people are classically thought to encode identity invariant to visual features. However, it remains largely unknown how visual information from higher visual cortex is translated into a semantic representation of an individual person. Here, we show that some MTL neurons are selective to multiple different face identities on the basis of shared features that form clusters in the representation of a deep neural network trained to recognize faces. Contrary to prevailing views, we find that these neurons represent an individual’s face with feature-based encoding, rather than through association with concepts. The response of feature neurons did not depend on face identity nor face familiarity, and the region of feature space to which they are tuned predicted their response to new face stimuli. Our results provide critical evidence bridging the perception-driven representation of facial featur...

Recent studies in adult humans have reported correlations between individual differences in people’s Social Network Index (SNI) and gray matter volume (GMV) across multiple regions of the brain. However, the cortical and subcortical loci identified are inconsistent across studies. These discrepancies might arise because different regions of interest were hypothesized and tested in different studies without controlling for multiple comparisons, and/or from insufficiently large sample sizes to fully protect against statistically unreliable findings. Here we took a data-driven approach in a pre-registered study to comprehensively investigate the relationship between SNI and GMV in every cortical and subcortical region, using three predictive modeling frameworks. We also included psychological predictors such as cognitive and emotional intelligence, personality, and mood. In a sample of healthy adults (n = 92), neither multivariate frameworks (e.g., ridge regression with cross-validatio...

How do people form impressions of others based on faces? Existing psychological theories argue that people attribute traits to others from faces along two or three dimensions. While these theories have now been incorporated into numerous empirical and theoretical studies, they were derived from a small set of trait attributions, which limits their generalizability and leaves the true nature of the psychological dimensions unclear. The present study applied deep neural networks to representatively sample an inclusive list of traits and faces, generating a comprehensive set of 100 traits and 100 faces that we administered in two large-scale preregistered studies. These comprehensive trait attributions (Study 1, 750,000 ratings) revealed a novel four-dimensional space: warmth, competence, female-stereotype, and youth-stereotype, challenging existing theories. Study 2 collecting dense individual-level data in seven different countries (2,100,000 trials) reproduced this four-dimensional ...

While inferences of traits from unfamiliar faces prominently reveal stereotypes, some facial inferences also correlate with real-world outcomes. We investigated whether facial inferences are associated with an important real-world outcome closely linked to the face bearer's behavior: political corruption. In four preregistered studies ( N = 325), participants made trait judgments of unfamiliar government officials on the basis of their photos. Relative to peers with clean records, federal and state officials convicted of political corruption (Study 1) and local officials who violated campaign finance laws (Study 2) were perceived as more corruptible, dishonest, selfish, and aggressive but similarly competent, ambitious, and masculine (Study 3). Mediation analyses and experiments in which the photos were digitally manipulated showed that participants' judgments of how corruptible an official looked were causally influenced by the face width of the stimuli (Study 4). The findi...