Representation Learning
on Networks
Jure Leskovec, William L. Hamilton, Rex Ying, Rok Sosic
Stanford University
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 1
Why networks?
Networks are a general
language for describing
and modeling complex
systems
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 2
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 3
Network!
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 4
Many Data are Networks
A B
Social networks Economic networks Biomedical networks
Information networks: Figure 3: Higher-order cluster in the C. elegans neuronal network (28). A: The 4-node
Internet Networks of neurons
“bi-fan” motif, which is over-expressed in the neuronal networks (1). Intuitively, this motif
Web & citations describes a cooperative propagation of information from the nodes on the left to the nodes on
the right. B: The best higher-order
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWWcluster
2018in the C. elegans frontal neuronal network
5 based on
Why Networks? Why Now?
§ Universal language for describing complex
data
§ Networks from science, nature, and technology
are more similar than one would expect
§ Shared vocabulary between fields
§ Computer Science, Social science, Physics,
Economics, Statistics, Biology
§ Data availability (+computational challenges)
§ Web/mobile, bio, health, and medical
§ Impact!
§ Social networking, Social media, Drug design
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 6
Machine Learning with Networks
Classical ML tasks in networks:
§ Node classification
§ Predict a type of a given node
§ Link prediction
§ Predict whether two nodes are linked
§ Community detection
§ Identify densely linked clusters of nodes
§ Network similarity
§ How similar are two (sub)networks
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 7
Example: Node Classification
? ?
?
?
Machine
Learning
?
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 8
Example: Node Classification
Classifying the
function of
proteins in the
interactome!
Image from: Ganapathiraju et al. 2016. Schizophrenia interactome with 504 novel
protein–protein interactions. Nature.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 9
Example: Link Prediction
? x
? Machine
Learning
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 10
Example: Link Prediction
Content
recommendation
is link prediction! ?
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 11
Machine Learning Lifecycle
§ (Supervised) Machine Learning
Lifecycle: This feature, that feature.
Every single time!
Raw Structured Learning
Model
Data Data Algorithm
Feature Automatically Downstream
Engineering learn the features prediction task
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 12
Feature Learning in Graphs
Goal: Efficient task-independent
feature learning for machine learning
in networks!
node 2 vec
u
𝑓: 𝑢 → ℝ&
ℝ&
Feature representation,
embedding
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 13
Example
§ Zachary’s Karate Club Network:
A B
Input Output
Image from: Perozzi et al. 2014. DeepWalk: Online Learning of Social
Representations. KDD.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 14
Why Is It Hard?
§ Modern deep learning toolbox is
designed for simple sequences or grids.
§ CNNs for fixed-size images/grids….
§ RNNs or word2vec for text/sequences…
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 15
Why Is It Hard?
§ But networks are far more complex!
§ Complex topographical structure
(i.e., no spatial locality like grids)
§ No fixed node ordering or reference point
(i.e., the isomorphism problem)
§ Often dynamic and have multimodal features.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 16
This talk
§ 1) Node embeddings
§ Map nodes to low-dimensional
embeddings.
§ 2) Graph neural networks
§ Deep learning architectures for graph-
structured data
§ 3) Applications
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 17
