Python for Data Science
Machine Learning in Python:
Clustering
Dr. Ilkay Altintas and Dr. Leo Porter
Twitter: #UCSDpython4DS
By the end of this video, you should be able to:
Python for Data Science
§ Articulate the goal of cluster analysis
§ Discuss whether cluster analysis is supervised or
unsupervised
§ List some ways that cluster results can be applied
Cluster Analysis Overview
Python for Data Science
Goal: Organize similar items into groups
Cluster Analysis Examples
Python for Data Science
• Segment customer base into groups
• Characterize different weather patterns
for a region
• Group news articles into topics
• Discover crime hot spots
Cluster Analysis
• Divides data into clusters
Python for Data Science
• Similar items are placed in same cluster
Intra-cluster
differences are
minimized
Inter-cluster differences are
v maximized
Similarity Measures
A A
Python for Data Science
B B
Euclidean Distance Manhattan Distance
Cosine Similarity
Normalizing Input Variables
Python for Data Science
Scaled Values
Weight
Height
Cluster Analysis Notes
Python for Data Science
Unsupervised
There is no ‘correct’
clustering
Clusters don’t come
with labels
Interpretation and analysis required to
make sense of clustering results!
Uses of Cluster Results
• Data segmentation
Python for Data Science
• Analysis of each segment can provide insights
science fiction
non-fiction
children’s
Uses of Cluster Results
• Categories for classifying new data
Python for Data Science
• New sample assigned to closest cluster
Label of closest
cluster used to
classify new
sample
Uses of Cluster Results
• Labeled data for classification
Python for Data Science
• Cluster samples used as labeled data
Labeled samples
for science fiction
customers
Uses of Cluster Results
• Basis for anomaly detection
Python for Data Science
• Cluster outliers are anomalies
Anomalies that
require further
v analysis
Cluster Analysis Summary
• Organize similar items into groups
Python for Data Science
• Analyzing clusters often leads to useful
insights about data
• Clusters require analysis and interpretation
Python for Data Science
Machine Learning in Python:
k-Means Clustering
Dr. Ilkay Altintas and Dr. Leo Porter
Twitter: #UCSDpython4DS
By the end of this video, you should be able to:
Python for Data Science
§ Describe the steps in the k-means algorithm
§ Explain what the ‘k’ stands for in k-means
§ Define cluster centroid
Cluster Analysis
• Divides data into clusters
Python for Data Science
• Similar items are in same cluster
Intra-cluster
differences are
minimized
Inter-cluster differences are
maximized
k-Means Algorithm
Select k initial centroids (cluster centers)
Python for Data Science
Repeat
Assign each sample to closest centroid
Calculate mean of cluster to determine new centroid
Until some stopping criterion is reached
centroid
X
(a) (b) (c)
X X k-Means
Python for Data Science
X X
Original samples Initial centroids Assign samples
(d) (e) (f)
X
X X X X
X
Re-calculate centroids Assign samples Re-calculate centroids
Choosing Initial Centroids
Issue:
Python for Data Science
Final clusters are sensitive to initial centroids
Solution:
Run k-means multiple times with
different random initial centroids,
and choose best results
Evaluating Cluster Results
error = distance between sample & centroid
Python for Data Science
X squared error = error2
Sum of squared errors between all
samples & centroid
Sum over all clusters WSSE
Within-Cluster Sum of
Squared Error
Using WSSE
Python for Data Science
WSSE1 < WSSE2 WSSE1 is better numerically
Caveats:
• Does not mean that cluster set 1 is
more ‘correct’ than cluster set 2
• Larger values for k will always reduce
WSSE
Choosing Value for k
• Approaches: k=?
Python for Data Science
• Visualization
• Application-Dependent
• Data-Driven
Elbow Method for Choosing k
“Elbow” suggests value for
Python for Data Science
k should be 3
Stopping Criteria
X
Python for Data Science
When to stop iterating?
• No changes to centroids
• Number of samples changing clusters
is below threshold
Interpreting Results
• Examine cluster centroids
Python for Data Science
• How are clusters different?
X
X Compare centroids
to see how clusters
are different
X
K-Means Summary
• Classic algorithm for cluster analysis
Python for Data Science
• Simple to understand and implement
and is efficient
• Value of k must be specified
• Final clusters are sensitive to initial
centroids
