The document describes an experiment applying EM and k-means clustering algorithms to unlabeled data from a .CSV file. It involves understanding the algorithms, applying k-means and hierarchical clustering in Python to the Iris dataset, plotting the results, and comparing the algorithms. K-means clusters the Iris data into 3 groups corresponding to the known classes, while hierarchical clustering also separates the data reasonably well into 3 clusters. K-means is more efficient but requires specifying the number of clusters, while hierarchical clustering is more flexible but less scalable.
The document describes an experiment applying EM and k-means clustering algorithms to unlabeled data from a .CSV file. It involves understanding the algorithms, applying k-means and hierarchical clustering in Python to the Iris dataset, plotting the results, and comparing the algorithms. K-means clusters the Iris data into 3 groups corresponding to the known classes, while hierarchical clustering also separates the data reasonably well into 3 clusters. K-means is more efficient but requires specifying the number of clusters, while hierarchical clustering is more flexible but less scalable.
The document describes an experiment applying EM and k-means clustering algorithms to unlabeled data from a .CSV file. It involves understanding the algorithms, applying k-means and hierarchical clustering in Python to the Iris dataset, plotting the results, and comparing the algorithms. K-means clusters the Iris data into 3 groups corresponding to the known classes, while hierarchical clustering also separates the data reasonably well into 3 clusters. K-means is more efficient but requires specifying the number of clusters, while hierarchical clustering is more flexible but less scalable.
The document describes an experiment applying EM and k-means clustering algorithms to unlabeled data from a .CSV file. It involves understanding the algorithms, applying k-means and hierarchical clustering in Python to the Iris dataset, plotting the results, and comparing the algorithms. K-means clusters the Iris data into 3 groups corresponding to the known classes, while hierarchical clustering also separates the data reasonably well into 3 clusters. K-means is more efficient but requires specifying the number of clusters, while hierarchical clustering is more flexible but less scalable.
Download as DOCX, PDF, TXT or read online from Scribd
Download as docx, pdf, or txt
You are on page 1/ 6
Department of Electronics & Telecommunications
Engineering ETEL71A-Machine Learning and AI Class: BE Name: Adya Kastwar UID : 2016120024 Sem: VII Experiment: Unsupervised learning algorithms
Objective: Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the same data set for clustering using k-Means algorithm. Compare the results of these two algorithms and comment on the quality of clustering. You can make use of /Python ML library classes/API in the program. Outcomes: 1. Understand unsupervised, semi-supervised learning and the methods of clustering. 2. Understand Expectation Maximization algorithms to maximize the likelihoods. 3. Apply K-means algorithm in Python to form clusters of unlabelled data. 4. Apply Hierarchical algorithm to form clusters and plot a dendogram. 5. Compare the unsupervised learning algorithms.
System Requirements: Linux OS with Python and libraries or R or windows with MATLAB
Task 1: Describe the clustering algorithms:
a. There are mainly 4 methods of clustering in unsupervised learning: Exclusive (K-means), Overlapping (Fuzzy K-means), Hierarchical (Agglomerative, Divisive) and Probabilistic (Expectation Maximization). 1) K-means Kmeans algorithm is an iterative algorithm that tries to partition the dataset into Kpre- defined distinct non-overlapping subgroups (clusters) where each data point belongs to only one group. It tries to make the inter-cluster data points as similar as possible while also keeping the clusters as different (far) as possible. It assigns data points to a cluster such that the sum of the squared distance between the data points and the cluster’s centroid (arithmetic mean of all the data points that belong to that cluster) is at the minimum. The less variation we have within clusters, the more homogeneous (similar) the data points are within the same cluster. 2) Fuzzy K-means Unlike K-Means, which seeks hard cluster, wherein each of the points belongs to one cluster, Fuzzy K-Means seeks the softer clusters for overlapping. A single point in a soft cluster can belong to more than one cluster with a certain affinity value towards each of the points. The affinity is in proportion with the distance of that point from the cluster centroid. Similar to K-Means, Fuzzy K-Means works on the objects that have the distance measure defined and can be represented in the n-dimensional vector space. 3) Agglomerative Hierarchical clustering Technique: In this technique, initially each data point is considered as an individual cluster. At each iteration, the similar clusters merge with other clusters until one cluster or K clusters are formed. 4) Divisive Hierarchical clustering: We consider all the data points as a single cluster and in each iteration, we separate the data points from the cluster which are not similar. Each data point which is separated is considered as an individual cluster. In the end, we’ll be left with n clusters. 5) Expectation maximization : EM is an iterative process that begins with a "naive" or random initialization and then alternates between the expectation and maximization steps until the algorithm reaches convergence. We try to find a number of gaussian distributions which can be used to describe the shape of our dataset. A critical point for the understanding is that these gaussian shaped clusters must not be circular shaped as for instance in the KNN approach but can have all shapes a multivariate Gaussian distribution can take. b. Note down the steps of K-means algorithm and Hierarchical agglomerative algorithm for clustering. State the methods to find similarities between the clusters. K means: 1. Specify number of clusters K. 2. Initialize centroids by first shuffling the dataset and then randomly selecting K data points for the centroids without replacement. 3. Keep iterating until there is no change to the centroids. i.e assignment of data points to clusters isn’t changing. Compute the sum of the squared distance between data points and all centroids. Assign each data point to the closest cluster (centroid). Compute the centroids for the clusters by taking the average of the all data points that belong to each cluster. Hierarchical agglomerative : Compute the proximity matrix Let each data point be a cluster Repeat: Merge the two closest clusters and update the proximity matrix Until only a single cluster remains Key operation is the computation of the proximity of two clusters
Task 2: Write a python code to implement K-means and Agglomerative algorithm to form clusters for the ‘Iris’ flower data. Assume the value of ‘K’ from the number of classes and remove the class column to use the data for unsupervised learning.
#kmeans #adya #importing libraries import numpy as np import matplotlib.pyplot as plt import pandas as pd
#dataset and features
dataset = pd.read_csv('iris.data') X = dataset.iloc[:, [2, 3]].values
#kmeans and finding value of k
from sklearn.cluster import KMeans wcss = [] for i in range(1, 11): kmeans = KMeans(n_clusters = i, init = 'k-means++', random_state = 42) kmeans.fit(X) wcss.append(kmeans.inertia_)
plt.plot(range(1, 11), wcss)
plt.title('The Elbow Method') plt.xlabel('Number of clusters') plt.ylabel('WCSS') plt.show()
#fit model from sklearn.cluster import AgglomerativeClustering hc = AgglomerativeClustering(n_clusters = 3, affinity = 'euclidean', linkage = 'ward') y_hc = hc.fit_predict(X)
#plot clusters plt.scatter(X[y_hc == 0, 0], X[y_hc == 0, 1], s = 100, c = 'red', label = 'Cluster 1') plt.scatter(X[y_hc == 1, 0], X[y_hc == 1, 1], s = 100, c = 'blue', label = 'Cluster 2') plt.scatter(X[y_hc == 2, 0], X[y_hc == 2, 1], s = 100, c = 'green', label = 'Cluster 3') #plt.scatter(X[y_hc == 3, 0], X[y_hc == 3, 1], s = 100, c = 'cyan', label = 'Cluster 4') #plt.scatter(X[y_hc == 4, 0], X[y_hc == 4, 1], s = 100, c = 'magenta', label = 'Cluster 5') plt.title('Clusters of flowers') plt.xlabel('petal length') plt.ylabel('petal width') plt.legend() plt.show() Task 3: Plot a dendogram for agglomerative clustering, plot the clusters formed and compare iterations required by both models. Task 4: State the pros and cons of both the algorithms. Kmeans Pros: 1) Fast, robust and easier to understand. 2) Relatively efficient: O(tknd), where n is # objects, k is # clusters, d is # dimension of each object, and t is # iterations. Normally, k, t, d << n. 3) Gives best result when data set are distinct or well separated from each other. Cons : 1) The learning algorithm requires apriori specification of the number of cluster centers. 2) The use of Exclusive Assignment - If there are two highly overlapping data then k-means will not be able to resolve that there are two clusters. 3) Euclidean distance measures can unequally weight underlying factors. 4) Unable to handle noisy data and outliers. Agglomerative hieracrchical Pros : Flexible Useful for smaller data Cons : Not very scalable Cannot be used on large data
Dataset: Iris flower data set with ‘petal length’ and ‘petal width’ attributes to limit it in 2-dimensions.
Conclusion: Both the algorithms look for similarities among data and both use the same approaches to decide the number of clusters. Both give almost similar accuracy of clustering. Kmeans requires prior specification of k. Elbow method was used to find out an approximate value of k for k means.
