[go: up one dir, main page]
Scribd
Upload
67 views14 pages

Machine Learning Algorithms: Amit Kumar Singh.b

Linear regression finds the best fit line to model the relationship between variables. K-nearest neighbors (KNN) algorithm uses distance calculation and majority voting of k-nearest data points to classify data. Support vector machines (SVM) find the optimal separating hyperplane between two classes by maximizing the margin between them. Logistic regression uses the sigmoid function to transform linear regression output into probabilities for binary classification.

Uploaded by

amit kumar singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
67 views14 pages

Machine Learning Algorithms: Amit Kumar Singh.b

Linear regression finds the best fit line to model the relationship between variables. K-nearest neighbors (KNN) algorithm uses distance calculation and majority voting of k-nearest data points to classify data. Support vector machines (SVM) find the optimal separating hyperplane between two classes by maximizing the margin between them. Logistic regression uses the sigmoid function to transform linear regression output into probabilities for binary classification.

Uploaded by

amit kumar singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

Machine Learning

Algorithms

Amit kumar singh.B


Linear regression
• For applying this algo there should be some sort of relation among x
and y
• Firstly we need to find best fit line.
• We have eqn of best fit line to be y=m*x+b
• Where m is slope and b is y-intercept
Linear regression efficiency
KNN
k-Nearest Neighbors
 Pros: High accuracy, insensitive to outliers, no assumptions about data
 Cons: Computationally expensive, requires a lot of memory
 Works with: Numeric values, nominal values
 Distance calculation with Gaussian method(father of geometry)
 Selecting the n nearest neighbours
Support Vector Machine
• It’s a binary classifier it basically divides only one group at a time
• Firstly find best separating hyper plane known as decision boundary
• It’s the best boundary as it has max width of points

• Its worst as the distance of point and decision boundary is min


SVM(understanding vectors)
• Vectors have both magnitude and direction
• Direction A

• Magnitude A
• If a is [4,5] then magnitude is 4**2+5**2=5
• Dot product of vector if a=[2,3] and b=[4,5]
• A.B=(2 4)+(3 5)=23
If there are 2 dimensions we can rely on to linear algebra. If the
dimensions are 63 63 then vector space can be used.
SVM hyper plane ( decision boundary)
• Vector w is any point perpendicular to decision boundary and vector
u a unknown data point
• If u.w+b>=0 then +
• If u.w+b<=0 then –
• Where b is bias
• U is the data set
• We need to find both w and b
• X (-ve class data).W+B=-1
• X (+ve class data).W+B=1
•Yi(class) it will be -1 and 1 for the respective classes
SVM hyper plane ( decision boundary)(part 2)
• Yi ( X i (-ve class data).W+B)=(-1 Yi) class -1
• Yi ( X i (+ve class data).W+B)=(1 Yi) class 1
• Then both equation would become
• Yi ( X i (both class data).W+B)=1
• Yi ( X i (both class data).W+B)-1=0
• Svm optimization
• We mainly focus on maximizing the width between both the hyper
plane
• Width=(x+-x-)-w/ w
• Maximize the width(b) .minimize the w .
• X =y (x.w+b)==1+b similarly X =y (x.w+b)==1-b
+ i - i
SVM optimization(first method upto 1993)
• So we introduce lagrangian
𝑛
equation
• L(w,b)=(1/2* W **2)-෍ α[Yi ( X i (both class data).W+B)-1]
𝑖=0
SVM Optimization (using CONVEX method)

W Magnitude of w
start with
`
maximum

Global minimum
SVM Conclusion
• We need to find the w(minimum) and b(maximum) so that it satisfies
the equation Yi(Xi.W+B)>=1
• Then use y=sign(x.w+b) for prediction
• For the hyper plane (-w[i]*x-b+Yi)/w[i+1]) if its 2d data then I will be 0
and Yi can be -1 for –ve boundary ,+1 for +ve boundary ,0 for decision
boundary
• For hyper plane consider the min ,max featureset value
• Initally the w should be [maxfeaterue ,maxfeature],and atleast 3 step
sizes like 10,1,0.1 percent of data
• Need to check w in all possible transformation [1,1],[1,-1],[-1,-1],
[-1,1]
KMeans (clustering) unsupervised learning
• Flat clustering(will buy or not)
• Hierarchal clustering(will , likely , not buyer)
• We need to choose centroids Kmeans comes by default with 8 cluster
• Centroids are formed by taking the distance from each points either
by Euclidean or norm methodology
• Each centroids are recognized by having iterations until no change in
centroids
• Here the classifier has fit,cluster_center_(centroids),labels_(voting
features)
Logistic Regression

• Sigmoid function (shown below), the standard linear formula is


transformed to the logistic regression formula (also shown below).
This logistic regression function is useful for predicting the class of a
binomial target feature.
• The Sigmoid Function Logistic Regression Formula
Logistic Regression Assumptions

• Any logistic regression example in Python is incomplete without


addressing model assumptions in the analysis. The important
assumptions of the logistic regression model include:
• Target variable is binary
• Predictive features are interval (continuous) or categorical
• Features are independent of one another
• Sample size is adequate – Rule of thumb: 50 records per predictor

You might also like