1
2
Wollo University ,Kombolicha Institute of Technology
Department of Software Engineering
Fundamental of Machine Learning
By Ashenafi Workie(MSc.)
KIOT@SE by Ashenafi Workie
Major chapters outlines
1 Chapter 1: Introduction to Machine Learning
2 Chapter 2: Classification based Supervised Learning
3 Chapter 3: Regression based Supervised Learning
4 Chapter 4: Unsupervised Learning
5 Chapter 5: Reinforcement Learning
6 Chapter 6: Advanced Machine Learning
4
Assessment Methods
• Assignments and Lab [15%]
• Quiz and Test [20%]
• Project and presentation [20%]
• Final Exam [45%]
5
Regression vs classification
6
Regression vs classification
7
What is Regression
▪ Regression is
▪ The difference between classification and regression is that in
regression our target variable is numeric with nominal as textual
and continuous.
▪ Companies may use this for doing things such as sales forecasts
or forecasting manufacturing defects.
▪ One creative example: predicting the probability of celebrity
divorce.
8
Regression
9
Regression
10
Regression
11
Regression
12
linear Regression
13
linear Regression
14
Linear Regression
▪ Linear regression: is probably the simplest approach for statistical
learning.
▪ Good starting point for more advanced approaches.
▪ Many fancy statistical learning techniques are extension of LR
.
▪ LR is very good to answer the following questions:
▪ Is there a relationship between two variables?
▪ How strong is the relationship?
▪ Which variable contribute the most?
▪ How accurately can we estimate the effect of each variable?
▪ How accurately can we predict the target?
15
linear Regression
16
linear Regression
17
Linear Regression
▪ Linear regression:
▪ Pros: Easy to interpret results, computationally inexpensive.
▪ Cons: Poorly models nonlinear data.
▪ Works with: Numeric values, nominal values.
▪ Our goal when using regression is to predict a numeric target
value.
▪ One way to do this is to write out an equation for the target value
with respect to the inputs.
18
linear Regression
19
linear Regression
20
linear Regression
21
linear Regression
22
linear Regression
23
linear Regression
24
Calculate the predicted values
25
Calculate the error and draw the line
26
linear Regression
27
More on Linear Regression
28
More on Linear Regression
29
Logistic Regression
30
Logistic Regression
31
Logistic Regression
32
Logistic Regression
33
Logistic Regression
34
Logistic Regression
35
Logistic Regression
36
Logistic Regression
37
Logistic Regression
38
Logistic Regression
39
Logistic Regression
40
Logistic Regression
❖ Stochastic gradient ascent can do as well as gradient ascent using far fewer
computing resources.
❖ Logistic regression:
❖ Pros: Computationally inexpensive, easy to implement, knowledge representation
easy to interpret.
❖ Cons: prone to underfitting, may have low accuracy.
❖ Works with: Numeric values, nominal values.
41
Logistic Regression(setting thourshold)
❖ We can treat a binary classification problem as a regression problem.
❖ The binary classes 1/0, and fit a regression algorithm.
❖ When a new unlabeled recorded comes in, try to guess the value with a
regression algorithm.
❖ If the value is higher than the midpoint (0.5), declare the prediction to be
the class 1, and vice versa.
42
Logistic Regression
❖ What if our data looks different as shown below; the line becomes
more slanted.
❖ This way, our algorithm is not skewed by the sample and it
manages to find the obvious visual threshold.
43
Logistic Regression
❖ Yb(x) is always going to be between 0 and 1.
❖ As opposed to linear regression approach; which can take on any
value.
44
Logistic Regression
45
Logistic Regression
46
End ….
47
