COSC 3107: Machine Learning

Lecture 1
Introduction to Machine Learning
COSC-3107 Machine Learning

Shahzad Hussain
Lecturer
Today’s Lecture Outline
1. Course Evaluation Detail

2. Introduction to Machine Learning

3. Machine Learning Case Studies

COSC-3107 Machine Learning
4. Machine Learning Course Journey!

5. Machine Learning Prerequisites

2 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
1. Course Evaluation Procedure
COSC-3107 Machine Learning
Course Evaluation Procedure

• Mid Term 30%

• Final Term 50%
• Sessional 20%
– Quizzes 5%
– Assignments 5%
COSC-3107 Machine Learning
– Term Project 10%
– Presentations
– Class behavior
• Attendance must be maintained as per rule (>=75%)

4 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Recommended Book
• Practical Machine Learning
with Python
Dipanjar Sarkar

Practical Implementation & Reference Book

COSC-3107 Machine Learning
• Introduction to Machine
Learning with Python
Andreas C. Muller & Sarach Guido

5
Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
1. Artificial Intelligence, Machine Learning, and Deep Learning

COSC-3107 Machine Learning

6 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Artificial Intelligence, Machine Learning and Deep Learning

• Promised future of intelligent chatbots, self-

driving cars, and virtual assistants where
deep learning will play crucial roles.

COSC-3107 Machine Learning

7 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Artificial Intelligence, Machine Learning and Deep Learning

• The field of artificial intelligence: the

effort to automate intellectual tasks
normally performed by humans

COSC-3107 Machine Learning

8 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Artificial Intelligence, Machine Learning and Deep Learning

• The dominant paradigm in AI from 1950s to

the late 1980s:
Expert System (known as symbolic AI).
– Intractable to figure out explicit rules for
solving complicated rules (e.g., image
classification, speech recognition, language translation)

COSC-3107 Machine Learning

An example of a rule-based system

9 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Artificial Intelligence, Machine Learning and Deep Learning

• Machine learning system is trained

rather than explicitly programmed
– Many examples (data) are needed for
finding out the rules automatically

Rules Classical Answers COSC-3107 Machine Learning

Data Programming

Data Machine Rules

Answers Learning

Machine Learning a new Programming Paradigm

10 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Artificial Intelligence, Machine Learning and Deep Learning

• To do machine learning we need three things:

– Input data points
– Examples of the expected output (=labels)
– A measure for goodness (objective function)
• Machine learning models are all about finding
appropriate representations for input data
– e.g., coordinate change
COSC-3107 Machine Learning

11 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
3. Introduction to Machine Learning
COSC-3107 Machine Learning

12 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Machine Learning as Inductive Inference
COSC-3107 Machine Learning
What is Machine Learning?
First Explanation:
• Development of algorithms which allow a
computer to “learn” specific tasks from
training examples.
• Learning means that the computer can not
only memorize the seen examples, but can
generalize to previously unseen instances.
COSC-3107 Machine Learning

• Ideally, the computer should use the examples

to extract a general “rule” how the specific
task has to be performed correctly.

14 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Deduction vs. Induction
• Who know what induction and deduction mean?

• Deduction inference is the process of reasoning from

one or more general statements (premises) to reach a
logically certain conclusion.
• Example:
– Premise 1: every person in this room is a student.
– Premise 2: every person student in older than 10 years.
– Conclusions: every person in this room is older than 10 COSC-3107 Machine Learning
years.
• If the premises are correct, then all conclusions
are correct as well.
• Nice in theory, Mathematics is based on this principle.
• But no natural way to deal with uncertainty regarding the
premises.
15 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Deduction vs. Induction
• Inductive inference: reasoning that
constructs or evaluates general prepositions
that are derived from specific examples.
• Example:
– We drop lots of things, very often.
– In all our experiments, the things fall downwards,
not upwards. COSC-3107 Machine Learning

– So we conclude that likely, things always fall

downwards when we drop them.

• Very important: we can never be sure,

our conclusion can be wrong!

16 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Deduction vs. Induction
• Humans do inductive reasoning all the
time: we draw uncertain conclusions from our
relatively limited experiences.
• Example:
– You come 10 minutes late to every lecture I give.
– The first 7 times I don’t complain. COSC-3107 Machine Learning
– You conclude that I don’t care and it won’t have any
consequences.
– BUT you cannot be sure ….

17 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Deduction vs. Induction
• Here comes now our second, more abstract
description of what machine learning is:

MACHINE LEARNING TRIES TO AUTOMATE

THE PROCESS OF INDUCTIVE INFERENCE.
COSC-3107 Machine Learning

18 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
What is Machine Learning?
• A Computer Program is said to learn from
experience E with respect to some class
of tasks T and performance measure P, if
its performance at tasks in T, as
measured by P, improves with
experience E.
COSC-3107 Machine Learning

19 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Defining the Task, T (Data)
• The task, T, which can be defined in a two-fold
approach.
• From a Problem Standpoint, the task, T, is
basically the real-world problem to be solved at
hand, which could be anything from finding the best
marketing or product mix to predicting infrastructure
failures.
COSC-3107 Machine Learning
• In the Machine Learning world, it is best if you can
define the task as concretely as possible such that
you talk about what the exact problem is which you
are planning to solve and how you could define or
formulate the problem into a specific Machine
Learning task.

20 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Defining the Task, T-Examples
• Classification or categorization: This typically encompasses the
list of problems or tasks where the machine has to take in data
points or samples and assign a specific class or category to each
sample. A simple example would be classifying animal images into
dogs, cats, and zebras.
• Regression: These types of tasks usually involve performing a
prediction such that a real numerical value is the output instead
of a class or category for an input data point.
– The best way to understand a regression task would be to take the case of a
real-world problem of predicting housing prices considering the plot area,
number of floors, bathrooms, bedrooms, and kitchen as input attributes for COSC-3107 Machine Learning
each data point.

• Anomaly detection: These tasks involve the machine going over

event logs, transaction logs, and other data points such that it
can find anomalous or unusual patterns or events that are
different from the normal behavior.
– Examples for this include trying to find denial of service attacks from
logs, indications of fraud, and so on.

21 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Defining the Experience, E (Training)
• At this point, you know that any learning
algorithm typically needs data to learn over
time and perform a specific task, which we
named as T.
• The process of consuming a dataset that consists
of data samples or data points such that a
learning algorithm or model learns inherent
patterns is defined as the experience, E which is COSC-3107 Machine Learning

gained by the learning algorithm.

• The idea of a model or algorithm gaining
experience usually occurs as an iterative
process, also known as training the model.

22 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Defining the Performance, P (Testing / Validation)
• The performance, P, is usually a quantitative measure
or metric that is used to see how well the algorithm or
model is performing the task, T, with experience, E.
• While performance metrics are usually standard metrics
that have been established after years of research and
development, each metric is usually computed specific
to the task, T, which we are trying to solve at any given
point of time.
• Performance measures include accuracy, precision, COSC-3107 Machine Learning
recall, F1 score, sensitivity, specificity, error rate,
misclassification rate, and many more.
• Performance measures are usually evaluated on training
data samples (used by the algorithm to gain experience, E)
as well as data samples which it has not seen or
learned from before, which are usually known as
validation and test data samples.
23 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
3. Machine Learning Pipeline
COSC-3107 Machine Learning
The Machine Learning Pipeline

Data ML Method Intelligence

COSC-3107 Machine Learning

25 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
4. Machine Learning Case Studies
COSC-3107 Machine Learning
Case Study 1: Predicting House Prices

Data ML Method Intelligence

$ = ??
price ($)

COSC-3107 Machine Learning

$ $
$
+ house features house size

27 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Case Study 2: Sentiment Analysis

Data ML Sentiment Intelligence

Analysis
Sushi was awesome,
the food was awesome, but
the service was awful. Score(x) < 0
Allreviews: COSC-3107 Machine Learning
“awful”

Score(x) > 0

“awesome”

28 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Case Study 3: Document retrieval

Data ML Clustering Intelligence

Methods

COSC-3107 Machine Learning

SPORTS WORLD NEWS

ENTERTAINMENT SPACE
29 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Case Study 4: Product Recommended

Data ML Methods Intelligence

Your past Recommended

purchases: items:
Customers

COSC-3107 Machine Learning

+ purchase Products
histories of
all customers
30 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Case Study 4: Product Recommended

Data ML Methods Intelligence

Your past Customers Recommended

purchases: features items: COSC-3107 Machine Learning
features

features

Products
+ purchase features

histories of features

all customers features

31 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Case Study 5: Product Recommended

Data ML Methods Intelligence

Deep Learning

Input images: Nearest neighbors:

Layer 1 Layer 2

1 1 COSC-3107 Machine Learning

x1 z1 y

x2 z
2

32 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology
Today’s Lecture Summary
1. Course Evaluation Detail

2. What are AI, ML and DL?

3. Introduction to Machine Learning

COSC-3107 Machine Learning
4. Machine Learning Case Studies

5. Machine Learning Course Journey!

6. Machine Learning Prerequisites

33 Shahzad Hussain, Lecturer, Khawaja Fareed University of Engineering and Information Technology