MACHINE

LEARNING

1
 Introduction to
MLlearning studies computer algorithms
• Machine for
learning to do things ( such as learning to complete a
task, or to make accurate predictions or to behave
intelligently).
• The learning that is being done is always based on
some sort of observations or data, such as examples
(the most common case in this course), direct
experience, or instruction.
• So in general, machine learning is about learning to do
better in the future based on what was experienced in
the past

2
 Machine Learning definition
• Arthur Samuel (1959). Machine Learning: Field of
study that gives computers the ability to learn
without being explicitly programmed.
• Tom Mitchell (1998) Well-posed Learning Problem: A
computer program is said to learn from experience E
with respect to some task T and some performance
measure P, if its performance on T, as measured by P,
improves with experience E.
3
 Machine Learning definition…

Herbert Simon
“Learning is any process by which a system improves
performance from experience.”
What is the task?
 Classification
 Problem solving / planning /
control
ALT:
Improve on task, T, with respect to
performance metric, P, based on
experience, E
4
 Introduction to ML..
Types of ML Problems

• Classification – The goal is to categorize objects into a fixed set of

categories

• Regression – Forming a model for predicting a real value

For example, how much will it rain tomorrow or how much a house
will sell for or the price of an item in the stock market or how much
the stock will go up

• Clustering – creating clusters from provided data for unsupervised

learning e.g. learning topics of a discussion in an online forum
5
 Introduction to
ML..
Examples of Classification Problems..

• Medical diagnosis: diagnose a patient as a sufferer or non-sufferer of some

disease

• Customer segmentation: predict, for instance, which customers will respond

to a particular promotion

• Fraud detection: identify credit card transactions (for instance) which may be
fraudulent in nature

• Weather prediction: predict, for instance, whether or not it will rain

tomorrow

6
 Introduction to
ML..
Application Areas of ML
 Database mining
• Large datasets from growth of automation/web.
• E.g., Web click data (click stream data), medical
records, biology, engineering
 Applications can’t program by hand.
• E.g., Autonomous helicopter, handwriting
recognition, most of Natural Language Processing
(NLP), Computer Vision.
 Self-customizing programs
• E.g., Amazon
 Understanding human learning (brain, real AI).
7
 Introduction to ML..
Typical Learning Problem New Example

Labeled Training
Prediction
Examples
ML Algorithm Rule
(Training set)
(hypothesis)

Predicted
Classification

Big Q: How do we represent H?

8
 Introduction to ML..
Typical Learning Problem-Illustration
Example: Create a Spam Filter
• Gather a large collection of example email messages which are
labelled spam or non-spam
• Feed these examples to a general purpose algorithm
• The algorithm produces a prediction rule (which is hopefully) capable
of classifying new email messages
• Then feed a new unseen example to the prediction rule
• You obtain the predicted classification for the new example

9
What is the Learning
Problem?
Example: Visual Object Categorization

 Images are given as examples, determine classes of given images

 A classification problem: predict category y based on image x.
 Little chance to “hand-craft” a solution, without learning.
 Applications: robotics, HCI, web search (a real image Google...)

10
LEARNING SYSTEMS/TYPES

11
ROTE LEARNING
• Rote learning is the memorization of
information based on repetition. The two
biggest examples of rote learning are the
alphabet and numbers. Slightly more
complicated examples include multiplication
tables and spelling words.
• In Rote learning, the idea is that one will be
able to quickly recall the meaning of the
material the more one repeats it.
12
Rote Learning
• At the high school level, the elements and their
chemical numbers must be memorized by rote.
• Is rote learning an outdated technique or is there a
valid place for its use in the classroom today?
Increasingly, rote learning is being abandoned for
newer techniques such as associative learning,
meta cognition, and critical thinking instead of
being used as a functional foundation to higher
levels of learning.

13
SUPERVISED/INDUCTIVE
LEARNING

14
 Types of Learning Tasks
Supervised Learning…
In supervised algorithms, the classes are predetermined
i.e. Examples have labels.
ML Task that infers a function from labeled training data.
Each example consists of an input vector and a desired
output value (class).
The derived function is then used to classify new
examples.
The learning algorithm is expected to generalize training
data to unseen situations (examples).

15
 Types of Learning Tasks
Supervised Learning…
Q..
Commonly supervised problems are either Classification or Regression
problems.
Differentiate between the two

Q. 1. You have a large inventory of identical items. You want to predict

how many of these items will sell over next 3 months.
2. You want to examine a set of customer accounts and decide whether
they have been hacked or not.
What are the type of the above learning problems?

16
 Types of Learning
Tasks
Supervised Learning Steps
• Data Ccollection - Start with training data for which we know the
correct outcome provided by a teacher or oracle.
• Representation - Choose how to represent the data.
• Modeling - Choose a class of hypotheses - a set of possible
explanations for the connection between images and categories. This
is our model of the problem.
• Estimation - Find best hypothesis you can in the chosen class. There
can be none!!!
• Model Selection - We may reconsider the class of hypotheses given
the outcome.
N/B: Each of these steps can make or break the learning outcome.

17
 UNSUPERVISED/DISCOVERY
LEARNING
Unsupervised Learning
 The Problem of trying to find hidden structure in unlabeled data.
 With unlabeled examples, there is no error or reward signal to evaluate
performance of potential solutions.
 Approaches include clustering (news.google.com-grouped links, same
story), density estimation etc.
 No specific/ explicit target outputs (class labels) and given a set of data,
the task is to establish the existence of classes or clusters in the data
 Example Applications: Market segmentation, SNA, Astronomical Data
Analysis, Organization of Computing Clusters etc.
 Introduce:: The Cocktail Party Problem
18
 Types of Learning Tasks
Unsupervised Learning

19
 Learning Tasks
What type of learning algorithms are appropriate for these problems?
 Given email labelled as Spam/not Spam, learn a Spam filter.
 Given a set of new articles found on the web, group them into set of
articles about the same story.
 Given a database of customer data, automatically discover market
segments and group customers into different market segments.
 Given a dataset of patients diagnosed as having diabetes or not, learn to
classify new patients as having diabetes or not.

20
LEARNING FROM EXPERIENCE
• Machine Learning (Mitchell 1997)
• Learn from past experiences
• Improve the performances of intelligent
programs
• Definitions (Mitchell 1997)
• A computer program is said to learn from
experience E with respect to some class of
tasks T and performance measure P, if its
performance at the tasks improves with
the experiences

21
Example 1: Text Classification

Classified text files

Text file 1 trade
Text file 2 ship
… …

Training

New text file Text classifier class

22
Example 2: Disease Diagnosis

Database of medical records

Patient 1’s data Absence
Patient 2’s data Presence
… …

Training

New patient’s Presence or

Disease classifier
data absence

23
LEARNING BY ANALOGY
• Reasoning by analogy generally involves abstracting
details from a particular set of problems and
resolving structural similarities between previously
distinct problems.
• Analogical reasoning refers to this process of
recognition and then applying the solution from the
known problem to the new problem. Such a
technique is often identified as case-based
reasoning.
• Analogical learning generally involves developing a
set of mappings between features of two instances.

24
ML ALGORITHMS
• Artificial Neural Network
• Naïve Bayes
• Support Vector Machines
• K-Nearest Neighbor
• Instance based Learning tech
• Maximum Entropy

25
LEARNING BY ANALOGY
• Learning by Analogy – This is where the system maps
the knowledge it has to the new problem, using
analogy.
Read more on this

• Go to Neural Networks (next)

26