This document provides an overview of machine learning concepts including:
1) It defines machine learning as algorithms that improve performance on tasks based on experience.
2) Common machine learning tasks are classification, regression, transcription, and machine translation.
3) Experience refers to datasets containing many examples that provide data for the machine learning system.
4) Performance is measured quantitatively depending on the specific task. Accuracy and error rate are common metrics.
5) Supervised learning uses labeled data to predict outputs, while unsupervised learning derives structure from unlabeled data. Applications include clustering, dimensionality reduction, classification, and regression.
This document provides an overview of machine learning concepts including:
1) It defines machine learning as algorithms that improve performance on tasks based on experience.
2) Common machine learning tasks are classification, regression, transcription, and machine translation.
3) Experience refers to datasets containing many examples that provide data for the machine learning system.
4) Performance is measured quantitatively depending on the specific task. Accuracy and error rate are common metrics.
5) Supervised learning uses labeled data to predict outputs, while unsupervised learning derives structure from unlabeled data. Applications include clustering, dimensionality reduction, classification, and regression.
This document provides an overview of machine learning concepts including:
1) It defines machine learning as algorithms that improve performance on tasks based on experience.
2) Common machine learning tasks are classification, regression, transcription, and machine translation.
3) Experience refers to datasets containing many examples that provide data for the machine learning system.
4) Performance is measured quantitatively depending on the specific task. Accuracy and error rate are common metrics.
5) Supervised learning uses labeled data to predict outputs, while unsupervised learning derives structure from unlabeled data. Applications include clustering, dimensionality reduction, classification, and regression.
This document provides an overview of machine learning concepts including:
1) It defines machine learning as algorithms that improve performance on tasks based on experience.
2) Common machine learning tasks are classification, regression, transcription, and machine translation.
3) Experience refers to datasets containing many examples that provide data for the machine learning system.
4) Performance is measured quantitatively depending on the specific task. Accuracy and error rate are common metrics.
5) Supervised learning uses labeled data to predict outputs, while unsupervised learning derives structure from unlabeled data. Applications include clustering, dimensionality reduction, classification, and regression.
Cairo University a.korani@fci-cu.edu.eg Course Overview
▪ Course Description ▪ Assessment Guidelines and Rules
▪ Keep your Mobile phones silent please.
▪ Alert me if I went faster than you could
capture the content.
▪ Additional bonus for impressive answers
Course Overview
▪ Introduction to machine learning
▪ Linear regression ▪ Logistic regression ▪ Naive Bayes ▪ KNN ▪ Decision Tree ▪ Support vector machines ▪ Neural Networks: ▪ Clustering REFERENCES • T. Mitchell, Machine Learning, McGraw-Hill
• Peter Flach, Machine Learning. The Art and
Science of Algorithms that Make Sense of Data.
• John D. Kelleher, Fundamental of Machine
Learning for predictive Data Analytic. Grading
Type grades ▪ 3 Assignments (regression, SVM, DT) 16 ▪ Mid term Exam 12 ▪ project 12 ▪ Final exam 60 Total 100 What is learning?
▪ ”The activity or process of gaining knowledge or skill by studying, practicing, being
taught, or experiencing something.” Merriam Webster dictionary ▪ “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” Tom Mitchell ▪ “Learning is any process by which a system improves performance from experience.” Herbert Simon
⮚ In other words ▪ instead of the programmer writing explicit rules for how to solve a given problem, the programmer instructs the computer how to learn from examples ▪ in many cases the computer program can even become better at the task than the programmer is! What is machine learning? ⮚Definition:
Machine = computer, computer program (in this course)
Learning = improving performance on a given task, based on experience / examples
Tom Mitchell: Algorithms that:
• improve their performance P • at task T • with experience E
A well-defined machine learning task is given by (P,E, T)
What is machine learning? Traditional Programming
Data Output Program
Machine Learning
Data Program Output EXAMPLES Example: Game Playing
T = playing Checkers P = win rate against opponents E = playing games against itself Example 1 : Tic-Tac-Toe Example 2 : face recognition Example 3: Prediction of search queries Example 4: Ranking search results Example 5 Example 6 Example 7 Problem setup Consider the case of Game playing, According to ML definition, it is a computer program that improve its performance on a given task with experience : T = playing Checkers P = win rate against opponents E = playing games against itself
So we need to separate
Task Experience Performance
1- Task
• Machine learning allows us to tackle tasks that
are too difficult to solve with fixed programs written and designed by human beings.
• Machine learning is interesting because
developing our understanding of machine learning entails developing our understanding of the principles that underlie intelligence. 1- Task (Cont.) Some of the most common machine learning tasks include the following:
1- Classification : In his type of task, the computer program is asked to specify
which of k categories some input belongs to. -Classification Algorithms attempt to map inputs into one of a set of classes (Colors, Good and Bad Credit Risks)
2- Regression : In this type of task, the computer program is asked to predict a
numerical value given some input. -Regression Algorithms attempt to mapinputs into continuous output (Integers,Real Numbers, Vectors, etc.) 1- Task (Cont.) Some of the most common machine learning tasks include the following: 3- Transcription: In this type of task, the machine learning system is asked to observe a relatively unstructured representation of some kind of data and transcribe it into discrete, textual form.
E.g.: optical character recognition (OCR), where the computer program is shown a photograph containing an image of text and is asked to return this text in the form of a sequence of characters.
4- Machine translation: In a machine translation task, the input already consists
of a sequence of symbols in some language, and the computer program must convert this into a sequence of symbols in another language. 2- Experience
• Experience is a dataset.
• A dataset is a collection of many examples.
• An example is a collection of features that have been
quantitatively measured from some object or event that we want the machine learning system to process. 3- performance level
In order to evaluate the abilities of a machine learning algorithm, we must
design a quantitative measure of its performance. Usually this performance measure P is specific to the task T being carried out by the system.
- Accuracy : Accuracy is just the proportion of examples for which the model produces the correct output.
- Error Rate : the proportion of examples for which the model produces an incorrect output. WHEN DO WE USE MACHINE LEARNING?
ML is used when: ❑• Human expertise does not exist (navigating on Mars) ❑• Humans can’t explain their expertise (speech recognition) ❑• Models must be customized (personalized medicine ( ❑• Models are based on huge amounts of data (genomics) WHEN DO WE USE MACHINE LEARNING? Some more examples of tasks that are best solved by using a learning algorithm
• Recognizing patterns: – Facial identities or facial expressions – Handwritten or spoken words – Medical images • Generating patterns: – Generating images or motion sequences • Recognizing anomalies: – Unusual credit card transactions – Unusual patterns of sensor readings in a nuclear power plant • Prediction: – Future stock prices or currency exchange rates Slide credit: Geoffrey Hinton Neural network Supervised Learning
❑In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output. ❑Supervised learning problems are categorized into "regression" and "classification" problems. ❑In a regression problem, we are trying to predict results within a continuous output, meaning that we are trying to map input variables to some continuous function. ❑ In a classification problem, we are instead trying to predict results in a discrete output. In other words, we are trying to map input variables into discrete categories. SUPERVISED LEARNING PARADIGM
Learning entries
Training usually entries are encoded Text, Images as vector (x1,x2,…..,xn) Omic data, Feature Etc… vectors Human expert labels each learning entries
Machine Labels learning algorithm
Feature Testing vector Text, Images Expected Predictiv Omic data, e Model Label Etc… Housing price prediction
250
175
750
Supervised Learning Regression:
“right answers” or “Labeled data” Predict continuous valued output
given (price) UNSUPERVISED LEARNING
❑Unsupervised learning, on the other hand, allows us to approach problems with
little or no idea what our results should look like. ❑We can derive structure from data where we don't necessarily know the effect of the variables. ❑We can derive this structure by clustering the data based on relationships among the variables in the data. ❑With unsupervised learning there is no feedback based on the prediction results, i.e., there is no teacher to correct you. SUPERVISED
X X X X o Y o o o X XX oo o X Unsupervised
o o oo ooo Y o oo oo o o X Unsupervised learning is where you only have input data and no corresponding output variables. Unsupervised Learning applications The expression of set of genes for a Gen certain individual es
Individual s We need to cluster the individuals based of the similarity of their genes.
[Source: Daphne Koller]
Unsupervised Learning applications
Gen es
Individuals Microarray data • Have a group of individuals • On each measure expression of a gene • Run algorithm to cluster individuals into types of people Unsupervised Learning applications UNSUPERVISED Learning Types
Leaning by interaction • Self driven car ML in Practice
▪ Understanding domain, prior knowledge, and goals
▪ Data integration, selection, cleaning, pre-processing, etc. ▪ Learning models ▪ Interpreting results ▪ Consolidating and deploying discovered knowledge ▪ Loop You’re running a company, and you want to develop learning algorithms to address each of two problems.
Problem 1: You have a large inventory of identical items. You want to predict how many of these items will sell over the next 3 months.
Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised.
Should you treat these as classification or as regression problems?
Treat both as classification problems.
Treat problem 1 as a classification problem, problem 2 as a regression problem.
Treat problem 1 as a regression problem, problem 2 as a classification problem.
Treat both as regression problems.
Of the following examples, which would you address using an unsupervised learning algorithm? (Check all that apply.)
Given email labeled as spam/not spam, learn a spam filter.
Given a set of news articles found on the web, group them
into set of articles about the same story.
Given a database of customer data, automatically discover
market segments and group customers into different market segments. Given a dataset of patients diagnosed as either having diabetes or not, learn to classify new patients as having diabetes or not.
