The document discusses machine learning and predictive modeling. It defines machine learning and describes the types of machine learning, including supervised learning techniques like regression and classification, and unsupervised learning techniques like clustering. It provides examples of implementing regression models, classification models, and clustering models. It also covers key considerations for developing machine learning models, such as visualizing data, addressing overfitting and underfitting, and the regularization process.
The document discusses machine learning and predictive modeling. It defines machine learning and describes the types of machine learning, including supervised learning techniques like regression and classification, and unsupervised learning techniques like clustering. It provides examples of implementing regression models, classification models, and clustering models. It also covers key considerations for developing machine learning models, such as visualizing data, addressing overfitting and underfitting, and the regularization process.
The document discusses machine learning and predictive modeling. It defines machine learning and describes the types of machine learning, including supervised learning techniques like regression and classification, and unsupervised learning techniques like clustering. It provides examples of implementing regression models, classification models, and clustering models. It also covers key considerations for developing machine learning models, such as visualizing data, addressing overfitting and underfitting, and the regularization process.
The document discusses machine learning and predictive modeling. It defines machine learning and describes the types of machine learning, including supervised learning techniques like regression and classification, and unsupervised learning techniques like clustering. It provides examples of implementing regression models, classification models, and clustering models. It also covers key considerations for developing machine learning models, such as visualizing data, addressing overfitting and underfitting, and the regularization process.
2. What is Machine Learning? 3. Type of Machine Learning Supervised Learning Regression Classification Unsupervised Learning Clustering 4. Implementation Regression Model Using Normal Equation Using Gradient Descent Classification Model Using Logistics Regression Using Naïve Bayesian Conditional Probability Clustering Using K Means
2 Introduction
Why Machine Learning?
My first thought: Is it because humans can’t learn any more??? Volume, Velocity and Variety of information drove the need for machine learning Big Data, Hadoop and analytic challenges It was easy for human to estimate the house price based on 3-4 features like location, size, age and interest rate Imagine when you have understand the factors( 30-40) that drives our user’s decision to buy an image Imagine when you have to detect a fraud attempt that has 100 – 300 features (attributes) and attempts are appearing at 100 messages/sec.
3 What is Machine Learning?
As defined in Wikipedia (Tom M. Mitchell) : A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E
E Successfully labeled xp observations
Train
Incoming new events, T ask
E Data set, xp Observation, % of correct (Experiences) classification
P erf
4 The Learning Model? with continuous knowledge update
Successfully Update Knowledge update
labeled observations E xp knowledge • Based on sample • Based on Filter • Real time/batch • Supervised update
Train Incoming new events, T ask
E Data set,
xp Observation, % of correct (Experiences) classification
P erf
X{x1,x2,…xn} f(x) y 5 Type of Machine Learning?
• Supervised Learning In supervised learning a supervisor provides a set of labeled data, also known as experience that is used to train the machine to build knowledge (i.e. a model/function)
(customer Age, Geo location, Profession,
social group,) Purchase Image A (Yes or No)
• Unsupervised Learning Find some structure or groups in the data. Also known as clustering technique Example :News groups, Cohesive groups in Facebook, customer groups
6 Examples of Supervised Learning where outcome is a continuous variable
• Outcome or yield (Y) is a continuous
variable, i.e. we are predicting numeric value Yah, this is a regression problem Example : Revenue from on-line sales for next year concurrent active user sessions
Here y is the outcome that depends on m features or
attributes X1, X2…...Xm
y f(x x ........ x ) 1 2 m
7 Examples of Supervised Learning where outcome is discrete variable
• Outcome or yield is a discrete variable
Got it , this is a Classification problem • Outcome has two classes Example : Here the outcome Y will have one of the two values : Yes or No, : True or False • Outcome has multiple classes Example : Classify email tone as Threat, Anger, Appreciate, Unhappy.. ; i.e. value of Y is either Threat or Anger or Appreciate
8 Regression Model, Visual Analysis On-line purchase of images per year (y)
4k (Dependent variable, Yield, result)
3k y 0 1 x1 y 0 x0 1 x1 2k x0 y ( 0 , 1 ) x1 1k Y T X
0 5 10 15 Passion for photography(x) (Feature, independent variable) 9 Classification Model, Visual Analysis Distance from decision boundary
200 Decision Purchase amount in $ (Feature)
Boundary 150
100
50
0 10 20 30 Time spend to search and make decision (In minutes) (Feature)
10 Clustering Model, Visual Analysis
e Type of Image purchased
20 30 40 50 60 60 70 Customer Age
11 Key considerations and concepts for implementation and development of Machine learning Models Visualize your data to identify features, understand and determine the model Carefully analyze Standard deviation or percentage of false positive or false negative Set users expectations on the probability of false conclusion, this is very important Keep a balance between Bias caused by under fitting and Variance cause by over fitting May decide to use Training, Validation and Test (60%, 20%,20%) steps Regularization process and selection of regularization parameter
12 A Few Machine Learning Techniques Normal Equation Regression Model
14 Linear Regression Model Introduction to Linear model
y f ( x) a bx f ( x) a b x f ( x) a b x b x 11 11 2 2 f ( x)b b x b x f ( x) b x b x b x 0 11 2 2 0 0 11 2 2 Where b a, x 1 0 0 f ( x)b x b x b x ........ b x 0 0 11 2 2 n n
Y is the outcome which depends on n features x1, x2,…xn,
where bi is the parameter of xi
15 Linear Regression Model Compact and Matrix representation of Linear model
n f ( x) β x β x β x ........ β x f ( x) x 0 0 1 1 2 2 n n i i i 0 f ( x)βTX β x 0 0 β x 1 1 β . , x . . . β n x n
16 Linear Regression Find beta values using Normal equation Solve analytically using well known Normal equation method
β (XTX)1 XT y Well know methodology Easy to implement Computationally less expensive for small to medium number of feature (<1000) No need to select Learning rate that is needed for Gradient Descent Pay attention on matrix inverse issue in case of linear dependency
17 Linear Regression Find beta values Using Gradient descent Cost function with m observations 1 m (i) (i) 2 C(β) (f (x ) y ) 2m i 1 β Let’s understand gradient descent technique using example of an ant who wants to reach the bottom of the hat δ 1 m (i) (i) (i) β :β α C( ) β :β α (f (x ) y ) x j j δβ j j m i 1 β j j Now iterate through the above formula to optimize cost function . Stop iteration when cost function does not change significantly between iterations J=1,2….n And α is the Learning Rate
18 Classification Model Solution using Logistic Regression
Classification Model Y, the outcome can get one of the two values; 0=negative (No), 1=positive (Yes) Email Spam, Email Threat, Fraudulent transaction Multi-class classification problem; Tone of the message The approach Compute the Decision Boundary based on training data and that boundary that you will use to classify as YES or NO depending on where (which side of the Decision Boundary) the observation falls Depending on the dimension (i.e. number of features) The Decision Boundary will be a line, a plane or a hyper plane (one degree less than the number of features)
19 Logistic Regression Model Distance from decision boundary Purchase amount in $ (Feature)
4 Decision Boundary 3
0 10 20 30 Time spend to search and make decision (In minutes) (Feature)
20 Logistic Regression Model f β (x) 1 T x 0 βT x 1 e f β (x) g(β T x) Predict Y=1 if g(z) ≥0.5, z ≥0 g(z) 1 Predict Y=0 if g(z) <0.5, z <0 1 e z T x 0 The Model 0<=f(x)<=1 The function known as Sigmoid function The Cost function become a convex function Decision boundary Now we need to optimize the cost function for Logistics regression for estimating beta using Gradient descent
21 Classification Model Solution using Bayesian Model • Based on conditional probability • It assumes features are statistically independent variable • We find the probability for a given observation to be classifies as class ci, for i=1,2…n • The observation will be classified as a type/category for which this conditional probability is maximum
p( x1, x2, ..., xn / ci ) p(ci )
p(ci /( x1, x2, ..., xn ) p( x1, x2, ..., xn ) p( x1, x2, ..., xn / ci ) p( x1 / ci ) p( x2 / ci ).......p( xn / ci ) p( x1 / ci ) p( x2 / ci ).......p( xn / ci ) p(ci ) p(ci /( x1, x2, ..., xn ) p( x1, x2, ..., xn ) 22 Clustering Model with K Means
