UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,

Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

A Decision Tree is a supervised machine learning model used for both classification and regression tasks. It
resembles a flowchart-like tree structure where:

 Each internal node represents a decision on a feature.

 Each branch represents the outcome of the decision.

 Each leaf node represents a class label (in classification) or a numerical value (in regression).

It splits the dataset into subsets based on the value of input features, making decisions step-by-step until a prediction
is made.

Simple Decision Tree Example

Suppose we want to classify whether a person buys a computer or not based on their Age and Income.

Example Dataset

Age Income Buys Computer

<30 High No

<30 Medium Yes

31-40 High Yes

>40 Low Yes

>40 Medium Yes

>40 High No

Example Decision Tree

[Age]

/ | \

<30 31-40 >40

/ \ \

[Income] Yes [Income]

/ \ / \

High Medium Low High

No Yes Yes No

🧠 How Classification Works in a Decision Tree

1. Select the best feature to split the data (based on impurity measures like Information Gain, Gini Index).

2. Split the data into subsets using that feature.

UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

3. Repeat the process recursively for each branch until:

o All records in a node belong to the same class.

o No more features remain.

4. Prediction: For a new instance, start at the root and follow the path based on the feature values until a leaf
node is reached → this is the predicted class.

🔍 Key Concepts in Decision Tree Classification

Term Description

Impurity How mixed the classes are in a node (lower is better).

Information Gain Reduction in impurity from a split.

Overfitting Deep trees may memorize training data. Use pruning to reduce it.

Greedy Search Trees are typically built using greedy top-down methods (e.g., ID3, C4.5, CART).

📘 Decision Trees for Classification – Summary

This section explains how decision trees are used to classify data by recursively splitting the dataset into subsets
based on features, aiming to increase the purity (i.e., reduce the mix of classes) at each node.

🔹 Key Concepts:

1. Root Node and Children:

o The tree starts at a root node and branches into child nodes based on feature values.

o Splitting continues until the nodes are pure or other stopping criteria are met.

2. Impurity and Entropy:

o The quality of a split is measured by how pure the resulting subsets are.

o The most common impurity measure used is Shannon entropy.

Entropy Formula:

The entropy for a node is given by:

Where:

 C = Number of classes

 Pi = Proportion of samples belonging to class i

UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

Entropy measures the uncertainty or disorder. It is zero when the node is pure (only one class present) and
maximum when classes are equally mixed.

🔹 Example in Text:

 For a 2-class problem (C=2), the entropy is computed using the probabilities p1p_1p1 and p2p_2p2.

 If one class dominates (e.g., all values are of class 1), entropy = 0.

Entropy

 The goal of the tree is to minimize entropy at each split.

 A lower entropy means a better (more informative) split.

 The algorithm chooses the feature that results in the maximum Information Gain, which is the reduction in
entropy.

Table 3.1: Probability Distributions and Entropy Values

This table shows 5 different cases of class distributions for a two-class classification problem (Class 1 and Class 2),
and their corresponding entropy values.

Case Class 1 (p₁) Class 2 (p₂) Entropy

1 1.00 0.00 0.000

2 0.75 0.25 0.811

3 0.50 0.50 1.000

4 0.25 0.75 0.811

5 0.00 1.00 0.000

Interpretation:

 Case 1 & 5 (Pure Node): Entropy is 0 when all data belongs to a single class.

 Case 3 (Equal distribution): Entropy is maximum (1) when the node is perfectly impure (50%-50%).

 Cases 2 & 4: Entropy is between 0 and 1, indicating partial impurity.

UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

Entropy(p1,p2)=−p1log2(p1)−p2log2(p2)

:
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

Case p1p_1p1 p2p_2p2 Entropy Comment

1 1.00 0.00 0.000 Pure class (Class 1 only)

2 0.75 0.25 0.811 Mixed, but mostly Class 1

3 0.50 0.50 1.000 Max uncertainty (equal split)

4 0.25 0.75 0.811 Mixed, but mostly Class 2

5 0.00 1.00 0.000 Pure class (Class 2 only)

UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

1. Splitting Rules

The splitting rule is the mechanism that determines how to divide the data at each node of the tree. The most
common methods for splitting data in decision trees include:

 Binary Splits (for classification):

o Each decision node typically splits the data into two branches based on a feature value or a
threshold. This is the most common type of split used in decision tree classifiers, where the data is
divided into two groups at each node based on a specific condition.

 Non-Binary Splits (for regression or multiclass classification):

UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

o Non-binary splits allow more than two partitions at a decision node, which can be useful for
handling multiclass classification or regression problems where there may be more complex
relationships to capture.

2. Criterion for Splitting

Decision trees use a splitting criterion to evaluate how well a feature divides the data at each node. The most
common splitting criteria are:

 Gini Impurity (used in CART):

o Measures how "impure" the node is. It is used to select the best feature and threshold that
minimizes the impurity. The formula for Gini impurity for a node ttt is:

Gini(t)=1−∑i=1Cpi2Gini(t) = 1 - \sum_{i=1}^{C} p_i^2Gini(t)=1−i=1∑Cpi2

where pip_ipi is the probability of class iii at node ttt.

 Entropy (Information Gain) (used in ID3, C4.5):

o Entropy measures the randomness or disorder in the node. It is used in decision tree algorithms like
ID3 and C4.5. The feature that maximizes the Information Gain (reduction in entropy) is selected for
splitting.

 Chi-Square (for categorical data):

o The Chi-Square test can be used to evaluate the independence between the feature and the class
labels. A feature with the highest Chi-Square statistic is chosen to split the data.

 Variance Reduction (for regression trees):

o When dealing with regression, variance reduction is used to split the data. The goal is to minimize
the variance within each child node.

3. Termination Condition

A decision tree classifier has termination conditions that define when the tree should stop growing. These conditions
prevent the tree from becoming too large and overfitting the data. Common termination conditions include:

 Maximum Depth:

o The tree stops growing once it reaches a specified depth (number of levels from the root to the leaf).

 Minimum Samples per Leaf:

o The tree stops splitting when a node has fewer than a predefined minimum number of samples (data
points) in the leaf node.

 Minimum Impurity Decrease:

o The tree stops growing if the decrease in the impurity (Gini or entropy) from a split is below a
threshold.

 No Further Improvement:
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

o If no further improvement can be made by splitting the data, the tree stops.

4. Class Labels

In decision tree classifiers, each leaf node corresponds to a class label. The label is assigned based on the majority
class in that leaf node. The class label for a leaf node is determined as follows:

 The class that occurs most frequently among the training samples in the node is assigned as the label.

 If there is a tie between multiple classes, various tie-breaking strategies (e.g., choosing the most frequent
class in the training data) can be used.

5. Classification Transparency

 Transparency (Interpretability):

o Decision trees are highly interpretable models, making it easy to understand how decisions are
made.

o The tree structure can be visualized, showing how data is split at each node based on the feature
values.

o This makes decision trees useful in scenarios where model explainability is important, such as in
healthcare or finance.

6. Handling Mixed Data Types

Decision tree classifiers can handle both numerical and categorical data types, which makes them very versatile.

 Numerical Data:

o For continuous features, decision trees can split based on inequalities (e.g., X1≤5X_1 \leq 5X1≤5).

 Categorical Data:

′X_2 = 'red'X2=′red′) or on a grouping of categories (e.g., X2∈{red,blue}X_2 \in \{\text{red}, \

o For categorical features, decision trees can split based on the equality of feature values (e.g., X2= ′red

text{blue}\}X2∈{red,blue}).

This flexibility allows decision trees to work well with datasets that contain a mix of numerical and categorical
features.

7. Eliminating Irrelevant Features

One of the benefits of decision trees is that they naturally select relevant features during the tree construction
process. Features that don't contribute much to reducing the impurity or improving the classification are less likely to
be chosen for splitting at higher levels of the tree.

However, irrelevant features can still be introduced into the tree if they are included in the dataset. In practice,
feature selection methods (such as recursive feature elimination) can be used to remove irrelevant or redundant
features before training the model.
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

8. Pruning a Decision Tree

Pruning is the process of removing branches from a fully grown tree to prevent overfitting. There are two main types
of pruning:

 Pre-pruning (Early Stopping):

o The tree is stopped from growing once certain conditions (such as maximum depth or minimum
samples per leaf) are met.

 Post-pruning (Cost-Complexity Pruning):

o After the tree is fully grown, branches that add little predictive value (i.e., those that lead to
overfitting) are removed. The process involves evaluating the performance of the tree on a validation
set and pruning nodes that degrade the model's performance.

Pruning helps to:

 Reduce overfitting.

 Improve generalization to unseen data.

The Bias-Variance Trade-off is a fundamental concept in machine learning and statistics that helps to explain the
sources of error in a model and how they influence its performance. It addresses the balance between two types of
errors that can occur when a model learns from data:

1. Bias:

Bias refers to the error introduced by approximating a real-world problem with a simplified model. In other words,
bias is the difference between the expected prediction of the model and the true values.

 High Bias: When the model is too simple, it makes strong assumptions about the data and doesn't capture
the underlying patterns well. This is known as underfitting. The model may miss important relationships and
produce consistent errors, leading to poor performance on both the training and test datasets.

 Low Bias: When the model is more complex, it can capture the subtle patterns in the data and make more
accurate predictions, reducing the error between the predicted and true values.

2. Variance:

Variance refers to the model's sensitivity to small fluctuations or changes in the training data. It measures how much
the model's predictions change when it is trained on different subsets of the data.

 High Variance: A model with high variance is overfitting the training data, meaning it performs well on the
training set but poorly on new, unseen data. This occurs when the model is too complex and captures not
only the true patterns but also the noise and random fluctuations in the data.

 Low Variance: A model with low variance has more stability across different training sets, meaning it
generalizes better to new data. However, if the variance is too low, it might be too simple and fail to capture
the complexities of the data (high bias).
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

The Trade-off:

The key idea behind the Bias-Variance Trade-off is that as you reduce bias, you increase variance, and as you reduce
variance, you increase bias. This trade-off exists because:

 Complex models (like deep neural networks or decision trees with many levels) tend to have low bias but
high variance, because they can learn intricate details from the training data but may also learn noise and fail
to generalize.

 Simpler models (like linear regression) tend to have high bias but low variance, because they make fewer
assumptions and don't overfit the data, but they might miss important patterns.

Visualizing the Trade-off:

Imagine you are fitting a curve to data:

 If you use a very simple model (e.g., a straight line for a complex dataset), the bias is high because the model
doesn't capture the complexity of the data. The model may consistently make large errors in prediction.

 If you use a very complex model (e.g., a very deep decision tree or a polynomial curve), the variance is high
because the model fits the noise in the training data. This leads to overfitting, where the model performs
poorly on new data.

The goal is to find the sweet spot—a model that balances bias and variance to minimize the total error.

Formula for Total Error:

The total error of a model can be expressed as:

Total Error=Bias2+Variance+Irreducible Error\text{Total Error} = \text{Bias}^2 + \text{Variance} + \text{Irreducible

Error}Total Error=Bias2+Variance+Irreducible Error

 Irreducible Error: This is the noise in the data that cannot be reduced by any model. It's the error inherent in
the data itself and is often due to factors that can't be modeled or predicted.

Examples of Bias and Variance in Different Models:

 High Bias, Low Variance (Underfitting):

o Model: Linear regression on complex data

o Problem: The model makes strong assumptions about the data (linearity) and cannot capture
complex patterns.

o Outcome: It performs poorly on both the training and test sets.

 Low Bias, High Variance (Overfitting):

o Model: Deep decision tree without pruning

o Problem: The tree learns all the details of the training data, including noise.

o Outcome: It performs well on the training set but poorly on the test set (because it overfits the
training data).
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

 Balanced Bias and Variance (Good Fit):

o Model: Random Forests (or any well-tuned ensemble method)

o Problem: A balance between bias and variance, where the model captures the essential patterns
without overfitting.

o Outcome: It performs well on both the training and test sets.

How to Manage the Bias-Variance Trade-off:

1. Choosing a Model:

o Start with a simple model (e.g., linear regression) and increase complexity (e.g., polynomial
regression, decision trees, neural networks) based on how well it generalizes to unseen data.

2. Cross-Validation:

o Use techniques like k-fold cross-validation to assess how well the model generalizes and helps to
identify overfitting or underfitting.

3. Regularization:

o Regularization methods like L1 (Lasso) or L2 (Ridge) help control overfitting by penalizing overly
complex models. This reduces variance by adding a penalty term to the model's cost function.

4. Pruning:

o For decision trees, pruning removes branches that have little predictive value and can help reduce
variance.

5. Ensemble Methods:

o Ensemble methods like Random Forests or Boosting help reduce variance without sacrificing bias by
combining predictions from multiple models.

6. Hyperparameter Tuning:

o Carefully tuning the hyperparameters of the model (such as max depth of trees, or learning rate in
boosting algorithms) helps to find the right balance between bias and variance.

Random Forests for Classification and Regression

Random Forests is a powerful ensemble learning technique that can be used for both classification
and regression tasks. It is based on creating a forest of decision trees and combining their
predictions to make a final decision. The primary strength of Random Forest is its ability to reduce
overfitting compared to a single decision tree and improve accuracy.

 Ensemble Learning: Random Forest is an ensemble method that uses multiple decision trees to
make predictions. The main idea is to combine the predictions from multiple models to reduce
variance (in the case of regression) or bias (in the case of classification).
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

 Bootstrapping (Bagging): Each tree in the forest is trained on a random subset of the training
data, selected with replacement. This technique is called bagging (Bootstrap Aggregating), which
helps reduce variance and overfitting.
 Feature Randomness: When constructing each tree, random subsets of features are considered
at each split (node). This helps in decorrelating the trees, making them more diverse and improving
the ensemble's performance.
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

Introduction to the Bayes Classifier

The Bayes Classifier is a probabilistic classifier based on Bayes' Theorem, which uses Bayes' Rule to make predictions
about the class of a given data point. The Bayes classifier is often used in situations where we want to classify an
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)

input into one of several categories based on prior knowledge of the distribution of data and the likelihood of each
class.

1. Bayes’ Theorem and Bayes’ Rule

Bayes' Theorem describes the probability of an event, based on prior knowledge of conditions that might be related
to the event. In classification, Bayes' Theorem provides a way to update the probability estimate for a class given new
evidence (i.e., new data).

Bayes' Rule is formally expressed as:

Where:

 P(C∣X)P(C | X)P(C∣X) is the posterior probability: the probability of class CCC given the features XXX.

 P(X∣C)P(X | C)P(X∣C) is the likelihood: the probability of observing the features XXX given that class CCC is
true.

 P(C)P(C)P(C) is the prior probability: the initial belief or probability of class CCC before seeing the data.

 P(X)P(X)P(X) is the evidence or marginal likelihood: the probability of observing the features XXX across all
classes.

Bayes' Rule allows us to compute the probability of a class CCC, given the data XXX, by incorporating:

1. The prior knowledge about the distribution of the classes.

2. The likelihood of the data given a class.

2. Naive Bayes Classifier

The Naive Bayes Classifier is a simple but powerful classification algorithm that applies Bayes’ Theorem with the
naive assumption that the features (attributes) are conditionally independent given the class label. In other words,
the Naive Bayes classifier assumes that each feature contributes independently to the probability of the class, which
simplifies the computation significantly.

For a given data point with features X=(x1,x2,…,xn)X = (x_1, x_2, \dots, x_n)X=(x1,x2,…,xn), the Naive Bayes classifier
predicts the class CCC that maximizes the posterior probability P(C∣X)P(C | X)P(C∣X):
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)
UNIT-III: Models Based on Decision Trees: Decision Trees for Classification, Impurity Measures,
Properties, Regression Based on Decision Trees, Bias–Variance Trade-off, Random Forests for Classification
and Regression.
The Bayes Classifier: Introduction to the Bayes Classifier, Bayes’ Rule and Inference, The Bayes Classifier
and its Optimality, Multi-Class Classification | Class Conditional Independence and Naive Bayes Classifier
(NBC)