ASSIGNMENT

1. Perform Exploratory Data Analysis (EDA) on Iris dataset.

Data analysis utilising certain visual techniques is known as exploratory data analysis
(EDA). With this method, we may obtain comprehensive details on the statistical summary of
the data. Moreover, we will be able to handle duplicate values and outliers as well as identify
any trends or patterns in the collection.
The Iris Dataset is known as the "Hello World" of data science. It has five columns, including
Species Type, Sepal Length, Sepal Width, and Petal Length. Iris is a blooming plant, and the
researchers have measured and digitally documented the varied characteristics of the many iris
flowers.

Example:

import pandas as pd
data = pd.read_csv("/Downloads/Iris.csv")
data.head()

Output:
2. What is Decision Tree? Draw decision tree by taking the example of Play
Tennis.

A decision tree is a type of supervised learning algorithm used in machine learning and data
mining to predict outcomes by creating a tree-like model of decisions and their possible
consequences.
The decision tree is made up of nodes and branches. The nodes represent decisions or actions,
and the branches represent the possible outcomes of those decisions.

The decision tree for this example would look something like this:
Weather
/ \
Sunny Overcast/Rainy
/ / \
Humidity? Play Don't Play
/ \ / \
High Normal Don't Play Don't Play
/ \
Don't Play Play

3. In k-means or KNN, we use Euclidean distance to calculate the distance

between nearest neighbours. Why not Manhattan distance?

Euclidean distance and Manhattan distance are both popular distance metrics used in various
machine learning algorithms, including k-means and KNN. While Euclidean distance is
commonly used in these algorithms, Manhattan distance can also be used, depending on the
problem and the dataset.
The main difference between Euclidean and Manhattan distances is the way they calculate
distance. Euclidean distance is the straight-line distance between two points in a Euclidean
space,
The reason Euclidean distance is commonly used in k-means and KNN is that it tends to work
well for datasets with continuous, real-valued features.
On the other hand, Manhattan distance is often used in situations where the features are not
continuous, but rather represent discrete or categorical values.

4. How to test and know whether or not we have overfitting problem?

Overfitting occurs when a machine learning model learns the training data too well and
becomes too complex, resulting in poor generalization performance on new, unseen data. To
test and know whether or not we have an overfitting problem, we can use the following
methods:
Holdout method: We can split our data into training and testing sets, where the training set is
used to train the model, and the testing set is used to evaluate the performance of the model.
Cross-validation: Cross-validation is a technique for estimating the performance of a model by
splitting the data into multiple folds, training the model on each fold
Learning curves: Learning curves plot the performance of a model on the training and
validation sets as a function of the number of training examples.
Feature selection: Feature selection is the process of selecting a subset of features that are most
relevant to the problem.

5. How is KNN different from k-means clustering?

KNN (K-Nearest Neighbors) and k-means clustering are two different algorithms used in
machine learning for different purposes. Here are some differences between them:
Purpose: KNN is a supervised learning algorithm used for classification and regression tasks,
while k-means is an unsupervised learning algorithm used for clustering tasks.
Input: KNN takes labeled data as input, where each data point is associated with a class or
regression value. K-means takes unlabeled data as input and groups similar data points into
clusters.
Algorithm: KNN works by finding the k-nearest neighbors of a new data point based on some
distance metric, and then assigning a label or regression value based on the majority class or
average value of those neighbors. K-means works
Distance metric: KNN typically uses Euclidean distance or other distance metrics to calculate
the similarity between data points. K-means also uses Euclidean distance or other distance
metrics to measure the similarity between data points and centroids.
Model complexity: KNN can have high model complexity, especially when the dataset is large
or the number of features is high. K-means has lower model complexity and is more scalable
to larger datasets.

6. Can you explain the difference between a Test Set and a Validation Set?

we use a dataset to train a model and evaluate its performance. However, we need to make sure
that the model is not just memorizing the training data,
The training set is used to train the model. The model learns from the training set by adjusting
its parameters to minimize the training error.
Purpose: The validation set is used to tune the hyperparameters of the model, while the test set
is used to evaluate the performance of the final model.
Use: The validation set is used multiple times during the training process to choose the best
hyperparameters, while the test set is used only once at the end to evaluate the final model.
Size: The validation set is typically smaller than the training set, while the test set is of a similar
size to the training set.

7. How can you avoid overfitting in KNN?

KNN (K-Nearest Neighbours) is a simple but powerful supervised learning algorithm that
can be prone to overfitting when the value of k is too small. Overfitting occurs when the model
learns the noise in the training data instead of the underlying patterns
Increase the value of k: KNN works by finding the k-nearest neighbours of a new data point
and assigning a label based on the majority class or average value of those neighbours
Normalize the data: KNN is sensitive to the scale and distribution of the input features. If the
features have different scales or distributions.
Feature selection: KNN can be sensitive to irrelevant or redundant features. Including too many
features in the model can lead to overfitting
Regularization: Regularization is a technique that adds a penalty term to the objective function
of the model to encourage simpler models that are less prone to overfitting.
8. What is Precision?

Precision is a metric used in machine learning to measure the accuracy of a binary classifier.
It is defined as the ratio of true positives to the total number of predicted positives In other
words, precision measures the proportion of positive predictions that are true positives.
Precision is often used in situations where the cost of false positives (FP) is high, such as in
medical diagnosis or fraud detection. A high precision means that the classifier is making fewer
false positive predictions, which can help to reduce the cost of false alarms and increase the
confidence in the positive predictions.

9. Explain How a ROC Curve works.

A Receiver Operating Characteristic (ROC) curve is a graphical representation of the

performance of a binary classifier at different classification thresholds. It is a useful tool for
evaluating the trade-off between the true positive rate (TPR) and false positive rate (FPR) of a
classifier at various thresholds.
The ROC curve is particularly useful when the cost of false positives and false negatives are
different, or when the classifier's decision threshold can be adjusted to optimize a specific
performance metric.

10.What is Accuracy?

Accuracy is a metric used in machine learning to measure how well a model correctly predicts
the class labels of samples. It is defined as the ratio of the number of correctly classified
samples to the total number of samples in the dataset.
Mathematically, accuracy can be expressed as:
Accuracy = (number of correctly classified samples) / (total number of samples)
11.What is F1 Score?

The F1 score is a metric used in machine learning to evaluate the performance of a binary
classifier. It combines precision and recall into a single score that represents the harmonic mean
of the two measures. The F1 score is a way of balancing precision and recall, and is often used
in situations where both measures are equally important.
Mathematically, the F1 score can be expressed as:
F1 score = 2 * (precision * recall) / (precision + recall)

12.What is Recall?

Recall, also known as sensitivity or true positive rate, is a metric used in machine learning to
evaluate the performance of a binary classifier. It measures the proportion of actual positive
samples that are correctly identified by the classifier
Mathematically, recall can be expressed as:
Recall = true positives / (true positives + false negatives)
where true positives are the number of samples that are correctly classified as positive, and
false negatives are the number of samples that are incorrectly classified as negative.

13.What is a Confusion Matrix, and Why do we Need it?

A confusion matrix is a table used to evaluate the performance of a machine learning model
for a binary classification problem. It shows the number of true positives (TP), false positives,
true negatives , and false negatives for a set of predicted and actual class labels.
The confusion matrix is useful because it provides a more detailed view of the performance of
a model beyond simple accuracy.
14.What do you mean by AUC curve?

The AUC (Area Under the Curve) curve is a graphical representation of the performance
of a binary classifier at various classification thresholds. It is a way to visualize the relationship
between the classifier's true positive rate
The AUC curve is commonly used to compare the performance of different classifiers on the
same dataset, as it provides a single score that summarizes the overall performance of the
classifier

15.What is Precision-Recall Trade-Off?

The precision-recall trade-off is a common challenge in machine learning where increasing

the precision of a classifier typically results in a decrease in recall, and vice versa. In other
words, there is often a trade-off between precision and recall when designing a classifier.
Precision is the fraction of true positive results among all positive results, while recall is the
fraction of true positive results among all samples that should have been identified as positive.

16.What are Decision Trees?

Decision Trees are a type of machine learning algorithm used for both classification and
regression tasks. The algorithm builds a model in the form of a tree structure that represents a
set of decisions and their possible consequences.
17.Explain the structure of a Decision Tree

A Decision Tree is a hierarchical model consisting of nodes and branches. The structure of
a Decision Tree can be divided into three main types of nodes:
Root Node: The root node represents the starting point of the decision tree. It is the first node
that is evaluated and contains the entire dataset.
Internal Nodes: Internal nodes represent the test on a feature or attribute. Each internal node
contains a condition that splits the data into two or more subgroups based on the value of a
feature.
Leaf Nodes: Leaf nodes represent the outcome or decision of the decision tree. Each leaf node
contains a class label or a regression value.

18.What are some advantages of using Decision Trees?

There are several advantages of using Decision Trees

Easy to Understand and Interpret
Non-parametric
Feature Selection
Handling Missing Values
Ensemble Methods

19.How is a Random Forest related to Decision Trees?

Random Forest is an ensemble learning method that uses multiple Decision Trees to improve
the accuracy and robustness of the model. Random Forest combines the predictions of multiple
Decision Trees to obtain a final prediction. Each Decision Tree in the Random Forest casts a
vote, and the final prediction is the mode of the votes.
20.How are the different nodes of decision trees represented?

Root Node: The Root Node is the topmost node in the Decision Tree and represents the entire
dataset.
Decision Nodes: Decision Nodes represent the features used to split the data and are
represented as a square or a rectangle.
Leaf Nodes: Leaf Nodes represent the final outcome or decision, and are represented as circles.
Branches: Branches connect the nodes and represent the decision rules or paths in the tree.
Splitting Criteria: Splitting Criteria represent the conditions used to split the data at a Decision
Node

21.What type of node is considered Pure?

In Decision Trees, a node is considered pure if all the data points in that node belong to the
same class or category. there is no more room for further splitting, as all the data points in the
node have the same label or class.

22.How would you deal with an Overfitted Decision Tree?

Pruning: Pruning involves removing branches or nodes from the tree that do not contribute
significantly to the model's accuracy.
Early Stopping: Early Stopping involves stopping the tree-building process before it becomes
too complex or overfit.
Ensemble Learning: Ensemble learning methods such as Random Forest or Boosted Trees can
be used to combine multiple Decision Trees and reduce the overfitting problem.
Feature Selection: Feature selection involves selecting a subset of the most important features
for building the tree.
Cross-Validation: Cross-validation involves splitting the data into training and validation sets
and evaluating the model's performance on the validation set.

23.What are some disadvantages of using Decision Trees and how would you
solve them?

Overfitting: Decision Trees are prone to overfitting, which occurs when the model is too
complex and fits the training data too closely.
Instability: Decision Trees can be unstable and sensitive to small variations in the data, which
can result in different trees being built for different subsets of the data.
Bias: Decision Trees can be biased towards features with more levels or categories, which can
lead to suboptimal splits and lower accuracy.
Handling Missing Values: Decision Trees can struggle to handle missing values, which can
result in biased or incomplete models.
Scalability: Decision Trees can be computationally expensive and time-consuming to build,
especially for large datasets or complex models.

24.What is Gini Index and how is it used in Decision Trees?

Gini Index is a metric used in Decision Trees to measure the impurity or randomness of a
split. It measures the probability of misclassifying a randomly chosen sample from a given
node if it were labeled randomly according to the distribution of labels in that node.
the Gini Index is used to evaluate the quality of a split when selecting the best feature to split
the data at each node The Gini Index is commonly used in binary classification problems, where
there are only two possible outcomes, but it can also be used in multi-class problems
25.How would you define the Stopping Criteria for decision trees?

Stopping criteria for decision trees refer to the conditions that determine when the tree-
building process should stop and the tree should be considered complete. There are several
stopping criteria that can be used in decision trees, including:
Maximum depth
Minimum number of samples per split
Maximum number of leaf nodes.
Minimum improvement in impurity
Early stopping

26.What is Entropy?

Entropy is a measure of the impurity or randomness of a set of samples with respect to their
target variable. It is often used as a criterion to select the best feature to split the data at each
node in decision trees.

27.How do we measure the Information?

Information can be measured using the concept of entropy. Entropy is a measure of the
uncertainty or randomness of a system. In information theory, entropy is used to quantify the
amount of information contained in a message or signal.

28.What is the difference between Post-pruning and Pre-pruning?

Pre-pruning involves setting stopping criteria before the tree is built. This means that the tree-
building process will stop when certain conditions are met, such as reaching a maximum depth
or minimum number of samples at a node.
Post-pruning, on the other hand, involves growing the tree to its maximum depth or size and
then pruning back the branches that are found to be overfitting. This is done by removing nodes
or branches that do not improve the performance of the tree on a validation set.

29.Compare Linear Regression and Decision Trees

Linear regression is a type of supervised learning algorithm used for predicting a

continuous output variable based on one or more input variables. It assumes a linear
relationship between the input variables and the output variable. Linear regression models are
simple and easy to interpret, and can provide useful insights.

Decision trees, on the other hand, are a type of supervised learning algorithm used for both
classification and regression tasks. They involve recursively splitting the data based on the
values of the input variables, in order to create a tree-like model that can be used to make
predictions. Decision trees are more flexible than linear regression models and can capture non-
linear relationships between the variables.

30.What is the relationship between Information Gain and Information Gain

Ratio?

Information Gain (IG) is a measure of the reduction in entropy achieved by splitting the data
based on a particular feature. It measures how much information a feature provides about the
classification of the data

Information Gain Ratio (IGR) is a modification of IG that addresses a limitation of IG, which
is that it tends to favor features with many values (i.e., features with high cardinality). IGR
adjusts IG by taking into account the intrinsic information of the feature, which is defined as
the ratio of the number of distinct values of the feature to the total number of samples.
31.Compare Decision Trees and k-Nearest Neighbours

Learning Process: Decision trees are a supervised learning algorithm that builds a tree-like
model to represent the decision rules, while k-NN is a lazy learning algorithm that stores all
the training examples and classifies new instances based on their distance to the k-nearest
neighbours.

Interpretability: Decision trees produce a set of rules that can be easily understood and
visualized, whereas k-NN does not produce explicit rules and can be difficult to interpret.

Model Complexity: Decision trees can become overly complex and prone to overfitting if not
properly pruned, while k-NN does not have a model complexity issue.

Handling of Outliers: Decision trees can be sensitive to outliers and noise in the data, whereas
k-NN can be robust to outliers as it is based on the distances to the k-nearest neighbours.

Parameter Tuning: Decision trees require tuning of parameters such as the maximum depth of
the tree, while k-NN requires tuning of the number of neighbours k.

32.While building Decision Tree how do you choose which attribute to split
at each node?

When building a decision tree, we want to select the attribute that will result in the highest
information gain or lowest impurity at each node. The information gain is a measure of the
amount of uncertainty or entropy that is reduced by splitting on a particular attribute. The
attribute with the highest information gain is the one that provides the most information about
the target variable and is therefore the most informative for making a decision.

The process of selecting the best attribute to split on is known as attribute selection or feature
selection. There are several different methods for attribute selection, including:
Information Gain
Gain Ratio
Gini Index
Chi-Squared Test

33.How would you compare different Algorithms to build Decision Trees?

Data Split: Split the dataset into training and test sets or use cross-validation to avoid
overfitting.

Train and Evaluate Models: Train the decision tree models using different algorithms such as
ID3, C4.5, CART, or Random Forest.

Performance Comparison: Compare the performance of the models based on the evaluation
metrics.

Statistical Analysis: Conduct a statistical analysis to determine if the performance difference

between the models is statistically significant,

Best Model:, select the best model based on the evaluation metrics and statistical analysis.

34.How do you Gradient Boost decision trees?

Gradient Boosting is a popular ensemble learning technique that combines multiple weak
learning models to create a stronger predictive model. When it comes to decision trees, gradient
boosting involves building a sequence of decision trees where each new tree improves the
predictions made by the previous trees.

Build the first decision tree: Train the first decision tree using the dataset and calculate the
residuals (the difference between the actual values and the predicted values).

Build subsequent decision trees: In each subsequent iteration, train a new decision tree using
the residuals from the previous tree as the target variable. The residuals represent the errors
that the previous tree could not predict.

Combine decision trees: Combine all the decision trees to create a final prediction by adding
up the predictions from each individual decision tree.
Regularization: To prevent overfitting, add regularization to the algorithm, such as shrinkage
or subsampling.
Hyperparameter tuning: Tune the hyperparameters, such as the number of trees, learning rate,
and depth of the tree, to optimize the model's performance.
Gradient Boosting is a powerful algorithm for building decision trees that can achieve high
accuracy on a wide range of problems. However, it can be computationally expensive and
requires careful hyperparameter tuning to prevent overfitting.

35.What are the differences between Decision Trees and Neural Networks?
Decision Trees and Neural Networks are both popular machine learning models, but they differ
in several ways:

Model structure: Decision Trees are hierarchical models that represent decisions in a tree-like
structure, while Neural Networks are composed of layers of interconnected nodes that process
data through nonlinear transformations.

Model complexity: Decision Trees are simpler models that can be easily visualized and
interpreted, while Neural Networks are more complex models that require more computational
resources and can be harder to interpret.

Input data: Decision Trees are suited for datasets with categorical and numerical features, while
Neural Networks are more suited for datasets with high-dimensional and continuous features.

Training process: Decision Trees use a divide-and-conquer approach to recursively split the
data based on the most informative features, while Neural Networks use gradient-based
optimization to iteratively update the model parameters to minimize the loss function.

Generalization: Decision Trees can be prone to overfitting, especially when the tree is deep,
while Neural Networks are more robust to overfitting, especially when regularization
techniques are used.

Both Decision Trees and Neural Networks have their strengths and weaknesses, and the choice
between them depends on the nature of the problem, the available data, and the desired trade-
offs between model complexity, interpretability, and accuracy.
GURURAJ VA