FAIML: Unit 5: Learning

Syllabus:
Types of Learning: Supervised, Unsupervised and Semi-Supervised Learning.

Supervised: Learning a Class from Examples, Types of supervised Machine

learning Algorithms,

Unsupervised: Types of Unsupervised Learning Algorithm,

Dimensionality Reduction: Introduction
to Dimensionality Reduction, Subset Selection, and Introduction to Principal
Component Analysis.

Q.1) What is Supervised and Unsupervised Learning?

Give real life example of supervised and unsupervised
learning.
Ans:

✅ 1. Supervised Learning
🔍 Definition:
Supervised Learning is a type of machine learning where the model is trained
using labeled data (data that includes both inputs and correct outputs).
🧠 The algorithm learns to map inputs to outputs by comparing its predictions to
the correct answers.

🎯 Goal:
Predict an output (label) based on input data.

📦 Examples:
FAIML: Unit 5: Learning 1
 Input (Features) Output (Label)

Email text Spam or Not Spam

House size, location Price of the house

Patient symptoms Disease diagnosis

🌍 Real-Life Examples:
1. Email Spam Filter

Input: Email content

Output: Spam or Not Spam

✅ The model is trained with thousands of labeled emails.

2. Loan Approval Prediction

Input: Salary, Credit Score, Age

Output: Approve or Reject

✅ Labeled with past approval records.

3. Face Recognition

Input: Image

Output: Person’s Name

✅ Trained with labeled faces.

✅ 2. Unsupervised Learning
🔍 Definition:
Unsupervised Learning is where the model is trained on unlabeled data — it finds
hidden patterns or groups in the data without predefined outputs.
🧠 The model explores the data to discover structure, like grouping or
associations.

🎯 Goal:
FAIML: Unit 5: Learning 2
 Find patterns, clusters, or structure in the data.

📦 Examples:
Data (No labels) Output

Customer purchasing behavior Customer segments

Website visit logs Browsing pattern groups

Images without categories Object grouping

🌍 Real-Life Examples:
1. Customer Segmentation

Input: Buying history of customers

Output: Grouping similar customers

❌ No labels, just behavior patterns

2. Market Basket Analysis

Input: Items bought together

Output: Product recommendations

❌ Finds hidden relationships (e.g., people who buy bread often buy
butter)

Q.2) What is Dimensionality Reduction? Explain

Subset Selection in detail.
Ans:
✅ Dimensionality Reduction:
🔍 Definition:
Dimensionality Reduction is the process of reducing the number of input features
(dimensions) in a dataset, while still retaining as much important information as

FAIML: Unit 5: Learning 3

 possible.

🎯 Why Dimensionality Reduction is Important?

To reduce model complexity

To improve model performance

To avoid overfitting

To make visualization easier

To decrease training time

📌 Example:
If your dataset has 100 features, and only 10 are actually useful, you can reduce it
to those 10 features to train a better model.

✅ Subset Selection (Feature Selection):

🔍 Definition:
Subset Selection is a dimensionality reduction technique where you choose a
subset of the most relevant original features and discard the unimportant ones.

Unlike other techniques (like PCA), subset selection keeps the features as they
are, without transforming them.

🎯 Goals of Subset Selection:

Select features that are most relevant to the output

Remove redundant or irrelevant data

Improve accuracy and training speed

📊 Example:
All Features After Subset Selection

Age, Height, Weight, Blood Group, Name Age, Weight, Blood Group

FAIML: Unit 5: Learning 4

 👉 Features like "Name" may be irrelevant, so they are removed.
🧠 Types of Subset Selection Methods:
✅ 1. Filter Methods
Use statistical techniques to score and select features.

Independent of any machine learning model.

Examples:

Correlation coefficient

Chi-square test

Information gain

✅ 2. Wrapper Methods
Use machine learning models to test different feature combinations.

Select the subset that gives the best model performance.

Examples:

Forward Selection

Backward Elimination

Recursive Feature Elimination (RFE)

✅ 3. Embedded Methods
Feature selection is done automatically during model training.

Examples:

Lasso Regression

Decision Tree feature importance

📊 Comparison Table:
FAIML: Unit 5: Learning 5
 Method Speed Accuracy Uses ML Model

Filter Fast Medium ❌ No

Wrapper Slow High ✅ Yes
Embedded Medium High ✅Y

Q.3) Write short note Principal Component Analysis.

Ans:

✅ Definition:
Principal Component Analysis (PCA) is a Dimensionality Reduction technique
that transforms a large set of variables into a smaller one that still contains most of
the information in the large set.

🧠 Steps of PCA:
✅ 1. Standardize the Data
Convert all features to the same scale (mean = 0, standard deviation = 1).

Ensures that no feature dominates due to its unit or scale.

✅ 2. Compute the Covariance Matrix

Measures how features vary with each other.

Helps identify relationships between features.

✅ 3. Find Principal Directions and Scores

Identify new directions (principal components) that capture the most variation.

Each direction has an importance score (how much variance it explains).

✅ 4. Select Top Principal Components

Sort components by their importance scores (highest first).

FAIML: Unit 5: Learning 6

 Choose top few that capture most of the data's information.

✅ 5. Transform Data to New Axes

Project the original data onto the selected components (new feature space).

Resulting dataset has fewer dimensions with minimal loss of information.

📊 Visual Analogy:
Imagine you have a cloud of data points in 3D (x, y, z). PCA finds the best plane or
line to project the data onto, so it becomes 2D or 1D — but still looks almost the
same!

Q.3) Explain different types of Learning Models in

Machine Learning
Ans:

🤖 Types of Learning Models in Machine Learning

Machine Learning models are mainly classified based on how they learn from
data. The major types are:

1️⃣ Supervised Learning

📌 Definition:
The model is trained on a labeled dataset, meaning the input comes with the
correct output.

🛠️ Goal:
Learn a mapping from input → output so that it can predict outputs for new inputs.
📊 Examples:
Predicting house prices based on size and location

Email spam detection

FAIML: Unit 5: Learning 7

 Classifying handwritten digits

📌 Types:
Classification: Predict categories (e.g., spam or not spam)

Regression: Predict continuous values (e.g., price of a house)

2️⃣ Unsupervised Learning

📌 Definition:
The model is given unlabeled data and must find patterns or structure on its own.
🛠️ Goal:
Discover hidden patterns, groupings, or features from the data.

📊 Examples:
Customer segmentation

Market basket analysis

Grouping similar documents

📌 Types:
Clustering: Group similar data points (e.g., K-Means)

Association: Find relationships between features (e.g., Apriori algorithm)

3️⃣ Semi-Supervised Learning

📌 Definition:
Uses a small amount of labeled data and a large amount of unlabeled data.
🛠️ Goal:
Improve learning accuracy using both labeled and unlabeled data.
📊 Examples:
Text classification with few labeled documents

Face recognition with limited labeled photos

📌 Why Useful?
FAIML: Unit 5: Learning 8
 Labeled data is expensive; semi-supervised learning uses unlabeled data to
improve performance.

4️⃣ Reinforcement Learning (RL)

📌 Definition:
An agent learns by interacting with an environment and getting rewards or
penalties.
🛠️ Goal:
Learn a sequence of actions to maximize total reward over time.
📊 Examples:
Game playing (e.g., AlphaGo)

Robotics

Self-driving cars

📌 Key Concepts:
Agent

Environment

Actions

Rewards

Q.4) Explain with example Forward and Backward

Selection method for Subset Selection.
Ans:

🔎 Subset Selection in Machine Learning

Subset Selection is a Dimensionality Reduction technique used to select the best
set of features (variables) that improve model performance and reduce
overfitting.

FAIML: Unit 5: Learning 9

 Two popular methods are:

1️⃣ Forward Selection

📌 Definition:
Start with no features, and add one feature at a time that improves the model the
most until no significant improvement is seen.

🧠 Steps:
1. Start with an empty model.

2. Test each feature individually.

3. Add the feature that improves performance the most.

4. Repeat until adding more features doesn't improve the model significantly.

✅ Example:
Imagine you have 4 features: A, B, C, D

Start with: No features

Try A → model accuracy = 70%

Try B → 65%

Try C → 72% ✅ (Best, so pick C)

Try adding A to C → 78% ✅

Try adding D → no improvement

Final selected features = C and A

2️⃣ Backward Selection

📌 Definition:
Start with all features, and remove one feature at a time that reduces model
performance the least (i.e., least important).

FAIML: Unit 5: Learning 10

 🧠 Steps:
1. Start with the full model (all features).

2. Remove one feature at a time and check performance.

3. Remove the one that least affects accuracy.

4. Repeat until performance starts dropping significantly.

✅ Example:
Features: A, B, C, D

Start with: A, B, C, D

Try removing A → accuracy = 90%

Try removing B → 88%

Try removing C → 91%✅ (remove C)

Try removing A → 89% ✅ (remove A)

Final selected features = B and D

Q.5) Why do we need dimensionality reduction? What

are its drawback
Ans:

🎯 Why Do We Need Dimensionality Reduction?

Dimensionality Reduction is important in Machine Learning and Data Science for
several reasons:

✅ 1. Improves Model Performance

Reduces noise and irrelevant features.

Makes models faster and more accurate by focusing only on important

variables.

FAIML: Unit 5: Learning 11

 ✅ 2. Reduces Overfitting
With fewer features, models are less likely to memorize the training data.

Helps generalize better to new/unseen data.

✅ 3. Enhances Visualization
High-dimensional data is hard to visualize.

Dimensionality reduction (e.g., to 2D or 3D) helps in plotting and

understanding the data.

✅ 4. Reduces Storage and Computation Cost

Less memory is needed.

Algorithms run faster on fewer dimensions.

✅ 5. Helps in Feature Selection & Engineering

Removes redundant or correlated features.

Simplifies datasets for easier analysis.

⚠️ Drawbacks of Dimensionality Reduction

❌ 1. Loss of Information
Reducing dimensions can remove important details or patterns.

May affect model accuracy if critical features are dropped.

❌ 2. Hard to Interpret
Transformed features (e.g., in PCA) may not have a clear meaning.

Makes it difficult to explain model decisions to stakeholders.

❌ 3. Risk of Oversimplification
FAIML: Unit 5: Learning 12
 Too much reduction might oversimplify the problem.

The model may miss complex relationships in the data.

❌ 4. Extra Computation (Initially)

Methods like PCA require matrix operations, which may be costly for large
datasets during training.

Q.6) What is unsupervised learning and explain types

of unsupervised learning algorithm with example.
Ans:

🤖 What is Unsupervised Learning?

📌 Definition:
Unsupervised learning is a type of machine learning where the model is trained on
unlabeled data — meaning, the data has no predefined categories or outputs.

🎯 Goal:
To find patterns, relationships, or structure in the data without any human-
provided labels.

🧠 Key Features:
No input-output mapping

Learns from hidden patterns

Mainly used for clustering, association, and dimensionality reduction

🧩Examples)
Types of Unsupervised Learning Algorithms (with

FAIML: Unit 5: Learning 13

 1️⃣ Clustering Algorithms
📌 Purpose:
Group similar data points into clusters.

✅ Example Algorithms:
K-Means Clustering

→ Groups customers based on buying behavior.

Hierarchical Clustering
→ Builds a tree of clusters (dendrogram).

DBSCAN
→ Detects clusters of varying shapes and sizes.

📍 Real-Life Example:
Grouping users on a streaming platform based on viewing habits.

2️⃣ Association Rule Learning

📌 Purpose:
Find relationships or patterns among items in large datasets.

✅ Example Algorithms:
Apriori Algorithm
→ Finds frequent itemsets (e.g., bread and butter bought together).

Eclat Algorithm
→ More efficient for sparse datasets.

📍 Real-Life Example:
Market Basket Analysis — "People who bought X also bought Y."

3️⃣ Dimensionality Reduction Algorithms

📌 Purpose:
FAIML: Unit 5: Learning 14
 Reduce the number of features in the data while preserving key information.

✅ Example Algorithms:
PCA (Principal Component Analysis)
→ Reduces correlated features into uncorrelated principal components.

t-SNE
→ Visualizes high-dimensional data in 2D/3D.

📍 Real-Life Example:
Compressing image data for facial recognition or speeding up model training.

Q.7) Explain 2 methods for dimensionality reduction.

Ans:

📉 Dimensionality Reduction Methods

Dimensionality reduction techniques help simplify high-dimensional datasets by
reducing the number of input variables (features) while preserving as much
important information as possible.

1️⃣ Principal Component Analysis (PCA)

📌 Definition:
PCA is a mathematical technique that transforms the original features into a new
set of uncorrelated features called principal components. These components
capture the maximum variance in the data.

🧠 Key Ideas:
It rotates the data to align it with directions of maximum variance.

The first principal component captures the most variance, the second
captures the next most, and so on.

FAIML: Unit 5: Learning 15

 You can choose the top k components that preserve most of the information.

✅ Example Use Case:

Reducing the number of pixel features in image recognition.

Speeding up training for large datasets.

📊 Advantages:
Improves performance and reduces overfitting.

Helps in data visualization (e.g., reducing 100 features to 2 or 3 for plotting).

2️⃣ Feature Selection (Subset Selection)

📌 Definition:
Feature selection involves selecting a subset of the most relevant original
features based on their importance or impact on model performance.

✨ Popular Techniques:
Forward Selection: Start with no features and add one at a time based on
performance improvement.

Backward Elimination: Start with all features and remove the least useful one
at each step.

Recursive Feature Elimination (RFE): Systematically remove features and

evaluate model accuracy.

✅ Example Use Case:

Selecting only the most important medical test results for disease prediction.

📊 Advantages:
Maintains interpretability since original features are retained.

Reduces training time and model complexity.

FAIML: Unit 5: Learning 16

 Q.8) Explain features with suitable examples. Explain Feature selection and
feature extraction.

Ans:

🧩 What is Feature Selection?

📌 Definition:
Feature Selection is the process of choosing only the most relevant features
from the original dataset to improve model performance.

🎯 Goal:
Remove irrelevant, noisy, or redundant features.

Reduce computation, avoid overfitting, and enhance accuracy.

🔍 Techniques:
Forward Selection

Backward Elimination

Filter Methods (e.g., correlation)

Wrapper Methods (e.g., RFE)

✅ Example:
From a dataset with 20 features, if only 5 features strongly affect output, we select
those and drop the rest.

🔄 What is Feature Extraction?

📌 Definition:
Feature Extraction creates new features by combining or transforming original
features — especially useful for dimensionality reduction.

🎯 Goal:
FAIML: Unit 5: Learning 17
 Transform high-dimensional data into a lower-dimensional space while
retaining important information.

⚙️ Common Methods:
Principal Component Analysis (PCA)

Linear Discriminant Analysis (LDA)

Autoencoders (Deep Learning)

✅ Example:
Instead of using 100 pixel values in an image, PCA can create 2–3 principal
components that represent the main variations in the image.

📊 Feature Selection vs. Feature Extraction

Aspect Feature Selection Feature Extraction

Selects a subset of original Creates new features from original

What it does
features data

New features may lose original

Interpretability Original features remain intact
meaning

Combining “Height” & “Weight” →

Example Choosing “Age” and “Salary” only
BMI

RFE, Filter, Wrapper, Embedded

Tools/Methods PCA, LDA, Autoencoders
methods

Q.8) Write difference between unsupervised and semi

supervised learning and supervised Learning.
Ans:

📚andDifference Between Supervised, Unsupervised,

Semi-Supervised Learning
Unsupervised Semi-Supervised
Aspect Supervised Learning
Learning Learning

FAIML: Unit 5: Learning 18

 Learns from small
🔍 Definition Learns from labeled
data (input + output)
Learns from
unlabeled data only
labeled + large unlabeled
data

Discover hidden Improve accuracy using

🧠 Goal Predict output/labels
for new data
patterns or few labels and many
groupings unlabeled examples

🗂️ Type of Data Fully labeled dataset

Completely
Partially labeled dataset
Used unlabeled dataset

🧪 Examples of Classification, Clustering, Web page classification,

Tasks Regression Association Medical image labeling

Decision Trees,
📊 Algorithms SVM, Logistic
K-Means, PCA,
Apriori
Semi-supervised SVM,
Graph-based methods
Regression

Predict values/classes
🎯 Output Predict specific
values or classes
Discover structure
in data (no labels)
using both labeled &
unlabeled data

🧑‍🏫 Human High – requires Low – no labeling Medium – requires few

Involvement labeled training data needed labeled samples

✅ Examples:
Supervised: Predicting house prices from features like size and location.

Unsupervised: Grouping customers based on buying behavior without

predefined categories.

Semi-Supervised: Classifying emails as spam or not using a few labeled

emails and many unlabeled ones.

FAIML: Unit 5: Learning 19