Decision Tree Analysis
George Easaw
What is a Decision Tree?
● Definition:
○ A decision tree is a flowchart-like structure used for
decision-making, where each node represents a decision
point and branches represent the possible outcomes.
● Key Components:
○ Root Node: The starting point of the tree.
○ Decision Nodes: Points where a choice is made.
○ Leaf Nodes: Final outcomes or classifications.
How Decision Trees Work
● Process:
○ Start at the root node.
○ Evaluate conditions at each decision node.
○ Follow the branches based on the outcome of the evaluation.
○ Reach a leaf node, which gives the final decision or
classification.
● Key Concepts:
○ Splitting: Dividing data into subsets.
○ Pruning: Removing branches to avoid overfitting.
Advantages and Disadvantages
● Advantages:
○ Easy to interpret and visualize.
○ Handles both numerical and categorical data.
○ Requires minimal data preprocessing.
● Disadvantages:
○ Prone to overfitting with complex trees.
○ Can be unstable with small changes in data.
○ May not perform well with imbalanced datasets.
Common Applications
● Classification:
○ Used to classify data into predefined categories (e.g., spam
detection, loan approval).
● Regression:
○ Used to predict continuous outcomes (e.g., predicting house
prices).
● Real-World Examples:
○ Customer segmentation.
○ Medical diagnosis.
○ Credit risk assessment.
Summary:
● Decision trees are a powerful tool for both classification
and regression tasks.
● They offer intuitive visualization and straightforward
implementation but must be carefully managed to avoid
overfitting.
Exercises
Play Tennis
Problem: Should you play tennis based on the weather?
Conditions:
● Sunny
● Overcast
● Rainy
● If it's Sunny or Overcast, Play.
● If it's Rainy, Don’t Play.
2. Go for a Walk
Problem: Should you go for a walk based on temperature?
Conditions:
● Hot
● Warm
● Cold
If it's Hot, Don’t Walk.
If it’s Warm or Cold, Walk.
Buy a Laptop
Problem: Should you buy a laptop based on price and features?
Conditions:
● Price: Expensive, Affordable
● Features: Basic, Advanced
If it’s Expensive, Don’t Buy.
If it’s Affordable and has Advanced features, Buy.
Choose a Dessert
Problem: Choose a dessert based on availability and preference.
Conditions:
● Available: Yes, No
● Preference: Ice Cream, Cake
If available, choose Ice Cream or Cake based on preference.
If not available, Don’t Choose.
Go to the Gym
Problem: Should you go to the gym based on how you feel?
Conditions:
● Feeling: Energetic, Tired
● Time: Morning, Evening
If Energetic, Go to the Gym.
If Tired, and it’s Evening, Go to the Gym.
Buy a Book
Problem: Should you buy a book based on interest and
reviews?
Conditions:
● Interest: High, Low
● Reviews: Positive, Negative
If Interest is High and Reviews are Positive, Buy.
Otherwise, Don’t Buy.
Watch a Movie
Problem: Should you watch a movie based on genre and
reviews?
Conditions:
● Genre: Action, Drama
● Reviews: Good, Bad
If Genre is Action and Reviews are Good, Watch.
Otherwise, Don’t Watch.
Choose a Restaurant
Problem: Choose a restaurant based on cuisine and rating.
Conditions:
● Cuisine: Italian, Indian
● Rating: High, Low
If Cuisine is Italian and Rating is High, Choose.
Otherwise, Don’t Choose.
