[go: up one dir, main page]
Scribd
Upload
6 views4 pages

decisiontree

The document outlines various inductive learning problems, including regression, binary classification, multiclass classification, and ranking. It explains the Decision Tree model as a supervised learning technique primarily used for classification, detailing its structure, terminology, and algorithmic process. Additionally, it discusses Attribute Selection Measures and Information Gain as methods for selecting the best attributes for decision nodes in a tree.

Uploaded by

mstdsproject2023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
6 views4 pages

decisiontree

The document outlines various inductive learning problems, including regression, binary classification, multiclass classification, and ranking. It explains the Decision Tree model as a supervised learning technique primarily used for classification, detailing its structure, terminology, and algorithmic process. Additionally, it discusses Attribute Selection Measures and Information Gain as methods for selecting the best attributes for decision nodes in a tree.

Uploaded by

mstdsproject2023
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

Some canonical Learning Problems:

There are a large number of typical inductive learning problems. The primary difference between
them is in what type of thing they’re trying to predict. Here are some examples
Regression:
Trying to predict a real value. For instance, predict the value of a stock tomorrow given its past
performance. Or predict Alice’s score on the machine learning final exam based on her
homework scores.
Binary Classification:
trying to predict a simple yes/no response. For instance, predict whether Alice will enjoy a
course or not. Or predict whether a user review of the newest Apple product is positive or
negative about the product.
Multiclass Classification:
trying to put an example into one of a number of classes. For instance, predict whether a news
story is about entertainment, sports, politics, religion, etc. Or predict whether a CS course is
Systems, Theory, AI or Other.
Ranking:
trying to put a set of objects in order of relevance. For instance, predicting what order to put web
pages in, in response to a user query. Or predict Alice’s ranked preferences over courses she
hasn’t taken.
The Decision Tree Model of Learning:
Decision Tree is a Supervised learning technique that can be used for both classification and
Regression problems, but mostly it is preferred for solving Classification problems. It is a tree-
structured classifier, where internal nodes represent the features of a dataset, branches represent
the decision rules and each leaf node represents the outcome.
In a Decision tree, there are two nodes, which are the Decision Node and Leaf Node. Decision
nodes are used to make any decision and have multiple branches, whereas Leaf nodes are the
output of those decisions and do not contain any further branches.
The decisions or the test are performed on the basis of features of the given dataset.
Decision Tree Terminologies:
Root Node: Root node is from where the decision tree starts. It represents the entire dataset,
which further gets divided into two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated further after
getting a leaf node.
Splitting: Splitting is the process of dividing the decision node/root node into sub-nodes
according to the given conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted branches from the tree.
Parent/Child node: The root node of the tree is called the parent node, and other nodes are
called the child nodes.
How does the Decision Tree algorithm Work?
The complete process can be better understood using the below algorithm:

Step-1: Begin the tree with the root node, says S, which contains the complete dataset.
Step-2: Find the best attribute in the dataset using Attribute Selection Measure (ASM).
Step-3: Divide the S into subsets that contains possible values for the best attributes.
Step-4: Generate the decision tree node, which contains the best attribute.
Step-5: Recursively make new decision trees using the subsets of the dataset created in step -3.
Continue this process until a stage is reached where you cannot further classify the nodes and
called the final node as a leaf node.
Example:
Suppose there is a candidate who has a job offer and wants to decide whether he should accept
the offer or Not.
So, to solve this problem, the decision tree starts with the root node (Salary attribute by ASM).
The root node splits further into the next decision node (distance from the office) and one leaf
node based on the corresponding labels.
The next decision node further gets split into one decision node (Cab facility) and one leaf node.
Finally, the decision node splits into two leaf nodes (Accepted offers and Declined offer).
Consider the below diagram:

Attribute Selection Measures:


While implementing a Decision tree, the main issue arises that how to select the best attribute for
the root node and for sub-nodes. So, to solve such problems there is a technique which is called
as Attribute selection measure or ASM. By this measurement, we can easily select the best
attribute for the nodes of the tree. There is popular technique is used that is Information Gain.
Information Gain:
When we use a node in a decision tree to partition the training instances into smaller subsets the
entropy changes. Information gain is a measure of this change in entropy.
Suppose S is a set of instances,
A is an attribute
Sv is the subset of S
v represents an individual value that the attribute A can take and Values (A) is the set of all
possible values of A, then

Entropy:
Entropy is a metric to measure the impurity in a given attribute. It specifies randomness in data.
Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)


Where,
S= Total number of samples
P(yes)= probability of yes
P(no)= probability of no

You might also like