PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
Bayesian Concept Learning
Importance of Bayesian Network:
Bayesian networks are powerful probabilistic graphical models that represent the
probabilistic relationships among a set of variables. They have several important
applications across various domains due to their ability to handle uncertainty and perform
reasoning under uncertainty. Here are some of the key reasons why Bayesian networks
are important.
1. Probabilistic Representation: Bayesian networks provide an intuitive and compact
way to represent the probabilistic relationships among variables. They use directed
acyclic graphs (DAGs) to represent the dependencies between variables, with each
node representing a random variable and edges representing conditional
dependencies.
2. Uncertainty Modeling: In real-world scenarios, uncertainty is prevalent, and data may
not always be complete or deterministic. Bayesian networks can handle uncertainty
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
effectively by allowing the inclusion of probabilistic information and dealing with
missing data.
3. Inference and Prediction: Bayesian networks enable efficient inference and
prediction. Given evidence or observed values for some variables, they allow us to
calculate the posterior probabilities of other variables in the network. This is valuable
for making predictions and decisions in uncertain environments.
4. Causal Reasoning: Bayesian networks can help identify causal relationships
between variables. By representing causal dependencies, they enable us to understand
how changes in one variable can affect others, which is crucial for decision-making
and policy analysis.
5. Diagnosis and Decision Support: Bayesian networks are commonly used for
diagnosis and decision support systems in various fields, including medicine, finance,
and engineering. They can model complex dependencies among symptoms,
observations, and potential causes, aiding in accurate diagnosis and decision-making.
6. Learning from Data: Bayesian networks can be learned from data, making them
valuable for discovering hidden relationships and structures in the data. Learning the
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
structure and parameters of the network from data allows for data-driven
modeling.
7. Risk Analysis: Bayesian networks can be used to assess risk and uncertainty in
various scenarios. By incorporating probabilistic information about different factors, they
help in evaluating the likelihood of different outcomes.
8. Decision Making under Uncertainty: Bayesian networks provide a systematic way to
analyze decisions under uncertainty. By incorporating probabilistic information, they
help in making informed decisions that consider potential outcomes and their
probabilities.
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
Bayesian Theorem
The Bayesian theorem, also known as Bayes' theorem or Bayes' rule, is a
fundamental concept in probability theory and statistics. It allows us to update the
probability of an event based on new evidence or information. The theorem is
named after the Reverend Thomas Bayes, who first formulated it in the 18th
century.
The Bayesian theorem states that the probability of a hypothesis H being true, given
some observed evidence E, can be calculated using the following formula:
P(H|E) = [P(E|H) *P(H)] / P(E)
Where:
0. Posterior Probability( P(H∣E) ):
This is the updated probability of the class given the observed features.
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
It is what we are trying to predict or infer in a classification task.
P(H|E) is the posterior probability, i.e., the probability of the hypothesis H being
true given the evidence E.
1. Likelihood(P(E∣H)):
Represents the probability of observing the given evidence (features) given that
the class is true.
In the Naive Bayes algorithm, a key assumption is that features are conditionally
independent given the class label. In other words, Naive Bayes works best with
discrete features.
P(E|H) is the likelihood, i.e., the probability of observing the evidence E given .
that the hypothesis H is true.
2. Prior Probability (P(H)):
In machine learning, this represents the probability of a particular class before
considering any features.
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
It is estimated from the training data.
P(H) is the prior probability, i.e., the probability of the hypothesis H being true
Before considering any evidence.
3. Evidence Probability( P(E) ):
This is the probability of observing the given evidence (features).
It serves as a normalization factor and is often calculated as the sum of the joint
probabilities over all possible classes.
P(E) is the marginal likelihood or evidence, i.e., the overall probability of
Observing the evidence E, regardless of any specific hypothesis.
The Bayesian theorem is especially useful when we have some prior knowledge or
Beliefs (prior probabilities) about the hypothesis, and we want to update those beliefs
based on new evidence.
One of the significant advantages of using the Bayesian approach is its ability to handle
uncertainty and incorporate new information in a principled way. As more evidence
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
becomes available, the posterior probability can be updated iteratively, making the
Bayesian theorem a powerful tool for decision-making, learning from data, and reasoning
under uncertainty. It finds applications in various fields, including machine learning,
artificial intelligence, medical diagnosis, finance, and more.
We Can understand another diagram:-
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
Example 1:-
Let us say P(Fire) means how often there is fire, and P(Smoke) means how often we
see smoke, then:
P(Fire|Smoke) means how often there is fire when we can see smoke
P(Smoke|Fire) means how often we can see smoke when there is fire
So the formula kind of tells us "forwards" P(Fire|Smoke) when we know
"backwards" P(Smoke|Fire)
Suppose:
dangerous fires are rare (1%)
but smoke is fairly common (10%) due to barbecues,
and 90% of dangerous fires make smoke
We can then discover the probability of dangerous Fire when there is Smoke:
P(Fire|Smoke) =P(Fire) P(Smoke|Fire)|P(Smoke)
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
P(Fire|Smoke) = ( 1% * 90% ) | 10%
= 9%
So it is still worth checking out any smoke to be sure.
Example:- 2
Picnic Day
You are planning a picnic today, but the morning is cloudy
Oh no! 50% of all rainy days start off cloudy!
But cloudy mornings are common (about 40% of days start cloudy)
And this is usually a dry month (only 3 of 30 days tend to be rainy or 10%)
What is the chance of rain during the day?
We will use Rain to mean rain during the day, and Cloud to mean cloudy morning.
The chance of Rain given Cloud is written P(Rain|Cloud)
So let's put that in the formula:
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
P(Rain|Cloud) = P(Rain) P(Cloud|Rain)| P(Cloud)
P(Rain) is Probability of Rain = 10%
P(Cloud|Rain) is Probability of Cloud, given that Rain happens = 50%
P(Cloud) is Probability of Cloud = 40%
P(Rain|Cloud) = (0.1 * 0.5)|0.4 = .125
Or a 12.5% chance of rain. Not too bad, let's have a picnic!
Bayes’ Theorem And Concept Learning
In the context of concept learning, Bayes' theorem is a fundamental concept
that allows us to make probabilistic inferences about hypotheses (concepts)
based on observed data. Concept learning is the process of acquiring knowledge
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
about concepts or categories from examples, and Bayes' theorem provides a
principled way to update our beliefs about the likelihood of different hypotheses
being true given the available evidence (data).
Let's reiterate the components of Bayes' theorem and how they relate to concept learning:
1. Hypothesis (Concept): In concept learning, a hypothesis represents a concept or
category that we want to learn from the data. For example, in a spam email classification
task, the hypothesis could be whether an email is spam or not. In image classification, the
hypothesis might be whether an image contains a specific object or not.
2. Evidence (Data): The evidence in concept learning consists of observed data or
examples. In supervised learning, the evidence is typically represented as input-output
pairs, where the input is a set of features or attributes describing the data, and the output
is the corresponding class label or category.
3. Prior Probability P(H): The prior probability represents our initial belief about the
probability of a hypothesis (concept) being true before considering any evidence. In
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
concept learning, it captures our initial knowledge or assumptions about the likelihood of
different hypotheses being correct
4. Likelihood P(E|H): The likelihood represents the probability of observing the
evidence (data) given that the hypothesis (concept) is true. It quantifies how well the
hypothesis explains the observed data. In concept learning, the likelihood is typically
modeled using a probability distribution that describes how likely the data is under the
given hypothesis.
5. Posterior Probability P(H|E): The posterior probability represents the updated
probability of the hypothesis (concept) being true given the observed evidence (data). It is
the quantity we want to calculate and interpret in concept learning. The posterior
probability is obtained by combining the prior probability and the likelihood using Bayes'
theorem.
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
Bayesian Belief Network
A Bayesian Belief Network (BBN), also known as a Bayesian Network or Bayesian
Graphical Model, is a probabilistic graphical model that represents probabilistic
relationships among a set of variables using a directed acyclic graph (DAG). BBNs
are widely used for reasoning under uncertainty, learning from data, and making
predictions in various fields, including artificial intelligence, machine learning, decision
analysis, and expert systems.
Key features of Bayesian Belief Networks:
1. Directed Acyclic Graph (DAG): The network structure consists of nodes (vertices)
representing random variables and directed edges (arcs) representing probabilistic
dependencies between the variables. The acyclic property ensures that there are no
cycles in the graph, which is crucial for proper inference.
2. Conditional Probability Tables (CPTs): Each node in the BBN has a conditional
probability table that quantifies the probability distribution of the node given its
PARUL INSTITUTE OF ENGINEERING & TECHNOLOGY
FACULTY OF ENGINEERING & TECHNOLOGY
parents (i.e., variables connected directly to it by incoming edges). CPTs encode the
probabilistic relationships between the variables in the network.
3. Inference and Reasoning: BBNs allow for efficient probabilistic inference and
reasoning. Given evidence (observed values for some variables), the network can be
used to calculate posterior probabilities of other variables in the graph, which is
useful for making predictions and decisions under uncertainty.
4. Learning from Data: BBNs can be learned from data, allowing the structure and
parameters of the network to be automatically derived from observed examples.
Learning BBNs from data is valuable for discovering causal relationships and
dependencies among variables.
5. Causal Reasoning: BBNs are particularly useful for representing causal
relationships between variables. The directed edges in the graph indicate causal
dependencies, which enables us to understand the causal impact of one variable on
another.
