Bayesian Network
Topics covered
● Introduction to bayesian networks
● Joint probability and conditional probability
● Bayes Theorem
● Inference
● Independence in Bayes networks
● Case studies
● Implementation
2
Introduction
● calculate probability of an uncertain cause given some observed evidence
○ For eg. disease by observing symptoms (complex)
● Reversed conditional probability- in real life practice
○ ie. probability of evidence given cause
● Bayesian Networks - a graphical model
● useful tools for dealing not only with uncertainty, but also with complexity and
(even more importantly) causality,
3
Bayesian Network
● a representation of a joint
probability distribution of a set
of random variables with a
possible mutual causal
relationship
● Consists of nodes, edges and
conditional probability
distribution - directed acyclic
graph
4
Joint Probability
● The probability that two or more events happen at the same time:
● Only if the events are independent
● The events are independent if the occurrence of one does not affect another
● Similarly 2 random variables are independent if the realization of one does not
affect the probability distribution of another.
5
6
Conditional Probability
7
Bayes Theorem
● Conditional Probability
○ P(a|b) = P(a,b)/P(b)
● Product Rule
○ P(a, b) = P(a|b)P(b)
● Bayes Theorem
○ P(b|a) = P(a,b)/P(a) = P(a|b)P(b)/P(a)
8
P(R, W) = P(R)P(W|R)
P(L, R, W) = P(L)P(R)P(W|R)
9
P(L, R, W, S) = P(L)P(R)P(W|R)P(S|W)
10
Bayesian Network Construction
Bayesian networks will be constructed in three consecutive steps:
Step 1:Define the network variables and their values.
● A query variable is one which we need to ask questions about, such as
compute its posterior marginal
● An evidence variable is one which we may need to asset evidence about.
● An intermediary variable is neither query nor evidence and is meant to aid the
modeling process by detailing the relationship between evidence and query
variables
The distinction between query, evidence and intermediary variables is not a
property of the Bayesian Network but of task at hand.
Bayesian Network Construction
Step 2:
Define the network structure (edges)
We will be guided by the causal interpretation of network structure.
The determination of network structure will be reduced to answering the following
question about each network variable X: what set of variable we regard as the
direct causes of X ?
Bayesian Network Construction
Step 3:
Define the network CPTs
The difficulty and objectivity of this step varies considerably from one problem to
another.
● CPTs can sometimes be determined completely form the problem statement
by objective considerations.
● CPTs can be reflection of subjective beliefs
● CPTs can be estimated from data.
Bayesian Network Construction
Example
The flu is an acute disease characterized by fever, body aches and pains and can
be associated with chilling and a sore throat. The cold is a bodily disorder
popularly associated with chilling and can cause a sore throat. Tonsillitis is
inflammation of the tonsils which leads to a sore throat and can be associated with
fever.
Our goal here is develop a Bayesian Network to capture this knowledge and then
use it to diagnose the condition of a patient suffering from some of the syptoms
mentioned above.
Alarm Example
11
Inference
● Given a bayesian network given by P(X, Y, Z), what is P(Y)?
○ First approach : Enumeration
■ A brute force approach, not practical in real life scenario
○ Second approach : Variable Elimination
■ Replacement by a function
■ Can be done in different ways each having different cost
■ Must be done in such a way that the cost is minimum
12
Enumeration
13
14
Variable Elimination
15
Alarm Example
16
Alarm Example
17
Independence in Bayes Nets
● Each variable is conditionally independent of its non descendant given its
parent
18
Independence in Bayes Nets
● Each variable is conditionally
independent of any other variable
given its Markovs blanket
○ Parent, children and children's parent
19
20
Case Studies
● Disease Diagnosis: Bayesian Networks are commonly used in the field of medicine for the detection and
prevention of diseases. They can be used to model the possible symptoms and predict whether or not a person is
diseased.
● Optimized Web Search: Bayesian Networks are used to improve search accuracy by understanding the intent of a
search and providing the most relevant search results. They can effectively map users intent to the relevant content
and deliver the search results.
● Spam Filtering: Bayesian models have been used in the Gmail spam filtering algorithm for years now. They can
effectively classify documents by understanding the contextual meaning of a mail. They are also used in other
document classification applications.
● Biomonitoring: Bayesian Networks play an important role in monitoring the quantity of chemical dozes used in
pharmaceutical drugs.
21
Implementation
● Problem definition: Monty Hall Problem
○ involves three doors, given that behind one of these doors is a car
○ the remaining two have goats behind them
○ So you start by picking a random door, say #2.
○ On the other hand, the host knows where the car is hidden and he opens another door, say #1 (behind which
there is a goat).
○ Here’s the catch, you’re now given a choice, the host will ask you if you want to pick door #3 instead of your
first choice i.e. #2.
○ Is it better if you switch your choice or should you stick to your first choice?
● Language Used: Python
22
Monty Hall Problem
The first step is to build a Directed Acyclic Graph.
The graph has three nodes, each representing the door
chosen by:
● The door selected by the Guest
● The door containing the prize (car)
● The door Monty chooses to open
23
References
● https://www.edureka.co/blog/bayesian-networks/#Bayesian%20Networks%20Application
● https://www.bu.edu/sph/files/2014/05/bayesian-networks-final.pdf
● https://www.slideshare.net/adnanmasood/bayesian-networks-primer-21281913
● https://www.slideshare.net/MassimilianoPatacchi/bayesian-networks-78623411
● https://www.youtube.com/watch?v=TuGDMj43ehw&t=56s
24
Thank You
