Probability Theory and Random
Processes (MA225)
Lecture Slides
Lecture 04
Independence
Observe that P(B1 |W ) = 9/34 < 1/2 = P(B1 ), whereas
P(B2 |W ) = 5/17 > 1/6 = P(B2 ). Thus the “occurrence of one
event is making the occurrence of a second event more or less likely”.
Def: Let A and B be two events. They are said to be
a) negatively associated if P(A ∩ B) < P(A)P(B) ,
b) positively associated if P(A ∩ B) > P(A)P(B) ,
c) independent if P(A ∩ B) = P(A)P(B) .
I If P(B) = 0 then A and B are independent.
I If P(B) = 1 then A and B are independent.
I In particular any event A is independent of S and φ.
Theorem: If A and B are independent, so are A and B c , Ac and B,
Ac and B c .
Def: A countable collection of events E1 , E2 , . . . are said to be
pairwise independent if Ei and Ej are independent for i 6= j.
Def: A finite collection of events E1 , E2 , . . . , En are said to be
independent(or mutually independent) if for any sub-collection
En1 , . . . , Enk of E1 , E2 , . . . , En ,
k
\ k
Y
P Enk = P(Eni ) .
i=1 i=1
Def: A countable collection of events E1 , E2 , . . . are said to be
independent if any finite sub-collection are independent.
Remarks
I To verify the independence of E1 , E2 , . . . , En we must check
2n − n − 1 conditions. For example, for n = 3, the conditions that
need to be checked are
P(E1 ∩ E2 ) = P(E1 )P(E2 ), P(E1 ∩ E3 ) = P(E1 )P(E3 ), P(E2 ∩ E3 ) =
P(E2 )P(E3 ), P(E1 ∩ E2 ∩ E3 ) = P(E1 )P(E2 )P(E3 ).
I Independence implies pairwise independence.
I Pairwise independence does not imply independence in general.
Example 1: Let S = {HH, HT , TH, TT }. Suppose all elementary
events are equally likely. Let E1 = {HH, HT }, E2 = {HH, TH} and
E3 = {HH, TT }. Then E1 , E2 , E3 are pairwise independent but not
independent because
1/4 = P(E1 ∩ E2 ∩ E3 ) 6= P(E1 )P(E2 )P(E3 ) = 1/8 .
I P(E1 ∩ E2 ∩ E3 ) = P(E1 )P(E2 )P(E3 ) is also not sufficient.
Example 2: Let S = {(i, j) : i = 1, . . . , 6, j = 1, . . . , 6}. Suppose all
elementary events are equally likely. Define E1 = {1st roll is 1, 2 or 3},
E2 = {1st roll is 3, 4 or 5} and E3 = {Sum of the rolls is 9}.
Def: Given an event C two events A and B are said to be
conditionally independent if P(A ∩ B|C ) = P(A|C )P(B|C ).
Example 3: A box contains two coins: a fair coin and one fake
two-headed coin (P(H)=1). You choose a coin at random and toss it
twice. Define the following events.
A= First coin toss results in a H. B= Second coin toss results in a H.
C= Coin 1 (regular) has been selected.
Then A and B are conditionally independent given C . Are A and B
independent?
