Chapter 4: Joint & Conditional Probability Distributions

4.1. Joint and Marginal Distributions

4.2. Conditional Distributions and Independence
4.3. Expectation
4.3.1. Covariance and Correlation
4.3.2. Conditional Expectation
4.3.3. Independence and Expectation
4.1. Joint and Marginal Distribution
4.1.1. Joint Probability Distribution:
❑ If X and Y are two random variables, the
probability distribution that defines their
simultaneous behavior is called a joint
probability distribution
(a) Joint Probability Distribution – Continuous Case
Definition: Two random variable are said to have joint
probability density function f(x,y) if

1. 0 ≤ 𝑓 𝑥, 𝑦
∞ ∞

2. න න 𝑓 𝑥, 𝑦 𝑑𝑥𝑑𝑦 = 1
−∞ −∞

3. 𝑃 𝑋, 𝑌 ∈ 𝐴 = න න𝑓 𝑥, 𝑦 𝑑𝑥𝑑𝑦
𝐴
Example: Movement of a particle

Assume that a particle moves within the region A bounded by the x -

axis, the line x = 1, and the line y = x.

Let (X, Y ) denote the position of the particle at a given time. The joint
density of X and Y is given by

f X Y (x, y) = 8xy for (x, y) ∈A

a) Find P (0.5 < X < 1, 0 < Y <0.5)

b) Find P (0 < X < 0.5, 0 < Y <0.5)

(b). The joint probability function – Discrete Case
p(x,y) = P[X = x, Y = y]

1. 0 ≤ 𝑝 𝑥, 𝑦 ≤ 1

2. ෍ ෍ 𝑝 𝑥, 𝑦 = 1
𝑥 𝑦

3. 𝑃 𝑋, 𝑌 ∈ 𝐴 = ෍ ෍ 𝑝 𝑥, 𝑦 𝑥, 𝑦 ∈ 𝐴
 Example 1: Plastic covers for CDs (Discrete joint pmf)

Measurements for the length and width of a rectangular plastic

covers for CDs are rounded to the nearest m m (so they are
discrete).
Let X denote the length and
Y denote the width.
The possible values of X are 129, 130, and 131 mm. The possible values
of Y are 15 and 16 m m (Thus, both X and Y arediscrete).

a) Find the probability that a CD cover has

length of 129mm
b) What is the probability distribution of X
and Y?
Example 2:
Consider the following table for two discrete random variables, which
gives P (X = x, Y = y).
y
1 2 3
1 0 1/6 1/6
x 2 1/6 0 1/6
3 1/6 1/6 0
4.1.2. Marginal Distributions
• The individual probability distribution of a random variable is referred to as its
marginal probability distribution.
• Marginal distributions describe how one variable behaves ignoring the other
variable.
(a). Marginal Distributions - Discrete Case
Let X and Y denote two discrete random variables with joint probability function
p(x,y) = P[X = x, Y = y]
Then
pX(x) = P[X = x] is called the marginal
probability function of X.
and
pY(y) = P[Y = y] is called the marginal
probability function of Y.
Note: Let y1, y2, y3, … denote the possible values of Y.
𝑝𝑋 𝑥 = 𝑃 𝑋 = 𝑥

= 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦1 ∪ 𝑋 = 𝑥, 𝑌 = 𝑦2 ∪ ⋯

= 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦1 + 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦2 + ⋯

= 𝑝 𝑥, 𝑦1 + 𝑝 𝑥, 𝑦2 + ⋯

= ෍ 𝑝 𝑥, 𝑦𝑗 = ෍ 𝑝 𝑥, 𝑦
𝑗 𝑦

Thus the marginal probability function of X, pX(x) is obtained

from the joint probability function of X and Y by summing
p(x,y) over the possible values of Y.
 Also
𝑝𝑌 𝑦 = 𝑃 𝑌 = 𝑦

= 𝑃 𝑋 = 𝑥1 , 𝑌 = 𝑦 ∪ 𝑋 = 𝑥2 , 𝑌 = 𝑦 ∪ ⋯

= 𝑃 𝑋 = 𝑥1 , 𝑌 = 𝑦 + 𝑃 𝑋 = 𝑥2 , 𝑌 = 𝑦 + ⋯

= 𝑝 𝑥1 , 𝑦 + 𝑝 𝑥2 , 𝑦 + ⋯

= ෍ 𝑝 𝑥𝑖 , 𝑦 = ෍ 𝑝 𝑥, 𝑦
𝑖 𝑥

Thus the marginal probability function of Y, pY(x) is obtained

from the joint probability function of X and Y by summing
p(x,y) over the possible values of X.
 Example 1: Find the marginal probability distribution of X and Y – discrete case
y
1 2 3 Mx
1 0 1/6 1/6
x 2 1/6 0 1/6
3 1/6 1/6 0
My

Example 2: Suppose we consider a survey given to 100 students in a Basic Algebra course at
ECC, with the following responses to the statement "I enjoy math."
1) Marginal distribution for gender
2) Marginal distribution for whether or not the
student enjoys math
3) What proportion of women strongly agreed with
the statement "I enjoy math"? 0.05
4) What proportion of women disagreed?
5) What proportion of men were neutral? 0.06
6) What proportion of men strongly agreed? 0.10
Example: Marginal distribution – discrete case
(b). Marginal Distributions – Continuous Case

Let X and Y denote two random variables with joint

probability density function f(x,y) then
∞

the marginal density of X is 𝑓𝑋 𝑥 = න 𝑓 𝑥, 𝑦 𝒅𝒚

−∞

∞
the marginal density of Y is 𝑓𝑌 𝑦 = න 𝑓 𝑥, 𝑦 𝒅𝒙
−∞
4.2. Conditional Distributions & Independence
4.2.1. Conditional Distributions
❑ Conditional distributions describe how one variable behaves
when the other variable is held fixed

(a). Conditional Distributions - Continuous Case

Let X and Y denote two random variables with joint probability density function
f(x,y) and marginal densities fX(x), fY(y) then
𝑓 𝑥, 𝑦
• the conditional density of Y given X = x 𝑓𝑌ȁ𝑋 𝑦ȁ𝑥 =
𝑓𝑋 𝑥

𝑓 𝑥, 𝑦
• the conditional density of X given Y = y 𝑓𝑋ȁ𝑌 𝑥 ȁ𝑦 =
𝑓𝑌 𝑦
(b) Conditional Distributions – Discrete Case
❑ Let X and Y denote two discrete random variables
with joint probability function
p(x,y) = P[X = x, Y = y]
Then
pX |Y(x|y) = P[X = x|Y = y] is called the conditional
probability function of X given Y = y

and
pY |X(y|x) = P[Y = y|X = x] is called the conditional
probability function of Y given X = x
…discrete case…
𝑝𝑋ȁ𝑌 𝑥 ȁ𝑦 = 𝑃 𝑋 = 𝑥ȁ𝑌 = 𝑦

𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝑝 𝑥, 𝑦
= =
𝑃 𝑌=𝑦 𝑝𝑌 𝑦
and
𝑝𝑌ȁ𝑋 𝑦ȁ𝑥 = 𝑃 𝑌 = 𝑦ȁ𝑋 = 𝑥

𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝑝 𝑥, 𝑦
= =
𝑃 𝑋=𝑥 𝑝𝑋 𝑥
• Conditional Independence
• Example: Continuing the plasticcovers...

a) Find the probability that a CD cover has a length of 130mm GIVEN the width
is 15mm.
b) Find the probability that a CD cover has width of 16 mm GIVEN the
length is 131 mm
Example
Let X & Y denote jointly distributed random variables with
joint density function then

𝐾 𝑥2 + 𝑦 0 ≤ 𝑥 ≤ 1,0 ≤ 𝑦 ≤ 1
𝑓 𝑥, 𝑦 = ቊ
0 otherwise

a) Find the value of K.

b) Determine the marginal distributions of X and Y.
c) The conditional distribution of 𝑓(𝑦ȁ𝑋 = 𝑥)
d) The conditional distribution of 𝑓(𝑥ȁ𝑌 = 𝑦)
 Conditional Mean and Independence
Conditional Mean and Variance - for CONTINUOUS random variables
The conditional mean of Y given X =x , denoted as E(Y |x) or µY |x,is
E(Y |x) = ‫𝑦 ׬‬f Y |x(y) dy

• Example 1: Conditional distribution - suppose (X, Y ) has a probability

density function f X Y (x, y) = x +y for 0 < x < 1, 0 < y < 1

a) Find f Y |x (y).
b) What is the conditional mean of Y given X = 0.5 (i.e., E(Y |x=0.5)?
4.2.2. Independence
• X is independent of Y means that knowing Y does not change our belief
about X.
• P(X|Y=y) = P(X)
• P(X=x, Y=y) = P(X=x) P(Y=y)
• The above should hold for all x, y
• It is symmetric and written as X ⊥ Y
(a). Independence – Discrete Case
Example: Conditional Independence - discrete case
A company that services air conditioner units in residences and office
blocks is interested in how to schedule its technicians in the most
efficient manner
• The random variable X, taking the values 1,2,3 and 4, is the service
time in hours
• The random variable Y, taking the values 1,2 and 3, is the number
of air conditioner units
a) Find the marginal distributions of X & Y
b) Does P Y |x (y) = P Y (y) for all x & y?
c) Does P X Y (x, y) = P X (x) P Y (y) for all x & y?
Conditional Independence – continuous case

Example:
Suppose the joint distribution of X and Y is f(x,y) = 1, where x is
in [0,1] and y is [0,1]. Are X and Y independent ?
4.3. Expectation
Mean from a Joint Distribution

• If X and Yare continuous random variables with joint probability density

function f X Y (x, y), then 𝐸 𝑋 = ‫𝑥 𝑌𝑋𝑓𝑥 ׬ ׬‬, 𝑦 𝑑𝑦𝑑𝑥

Example: Consider f X Y (x, y) = 8xy, for 0 < x <1 and 0.5 < y <1

Find E(X)
4.3.1. Covariance
❑ Covariance between X and Y refers to a measure of how much
two variables change together.

❑ Covariance indicates how two variables are related.

❑ A positive covariance means the variables are positively

related, while a negative covariance means the variables are
inversely related.

❑ The formula for calculating covariance of sample data is shown

below.
Covariance
Example: Covariance (refer to air conditioner maintenance)

Find
1. E(X) 4. Cov (X,Y)
2. E(Y) 5. Var (X)
3. E(XY) 6. Var (Y)
Correlation
• The most familiar measure of dependence between two
quantities is the Pearson product-moment correlation
coefficient, or "Pearson's correlation." It is obtained by
dividing the covariance of the two variables by the product of
their standard deviations.

• The correlation is a measure of the linear relationship between

X and Y.
• It is obtained by dividing the covariance by the product of the
two standard deviations, i.e.,
 n
Correlation
 (x i − x )( yi − y )
r ( x, y ) = i =1

var( xi − x ) var( yi − y )
Covariance vs. Correlation
• Covariance indicates the direction of the linear relationship
between variables while correlation measures both the
strength and direction of the linear relationship between
two variables.
Example: refer to air conditioner maintenance

Find the correlation between X and Y

END OF CHAPTER 4