Adama Science and Technology University

School of Electrical Engineering and Computing

Department of Electronics and Communication Engineering

Probability and Random Process (ECEg3103)

Chapter 3 Random Variables

By
Gemechu D.
Random Variables

Outline

 Introduction

 The Cumulative Distribution Function

 Probability Density and Mass Functions

2
Introduction
 A random variable X is a function that assigns a real number
X(ω) to each outcome ω in the sample space Ω of a random
experiment.
 The sample space Ω is the domain of the random variable and the
set RX of all values taken on by X is the range of the random
variable.
 Thus, RX is the subset of all real numbers.


A
X ()  x
Re al Line
x B

3
Introduction Cont’d……
 If X is a random variable, then {ω: X(ω)≤ x}={X≤ x} is an event
for every X in RX.
Example: Consider a random experiment of tossing a fair coin three
times. The sequence of heads and tails is noted and the sample
space Ω is given by:
  {HHH , HHT , HTH , THH , THT , HTT , TTH , TTT}
Let X be the number of heads in three-coin tosses. X assigns
each possible outcome ω in the sample space Ω a number from
the set RX={0, 1, 2, 3}.
 : HHH HHT HTH THH THT HTT TTH TTT
X ( ) : 3 2 2 2 1 1 1 0

4
The Cumulative Distribution Function

 The cumulative distribution function (cdf) of a random variable X

is defined as the probability of the event {X≤ x}.
FX ( x)  P( X  x)
Properties of the cdf, FX(x):
 The cdf has the following properties.

i. FX ( x) is a non - negative function, i.e.,

0  FX ( x)  1
ii. lim FX ( x )  1
x 

iii. lim FX ( x )  0
x  

5
The Cumulative Distribution Function Cont’d…..

iv. FX ( x) is a non - decreasing function of X , i.e.,

If x1  x2 , then FX ( x1 )  FX ( x2 )

v. P( x1  X  x2 )  FX ( x2 )  FX ( x1 )
vi. P( X  x)  1  FX ( x)

Example:
Find the cdf of the random variable X which is defined as the
number of heads in three tosses of a fair coin.

6
The Cumulative Distribution Function

Solution:
 We know that X takes on only the values 0, 1, 2 and 3 with
probabilities 1/8, 3/8, 3/8 and 1/8 respectively.
 Thus, FX(x) is simply the sum of the probabilities of the
outcomes from the set {0, 1, 2, 3} that are less than or equal to x.
0, x  0
1 / 8, 0  x  1

 FX ( x)  1 / 2, 1  x  2
7 / 8, 2  x  3

1, x  3

7
Types of Random Variables
 There are two basic types of random variables.
i. Continuous Random Variable
 A continuous random variable is defined as a random variable
whose cdf, FX(x), is continuous every where and can be written as
an integral of some non-negative function f(x), i.e.,

FX ( x)   f (u )du


ii. Discrete Random Variable

 A discrete random variable is defined as a random variable whose
cdf, FX(x), is a right continuous, staircase function of X with
jumps at a countable set of points x0, x1, x2,……

8
The Probability Density Function
 The probability density function (pdf) of a continuous random
variable X is defined as the derivative of the cdf, FX(x), i.e.,
dFX ( x)
f X ( x) 
dx
Properties of the pdf, fX(x):

i. For all values of X , f X ( x)  0


ii. 

f X ( x)dx  1
x2
iii. P ( x1  X  x2 )   f X ( x)dx
x1

9
The Probability Mass Function
 The probability mass function (pmf) of a discrete random
variable X is defined as;

PX ( X  xi )  PX ( xi )  FX ( xi )  FX ( xi 1 )

Properties of the pmf, PX (xi ):

i. 0  PX ( xi )  1, k  1, 2, .....

ii. PX ( x)  0, if x  xk , k  1, 2, .....

iii. Pk
X ( xk )  1

10
Calculating the Cumulative Distribution Function

 The cdf of a continuous random variable X can be obtained by

integrating the pdf, i.e.,
x
FX ( x)   f X (u )du


 Similarly, the cdf of a discrete random variable X can be obtained by

using the formula:

FX ( x)  P
xk  x
X ( xk )

11
 Two Random Variables
 In many applications, it is very important to study two or more
random variable defined on the same sample space.
 In this lecture, we will consider only two random variables.
Let Ω be the sample space of a random experiment and let X and
Y be two random variables.
 Then, the pair (X, Y) is called a two-dimensional random variable if
each of X and Y associates a real number with every element of Ω.
 Thus, a two-dimensional random variable (X, Y) is a function that
assigns a point (x, y) in the xy-plane to each possible outcome ω in
the sample space.

30
 Example: 1
Consider the experiment of tossing a coin twice. The
sample space is
S = {HH, HT, TH, TT}.

Let X denotes the number of heads obtained in the first toss

and Y denote the number of heads in the second toss. Then

(X, Y) is a two-dimensional random variable or Bivariate random variable.

The range space X is Rx, = (0, 1) and Y is RY, = (0, 1)

The range space of RXY is {(1,1), (1,0), (0,1), (0,0)} which is finite and so (X, Y) is a two-
dimensional discrete random variables.

13
 Example 2
Consider an experiment of tossing a fair coin twice. Let (X, Y) be a bivariate r.v.,
where X is the number of heads that occurs in the two tosses and Y is the
number of tails that occurs in the two tosses.
(a) What is the range Rx of X, Ry of Y?
(b) Find and sketch the range Rxy of (X, Y).
(c) Find P(X = 2, Y = 0), P(X = 0, Y = 2), and P(X = 1, Y = 1).
The sample space S of the experiment is S = {HH, HT, TH, TT) (a) Rx, = (0, 1,2)
, RY, = (0, 1,2)
(b) RXY = ((2, O), (1, I), (0, 2)) which is sketched in Fig. 3-2.
(c) Since the coin is fair, we have
P(X = 2, Y = 0) = P(HH} = 1/4 P(X = 0, Y = 2) = P{TT) = 1/4 P(X= 1, Y = 1)= P{HT,
TH} = 1/2

14
 The Joint Cumulative Distribution Function
 The joint CDF of two random variables X and Y denoted by
FXY(x, y) is a function defined by:

FXY (x, y)  P[ X ()  x and Y ()  y]

 FXY (x, y)  P( X  x, Y  y)
where x and y are arbitraryreal numbers.
Properties of the Joint CDF, FXY(x, y):

i. 0  FXY (x, y)  1
ii. lim
x
F XY ( x, y)  F XY (, )  1
y

lim F XY ( x, y)  F XY (, )  0
iii. x
y  
15
The Joint Cumulative Distribution Function Cont’d…

iv. lim F XY ( x, y)  F XY ( , y)  0

x 

v. lim F X Y ( x, y)  F X Y ( x,)  0
y 

v i . P ( x1  X  x 2 , Y  y )  F XY ( x 2 , y )  F XY ( x1 , y)

v i i . P ( X  x, y1  Y  y 2 )  F XY ( x, y 2 )  F XY ( x, y1 )

vii. P ( x1  X  x 2 , y1  Y  y 2 )  F XY ( x 2 , y 2 )

 F XY ( x 2 , y1 )  F XY ( x1 , y 2 )  F XY ( x1 , y1 )

16
The Joint Probability Density Function

The joint probability function (PDF) of two continuous random

variables X and Y is defined as:

 FXY (x, y)
2
f XY (x, y) 
xy

 Thus, the joint cumulative distribution function (CDF) is given

by:

FXY (x, y)  - - f XY (x, y)dxdy

y x

17
The Joint Probability Density Function Cont’d…..

Properties of the Joint pdf, fXY(x, y):

1. f XY (x, y)  0
 
2.  
- -
f XY (x, y)dxdy  1
y2 x2

3. P(x1  X  x2 , y1  Y  y2 )  y 1 
x1
f XY (x, y)dxdy

18
The Joint Probability Mass Function
The joint probability mass function (pmf) of two discrete random
variables X and Y is defined as:
PXY (xi , y j )  P( X  xi , Y  y j )
The joint cdf can be written as:

FXY (x, y)   P XY (xi , y j )

xi  x y j  y

Properties of the Joint pmf, PXY (xi , yj ):

1. 0  PXY (xi , y j )  1
2.  P XY (xi , y j )  1
xi yj

19
 Conditional Distributions
i. Conditional Probability Density Functions
 If X and Y are two continuous random variables with joint pdf
fXY(x, y), then the conditional pdf of Y given that X=x is defined by:

f XY ( x, y)
f ( y / x)  ,
Y/X f X ( x)  0
f X ( x)

 Similarly, the conditional pdf of X given that Y=y is defined by:

f XY ( x, y)
f( x / y)  ,
X /Y f Y ( y)  0
f Y ( y)

20
 Condition Distributions Cont’d……
ii. Conditional Probability Mass Functions
 If X and Y are two discrete random variables with joint pmf
PXY(xi , yj), then the conditional pmf of X given that Y=yj is
defined by:
PXY ( x i , y j )
PX / Y ( x i / y j )  , PY ( y j )  0
PY ( y j )
 Similarly, the conditional pmf of Y given that X=xi is defined by:

PXY ( x i , y j )
PY / X ( y j / x i )  , PX ( x i )  0
PX ( x i )

21
 Marginal Statistics of Two Random Variables

 In the case of two or more random variables, the statistics of

each individual variable are called marginal statistics.
i. Marginal cdf of X and Y
F X ( x)  lim F XY ( x, y)  F XY ( x, )
y

FY ( y)  lim F XY ( x, y)  F XY (, y)
x 

ii. Marginal pdf of X and Y


f X ( x)   -
f XY ( x, y )dy

fY ( y)   -
f XY ( x, y)dx
40
Marginal Statistics of Two Random Variables Cont’d…..

iii. Marginal PMF of X and Y

P( X  x i )  PX ( x i )  P XY ( xi , y i )
yj

P(Y  y j )  PY ( y j )   PXY ( x i , y i )
xi

23
 Independence of Two Random Variables

 If two random variables X and Y are independent, then

i. from the joint cdf

F XY ( x, y)  F X ( x) FY ( y )

ii. from the joint pdf

f XY ( x, y)  f X ( x) f Y ( y)

iii.from the joint pmf

PXY ( xi , y j )  PX ( xi )PY ( y j )
24
 Examples on Two Random Variables

Example-1:The joint pdf of two continuous random variables X and Y is

given by:
kxy , 0  x  1, 0  y  1
f XY (x, y)  
0 , otherwise
where k is a constant.
a. Find the value of k.
b. Find the marginal pdf of X and Y.
c. Are X and Y independent?
d. Find P( X  Y  1)
e. Find the conditiona l pdf of X and Y.

25
Examples on Two Random Variables Cont’d……
Solution:
  1 1
a.  
- 
f XY (x, y)dxdy  1    kxydxdy  1
0 0

1  x2 1
 k 0 
y  2  0  1

k 1  y 2 1 k
   1
2 0 ydy  k  
 4  0 4
k  4

26
Examples on Two Random Variables Cont’d……
Solution:

b. Marginal pdf of X and Y

i. Marginal pdf of X
 1
f X (x)   f XY (x, y)dy  0 4xydy
 y2 1
 f X (x)  4x    2x
 2 0
2x, 0x1
 f X (x)  
0, otherwise

27
Examples on Two Random Variables Cont’d……
Solution:

b. Marginal pdf of X and Y

ii. Marginal pdf of Y
 1
f Y ( y)   f XY (x, y)dx  0 4xydx
 x2 1
 f Y ( y)  4 y    2 y
 2 0
 2 y, 0y1
 f Y ( y)  
0, otherwise

28
 Examples on Two Random Variables Cont’d……
Solution:
c. f XY (x, y)  f X (x) fY ( y)
 X and Y are independent
1 1 y 1  x2 1
d. P( X  Y  1)  0 0 4xydxdy  0 4 y 2  0dy
1 1
  4 y[1/ 2(1 y) ]dy   2( y  2 y  y )dy
2 2 3
0 0

 2( y 2 / 2  2 y3 / 3  y 4 / 4)  1/ 6
 P( X  Y  1)  1/ 6

29
 Examples on Two Random Variables Cont’d……
Solution:

e. Conditiona l pdf of X and Y

ii. Conditiona l pdf of Y
f XY (x, y) 4xy
f ( y / x)    2y
Y/X
f X (x) 2x

2 y, 0  x  1, 0 
 fY / X ( y / x)  
0, y1
otherwise

30
 Examples on Two Random Variables Cont’d……

Example-2:
The joint pdf of two continuous random variables X and Y is
given by:
0<y<1,
k, 0  y  x  1
f X (x)   y<x<1
0, otherwise
where k is a constant.
a. Determine the value of k.
b. Find the marginal pdf of X and Y.
c. Are X and Y independent?
d. Find P(0  X  1/ 2)
e. Find the conditiona l pdf of X and Y.
31
 Examples on Two Random Variables Cont’d……
Solution:
  1 1
a.  
-  f XY (x, y)dxdy  1  0 y kdxdy  1
1 1
 k  x   1
0 y
1  y 2  1  k  1
 k 0 (1 y)dy  k  y  2 
 0
k  2 2

32
Examples on Two Random Variables Cont’d……
Solution:

b. Marginal pdf of X and Y

i. Marginal pdf of X
 x
f X (x)   f XY (x, y)dy  0 2dy

 f (x)  2 y   2x
x
X
0
2x, 0  x 1
 f X (x)  
0, otherwise

33
Examples on Two Random Variables Cont’d……
Solution:

b. Marginal pdf of X and Y

ii. Marginal pdf of Y
 1
f Y ( y)   f XY (x, y)dx  y 2dx

 f ( y)  2x   2(1  y)
1
Y
y
2(1  y), 0y1
 f Y ( y)  
0, otherwise
34
Examples on Two Random Variables Cont’d……
Solution:
c. f XY (x, y)  f X (x) fY ( y)
 X and Y are not independent
1/ 2 x

d. P(0  X  1/ 2)  0 
0
f XY (x, y)dydx
1/ 2 x 1/ 2 x
 0  2dydx  
0 0
(2 y) dx
0
1/ 2 1/ 2
 0 2xdx  x2 0  1/ 4

 P(0  X  1/ 2)  1/ 4
35
 Examples on Two Random Variables Cont’d……
Solution:

e. Conditiona l pdf of X and Y

i. Conditiona l pdf of X
f XY (x, y)
f (x / y)   2 1
X /Y 
fY ( y) 2(1 y) (1 y)
 1
 , 0  y  x 1
 f X /Y (x / y)  1
y  otherwise
0,
36
 Examples on Two Random Variables Cont’d……
Solution:

e. Conditiona l pdf of X and Y

ii. Conditiona l pdf of Y
f XY (x, y) 2 1
f ( y / x)   
Y/X
f X (x) 2x x
1 ,
 0yx
 fY / X ( y / x)   x
0,
1 otherwise

37
 Examples on Two Random Variables Cont’d……

Example-3:
The joint pmf of two discrete random variables X and Y is given
by:

k(2xi  y j ) , xi  1, 2; y  1, 2
PXY (xi , y j )  
0 , otherwise
where k is a constant.
a. Find the value of k.
b. Find the marginal pmf of X and Y.
c. Are X and Y independent?

38
Examples on Two Random Variables Cont’d……
Solution:

a.  P XY (xi , y j )  1
xi yj

  k (2x i  y j)  1
xi 1 y j 1

 k[(2 1)  (2  2)  (4 1)  (4  2)]  1

 18k  1

 k  1/18
39
Examples on Two Random Variables Cont’d……
Solution:
b. Marginal pmf of X and Y
i. Marginal pmf of X
2
PX (xi )   PXY (xi , y j )   (2x i  y j )
1
yj y j 1 18
1 1
 Px (xi )  (2xi 1)  (2xi  2)
18 18
 1 (4x  3),
xi  1, 2
i

 PX (xi )  18
0, otherwise
40
Examples on Two Random Variables Cont’d……
Solution:
b. Marginal pmf of X and Y
ii. Marginal pmf of Y
2
PY ( y j )   PXY (xi , y j )   (2x i  y j )
1
xi xi 1 18
1 1
 P ( y )  (2  y )  (4  y )
Y j j j
18 18
 1 (2 y
 j  6), y j  1, 2
 PY ( y j )  18
0, otherwise
41
 Examples on Two Random Variables Cont’d……
Solution:

c. PXY (xi , y j )  PX (xi )PY ( y j )

 X and Y are not independent.

42