Communication

Systems

Random Process

By
Engr. Jawwad Ahmad (Ph.D.)

1
 Today’s Goal
 Overview of basic terminologies

 Random Process

 Types of RP

Engr. Dr. Jawwad Ahmad 2

 Probability Theory & Axioms
 Probability is the measure of the likelihood that an event will
occur.
 Probability theory establishes a mathematical framework for
the study of random phenomena.
 Probability can be reduced to three axioms. This was first
done by the mathematician Andrei Kolmogorov.

 Set of outcomes of any event A, called the sample space S.

Engr. Dr. Jawwad Ahmad 3

 Probability Theory & Axioms
 The probability of the event A, is denoted by P(A) such that

N ( A)
P ( A)  lim
 here, N(A) is the frequency of eventNA in N independent trails.
N 

 The first axiom is that the probability of any event is a

nonnegative real number.
P( A)  0
 The second axiom is that the probability of the entire sample
space is one.
P(S )  1

Engr. Dr. Jawwad Ahmad 4

 Probability Theory & Axioms
 The third axiom deals with mutually exclusive events.
If A1 and A2 are mutually exclusive or disjoint, meaning that
they have an empty intersection then their union is given as

P( A1  A2 )  P ( A1 )  P ( A2 )

A1 A2

Engr. Dr. Jawwad Ahmad 5

 Conditional Probability &
Independent Events
 One often comes across a situation where the probability of
one event is influenced by the outcome of another event.

 This type of condition comes under conditional probability.

 Denoted by P(B|A) termed as “probability of B given A”, that

is probability of event B when it is known that event A has
occurred.

 Mathematical expression is given as

P ( AB)

P( B | A) 
P ( A)
Engr. Dr. Jawwad Ahmad 6
 Conditional Probability &
Independent Events
 If two events have joint probability then it can be written as
P( AB)  P ( A | B) P ( B)  P( B | A) P( A)
 Two events are called independent event when they are
statistically independent, that is, they do not depend on each
other, represented as
P( AB)  P( A) P ( B)
Three fair coins are tossed simultaneously. The events are:
A : Exactly 2 Heads B: At Least 2 Heads C: At Most 2 Heads.
Find the probability that P(B|C) and P(C|B)? [3/7, ¾]

Engr. Dr. Jawwad Ahmad 7

 Motivation to study Random
Variables/Processes
 Communication System depends on random signals
(e.g. Information, channel, noise etc.)
 Treatment of signal properties (e.g. power, energy,
error probability) is dependent on analyzing random
signals.

A random variable is
“a number that you don’t know… yet”
Sam Savage, Stanford University

Engr. Dr. Jawwad Ahmad 8

 Random Variables
 Random Variable is a function that assigns real numbers
(values) to the outcomes of a random experiment under certain
well defined rule.
 A random variable X(A) represent the functional relationship
between a random event A and a real number.
 Despite the name, a random variable is neither random nor a
variable. Instead, it’s a function that generates numbers from the
outcomes of a chance experiment.

Engr. Dr. Jawwad Ahmad 9

 Fundamental Concept of Random Variables
Do
m
ai
n

Rule
Non-Animal = 0
Animal = 1
Range
0 1
 Random Experiment
 Domain (Sample Space)
 Function Rule
 Range
Engr. Dr. Jawwad Ahmad 10
 Discrete & Continuous RVs
 If sample space contains a countable
number of sample points, then random
variable is called discrete random
variable, that have a countable number of
distinct values.

 A continuous random variable may take

on any value within a certain range of
the real line, rather than being restricted to a
countable number of distinct points.

Engr. Dr. Jawwad Ahmad 11

 Characteristics of Random Variables
 Usually the outcome is considered under known range, say
random variable X, deal with numerical-valued events.

 If the numerical-value constant is replace by the independent

variable x, then probability functions came into being that helps to
calculate probabilities of numerical-valued events.

PX ( x)  P( X  x)

Engr. Dr. Jawwad Ahmad 12

 Characteristics of Random Variables
 There are two description / characterization of random variable.
 Probability Mass Function (PMF) or Probability Density Function
(PDF).

 Commutative Distribution Function (CDF).

 In PMF/PDF characterization all probabilities are mentioned

with which the random variable assigns all possible values.
 CDF is a function in which probabilities
 are
i added.

  PX ( xk )
 k  
FX ( x )  PX ( X  x )   x
 PX ( ) d
 
 
Engr. Dr. Jawwad Ahmad 13
 Characteristics of Random Variables
 Three fair coins are tossed simultaneously. Assuming random
variable is X.
 Domain of X : {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}

 Rule : Assigns Number of Heads

 Range : {0, 1, 2, 3} X 0 1 2 3
P(X) 1/8 3/8 3/8 1/8
 Another useful function
F(X) 1/8 4/8 7/8 8/8
relating CDF and PDF.

dFX ( x)
PX ( x) 
dx

Engr. Dr. Jawwad Ahmad 14

Engr. Dr. Jawwad Ahmad 15
 Parameters of Random Variable
Ensemble Average / Mean Value
 The first moment of a probability distribution of a random
variable X is called mean value mX, or expected value of a random
variable X, normally denoted by E[X].
n 
E[ X ]  mx  X   xi pi   x p X ( x) dx
i1   
    
Discrete
Continuous
Mean Square Value
 The second moment of a probability distribution is the mean-
square value of X, denoted by E[X2], gives average of square
values. n 
E[ X 2 ]  X 2   xi2 pi   x 2 p X ( x) dx
i1   
    
Discrete Continuous

Engr. Dr. Jawwad Ahmad 16

 Parameters of Random Variable
Variance
 Central moments are the moments of the difference between X
and mX and the second central moment is the variance of X.

 It gives the spread in the values assigned by random variable.

2
VarX  E[ X ]  E [ X ]  X  X
2 2 2

n 
  ( xi  m X ) 2 pi   ( x  m X ) 2 p X ( x) dx
i 1       
       
Discrete Continuous

Engr. Dr. Jawwad Ahmad 17

 Parameters of Random Variable
Problem
 Three fair coins are tossed simultaneously. Assuming random
variable is X.
 Domain of X : {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}

 Rule : Assigns Number of Heads

X 0 1 2 3
 Range : {0, 1, 2, 3}
P(X) 1/8 3/8 3/8 1/8
E[ X ]  1.5
E[ X ]  5.25
2 X2 0 1 4 9

VarX  0.67

Engr. Dr. Jawwad Ahmad 18

 Types of Random Variable
 There are many types of random variables depending upon their
distributions.
 Some of them are listed here:
 Uniform Random Variable
 Binomial Random Variable
 Poisson Random Variable
 Normal / Gaussian Random Variable
 Exponential Random Variable
 Rayleigh Random Variable
 Laplace Random Variable

Engr. Dr. Jawwad Ahmad 19

 Random / Stochastic Process
 Stochastic process maps random experimental outcomes
into real functions of time.
 Also defined as “time indexed random variable”.
 A random process Ψ(X, t) can be viewed as a function of
two variables: a random event X and time.

Engr. Dr. Jawwad Ahmad 20

 Random / Stochastic Process
 The collection of time functions is known as Ensemble and
each member is called a Sample Function.

Engr. Dr. Jawwad Ahmad 21

 Ensemble Averages
Ensemble Average / Mean Value
 The expected value of a random process Ψ, normally denoted by
E[Ψ]. 
E[ ( x, t )]    ( x, t ) p X ( x) dx


Mean Square Value

 The second moment of a random process is the mean-square
value of Ψ, denoted by E[Ψ 2].

E[ ( x, t )]  
2 2
( x, t ) p X ( x) dx


Engr. Dr. Jawwad Ahmad 22

 Ensemble Averages
Variance
 Central moments are the moments of the difference between Ψ
and its mean and the second central moment is the variance of Ψ.
Var   E[  ]  E [ ] 2 2

Auto-Correlation
 It is a parameter that gives the measurement of interdependence.
It is denoted by RΨ (t1, t2).

R (t1 , t 2 )  E[ ( x, t1 )   ( x, t 2 )]

Engr. Dr. Jawwad Ahmad 23

 Random Process
Problem
 A random process is defined by V(X, t) = 3X + t where X is a
random variable with zero mean and mean square value of 3. Find
the mean, mean square and co-relation function for the process.
E[V ( X , t )]  E[3 X  t ]
 E[3 X ]  E[t ]
 3E[ X ]  t
 3 0  t
E[V ( X , t )]  t

Engr. Dr. Jawwad Ahmad 24

 Random Process
Problem

E[ 3 X  t  ]
2
2
E[V ( X , t )] 
 E[9 X ]  2 E[3 X t ]  E[t ]
2 2

 9 E[ X 2 ]  6 E[ X ]E[t ]  t 2
 93  6 0t  t2
E[V 2 ( X , t )]  27  t 2

Engr. Dr. Jawwad Ahmad 25

 Random Process
Problem
RV [t1  t 2 ]  E[V  3 X  t1  V  3 X  t 2  ]
 E[9 X  3 X  t1  t 2   t1  t 2 ]
2

 9 E[ X ]  3E[ X ]E[t1  t 2 ]  E[t1  t 2 ]

2

 9  3  3  0   t1  t 2   t1  t 2
RV [t1  t 2 ]  27  t1  t 2
It is time
dependent process.
 Suppose t 1 = t2 = t

RV [t 2 ]  27  t 2  E[V 2 ( X , t )]

Engr. Dr. Jawwad Ahmad 26

Convolution Cross-correlation Autocorrelation

Engr. Dr. Jawwad Ahmad 27

 Stationary Random Process
 Stationary process is a random process, the mean of which is
independent of time.
 That is the mean of the process should be independent of time.

 Consider a three phase waveform with 120o phase difference .

 If we consider Φ as a random variable having uniform

distribution over 0 to 2π then random process is defined as

V (t , )  A cos[ct  ]

Engr. Dr. Jawwad Ahmad 28

 Stationary Random Process
 To check the process for stationary, compute its mean.

E V (t , )
2
 
0
A cos[c t   ] p ( )d
2 1
 
0
A cos[c t   ] d
2
A
E V (t , )
2

2 0
cos[ct   ] d
c t    z
A
E V (t , )
 c t  2

2 c t
cos[ z ] dz
 Let then dΦ = dz while zL = ωct and zU = ωct + 2 .
Engr. Dr. Jawwad Ahmad 29
 Stationary Random Process
 A
EV (t , )
 c t  2
   sin[ z ] ct
 2 
 A
EV (t , )     sin  ct  2   sin  ct  
 2 
 Since sin ωct + 2π = sin ωct
 A
EV (t , )     sin  ct   sin  ct  
 2 
EV (t , )  0

 Hence the process is Stationary.

Engr. Dr. Jawwad Ahmad 30
 Ergodic Random Process
 Ergodic process is a random process in which time average
equal to the ensemble average.
 That is the power of the process should be independent of
time.
 For the process to be Ergodic, it must be Stationary.

 Again, consider the three phase motor problem and

calculate its power.

Engr. Dr. Jawwad Ahmad 31

 Ergodic Random Process

E V (t ,  )
2
  
0
2
A cos [c t   ] p ( )d
2 2

2 1
 0 A cos [ct  ] 2 d
2 2

2
A 2 1
  1  cos[ 2c t  2] d
2 0 2

 
2 2
A 2  1 A 2
E V (t ,  )   d   cos[ 2c t  2 ] d
2

2 0 2 2 0

Engr. Dr. Jawwad Ahmad 32

 Ergodic Random Process
 Since integral of sinusoid over a whole period is zero,
therefore


E V 2 (t ,  )  
A2 2
2 2


 
2
A
E V 2 (t ,  )   0
 Hence the process is Ergodic. 2

Engr. Dr. Jawwad Ahmad 33

 Autocorrelation of an Energy Signal
 The autocorrelation function of a real-valued energy signal has the
following properties:

 R x (t ) =R x (-symmetrical
t) in about zero

 R x (t ) �R x (0)maximum
for all tvalue occurs at the origin

 autocorrelation and ESD form a Fourier

R x (t ) � y x (f)
transform pair, as designated by the double-
headed arrows value at the origin is equal to the energy of
�
 � (t) dt
2
R x (0)
the signal. x
�

Engr. Dr. Jawwad Ahmad 34

 Autocorrelation of an Energy Signal
 Correlation is a matching process; autocorrelation refers to the matching of
a signal with a delayed version of itself.

 Autocorrelation function of a real-valued energy signal x(t) is defined as:

�
R x (t ) = x(t) x (t + t ) dt
� for -� < t < �
�
 The autocorrelation function Rx(τ) provides a measure of how closely the
signal matches a copy of itself as the copy is shifted

 τ units in time.

 Rx(τ) is not a function of time; it is only a function of the time difference τ

between the waveform and its shifted copy.

Engr. Dr. Jawwad Ahmad 35

 Autocorrelation of a Power Signal
 Autocorrelation function of a real-valued power signal x(t) is defined as:

T /2
1
R x (t )  lim � x(t) x (t + t ) dt for -� < t < �
T �� T T / 2

 When the power signal x(t) is periodic with period T0, the autocorrelation
function can be expressed as
T0 / 2
1
R x (t )  �x(t) x (t + t ) dt for -� < t < �
T0 T0 / 2

Engr. Dr. Jawwad Ahmad 36

 Autocorrelation of a Power Signal
 The autocorrelation function of a real-valued periodic signal has the
following properties similar to those of an energy signal:

 R x (t ) =R x (-symmetrical
t) in about zero

 R x (t ) �R x (0)maximum
for all t value occurs at the origin

 R x (t ) � Gx (f)
autocorrelation and PSD form a
Fourier transform pair
T0 / 2
 1 value2 at the origin is equal to the
R x (0)  �
T  T0the
x (t) dt
average power0 of / 2 signal

Engr. Dr. Jawwad Ahmad 37

Properties of LTI System

Engr. Dr. Jawwad Ahmad 38

Relation Between Correlation & PSD

Engr. Dr. Jawwad Ahmad 39

Thank you

Engr. Dr. Jawwad Ahmad 40