Wong & Lok: Theory of Digital Communications 1.

Review

Chapter 1

Review on Background Materials

We start by reviewing some important concepts which will be needed in the following chapters.

1.1 Signals and Systems

1.1.1 De£nitions

Continuous-time signal A continuous-time signal x(t) is a function for which the independent vari-
able, namely time t, takes the real numbers.

Discrete-time signal A discrete-time signal x(n) is a sequence, i.e., the independent variable n takes
only the integers.

System A system is an operator that takes a signal as its input and produces a signal as its output.

Continuous-time system A continuous-time system is a system whose inputs and outputs are continuous-
time signals.

Discrete-time system A discrete-time system is a system whose inputs and outputs are discrete-time
signals.

Consider continuous-time systems. (Similar for discrete-time systems.)

1.1
Wong & Lok: Theory of Digital Communications 1. Review

Linearity A system T is linear if and only if for any input signals x1 (t) and x2 (t) and any two scalars
α and β,
T [αx1 (t) + βx2 (t)] = αT [x1 (t)] + βT [x2 (t)].

Time-invariance A system is time-invariant if and only if for all x(t) and all values of t0 , its response
to x(t − t0 ) is y(t − t0 ), where y(t) is the response of the system to x(t).

Causality A system is causal if and only if its output at any time t0 depends on the input at times up
to (and possibly including) t0 only.

1.1.2 Fourier Transform

If a signal x(t) satis£es certain conditions, then the (continuous-time) Fourier transform of x(t), de-
noted by X(f ) is de£ned by
Z ∞
X(f ) = x(t)e−j2πf t dt. (1.1)
−∞

The signal x(t) can be recovered from its Fourier transform by taking the inverse Fourier transform
Z ∞
x(t) = X(f )ej2πf t df. (1.2)
−∞

Properties of the Fourier transform

1. Linearity
αx1 (t) + βx2 (t) ←→ αX1 (f ) + βX2 (f )

2. Duality
X(t) ←→ x(−f )

3. Convolution
x1 ∗ x2 (t) ←→ X1 (f )X2 (f )

4. Scaling (a 6= 0) Ã !
1 f
x(at) ←→ X
|a| a

1.2
Wong & Lok: Theory of Digital Communications 1. Review

5. Time-shift
x(t − t0 ) ←→ X(f )e−j2πf t0

6. Modulation
x(t)ej2πf0 t ←→ X(f − f0 )

7. Differentiation
dn
x(t) ←→ (j2πf )n X(f )
dtn
8. Parseval’s relation
Z ∞ Z ∞
x1 (t)x∗2 (t)dt = X1 (f )X2∗ (f )df
−∞ −∞

1.1.3 Delta function

The delta function is not a function in strict mathematical sense. Nevertheless, it is convenient to de£ne
and use it to obtain various results. The delta function δ(t) is “de£ned” by the following properties:

δ(t) = 0 for all t 6= 0 (1.3)

and
Z ∞
x(t)δ(t)dt = x(0) (1.4)
−∞

for any function x(t), which is continuous at t = 0.

Response of a linear time-invariant (LTI) system

The convolution of any signal x(t) with δ(t) is the original x(t)
Z ∞
x ∗ δ(t) = x(s)δ(t − s)ds = x(t). (1.5)
−∞

Consider an LTI system T [·]:

T [x(t)] = T [x ∗ δ(t)] (1.6)

·Z ∞ ¸
= T x(s)δ(t − s)ds
−∞
Z ∞
= x(s)T [δ(t − s)]ds
−∞
Z ∞
= x(s)h(t − s)ds (1.7)
−∞

1.3
Wong & Lok: Theory of Digital Communications 1. Review

where h(t) = T [δ(t)] is called the impulse response of the LTI system. Therefore, the response of an
LTI system to an input signal is the convolution of the input signal and the impulse response.

Some useful Fourier transform pairs involving the delta function

δ(t) ←→ 1

1 ←→ δ(f )

δ(t − t0 ) ←→ e−j2πf t0

ej2πf0 t ←→ δ(f − f0 )
1 1
cos(2πf0 t) ←→ δ(f − f0 ) + δ(f + f0 )
2 2
1 1
sin(2πf0 t) ←→ δ(f − f0 ) − δ(f + f0 )
2j 2j

Fourier transform of periodic signals

Let x(t) be a periodic signal of period T0 , and let {xn } be the Fourier coef£cients, i.e.,
∞
X
x(t) = xn ej2πnt/T0 . (1.8)
n=−∞

Taking Fourier transform on both sides, we have

∞
X µ ¶
n
X(f ) = xn δ f − . (1.9)
n=−∞ T0
A period signal of special interest is the impulse train
∞
X
x(t) = δ(t − nT0 ) (1.10)
n=−∞

Its Fourier coef£cients are given by

1 Z T0 /2 1
xn = x(t)e−j2πnt/T0 dt = . (1.11)
T0 −T0 /2 T0
Therefore,
∞
X ∞
1 X
δ(t − nT0 ) = ej2πnt/T0 . (1.12)
n=−∞ T 0 n=−∞

This result is sometimes called the Poisson’s sum formula. The Fourier transform of the impulse train
is, thus, given by
∞
X ∞ µ ¶
1 X n
δ(t − nT0 ) ←→ δ f− . (1.13)
n=−∞ T0 n=−∞ T0

1.4
Wong & Lok: Theory of Digital Communications 1. Review

Signum function, unit step function, and integration property

The signum function sgn(t) is de£ned by





 1 if t > 0,


sgn(t) = 0 if t = 0, (1.14)





 −1 if t < 0.
It can be viewed as the limit of the antisymmetric double exponential pulse






e−at if t > 0,

x(t) = 0 if t = 0, (1.15)





 −eat if t < 0,

where a > 0. The Fourier transform of x(t) can be computed as

−j4πf
X(f ) = . (1.16)
a2 + 4π 2 f 2
Taking the limit as a ↓ 0, we have
1
sgn(t) ←→ . (1.17)
jπf
The unit step function u(t) is de£ned by




 1 if t > 0,


u(t) =  1 (1.18)
2
if t = 0,




 0 if t < 0.
It can be expressed as
1 1
u(t) = sgn(t) + . (1.19)
2 2
Therefore, we have the Fourier transform pair
1 1
u(t) ←→ + δ(f ). (1.20)
j2πf 2
The integration of a function x(t) can be expressed as
Z t
x(τ )dτ = x ∗ u(t). (1.21)
−∞

By the convolution property and the Fourier transform of u(t), we have

Z t X(f ) 1
x(τ )dτ ←→ + X(0)δ(f ). (1.22)
−∞ j2πf 2

1.5
Wong & Lok: Theory of Digital Communications 1. Review

1.1.4 Discrete Time Fourier Transform

Given a discrete time signal x(n), its discrete-time Fourier transform (DTFT) is given by
∞
X
X(ω) = x(n)e−jωn (1.23)
n=−∞

Notice that X(ω) is always periodic with period 2π. Therefore, the “frequency” range of interest for
any discrete time signal is [−π, π). Conversely, given X(ω), x(n) can be recovered by

1 Zπ
x(n) = X(ω)ejωn dω. (1.24)
2π −π

Properties of the DTFT

1. Linearity
αx1 (n) + βx2 (n) ←→ αX1 (ω) + βX2 (ω)

2. Time-shift
x(n − n0 ) ←→ X(ω)e−jωn0

3. Frequency-shift
x(n)ejω0 n ←→ X(ω − ω0 )

4. Convolution
x ∗ h(n) ←→ X(ω)H(ω)

5. Multiplication
1 Zπ
x1 (n)x2 (n) ←→ X1 (ω − ν)X2 (ν)dν
2π −π

6. Parseval’s relation
∞
X 1 Zπ
x1 (n)x∗2 (n) = X1 (ω)X2∗ (ω)dω
−∞ 2π −π

1.6
Wong & Lok: Theory of Digital Communications 1. Review

Response of an LTI system

The dirac-delta function δ(n) is de£ned by



 1 if n = 0,
δ(n) =  (1.25)
 0 if n 6= 0.

Notice that any discrete time signal x(n) can be written as

∞
X
x(n) = x(m)δ(n − m). (1.26)
m=−∞

Consider an LTI system T [·]. Let h(n) be its impulse response, i.e., its response to δ(n). Then
" ∞
#
X
T [x(n)] = T x(m)δ(n − m)
m=−∞
∞
X
= x(m)T [δ(n − m)]
m=−∞
∞
X
= x(m)h(n − m)
m=−∞
= x ∗ h(n) (1.27)

Therefore, the response of an LTI system to an input is the convolution of the input and the impulse
response.

1.1.5 Z-Transform

Given a discrete time signal x(n), its Z-transform is given by

∞
X
X(z) = x(n)z −n (1.28)
n=−∞

if the series converges. The region in the complex plane where the series converges is called the region
of convergence (ROC). Notice that by putting z = e−jω , we obtain the DTFT of x(n). Consider the
following examples:

1. x(n) = δ(n). Then X(z) = 1 and the ROC is the entire complex plane.

2. Let 

 an n ≥ 0,
x(n) = 
 0 n < 0.

1.7
Wong & Lok: Theory of Digital Communications 1. Review

Then
∞
X ∞
X
n −n 1
X(z) = a z = (az −1 )n =
n=0 n=0 1 − az −1
−1
where the last step holds only if |az | < 1. Therefore, the ROC is |z| > |a|.

3. Let 

 −an n < 0,
x(n) =

 0 n ≥ 0.
Then
−1 −1 ∞
X X X a−1 z 1
X(z) = − an z −n = − (az −1 )n = − (a−1 z)n = − =
n=−∞ n=−∞ n=1 1−a z
−1 1 − az −1

where the second last step holds only if |a−1 z| < 1. Therefore, the ROC is |z| < |a|.

Notice that the expressions for X(z) are the same for examples 2 and 3. However, the ROC’s are
different.
Conversely, given X(z) and its ROC, x(n) can be recovered by

1 I
x(n) = X(z)z n−1 dz (1.29)
2πj

where the contour integral is over any simple contour in the interior of the ROC of X(z) that circles
the origin exactly once in the counterclockwise direction. Notice that in many cases, there are simpler
methods to perform inverse Z-transforms.

Properties of the Z-transform

1. Linearity
αx1 (n) + βx2 (n) ←→ αX1 (z) + βX2 (z)

2. Time-shift
x(n − n0 ) ←→ X(z)z −n0

In particular, a unit delay in time translates into the multiplication of the z-transform by z −1 .

3. Convolution
x ∗ h(n) ←→ X(z)H(z)

1.8
Wong & Lok: Theory of Digital Communications 1. Review

Z-transforms of some common sequences

Let 

 1 n ≥ 0,
u(n) =  (1.30)
 0 n < 0.
We consider the Z-transforms of some common sequences:
x(n) X(z) ROC
δ(n) 1 all z
1
an u(n) 1−az −1
|z| > |a|
1
−an u(−(n + 1)) 1−az −1
|z| < |a|
az −1
nan u(n) (1−az −1 )2
|z| > |a|
1−az −1 cos(ω0 )
an cos(ω0 n)u(n) 1−2az −1 cos(ω0 )+a2 z −2
|z| > |a|
az −1sin(ω0 )
an sin(ω0 n)u(n) 1−2az −1 cos(ω0 )+a2 z −2
|z| > |a|

Inverse Z-transform with partial fractions

If X(z) is a rational function, its inverse Z-transform can be conveniently found by partial fraction
expansion followed by table look-up as illustrated by the following example where the ROC of X(z)
is |z| > 0.5:

10z 2 − 3z
X(z) =
10z 2 − 9z + 2
10 − 3z −1
=
10 − 9z −1 + 2z −2
4 −5
= +
2−z −1 5 − 2z −1
2 −1
= + (1.31)
1 − 0.5z −1 1 − 0.4z −1

Therefore,
x(n) = 2(0.5)n u(n) − (0.4)n u(n). (1.32)

Implementation of discrete-time LTI systems

Recall that when a discrete-time signal passes through a discrete-time LTI system, the output is the
convolution of the signal and the impulse response of the system. Consider a £nite impulse response

1.9
Wong & Lok: Theory of Digital Communications 1. Review

x(n) z
-1
z
-1 ... z
-1

h0 h1 h2 hK-1

y(n)

Figure 1.1: Direct form implementation of a discrete-time FIR system

(FIR) system, i.e., the impulse response h(n) has a £nite length, say K. Then the output signal y(n)
and the input signal x(n) are related by
K−1
X
y(n) = h(k)x(n − k). (1.33)
k=0

One way to implement the discrete time system is the direct form implementation as shown in Fig-
ure 1.1. Notice that only delays, multiplications, and additions are required. All these operations can
be conveniently implemented.

1.2 Sampling
To process a continuous-time signal digitally, we £rst need to convert it to a discrete-time signal.
Sampling is a common conversion method. As shown in Figure 1.2, a continuous-time signal x(t) is
sampled at every Ts seconds to obtain the discrete-time samples x(nTs ) for every integer n. These
ordered samples form a discrete-time signal.

1.10
Wong & Lok: Theory of Digital Communications 1. Review

x( t )

t
Ts

Figure 1.2: Sampling of a continuous-time signal

X( f )

-W W f

Figure 1.3: Spectrum of a bandlimited signal

1.2.1 Sampling Theorem

Main ideas:

• If the signal x(t) is bandlimited to W (Hz), i.e., X(f ) = 0 for |f | ≥ W (see Figure 1.3), then it
1 1
suf£ces to sample it at intervals T s = 2W
. In other words, the sampling rate fs = Ts
can be as
low as 2W .

1
• If x(t) is bandlimited to W and is sampled at intervals Ts ≤ 2W
, then it is possible to perfectly
reconstruct x(t) from its samples with suitable interpolating signals.

1.11
Wong & Lok: Theory of Digital Communications 1. Review

Xδ ( f )

-f s -W -f s -f s +W -W W f s -W fs f s +W f

Figure 1.4: Spectrum of xδ (t)

Theorem:

Let x(t) be bandlimited to W , i.e., X(f ) = 0 for |f | ≥ W . Sample x(t) at intervals Ts , where
1
Ts ≤ 2W
, to yield the sequence {x(nTs )}∞
n=−∞ . Then

∞
X
x(t) = 2W 0 Ts x(nTs )sinc[2πW 0 (t − nTs )] (1.34)
n=−∞

1 sin x
where W 0 is any number satisfying W ≤ W 0 ≤ Ts
− W , and sinc x is de£ned by x
.
1
Special case: When Ts = 2W
,
∞
X µ ¶
t
x(t) = x(nTs ) sinc π −n . (1.35)
n=−∞ Ts

Proof:

Let
∞
X
xδ (t) = x(nTs )δ(t − nTs ). (1.36)
n=−∞

Notice that
∞
X
xδ (t) = x(t) δ(t − nTs ). (1.37)
n=−∞

Taking Fourier transform, we have

∞
" #
1 X n
Xδ (f ) = X(f ) ∗ δ(f − )
Ts n=−∞ Ts
∞ µ ¶
1 X n
= X f− . (1.38)
Ts n=−∞ Ts
1
We see from Figure 1.4 that we need Ts ≤ 2W
or fs ≥ 2W . Otherwise, copies of the original signal

1.12
Wong & Lok: Theory of Digital Communications 1. Review

spectrum would overlap. The overlapping of replica of the original spectrum is called aliasing. To get
back X(f ), we can apply a low-pass £lter H(f ) with a passband from −W 0 to W 0 where
1
W ≤ W0 ≤ −W (1.39)
Ts
together with an appropriate gain. Take H(f ) as the ideal low-pass £lter given by


 1 if |f | ≤ W 0
H(f ) = (1.40)

 0 if |f | > W 0
and Ts as the gain. Then
X(f ) = Ts Xδ (f )H(f ). (1.41)

Taking inverse Fourier transform, we have

" ∞
#
X
x(t) = Ts x(nTs )δ(t − nTs ) ∗ [2W 0 sinc(2πW 0 t)]
n=−∞
∞
X
= 2W 0 Ts x(nTs )sinc [2πW 0 (t − nTs )] . (1.42)
n=−∞

Remark:

1
• The minimum sampling rate 2W
is called the Nyquist sampling rate.

• The frequency band between two adjacent copies of X(f ) in Xδ (f ) is called a guard band. Its
size is ( T1s − W ) − W = fs − 2W .

1.2.2 Relationship between CTFT and DTFT

We can represent a continuous-time signal by its continuous-time Fourier transform (CTFT). For the
discrete-time signal obtained by sampling the continuous-time signal, we have a corresponding repre-
sentation, namely, the discrete-time Fourier transform (DTFT). It is natural to ask what the relationship
between the CTFT of the continuous-time signal and the DTFT of its sampled version is.
Consider a continuous-time signal xa (t) with CTFT Xa (Ω) where Ω = 2πf . It is sampled with a
sampling interval Ts to give the sequence xa (nTs ). We treat this sequence as our discrete-time signal
x(n). Recall the modulated impulse train xa,δ (t) de£ned by
∞
X ∞
X
xa,δ (t) = xa (t) δ(t − nTs ) = xa (nTs )δ(t − nTs ). (1.43)
n=−∞ n=−∞

1.13
Wong & Lok: Theory of Digital Communications 1. Review

Its CTFT is given by

Z ∞
Xa,δ (Ω) = xa,δ (t)e−jΩt dt
−∞
X∞ Z ∞
= xa (nTs ) δ(t − nTs )e−jΩt dt
n=−∞ −∞

X∞
= x(n)e−jΩnTs
n=−∞
= X(ΩTs ). (1.44)

Equivalently,
µ ¶
ω
X(ω) = Xa,δ . (1.45)
Ts
Hence, the DTFT of the sample sequence x(n) is just a normalized version of the CTFT of the modu-
lated impulse train xa,δ (t). By the previous expression for Xa,δ (Ω) (see (1.38)), we have
∞
à !
1 X ω − 2πk
X(ω) = Xa . (1.46)
Ts k=−∞ Ts

Therefore, the DTFT of the sequence of samples has the same shape as the folded spectrum of the
continuous-time signal.

1.3 Gaussian Random Variables

1.3.1 Gaussian Random Variables

De£nition:

A Gaussian random variable is any continuous random variable with a probability density function of
the form
1 2 2
fX (x) = √ e−(x−µ) /2σ , (1.47)
2πσ 2

where µ is a constant, and σ > 0.

Properties:

Let X be a Gaussian random variable with the density function shown in (1.47).

1.14
Wong & Lok: Theory of Digital Communications 1. Review

1. The mean and the variance of X are µ and σ 2 , respectively. Hence, a Gaussian random variable
is completely speci£ed by its mean and variance.

2. A zero-mean unit-variance Gaussian random variable has the density function

1 2
Z(x) = √ e−x /2 . (1.48)
2π

The probability distribution function of a zero-mean unit-variance Gaussian random variable is

given by
Z x
Φ(x) = Z(u)du. (1.49)
−∞

Very often, it is convenient to use the Q-function de£ned by

Q(x) = 1 − Φ(x). (1.50)

The Q-function gives the area under the tail of Z(x). It will be used frequently in the following
chapters. We note that the Q-function is monotone decreasing and is bounded by

1 2
Q(x) ≤ e−x /2 (1.51)
2

for x ≥ 0. Moreover, the bound in (1.51) is the best of this type in the sense that


 ∞
Q(x) if a > 1
lim = (1.52)
x→∞ exp{−ax2 /2}  0 if a ≤ 1

3. For the Gaussian random variable X with mean µ and variance σ 2 ,

Z x
Pr(X ≤ x) = FX (x) = fX (u)du. (1.53)
−∞

Changing the dummy argument by putting v = (u − µ)/σ, we get

µ ¶
x−µ
Pr(X ≤ x) = Φ . (1.54)
σ

Similarly, we have
µ ¶
x−µ
Pr(X > x) = Q . (1.55)
σ

1.15
Wong & Lok: Theory of Digital Communications 1. Review

4. The Gaussian distribution is widely tabulated and is also available in most mathematical soft-
wares. For example, in Matlab, we can use the error function (erf) and the complementary error
function (erfc) to £nd values of Φ(·) and Q(·). The relationships are given by
" Ã !#
1 x
Φ(x) = 1 + erf √ (1.56)
2 2
and à !
1 x
Q(x) = erfc √ . (1.57)
2 2
5. For any constant a 6= 0, aX is also a Gaussian random variable with mean aµ and variance a2 σ 2 .

1.3.2 Jointly Gaussian Random Variables

De£nition:

Two random variables X and Y are jointly Gaussian if their joint probability density function is of the
form
 
1 ( x−µ
σX
) − 2ρ( x−µ
X 2
σX
X
)( y−µ
σY
Y
) + ( y−µ
σY
Y 2
)
fXY (x, y) = √ exp  , (1.58)
2πσX σY 1 − ρ2 −2(1 − ρ2 )
where µX , µY , σX > 0, σY > 0, and −1 < ρ < 1 are constants.

Properties:

2
1. X and Y are Gaussian random variables with means µX and µY , and variances σX and σY2 .

2. The parameter ρ is the correlation coef£cient given by

E[(X − µX )(Y − µY )]
ρ= . (1.59)
σX σY
Therefore, two jointly Gaussian random variables are completely speci£ed by their means, vari-
ances and correlation coef£cient.

3. The random variables X and Y are uncorrelated if and only if ρ = 0. In this case, the joint
density function reduces to
1 2 /2σ 2 1 2 2
fXY (x, y) = q e−(x−µX ) X ·q e−(y−µY ) /2σY = fX (x)fY (y). (1.60)
2
2πσX 2πσY2
Hence, uncorrelated Gaussian random variables are also independent.

1.16
Wong & Lok: Theory of Digital Communications 1. Review

4. Given any two constants a and b (not both zeros), aX + bY is also a Gaussian random variable
with mean aµX + bµY and variance a2 σX
2
+ 2abρσX σY + b2 σY2 .

5. Given any invertible 2 × 2 matrix A, the random variables U and V de£ned by

   
 U   X 
  = A  (1.61)
V Y

are jointly Gaussian.

6. Suppose X and Y are uncorrelated jointly Gaussian random variables with the same variance
σ 2 . If we write X and Y as the real and imaginary parts, respectively, of the complex notation

Z = X + jY, (1.62)

then Z is a (symmetric) complex Gaussian random variable. Strictly speaking, Z is not a random
variable. Its probabilistic behavior is actually governed by the joint density function of X and
Y . However, for convenience, we usually associate Z with a mean and a variance as below:

4
E[Z] = µX + jµY = µZ (1.63)
1
E [(Z − µZ )(Z − µZ )∗ ]
4
var[Z] =
2
1
= E [((X − µX ) + j(Y − µY )) ((X − µX ) + j(Y − µY ))∗ ]
2
= σ2. (1.64)

We also associate Z with the “density function”

à !
1 |z − µZ |2
fZ (z) = exp − , (1.65)
2πσ 2 2σ 2

which is just a compact form of (1.58) when ρ = 0.

Sometimes, we also use the notion of “complex-valued random variable” for other non-Gaussian
random variables.

1.17
Wong & Lok: Theory of Digital Communications 1. Review

1.3.3 Gaussian Random Vectors

De£nition:

The random variables X1 , X2 , . . . , Xn are jointly Gaussian if their joint probability density function is
of the form
· ¸
1 1
fX (x) = q exp − (x − µX )T CX −1 (x − µX ) , (1.66)
(2π)n/2 det(CX ) 2

where X = [X1 , X2 , . . . , Xn ]T , x = [x1 , x2 , . . . , xn ]T , µX = [µX1 , µX2 , . . . , µXn ]T is a constant n × 1

vector, det(·) is the determinant operation, and CX is an n × n positive-de£nite symmetric matrix.
Instead of saying that X1 , X2 , . . . , Xn are jointly Gaussian random variables, it is also customary to
say that X is a Gaussian random vector.

Properties:

1. Each Xi is a Gaussian random variable with mean equal to µXi , and variance equal to the (i, i)-th
entry of CX .

2. The covariance of Xi and Xj , E[(Xi − µXi )(Xj − µXj )], is given by the (i, j)-th entry of CX ,
which is, thus, called the covariance matrix. Hence, a Gaussian random vector is completely
speci£ed by its mean and its covariance matrix.

3. The random variables X1 , X2 , . . . , Xn are uncorrelated if and only if CX is a diagonal matrix.

In this case, the joint density function factors into the product of the marginal density functions.
Hence, uncorrelated Gaussian random variables are also independent.

4. Given any non-zero n × 1 constant vector a, aT X is a Gaussian random variable with mean
aT µX and variance aT CX a.

5. Given any invertible n × n matrix A, the random vector Y de£ned by

Y = AX (1.67)

is a Gaussian random vector with mean AµX and covariance matrix ACX AT .

1.18
Wong & Lok: Theory of Digital Communications 1. Review

1.3.4 Related Random Variables

Let X and Y be zero-mean uncorrelated (independent) jointly Gaussian random variables with the
same variance σ 2 . De£ne the random variables
√
R = X2 + Y 2

Θ = arctan(Y /X) (1.68)

where the ranges of R and Θ are [0, ∞) and (−π, π], respectively. We note that the transformation
from (X, Y ) to (R, Θ) is non-singular and

X = R cos Θ

Y = R sin Θ. (1.69)

If we interpret (X, Y ) as the rectangular coordinates, then (R, Θ) is the corresponding polar coordi-
nates. If we interpret (X, Y ) as the complex number Z in (1.62), then R and Θ are |Z| and arg(Z),
respectively.

Properties:

1. R and Θ are independent.

2. R is Rayleigh distributed, i.e., its density function is

à !
r r2
fR (r) = 2 exp − 2 , r ≥ 0. (1.70)
σ 2σ
3. θ is a uniform random variable, i.e., its density function is
1
fΘ (θ) = , − π < θ ≤ π. (1.71)
2π
4. The random variable W = R2 is exponentially distributed, i.e., its density function is
µ ¶
1 w
fW (w) = exp − , w ≥ 0. (1.72)
2σ 2 2σ 2
5. If X and/or Y are not zero-mean, then R is Rician distributed, i.e.,
à ! µ ¶
r r 2 + s2 rs
fR (r) = 2 exp − 2
I0 , r ≥ 0, (1.73)
σ 2σ σ2
1 R 2π
where s2 = µ2X + µ2Y and I0 (x) = 2π 0 exp(x cos θ)dθ is the zeroth order modi£ed Bessel
function of the £rst kind. We note that Θ is neither uniform nor independent of R in this case.

1.19
Wong & Lok: Theory of Digital Communications 1. Review

1.3.5 Central Limit Theorem

There are many different versions of this well-known and extremely useful theorem. Here we state the
simplest version:
Suppose {Xn } is a sequence of independent and identically distributed (iid) random variables with
£nite mean µ and £nite positive variance σ 2 . If Sn = X1 + X2 + · · · + Xn , then the random variables

Sn − nµ
Yn = √ (1.74)
σ n

converge to a zero-mean unit-variance Gaussian random variable in distribution, i.e., the distribution
functions of Yn converge (pointwise) to Φ(x) de£ned in (1.49).

1.4 Random Processes

1.4.1 General De£nitions

Random process

A random process is an indexed set of random variables de£ned on the same probability space.

• In communications, the index is usually a time index.

• If n(t) is a random process, then at any given time t0 , n(t0 ) is a random variable.

• In many cases, it is convenient to view a random process as a mapping de£ned on the sample
space. It maps an outcome to a function of time.
For example, Ω = {0, 1}.

0 → cos(t),

1 → sin(t).

Notice that at any given time t0 , we have a random variable n(t0 ) that maps 0 to cos(t0 ) and 1
to sin(t0 ).

1.20
Wong & Lok: Theory of Digital Communications 1. Review

Mean of a random process

At any given time t0 , n(t0 ) is a random variable. We can determine the mean of this random variable.
We denote this mean by µn (t0 ). If we calculate the mean at each time t, we get the mean of the process

µn (t) = E[n(t)] (1.75)

which is function of time. It gives the mean of the process at time t.

Autocorrelation function

At any given t1 and t2 , n(t1 ) and n(t2 ) are random variables. We can determine the correlation of these
random variables. We denote this correlation by Rn (t1 , t2 ). If we calculate the correlation at each pair
of time instants t and s, we get the autocorrelation function of the process

Rn (t, s) = E[n(t)n(s)] (1.76)

which is a function of two time variables t and s. Setting s = t, we get

Rn (t, t) = E[n2 (t)]. (1.77)

It represents the (ensemble average) power of the process at time t.

Autocovariance function

At any given t1 and t2 , n(t1 ) and n(t2 ) are random variables. We can determine the covariance of these
random variables. We denote this covariance by Cn (t1 , t2 ). If we calculate the covariance at each pair
of time instants t and s, we get the autocovariance function of the process

Cn (t, s) = E[(n(t) − µn (t))(n(s) − µn (s))] (1.78)

which is a function of two time variables t and s.

Cross-correlation function and cross-covariance function

Suppose that we are given two random processes n(t) and m(t). At any given t1 and t2 , n(t1 ) and
m(t2 ) are random variables. We can determine the correlation and the covariance of these random

1.21
Wong & Lok: Theory of Digital Communications 1. Review

variables. We denote them by Rnm (t1 , t2 ) and Cnm (t1 , t2 ). If we perform the calculations at each pair
of time instants t and s, we get the cross-correlation function

Rnm (t, s) = E[n(t)m(s)], (1.79)

and the cross-covariance function

Cnm (t, s) = E[(n(t) − µn (t))(m(s) − µm (s))] (1.80)

of the processes. Rnm (t, s) and Cnm (t, s) are functions of the two variables t and s.

1.4.2 Wide-Sense Stationary (WSS) Random Processes

De£nitions

• A random process n(t) is wide-sense stationary (WSS) if the following conditions hold:

1. µn (t) does not depend on t, i.e., µn (t) takes the same value for all t. (We may just represent
it by µn .)

2. Rn (t, s) depends only on the difference τ = t − s, but not on t or s individually. (We may
just write Rn (τ ).)

Notice that if n(t) is WSS, then the (ensemble average) power of the process at any time t is
Rn (t − t) = Rn (0), and does not depend on the time t.

• Two random processes n(t) and m(t) are jointly WSS if the following conditions hold:

1. The processes n(t) and m(t) are each WSS.

2. Rnm (t, s) depends only on the difference τ = t − s, but not on t or s individually. (We
may just write Rnm (τ ).)

• A random process n(t) is wide-sense cyclostationary (WSCS) with period T if the following
conditions hold:

1. µn (t + kT ) = µn (t)

1.22
Wong & Lok: Theory of Digital Communications 1. Review

2. Rn (t + kT, s + kT ) = Rn (t, s)

for every t, s and every integer k.

Suppose n̄(t) = n(t − ∆) where n(t) is a WSCS process and ∆ is a uniform random variable in
the interval [0, T ) independent of n(t). Then n̄(t) is a WSS process and

1ZT
µn̄ = µn (t)dt (1.81)
T 0
1ZT
Rn̄ (τ ) = Rn (t + τ, t)dt. (1.82)
T 0

Response of an LTI system to a WSS random process

Suppose x(t) is a WSS random process with mean µx and autocorrelation function Rx (τ ). We pass
x(t) through an LTI £lter with impulse response h(t) and obtain the output process y(t). Then the
output
Z ∞
y(t) = x ∗ h(t) = x(τ )h(t − τ )dτ. (1.83)
−∞

We can determine the mean of y(t)

µy (t) = E[y(t)]
·Z ∞ ¸
= E x(τ )h(t − τ )dτ
−∞
Z ∞
= E[x(τ )]h(t − τ )dτ
−∞
Z ∞
= µx h(t − τ )dτ
−∞
Z ∞
= µx h(τ )dτ. (1.84)
−∞

Moreover, the autocorrelation function of y(t)

Ry (t, s) = E[y(t)y(s)]
·Z ∞ Z ∞ ¸
= E x(τ )x(ν)h(t − τ )h(s − ν)dτ ν
−∞ −∞
Z ∞ Z ∞
= E[x(τ )x(ν)]h(t − τ )h(s − ν)dτ ν
−∞ −∞
Z ∞ Z ∞
= Rx (τ − ν)h(t − τ )h(s − ν)dτ ν
−∞ −∞

= Rx ∗ h ∗ h̃(t − s) (1.85)

1.23
Wong & Lok: Theory of Digital Communications 1. Review

where h̃(t) = h(−t). Since Ry (t, s) depends only on the difference τ = t − s but not on t or s
individually, we may just write Ry (τ ). From (1.84) and (1.85), we see that y(t) is also WSS. It is
straightforward to check that Rxy (t, s) = Rx ∗ h̃(t − s) depends only on the difference τ = t − s, but
not on t or s individually. (We may just write Rxy (τ ).) Therefore, x(t) and y(t) are jointly WSS.

1.4.3 Power Spectral Densities of WSS Random Processes

De£nition:

Let x(t) be a WSS random process. Its power spectral density, denoted by Φx (f ) is de£ned as the
Fourier transform of Rx (τ ), i.e.,
Z ∞
Φx (f ) = Rx (τ )e−j2πf τ dτ. (1.86)
−∞

Properties:

1. Rx (τ ) can be obtained from the inverse Fourier transform of Φx (f ):

Z ∞
Rx (τ ) = Φx (f )ej2πf τ df. (1.87)
−∞

2. The power of x(t) is given by

Z ∞
Rx (0) = Φx (f )df. (1.88)
−∞

3. Let y(t) be the output when x(t) is passed through an LTI £lter with impulse response h(t).
Similarly, we de£ne the power spectral density of y(t), Φ y (f ), to be the Fourier transform of
Ry (τ ), and the power of y(t) can be obtained by the integral of Φy (f ). Since Ry (τ ) = Rx ∗ h ∗
h̃(τ ),
Φy (f ) = Φx (f )|H(f )|2 (1.89)

where H(f ) is the Fourier transform of h(t).

4. Consider that the LTI £lter above is a bandpass £lter with a very narrowband passband around
f0 , i.e., 

 1 ∆f ∆f
if f0 − 2
< f < f0 + 2
H(f ) =  (1.90)
 0 otherwise.

1.24
Wong & Lok: Theory of Digital Communications 1. Review

The output power is given by

Z ∞
Ry (0) = Φy (f )df
−∞
Z ∞
= Φx (f )|H(f )|2 df
−∞

≈ Φx (f0 )∆f. (1.91)

The last approximation is valid if ∆f is small enough so that Φx (f ) is essentially constant in the
passband. The output power per Hz is given by

Ry (0)
= Φx (f0 ). (1.92)
∆f

Physically, the output power should be the power of x(t) around f0 (more precisely, the power
∆f ∆f
of x(t) from f0 − 2
to f0 + 2
). The output power per Hz should be the power of x(t) per
Hz around f0 . Hence, Φx (f0 ) is the power of x(t) per Hz around f0 , and is, therefore, called the
power spectral density.

1.4.4 Special processes

White processes

A WSS random process n(t) is white if µn = 0 and Rn (τ ) is a scalar multiple of the delta function
δ(τ ).

1. Equivalently, a WSS random process n(t) is white if µn = 0 and Φn (f ) is a constant because

the Fourier transform of a delta function is a constant.

2. For t1 6= t2 , the random variables n(t1 ) and n(t2 ) are uncorrelated.

3. The power of the process is Rn (0) = ∞. Hence, a white process has in£nite power. Of course,
this is just a convenient mathematical model, and does not exist in reality. Nevertheless, this
model is useful especially in modeling thermal noise.

1.25
Wong & Lok: Theory of Digital Communications 1. Review

Gaussian random processes

A random process x(t) is a Gaussian random process if for all k and for all (t1 , t2 , . . . , tk ), the random
variables (x(t1 ), x(t2 ), . . . , x(tk )) are jointly Gaussian.
Two random processes x(t) and y(t) are jointly Gaussian if for all k1 and k2 and for all (t1 , t2 , . . . , tk1 )
and (s1 , s2 , . . . , sk2 ), the random variables (x(t1 ), x(t2 ), . . . , x(tk1 ), y(s1 ), y(s2 ), . . . , y(sk2 )) are jointly
Gaussian.

1. It can be shown that if a Gaussian random process x(t) is passed through an LTI system, then
the output process y(t) is also a Gaussian random process. Actually, x(t) and y(t) are jointly
Gaussian random processes.

2. In particular, a time sample of the output process y(t) is a Gaussian random variable.

3. More generally, if a random process x(t) is passed through a bank of LTI £lters, the outputs of
the £lters are jointly Gaussian random processes.

4. Very often, we model the thermal noise as an additive white Gaussian random process. This
means that the noise is assumed to add onto the signal, the noise is assumed to be a white
process, and the noise is assumed to be a Gaussian random process. We then call the noise
Additive White Gaussian Noise (AWGN).

Complex-valued random processes

Given two random processes x(t) and y(t), we can de£ne the complex-valued random process z(t) as

z(t) = x(t) + jy(t). (1.93)

Like complex-valued random variables, z(t) is not a random process. However, it is sometimes conve-
nient to treat it as if it were a random process. We can de£ne the mean, autocorrelation function, and
autocovariance function of z(t) by extending the corresponding de£nitions for a random process. For
example, the autocorrelation function of z(t) is

1
Rz (t, s) = E[z(t)z(s)∗ ]
2

1.26
Wong & Lok: Theory of Digital Communications 1. Review

1
= E [(x(t) + jy(t))(x(s) − jy(s))]
2
1
= {Rx (t, s) + Ry (t, s) + j[Ryx (t, s) − Rxy (t, s)]} . (1.94)
2

If x(t) and y(t) are jointly WSS, Rz (t, s) depends only on the difference τ = t − s, but not on t or
s individually. (We may just write Rz (τ ).) In this case, we say z(t) is WSS. Cross-correlation and
covariance functions of two complex-valued random processes can be de£ned in a similar way. We
note that our convention is to include the factor 1/2 in all second-order statistics involving complex-
valued random variables and processes. The power spectral density of a WSS complex-valued random
process is de£ned as the Fourier transform of its autocorrelation function.

1.4.5 Sampling Theorem for Bandlimited WSS Random Processes

Suppose x(t) be a WSS random process. If its power spectral density Φx (f ) = 0 for |f | ≥ W , then
x(t) is called bandlimited to W .
As an immediate consequence of (1.34), the autocorrelation function satis£es
∞
X
Rx (τ ) = 2W 0 Ts Rx (nTs )sinc[2πW 0 (τ − nTs )] (1.95)
n=−∞

1
where W 0 is any number satisfying W ≤ W 0 ≤ Ts
− W . Based on this, if we construct the random
process
∞
X
x̃(t) = 2W 0 Ts x(nTs )sinc[2πW 0 (t − nTs )] (1.96)
n=−∞

from the time samples of x(t), then x̃(t) is WSS and

h i
E |x(t) − x̃(t)|2 = 0 (1.97)

We note that the equality in (1.97) should be interpreted as the mean-square convergence of the series
of random variables represented by (1.96) to the random variable x(t0 ).

1.27