EE 8107 Digital Communications

Course Notes

Department of Electrical and Computer Engineering

Ryerson University

c
°Dr. A. Anpalagan
Fall 2008
Contents
1 Overview of a Digital Communication System 5

2 Communication Channel Models 7

3 Probability Theory 9
3.1 Event Space and Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Statistics of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.4 Useful Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Random Processes 14
4.1 Notion of Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2 Stationary Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3 Statistical Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.4 Wide Sense Stationary: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.5 Power Spectrum Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.6 Input/Output Relationship in LTI Systems . . . . . . . . . . . . . . . . . . . . . 18
4.7 Fourier Transform and Properties: . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5 Complex Signals and Systems 20

5.1 Bandpass Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.2 Bandpass Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.3 Response of a Bandpass System to a Bandpass Signal . . . . . . . . . . . . . . . 23
5.4 Bandpass Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.5 Bandpass White Noise Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6 Vector and Signal Space 27

6.1 Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.2 Signal Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7 Source Coding 31

2
 7.1 Information Source Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
7.2 Nyquist Sampling Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
7.3 Measure of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
7.3.1 Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
7.3.2 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
7.4 Coding for Discrete Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.4.1 Fixed Length Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
7.4.2 Variable Length Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
7.5 Coding for Analog Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8 Channel Capacity 43
8.1 Channel Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
8.2 Channel Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
8.3 Capacity of BSC and AWGN Channels . . . . . . . . . . . . . . . . . . . . . . . 46
8.4 Shannon’s Channel Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

9 Digitally Modulated Signals 50

9.1 Memoryless Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
9.1.1 Amplitude Shift Keying (ASK) . . . . . . . . . . . . . . . . . . . . . . . 51
9.1.2 Phase Shift Keying (PSK) . . . . . . . . . . . . . . . . . . . . . . . . . . 53
9.1.3 Quadrature PAM (QAM) . . . . . . . . . . . . . . . . . . . . . . . . . . 54
9.1.4 Frequency Shift Keying (FSK) . . . . . . . . . . . . . . . . . . . . . . . . 55
9.2 Modulation with Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
9.2.1 Continuous Phase FSK (CPFSK) . . . . . . . . . . . . . . . . . . . . . . 58
9.2.2 Minimum Shift Keying (MSK): . . . . . . . . . . . . . . . . . . . . . . . 60

10 Spectra of Digital Signals 62

10.1 Spectral Shaping without Precoding . . . . . . . . . . . . . . . . . . . . . . . . . 63
10.1.1 Rectangular pulse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10.1.2 Raised cosine pulse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10.2 Spectral Shaping with Precoding . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3
 10.3 PSD of Bandpass Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

11 Receiver Design for AWGN Channel 66

11.1 Demodulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
11.1.1 Correlator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
11.1.2 Matched Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
11.2 Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
11.2.1 MAP, ML and MD Detectors . . . . . . . . . . . . . . . . . . . . . . . . 74
11.3 MAP Detector for Non Equiprobable Symbols . . . . . . . . . . . . . . . . . . . 76
11.4 Maximum-Likelihood Sequence Detector (MLSD) . . . . . . . . . . . . . . . . . 77

12 Performance Evaluation in AWGN Channel 80

12.1 Error Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
12.1.1 Binary PAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
12.1.2 M-ary Orthogonal Signals . . . . . . . . . . . . . . . . . . . . . . . . . . 82
12.1.3 M-ary PAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
12.1.4 M-ary PSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.2 Bandwidth Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.2.1 Bandwidth Limited Case . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
12.2.2 Power Limited Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
12.3 Analog vs Digital Repeaters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

13 Signal Design for Band-limited Channels 92

13.1 Inter Symbol Interference (ISI) . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
13.2 Nyquist Pulse Design Criterion for No ISI . . . . . . . . . . . . . . . . . . . . . 95
13.3 Band-limited Signal Design with Controlled ISI . . . . . . . . . . . . . . . . . . 101

4
1 Overview of a Digital Communication System

• Any communication systems involve, source ⇒ sink via a channel.

• Source (or sink) can be analog (e.g., audio) or digital (e.g., fax).

• In digital communication, a sequence of binary digits (discrete in time and

finite number of them) is transmitted.

• Goal: efficient and reliable communication over noisy channel.

Information Source Channel Digital

source encoder encoder modualtor
comunication

Basic elements of a digital communication system Channel

Output Source Channel Digital

transducer decoder decoder demodualtor

Signal bandwidth: The (base-band) signal is said to have bandwidth of B

Hz if most (e.g., 95%) of the energy of the signal is contained in frequency
components smaller than B Hz. And, pass-band DSB signals have twice the
bandwidth of the base-band signals.
Coders: Source coder does compression for efficient transmission (i.e. removes
redundancy) and channel coder prepares for reliable transmission over the noisy
channel (i.e. may increase data rates for error detection/correction.)
Digital modulator: It is an interface to the channel, i.e., (digital) 0101011
to (analog) waveforms, e.g., modem. M-ary modulation: Take a b-bit sequence

5
and transmit one of M = 2b possible waveforms (a symbol is transmitted).
♠ Example with b = 2.
Rate is b-fold decreased. How about average power and bandwidth ?.
Communication channel: Examples are microwave radio, fiber or cable.
Thermal noise in the channel corrupts the waveforms and symbol by symbol
detection is done to declare the transmitted symbol in digital communication.
Multiple access interference (MAI) is also a cause for impairment, for example,
wireless cellular systems and Ethernet.
System performance: A digital communication system can be characterized
by performance measure such as

1. bit rate (copper wire (kbps), optical fiber (Gbps), cellular radio (kbps))

2. bit error rate (e.g., 10−3 in telephony), 10−8 in mission-critical apps.

3. propagation and processing delay (important in real-time applications such

as audio and video)
Bit error rate (BER) is dependent on:
(i) coding techniques (ii) transmitted waveforms (or signals) (iii) transmit
power (iv) channel characteristics (v) modulation/demodulation and (vi)
receiver design.

Advantages of digital communication: higher capacity, store/forward ca-

pability, switching/networking facility, high fidelity and enhanced services using
advanced DSP techniques.

6
2 Communication Channel Models

• Problem arises when the noise and the signal of interest occupy the same
frequency band. Unfortunately, noise occupy the entire band!

• A major source of noise is (additive) thermal noise.

• Proper system design (i.e., waveform, modulation, receiver etc.) can min-
imize the adverse effect of the noise.

• Mathematical models and tools help us understand channel characteristics

and design the signal/system accordingly.

Additive Noise Channel: Signal is corrupted by additive noise process

n(t). Generally, characterized statistically as additive Gaussian noise process
No
(AWGN) with flat power spectrum density of 2 .

r(t) = αs(t) + n(t)

Linear Time-invariant Channel: Front-end filters are often used in com-

munication systems to eliminate the out-of-band noise or signal. They are
generally characterized as LTI systems.
Z +∞
r(t) = s(t) ? c(t) + n(t) = c(τ )s(t − τ )dτ + n(t)
−∞

Linear Time-variant Channel: Example: multipath signals that arrive with

different delay and attenuation in different environment.

r(t) = s(t) ? c(τ ; t) + n(t)

7
 Linear time−
s(t) r(t)
variant filter +
c(τ ; t)

channel n(t)

where c(τ ; t) is the impulse response of the channel (filter) at time t for an
impulse applied at time t − τ .

A radio multipath channel is characterized with its impulse response as,

L
X
c(τ ; t) = ak (t)δ(τ − τk )
k=1

where ak (t) represents time-varying attenuation factor for the L multipaths and
τk are the corresponding delays. Hence,
L
X
r(t) = ak (t)s(t − τk ) + n(t)
k=1

Received signal consists of L multipath components each characterized by an

attenuation and a delay component.

c(τ ; t)
1
Multipath
0.4 0.3
channel
0.2

0 to t 0 τ1 τ 2 τ 3 t

8
3 Probability Theory

An important mathematical tool in the design and evaluation of digital com-

munication systems. Examples:

1. statistical modelling of information sources (e.g., apriori probability of

symbol transmission). The occurrence of A’s in a fax message?

2. characterization of communication channels (e.g., corruption of bits in bi-

nary symmetric channel)

3. evaluation of different modulation schemes (e.g., probability of bit error)

4. design of optimum receivers (e.g., matched filter in AWGN channel)

Probability as a long term average. E.g., the probability of a person talking

more than 3 mins in a phone conversation? or the probability of a bit being
corrupted?

3.1 Event Space and Probability

Notion of sample space (S) and event space (A, B).

A. Mutual Exclusion: Two events, A and B are mutually exclusive if A∩B =

Ø. If Ai ∩ Aj = Ø, i 6= j = 1, 2, ... The total probability,
µ[ ¶ X
P (S) = P Ai = P (Ai ) = 1
i i

It is easy to visualize with Venn diagrams.

B. Joint Events and Probability: Let Ai , i = 1, 2, ...n, and Bj , j = 1, 2, ...m,

then the joint event (Ai , Bj ) has, 0 ≤ P (Ai , Bj ) ≤ 1. If events Ai , i =

9
 1, 2, ...n are mutually exclusive, then
n
X
P (Ai , Bj ) = P (Bj )
i=1

♠ Example: Ai ∈ {1, 2, ..., 6} and Bj ∈ {H, T }.

C. Conditional Probability: Knowledge of the outcome of one (related)

event affects the probability of another event.
P (A, B)
P (A|B) = , P (B) > 0
P (B)
Also important to note: P (A, B) = P (A|B)P (B) = P (B|A)P (A)

• If A and B are mutually exclusive, P (A|B) = 0

• If A and B are statistically independent, P (A|B) = P (A)

D. Bayes’ Theorem: If Ai , i = 1, 2, ..., n, are mutually exclusive events, and

B is any event with P (B) > 0, then
P (Ai , B) P (B|Ai )P (Ai )
P (Ai |B) = = Pn
P (B) j=1 P (B|Aj )P (Aj )

Since B = B ∩ S, P (B) = P (B ∩ A1 ) + ... + P (B ∩ An ) and P (B ∩ Ai ) =

P (B|Ai )P (Ai ). Note that (A ∩ B) = (A, B)
Bayes’ theorem is extremely useful. Examples:

• in computing the bit error rate in binary symmetric channel (BSC)

• in deriving the optimal receiver structure with apriori probabilities.

♠ Example - BSC:
Assuming equal probability for input bits,

Pe = P (X = 0|Y = 1)

10
 1−p
0 0

p
Input (X) Output (Y)
p

1 1
1−p

Figure 1: Binary Symmetry Channel

P (Y = 1|X = 0)P (X = 0)
Pe = =p
P (Y = 1|X = 0)P (X = 0) + P (Y = 1|X = 1)P (X = 1)
E. Statistical Independence: Occurrence of A does not depend on the
occurrence of B. Hence, P (A|B) = P (A) ⇒ P (A, B) = P (A)P (B)

In general, Aj , j = 1, 2, ....n are statistically independent, if the joint probability

is the product of the marginal probabilities.

3.2 Random Variables

A. Definition: Sample space S. Elements, s ∈ S. Function X(s) maps S into

real line (<). X(s) (or conveniently X) is called random variable (RV).
Randomness is in s not in X.

X(s)

Real line

Sample space

Figure 2: Definition of Random Variable

B. Distributions:

11
 • Cumulative Density Function (CDF): F (x) = P (X ≤ x), x ∈ <
dF (x)
• Probability Density Function (PDF): p(x) = dx , x ∈<

C. Multiple RVs:
Z x1 Z x2
F (x1 , x2 ) = P (X1 ≤ x1 , X2 ≤ x2 ) = p(u1 , u2 )du2 du1
−∞ −∞

where p(u1 , u2 ) is the joint PDF. Marginal PDF is obtained by integrating

over other variables.
Conditional Probability
R x1
−∞ p(u1 , x2 )du1
F (x1 |x2 ) = P (X1 ≤ x1 |X2 = x2 ) =
p(x2 )
p(x1 , x2 )
p(x1 |x2 ) =
p ( x2 )
Straightforward extension to multidimensional random variables.
Statistical Independence
CDF and PDF factor into product of marginal probabilities.

F (x1 , ...., xn ) = F (x1 )F (x2 )...F (xn )

p(x1 , ...., xn ) = p(x1 )p(x2 )...p(xn )

D. Functions of RVs: Y = g(X). Real roots of y = g(x) are x1 , ...xn . Then,

PDF of random variable Y is:
Xn
pX (xi )
pY (y) =
i=1
|g 0 (xi )|

3.3 Statistics of Random Variables

A. Moments: Let Y = g(X). The nth moment of Y = g(X) is

Z ∞
E(Y n ) = [g(x)]n p(x)dx
−∞

12
 R∞
• If n = 1 and Y = X, then mean of X, E(X) ≡ mx = −∞ xp(x)dx
R∞
• If n = 2 and Y = X − mx , then variance of X, σx2 = −∞ (x −
mx )2 p(x)dx

B. Joint Moments:

• Correlation:
Z ∞ Z ∞
E(Xi Xj ) = xi xj p(xi , xj )dxi dxj
−∞ −∞

• Covariance:

µij = E[(Xi − mi )(Xj − mj )] = E(Xi Xj ) − mi mj

If E(Xi Xj ) = 0, then RVs Xi and Xj are orthogonal.

If E(Xi Xj ) = mi mj , then RVs Xi and Xj are uncorrelated.

Statistical independence implies uncorrelation; however, uncorrelation does

not necessarily imply independence.

C. Characteristic Function: Defined as a statistical average:

Z ∞
jwX
ψX (jw) = E(e )= ejwx p(x)dx
−∞

By inverse Fourier transform,

Z ∞
1
p(x) = ψX (jw)e−jwx dw
2π −∞

The moments can be determined from the characteristic function. Since,

dn ψX (jw)
E(X n ) = (−j)n |w=0
dwn

13
 They are useful in determining PDF of a sum of statistically independent
P
RVs. Let Y = ni=1 Xi , then it can be shown that,
n
Y
ψY (jw) = ψXi (jw)
i=1

If Xi ’s are independent identically distributed (iid), then ψY (jw) = [ψX (jw)]n

3.4 Useful Probability Distributions

2 2
A. Gaussian (normal): The PDF is, p(x) = √ 1 e−(x−mx ) /2σ . The CDF can
2πσ

be written as, µ ¶
x − mx
F (x) = 1 − erfc √
2σ
R∞ 2
where erfc(x) = √2 e−t dt. Another function often used in Q(x) =
π x
1
2 erfc ( √x2 ).

The sum of n statistically independent Gaussian RVs (Xi ’s) is also a

P P P
Gaussian RV. Let Y = ni=1 Xi . Then, my = ni=1 mx , σy2 = ni=1 σi2 .

B. Lognormal: Let X = ln Y, where X is normally distributed with mean m

and variance σ 2 .


√ 1 (lny−m)/2σ 2
2πσy
e , y ≥ 0,
p(y) =

0, otherwise

Used in modelling the shadowing effect in mobile radio communications.

4 Random Processes

Some random phenomena evolve over time. Examples are:

14
 • thermal noise voltages at radio receivers

• audio signals generated during phone conversations

4.1 Notion of Random Process

• A random (or stochastic) process is simply a mapping from the sample

space S to a set of functions defined on T, where T is the parametric space
(e.g., time).
♠ Example: cos (wt + θ), where 0 ≤ θ ≤ 2π.

• A sample function is a realization of the random process.

• An ensemble (or collection) of sample functions forms a random process.

X(t)

θ= 0
x x
x x

X1(t) X2(t)
θ = π/2
t
x
x X4(t)
X3(t)
θ= π x
x
θ =3/2π t = t t = t2
1

4.2 Stationary Random Process

Let X(t) be a random process. Then, X(ti ), (i = 1, 2, ...., n) are random vari-
ables and can be characterized by their joint PDF, p(xt1 , ..., xtn ).

15
 If
p(xt1 , ..., xtn ) = p(xt1 +t , ..., xtn +t ), ∀t, n

then X(t) is said to be strict sense stationary (SSS). That is, the process is
statistically indifferent with respect to any time shift.

4.3 Statistical Averages

Ensemble average: Let X(t) be a random process and X(ti ) ≡ Xti . The nth
moment of the RV Xti is:
Z ∞
E(Xtni ) = xnti p(xti )dxti
−∞

If the process is SSS, PDF as well as the moment (hence, mXti )is indepen-
dent of time.

Autocorrelation: Consider 2 RVs, X(ti ), i = 1, 2.

Z ∞Z ∞
E(Xt1 Xt2 ) ≡ φ(t1 , t2 ) = xt1 xt2 p(xt1 , xt2 )dxt1 dxt2
−∞ −∞

If X(t) is SS stationary, p(xt1 , xt2 ) = p(xt1 +t , xt2 +t ). As a result, the auto-

correlation function is independent of the time instants, t1 and t2 . Rather,
depends on the time difference (t1 − t2 ). That is,

E(Xt1 Xt2 ) = φ(t1 , t2 ) = φ(t1 − t2 ) = φ(τ ).

Note that φ(τ ) = φ(−τ ). Important to note that,

φ(0) = E(Xt2 )

is the average power of the process X(t).

16
Autocovariance:

µ(t1 , t2 ) = E{[Xt1 − m(t1 )][Xt2 − m(t2 )]} = φ(t1 , t2 ) − m(t1 )m(t2 )

where m(ti ), i = 1, 2, is the mean of Xti . If X(t) is SS stationary,

µ(t1 , t2 ) = µ(t1 − t2 ) = µ(τ ) = φ(τ ) − m2

4.4 Wide Sense Stationary:

Wide sense stationary (WSS) is less stringent than strict-sense stationary (SSS)
in that mean and autocorrelation of the process are constant and de-
pendent on time difference respectively, but that is non-stationary process
(i.e., statistics such as PDF may not be stationary).

Stationary Gaussian Process: Let X(t) be Gaussian process. Then RVs

X(ti ), i = 1, 2, ..., n, at time instants ti are jointly Gaussian with mean m(ti ) =
m and autocovariance µ(ti , tj ) = µ(ti − tj ). Gaussian random process is com-
pletely characterized by mean and autocovariance functions. If Gaussian process
is WSS, it is SSS too since the PDF depends on mean and autocovariance.

4.5 Power Spectrum Density

• Frequency contents of any signal can reveal many features of a signal.

• Note the difference between deterministic signal vs random signal (e.g.,

cos (ωt) vs cos (ωt + θ)).

• Sample function of a random process is a random signal.

17
 • Fourier transform of the (deterministic) signal and Fourier transform of the
autocorrelation of the (random) signal are respectively used in frequency-
domain analysis.

The spectral characteristics (i.e., distribution of power at various frequency

components) of a stationary random process is computed by taking the Fourier
transform of the autocorrelation function. That is,
Z ∞
Ψ(f ) = φ(τ )e−j2πf τ dτ
−∞

Z ∞
φ(τ ) = Ψ(f )ej2πf τ df
−∞
R∞
The average power of the random signal is, φ(0) = −∞ Ψ(f )df = E(|Xt2 |).
Whether random process is real or complex, Ψ(f ) is real, that is, it contains
real information about the actual frequency contents of the (base-band or pass-
band) signal.

4.6 Input/Output Relationship in LTI Systems

Let x(t) be the sample function of the stationary random process X(t) and
h(t) be the impulse response of a LTI filter. Note that X(t) may be a complex
process in general. The output y(t) is a sample function of a process Y (t).
Hence,
Z ∞
y(t) = h(τ )x(t − τ )dτ
−∞

By convolution (a linear operation),

Z ∞ Z ∞
my = E[Y (t)] = h(τ )E[X(t − τ )]dτ = mx h(τ )dτ = mx H(0)
−∞ −∞

18
where H(0) is the frequency response of the system at f = 0. Therefore, the
mean of the output process is constant.
The autocorrelation function of the output signal,

1
φyy (t1 , t2 ) ≡ E(Yt1 , Yt∗2 )
2
Z Z
1 ∞ ∞
φyy (t1 , t2 ) = h(β)h∗ (α)E[X(t1 − β)X ∗ (t2 − α)]dαdβ
2 −∞ −∞
Since X(t) is stationary so is Y (t). Therefore,
Z ∞Z ∞
φyy (τ ) = h(β)h∗ (α)φxx (τ + α − β)dαdβ
−∞ −∞

If a WSS random process is passed through a LTI filter, then the

output process is also a WSS process.

It can be shown that the power spectrum density of the output is given by

Ψyy (f ) = Ψxx (f )|H(f )|2

Z ∞ Z ∞
Hence, φyy (τ ) = Ψyy (f )ej2πf τ df = Ψxx (f )|H(f )|2 ej2πf τ df
−∞ −∞

The average power of the output signal is

Z ∞
φyy (0) = Ψxx (f )|H(f )|2 df
−∞

♠ Example From text book, 2.2.1

Note the filtering of the (wideband) white noise results in (narrowband) noise
and the finite average power at the filter output.

4.7 Fourier Transform and Properties:

See the table.

19
5 Complex Signals and Systems

• Many digital signals are transmitted over a channel using some kind of
carrier modulation (e.g., DSB). Modulation is to generate bandpass (or
passband) signals from baseband signals.

• Representation of different forms of digitally modulated signals are impor-

tant to understand their spectral characteristics and hence efficient and
reliable transmission through the communication channels.

• We want to reduce all bandpass signals/systems to lowpass equivalent sig-

nals/systems for easy and consistent analysis. In doing so, bandpass sig-
nals/systems are represented as complex signals/systems.

♠ Time/Frequency domain description of lowpass and bandpass signals.

5.1 Bandpass Signals

|S(f)|

f
−fc 0 +f c

Note that

(a) ejθ = cosθ + jsinθ and e−jθ = cosθ − jsinθ

(b) Re(ζ) = 12 (ζ + ζ ∗ )

(c) cos2 (θ) + sin2 (θ) = 1

20
(d) cos2 (θ) − sin2 (θ) = cos(2θ) sin(2θ) = 2 sinθcosθ

(e) cos(A + B) = cosAcosB − sinAsinB

(f ) sin(A + B) = sinAcosB + cosAsinB

Let s(t) be a real-valued bandpass signal and fc be the carrier frequency. Also
let s+ (t) = s(t) + jŝ(t), where ŝ(t) is the Hilbert transform of s(t). Therefore,

S+ (f ) = 2u(f )S(f )

Hilbert transformer (or filter) is a 90o phase-shifter for all frequencies in the
input signal. It is given as,


−j, f > 0,
1
h(t) = and H(f ) =
πt 
+j, f < 0.

♠ Explanation of Hilbert transform using cos (wo t).

The analytical signal s+ (t) is a (complex) bandpass signal. By frequency-
shifting, we can obtain an equivalent lowpass signal. Let Sl (f ) = S+ (f + fc ).
Hence, the equivalent time-domain signal is

sl (t) = s+ (t)e−j2πfc t = [s(t) + jŝ(t)]e−j2πfc t

⇒ s(t) + jŝ(t) = sl (t)ej2πfc t

In general, the equivalent lowpass signal sl (t) is complex-valued and hence,

sl (t) = x(t) + jy(t),

where x(t) and y(t) are lowpass real signals. Note that s(t) is bandpass real signal
and sl (t) is lowpass equivalent of bandpass signal s(t).

21
 s l (t)
s(t) + j s(t)

fc

Quadrature Components: By equating real and imaginary parts in s+ (t) we

get,
s(t) = x(t)cos2πfc t − y(t)sin2πfc t

ŝ(t) = x(t)sin2πfc t + y(t)cos2πfc t

where x(t) and y(t) are quadrature components of s(t).

Envelope: Physical meaning of envelope? Example: m(t)cos(2πfc t), where

m(t) is a baseband signal.
Complex Envelope
s(t) = Re[sl (t)ej2πfc t ]

sl (t) is called complex envelope of real signal s(t).

Real Envelope
s(t) = a(t)cos[2πfc t + θ(t)]
µ ¶
p −y(t)
where a(t) = x2 (t) + y 2 (t) is the envelope and θ(t) = tan−1 x(t) is
the phase of the real signal s(t).

Frequency Response: Frequency response of the bandpass signal, s(t), and

equivalent lowpass signal, sl (t), are related via,
Z +∞ Z +∞
1
S(f ) = s(t)e−j2πf t dt = [sl (t)ej2πfc t + s∗l e−2πfc t ]e−j2πf t dt
−∞ 2 −∞

22
 1
S(f ) = [Sl (f − fc ) + Sl∗ (−f − fc )]
2
R +∞
Note that Sl∗ (f ) = −∞ s∗l (t)ej2πf t dt.

The energy in the narrowband bandpass signal is,

Z Z
1 ∞ 2 1 ∞
εs = s (t)dt = |sl (t)|2 dt
2 −∞ 2 −∞

5.2 Bandpass Systems

A real filter h(t) has:

H(f ) = Hl (f − fc ) + Hl∗ (−f − fc )

where H ∗ (−f ) = H(f ) and



H(f ), f > 0
Hl (f − fc ) =

0, f < 0.

Signal h(t) is given by,

h(t) = 2Re[hl (t)ej2πfc t ]

where the equivalent lowpass channel (or system or filter), hl (t), is in general,
complex-valued.

5.3 Response of a Bandpass System to a Bandpass Signal

Let s(t) be bandpass (input) signal and sl (t) is the equivalent lowpass signal.
Also, let h(t) be bandpass (system) response and hl (t) be equivalent lowpass
response.

23
 s(t) r(t)
h(t)

sl(t) rl (t)
hl (t)

We have just seen how s(t) and sl (t) (or h(t) and hl (t) are related in time
and frequency domain. The output of the filter is given by

r(t) = Re[rl (t)ej2πfc t ]

where r(t) = s(t) ? h(t).

It can be shown that
rl (t) = sl (t) ? hl (t)

Also ,
R(f ) = S(f )H(f ) and Rl (f ) = Sl (f )Hl (f )
1
R(f ) = [Rl (f − fc ) + Rl∗ (−f − fc )]
2
Note: The above equivalent characterization of signals and systems are valid
for narrowband signals and systems only. Otherwise, the energy in both sig-
nals/systems will not be equal.

5.4 Bandpass Random Process

Consider a stationary bandpass process. Similar to signals, relationships be-

tween autocorrelation function and power spectrum density of bandpass and
equivalent lowpass process can be derived.

24
 Let n(t) be a sample function of a narrowband WSS process with zero mean
and PSD of Φnn (f ).

Autocorrelation

n(t) = a(t)cos[2πfc t + θ(t)]

= x(t)cos2πfc t − y(t)sin2πfc t

= Re[z(t)ej2πfc t ],
p
where a(t) = x2 (t) + y 2 (t). Since n(t) is stationary,

φxx (τ ) = φyy (τ ) and φxy (τ ) = −φyx (τ )

Note that z(t) is low pass equivalent of n(t). It can be shown that

φnn (τ ) = φxx (τ )cos2πfc τ − φyx (τ )sin2πfc τ

Note the relationship between the autocorrelation function of the bandpass

process and the autocorrelation and cross-correlation functions of the quadra-
ture components of the process.

Power Spectrum

The lowpass equivalent process, z(t) = x(t) + jy(t). Therefore,

1 1
φzz (τ ) = E[z ∗ (t)z(t + τ )] = [φxx (τ ) + φyy (τ ) − jφxy (τ ) + jφyx (τ )]
2 2

φzz (τ ) = φxx (τ ) + jφyx (τ )

Hence,
φnn (τ ) = Re[φzz (τ )ej2πfc τ ]

25
That is, autocorrelation function of the bandpass random process is uniquely
determined from the autocorrelation of the equivalent lowpass process.
The power spectrum density is given by,
Z ∞
Φnn (f ) = {Re[φzz (τ )ej2πfc τ ]}e−j2πf τ dτ
−∞

1
Φnn (f ) = [Φzz (f − fc ) + Φzz (−f − fc )]
2
Note that the autocorrelation function, φzz (τ ), is an even function.

5.5 Bandpass White Noise Process

φnn(f)

No
2
B B
−f c +f c f

• White noise has a constant power spectrum density over the entire fre-
quency range, i.e., wideband noise signal. Hence, it cannot be expressed
in terms of quadrature components.

• Bandpass noise with constant spectrum density is achieved by passing the

white noise through an ideal bandpass filter.

• Now that we have a bandpass signal, we can use any of the above three
equivalent lowpass random process representation to analyze it.

26
6 Vector and Signal Space

6.1 Vector Space

A vector v = {v1 ...vn } in terms of its basis vectors, ei , 1 ≤ i ≤ n,

n
X
v= vi ei
i=1

vi is the projection of v onto the unit vector ei .

Inner Product: Let v1 = {v11 ...v1n } and v2 = {v21 ...v2n }.
n
X
v1 .v2 = v1i v2i
i=1

The inner product can also be defined as

(v1 .v2 ) =k v1 kk v2 k cosθ

where θ is the angle between the vectors.

k v1 + v2 k2 =k v1 k2 + k v2 k2 +2v1 .v2

If v1 .v2 = 0, v1 and v2 are orthogonal. Example is (1, 0, 0), (0, 2, 0), (0, 0, 1) in
3-D vector space.
Norm (or Length) of v is denoted by k v k and defined by
v
u n
uX
k v k= (v.v)1/2 = t vi2
i=1

Orthonormal: If two vectors are orthogonal and each has a unity norm, they
are orthonormal.

27
Gram-Schmidt Procedure for Orthonormalization of Vectors

Given a set of n-dimensional vectors, construct a set of orthonormal vectors.

v1
1. Select any vector (v1 ) and normalize it, u1 = kv1 k

2. Select the next vector (v2 ), subtract the projection of v2 onto u1 . Thus,
0
0 0 u2
u2 = v2 − (v2 .u1 )u1 and then normalize u2 . Hence, u2 = 0
ku2 k

3. Repeat until n orthonormal vectors are constructed.

♠ Orthonormalize (0, 0, 2), (0, 1, 1), (1, 0, 0)

6.2 Signal Space

Let x1 (t) and x2 (t) be (generally complex) signals. The inner product,
Z ∞
(x1 (t), x2 (t)) ≡ x1 (t)x∗2 (t)dt
−∞

• Orthogonal if (x1 (t), x2 (t)) = 0

• The norm is defined as

µZ +∞ ¶1/2
kx(t)k = |x(t)|2 dt
−∞

• A signal set is linearly independent if no signal can be represented in terms

of other signals.

Orthogonal Expansion of Signals

Let s(t) be a real-valued signal with finite energy,
Z ∞
²s = |s(t)|2 dt
−∞

28
Also let fn (t) be a set of K orthonormal functions. That is,

Z ∞ 
0, m 6= n,
fn (t)fm (t)dt =
−∞ 
1, m = n.

PK
s(t) is approximated by s̄(t) = k=1 sk fk (t), where {sk , 1 ≤ k ≤ K} are
coefficients in the approximation. Hence, the error in the approximation is

e(t) = s(t) − s̄(t)

Find {sk } so as to minimize the energy of the error signal ²e .

Z ∞· K
X ¸2
²e = s(t) − sk fk (t) dt
−∞ k=1

By mean-square-error (MSE) criterion, minimum ²e is obtained when the error

is orthogonal to each of the (basis) functions in the series expansion. Therefore,
Z ∞· K
X ¸
s(t) − sk fk (t) fn (t)dt = 0
−∞ k=1
Z ∞
⇒ sn = s(t)fn (t)dt, n = 1, ..., K
−∞

s̄(t) is obtained by projecting the signal s(t) onto K-dimensional signal space.
When ²e = 0, error signal has no energy and the set fn (t), (n = 1, ..., K) is a
complete set. Then,
Z ∞ K
X
2
²s = |s(t)| dt = s2k
−∞ k=1

♠ Find four orthogonal signals, cos(2πt), sin(2πt), ...

29
Gram-Schmidt Procedure for Orthonormalization of Signals

Let {si (t), i = 1, ..., M } be a set of finite energy signal waveforms. In general,
the orthonormalization of the kth function gives us,
0
fk (t)
fk (t) = √
²k

where
k−1
X
0
fk (t) = sk (t) − cik fi (t)
i=1
and
Z ∞
cik = sk (t)fi (t)dt, i = 1, ..., k − 1
−∞

Once orthonormal signal set is identified, then any signal in the signal space
can be expressed in terms of a linear combination of the orthonormal signals.

♠ How to develop basis functions (or signals)?


 


 1, 0 < t < 1,

1, 0 < t < 2, 

s1 (t) = s2 (t) = −1, 1 < t < 2,

0, otherwise. 




0, otherwise.


 

 1, 0 < t < 2,

 
−1, 0 < t < 3,
s3 (t) = −1, 2 < t < 3, s4 (t) =

 
0,

 otherwise.

0, otherwise.
q
1
1. f1 (t) = 2 s1 (t)
q
1
2. c12 = 0, therefore, f2 (t) = 2 s2 (t)

30
 √
3. c13 = 2 and c23 = 0, hence,


−1, (2 ≤ t ≤ 3),
0 √
f3 (t) = s3 (t) − 2f1 (t) =

0, otherwise.
0
f3 (t) = f3 (t)
0
4. f4 (t) = 0, eliminate this signal from the orthogonal set.

In signal form,
√
s4 (t) = − 2f1 (t) + f3 (t)

and in vector form,

√
s4 (t) = (− 2, 0, 1)

7 Source Coding

• Analog sources have infinite number of output values whereas digital

sources have finite number of output values. Sampling (in x-axis) and
quantization (in y-axis) produces digital signals from analog signals.

• Source encoder generates efficient digital sequence (i.e., binary: 0’s and
1’s) for transmission/storage.

7.1 Information Source Models

• Source output is random and hence is characterized by statistical terms.

• Binary source emits a binary sequence of the form 1010001001101 ... where
the output alphabet set consists of two letters {0, 1}, with transmitted
waveforms, for example, x0 (t) = cos (2πt) and x1 (t) = sin (2πt), 0 ≤ t ≤
T.
31
 • In general, a digital information source emits a sequence of letters (or sym-
bols) selected from a finite alphabet set of L possible letters (or symbols),
{x1 , ..., xL }.

♠L = 16, with 4 bits/symbol.

Let pk be the occurrence probability of a symbol xk , (1 ≤ k ≤ L). Hence,

L
X
pk = P (X = xk ), and pk = 1
k=1
• Source output sequence may be statistically independent (e.g., pixels in a
frame of a video clip with space).

They are known as Discrete Memoryless Sources (DMS) for which joint
PDFs can be expressed as the product of (individual) marginal PDFs.

• Source output sequence may be statistically dependent (e.g., English text).

We can have a mathematical model based on (statistically) stationary

property for which joint probabilities of any output sequence are invariant
with respect to time shift.

7.2 Nyquist Sampling Theorem

Let X(t) be a stationary random process with power spectral density ΦXX (f )
and ΦXX (f ) = 0 for |f | ≥ W (i.e., band-limited). By sampling theorem,
X∞ µ ¶ n
n sin [2πW (t − 2W )]
X(t) = X n
n=−∞
2W [2πW (t − 2W )]
where discrete samples X(m), m = 1, 2, ... are taken at the Nyquist sampling
rate of 2W samples/s.
¶ µ
n
♠ Bandwidth, sinc(x), construction of X(t) from X .
2W

32
It is important to note the following:

• Analog output, X(t) ⇒ equivalent discrete-time output, X(m).

• Output samples are generally continuous and cannot be represented in

digital form without loss in precision (lossy quantization process).

7.3 Measure of Information

Let xi and yj (i = 1, ..., n, and j = 1, ..., m) be two random variables. Let

P (X = xi |Y = yj ) ≡ P (xi |yj ).

What if X and Y are independent (e.g., raining in Toronto and NASDAQ index
going up or down) or dependent (e.g., wearing winter jackets in a snowy day)?
If X and Y are independent, then

P (xi |yj ) = P (xi ), no information is carried by yi about xi .

If X and Y are totally dependent (P (xi ) = P (yj )), then

P (yj )
P (xi |yj ) = = 1, full information is carried by yi about xi .
P (yj )
Hence, P (xi |yj ) carries some information!

7.3.1 Mutual Information

Mutual information between random variables xi and yj is defined by,

· ¸
P (xi |yj )
I(xi ; yj ) = log
P (xi )
From the above definition, statistically independent events carry no informa-
1
tion, i.e., I(.) = 0. Totally dependent events carry I(xi ; yj ) = log P (xi ) =

33
− log P (xi ) amount of information. Note that 0 ≤ P (xi ) ≤ 1. Also it can be
seen that, I(X; Y ) = I(Y ; X).

Self-Information: of the event X = xi ,

I(xi ) = − log P (xi )

Highly (low) probable event carries less (more) information.

Additivity Property: Let a discrete information source emit binary digits
(0 or 1) every T seconds with equal probability. Hence, the information
measure (content) is

I(xi ) = −log2 P (xi ), xi ∈ {0, 1}

1
= −log2 = 1 bit (note the base of 2)
2
Let us now consider a block of K binary digits that are statistically in-
dependent (DMS model). There are M = 2K possible K-bit blocks, each
with equal occurrence probability of 2−K . Note that

p(x1 ...xN ) = p(x1 )...p(xN )

Hence the self-information of the block is

I(x1 ...xK ) = −log2 (2−K ) = K bits.

Hence, the logarithmic measure of information possess the additive prop-

erty for independent discrete sources.

34
 P(X=0) = P(X=1) = 0.5
1−po
0 0
p
1
Input (X) Output (Y)
po
1 1
1−p1

Information Carried by Binary Input Binary Output (BIBO) Channel

Given that Y = 0, what is the mutual information about the occurrence of

X=0?
P (x|y)
I(x; y) = log2
P (x)
Since I(x; y) = I(y; x),

P (Y = 0|X = 0)
I(X = 0; Y = 0) = I(Y = 0; X = 0) = log2
P (Y = 0)

Also,

P (Y = 0) = P (Y = 0|X = 0)P (X = 0) + P (Y = 0|X = 1)P (X = 1),

1
⇒ P (Y = 0) = (1 − po + p1 )
2
· ¸
2(1 − po )
Hence, I(0; 0) = log2
1 − po + p1
If po = p1 = 0 (noiseless channel), then I(0, 0) = 1.
If po = p1 = 21 , then I(0, 0) = 0 (useless channel).

35
7.3.2 Entropy

Average mutual information is defined as,

n X
X m n X
X m · ¸
P (xi , yj )
I(X; Y ) = P (xi , yj )I(xi ; yj ) = P (xi , yj )log
i=1 j=1 i=1 j=1
P (xi )P (yj )

If X and Y are independent, I(X; Y ) = 0.

The entropy of X, H(X), is defined as average self-information per letter
(or symbol) where X represents all possible output letters (or symbols) from a
source. That is,
n
X n
X
H(X) = P (xi )I(xi ) = − P (xi )log P (xi ).
i=1 i=1

If P (xi ) = 1/n, ∀i, then H(X) = log n. If n = 16, H(X) = 4 bits/symbol.

It can be shown that, H(X) ≤ log n. That is, the entropy of a discrete
source is maximum when the output symbols are equally probable.
Let us consider a BSC where po = p1 = p. Also let P (X = 0) = q and
P (X = 1) = 1 − q. Hence,

H(X) = −qlog (q) − (1 − q)log (1 − q).

From the figure, H(X) is maximum when q = 21 .

Conditional entropy is defined as
n X
X m n X
X m
H(X|Y ) = P (xi , yj )I(xi |yj ) = − P (xi , yj )log P (xi |yj )
i=1 j=1 i=1 j=1

• H(X|Y ) can be interpreted as the average amount of uncertainty (condi-

tional self-information) in X after we observe Y .

36
 Figure 3.1−1
Binary entropy function.

• Uncertain events have large information (surprises!) and certain events

(i.e., highly predictable) have small information (no excitement!). Hence,
entropy of events with near-zero probability have high information.

• H(X) can be interpreted as average amount of uncertainty (self-information)

prior to the observation

• It can be shown that the mutual information, I(X; Y ),

I(X; Y ) = H(X) − H(X|Y )

Therefore, I(X; Y ) is the average amount of uncertainty (mutual informa-

tion) provided by X by the observation of Y .

• Note that conditioning decreases the uncertainty.

7.4 Coding for Discrete Sources

• H(X) measures the average information in the discrete source output.

• Encoding process: representing the source output efficiently by a sequence

of binary digits (i.e., compression).

37
 • Coding efficiency : compare the average number of binary digits per output
symbol with source entropy, H(X). It is defined as H(X)/R̄ where R̄ is
the average number of bits required to represent a symbol.

Let us consider a DMS source that produces an output symbol xi , i = 1, ..., L,

every T seconds. Each symbol occurs at the source output with probability,
P (xi ). The average self information (i.e., entropy) is,

H(X) ≤ log2 L.

Effective source rate is H(X)/T bits/s.

7.4.1 Fixed Length Coding

Consider a block encoding scheme in which a unique set of R binary digits

(i.e., code words) are assigned to each symbol. Total number of symbols is L
(assume L is a power of 2). Hence, R = log 2 L. Hence, the average number of
bits/symbol,
L
X
R̄ = R × P (xi ) = R.
i=1
Since, H(X) ≤ log2 L, R ≥ H(X). In fixed-length coding scheme with
H(X) H(X)
equiprobable occurrences, the coding efficiency is R̄
= R =1
However, when L is not a power of 2, then R = blog2 Lc + 1 and the coding
log2 L
efficiency is ≤ . Then, if L À 1, the efficiency is high and if L is small,
blog2 Lc+1
efficiency is low.

38
 Shannon Source Coding Theorem:
Let X be an alphabet set (consisting of output symbols) with entropy H(X).
Block of J symbols are encoded into code words of length N digits from a binary
alphabet. The symbols are not uniquely encoded and therefore decoding error
occurs.
The probability of a block decoding failure (Pb ) can be made very small if

N
R̂ = ≥ H(X)
J

and J is sufficiently large.

In other words, the average number of bits per symbol required to encode the
output of a DMS with near-zero decoding error probability is lower bounded
by its entropy (i.e., average information content).
Let X = {x0 , x1 , x2 } and it is given that H(X) = 3/4. If J = 3, N = 2,
i.e., a block of 3 symbols (x0 x1 x2 , x0 x2 x1 , x1 x0 x2 ) is encoded with unique code
word (11, 01, 10) respectively. The rest of the possible blocks are encoded to one
2
code word (00). In this example, J = 3, N = 2 and H(X) = 3/4 > 3 = N/J.
Therefore, decoding failure probability will approach to 1. On the other hand,
3
if J = 3, N = 3, then H(X) = 3/4 < 3 = 1 and the decoding failure can be
made smaller and, in fact to zero because of unique coding.

7.4.2 Variable Length Coding

When source symbols are not equally probable, use variable-length code words.
Idea is to assign short code words for high probable source symbols (known as
entropy coding).

39
 ♠ Example: Variable Length Coding

Symbol P (ak ) Code I Code II Code III

1
a1 2 1 0 0
1
a2 4 00 10 01
1
a3 8 01 110 011
1
a4 8 10 111 111

Code I has problem in decoding. What is the decoded sequence if the received
sequence is 001001 ? a2 a1 a2 a1 or a2 a4 a3 ?
The prefix condition requires that no code word is a prefix of any other code
word. Code III is also not uniquely decodable.

Huffman Coding Algorithm

(a) Variable-length code words, (b) prefix condition will be satisfied, (c) aver-
age number of binary digits to represent the source symbols is minimum, (d)
uniquely and instantaneously decodable, (e) symbol probabilities are needed
and (f) code words may not be unique but all achieve the same efficiency.

♠ Example using Huffman Coding

P7
The average number of bits/symbol, R̄ = k=1 nk P (xk ), where xk is the symbol
and nk is number of bits in the code word to represent xk .
It can be seen that H(X) = 2.11 and R̄ = 2.21 in the below example
(efficiency = 95%).

40
 Symbol Probability Code word
x1 0.35 00
x2 0.30 01
x3 0.20 10
x4 0.10 110
x5 0.04 1110
x6 0.005 11110
x7 0.005 11111

Figure 3.3−4
An example of variable−length source
encoding for a DMS.

Instead of encoding symbol-by-symbol, if we do the encoding blocks of sym-

bols (e.g., pair), R̄ can be decreased (see example 3.3.3).

7.5 Coding for Analog Sources

• When X(t) is a band-limited stationary random process, the sampling

theorem allows us to represent X(t) by a sequence of uniform samples
taken at the Nyquist rate.

41
 • Samples are then quantized in amplitude and encoded with finite number of
bits. Output is grouped into L levels with R bits/sample where R = log 2 L.
Example, in PCM (used in speech coding), L = 4096, R = 12.

• Entropy (Huffman) coding can be used to efficiently represent (i.e., com-

press) the discrete sequences. However, distortion is also introduced, be-
cause continuous levels are converted into discrete levels.

Rate Distortion Function: Let xk and x̂k be actual and quantized values.
Squared-error distortion is: d(xk , x̂k ) = (xk − x̂k )2 . With n samples,
n
1X
d(Xn , X̂n ) = d(xk , x̂k )
n
k=1

Expected value of d(Xn , X̂n ) is defined as distortion D. Therefore,

n
1X
D = E[d(Xn , X̂n )] = E[d(xk , x̂k )] = E[d(x, x̂)]
n
k=1

(assuming uniform quantization).

Rate distortion function R(D) is defined as the minimum number of bits
per source output that is required to represent the output X of the memo-
ryless source with distortion D.
Rate Distortion Function for Memoryless Gaussian Source


 1 log 2 (σx2 /D), 0 ≤ D ≤ σx2
2
RG (D) =

0, D > σx2 .

where σx2 is the variance of the Gaussian source output. Distortion-rate function
(for the above Gaussian source) is, Dg (R) = 2−2R σx2 . In dB, −6R + 10log 2
10 σx .

Note that mean-square error distortion decreases at a rate of 6dB/bit. What

happens when D > σx2 ?

42
8 Channel Capacity

8.1 Channel Coding

Introduce redundancy in the binary information sequence at the channel en-

coder that can be used at the receiver to overcome the effects of noise and
interference encountered in the physical channel.

k raw bits → n coded bits (or code word)

n
Code rate is denoted
µ ¶ as (k,n). The raw rate becomes k with the amount of
redundancy = n−k k %. Example: a1 a2 → a1 a2 a1 ⊕ a2 .

• Sequence of coded bits are input into the modulator which transmits one
symbol for each r bits (using one waveform from M = 2r possible wave-
forms). Orthogonal signals can make Pe smaller but with larger bandwidth

0 T 0 T
M = 4, r = 2
0 T 0 T
B=W B = 2W

W
expansion factor, Be = R, where R is the channel bit rate. Be grows ex-
ponentially with block length, r.

• Demodulator produces an estimate of the transmitted data symbols (bi-

nary or M-ary / scalar or vector) that have been possibly corrupted.

43
 • Detector outputs 0’s and 1’s (based on soft/hard decision rule).

• Channel decoder exploits the redundancy available in the code word to

compensate for the channel impairments.

8.2 Channel Models

Channel Demodulator, Channel

Modulator Channel
data encoder detector decoder data
Composite Channel

Discrete-Input Discrete-Output Channels

BIBO and BSC are examples. In general, we can have q−ary inputs and Q-ary
outputs. q = Q = 2 gives BIBO channel.

Discrete-Input Continuous-Output Channels

Let a channel encoder output consists of symbols selected from a finite alphabet
set, X = {x0 , ..., xq−1 } and Q = ∞ (un-quantized input to the channel decoder).
The composite channel can be characterized by the conditional PDFs as ,

P (y|X = xk ), k = 0, 1, ..., q − 1.

Let us consider a AWGN channel. Then,

Y =X +G

44
where G is a zero-mean Gaussian RV with variance σ 2 . For a given X = xk , Y
is Gaussian with mean xk and variance σ 2 . Therefore,

1 2 2
P (y|X = xk ) = √ e−(y−xk ) /2σ
2πσ
If we transmit n symbols through the memoryless channel, we can say,
n
Y
P (y1 , ..., yn |X = x1 , ..., Xn = xn ) = P (yi |Xi = xi )
i=1

Continuous-Input Continuous-Output Channels

Let us suppose that the inputs to the channel and outputs from the channel
are continuous. Let x(t) be a band-limited (to W ) input process to a AWGN
channel (with PSD of 21 N0 ). Let n(t) be the sample function of the noise process.
Hence, the output process,

y(t) = x(t) + n(t)

By expanding the signals with orthornormal functions,

X X X
x(t) = xi fi (t) n(t) = ni fi (t) y(t) = yi fi (t)
i i i

where {xi }, {ni }, {yi } are the sets of coefficients in the corresponding expansion.
The functions {fi (t)} form a complete orthonormal set over (0, T ), i.e.,

Z T 
1, i = j,
fi (t)fj (t)dt =
0 
0, i 6= j.

• Any orthonormal functions can be used in the expansion of white Gaussian

n(t)

45
 • Note that ni ’s are uncorrelated and hence statistically independent Gaussian
RVs. Hence, yi ’s are also statistically independent Gaussian RVs. Hence,
Z T Z T
yi = y(t)fi (t)dt = [x(t) + n(t)]fi (t)dt = xi + ni
0 0

• Now we can work with the coefficients in characterizing the channel. Hence
1 2 2
P (yi |xi ) = √ e−(yi −xi ) /2σi , i = 1, 2, ....
2πσi
• Continuous waveform channel has been reduced to equivalent discrete
channel.

• Samples of x(t) and y(t) can be taken at Nyquist rate of 2W samples/s.

Hence, in an interval of T , a total of 2W T samples will be taken. Hence,
2W T transition probabilities are enough to describe the AWGN channel.

• The variance of yi is, σi2 = 21 N0 . σi2 = E[yi2 ] = Average power contained in

sample yi .

8.3 Capacity of BSC and AWGN Channels

Consider a discrete memoryless channel having an input alphabet X = {x0 , ..., xq−1 }
and an output alphabet Y = {y0 , ..., yQ−1 } with the set of transition probabili-
ties, P (yi |xi ).
The mutual information provided about X = xi by the occurrence of Y = yi
is, · ¸
P (yi |xi )
I(xi ; yi ) = log
P (yi )
The probability of Y = yi is,
q−1
X
P (yi ) = P (xk )P (yi |xk ).
k=0

46
Therefore, the average mutual information provided by the output Y about the
input X is given by,
q−1 X
X Q−1 · ¸
P (yi |xj )
I(X; Y ) = P (xj )P (yi |xj )log
j=0 i=0
P (yi )

• Channel characteristics determine P (yi |xj )

• Probabilities of input symbols, P (xj ), ∀j, depend on the channel encoder

• Goal is to maximize the average mutual information, I(X; Y ). If we do

it over all P (xj )’s, then the maximized quantity depends only on channel-
dependent P (yi |xj ).

The channel capacity is defined as,

C = max I(X; Y )
P (xj )
Pq−1
under the constraints that P (xj ) ≥ 0 and j=0 P (xj ) = 1. The units of C is
bits/symbol and the channel rate is C/T , if a symbol enters the channel every
T seconds.
♠ Capacity of a BSC

Given that P (0|1) = P (1|0) = p and P (0) = P (1) = 0.5. The capacity of a
BSC is
C = plog 2p + (1 − p)log 2(1 − p) = 1 − H(p)

where H(p) is the source entropy.

♠ AWGN Channel: Discrete Input and Continuous Output

Transition probabilities are given as,

1 2 2
P (y|X = xk ) = √ e−(y−xk ) /2σ
2πσ
47
 Figure 7.1−4
The capacity of a BSC as a function of the error
probability p.

where X = {x0 , ..., xq−1 } and Y = {−∞, +∞}. Therefore,

q−1 Z ∞
X P (y|xi )
C = max P (y|xi )P (xi )log2 dy
P (xi )
i=0 −∞
P (y)
where
q−1
X
P (y) = P (y|xk )P (xk )
k=0
For example, let X = {A, −A} and P (A) = P (−A) = 0.5. I(X; Y ) is maxi-
mized when X’s are equi-probable.
Therefore, the capacity of binary-input AWGN memoryless channel is,
Z · ¸ Z · ¸
1 ∞ P (y|A) 1 ∞ P (y| − A)
C= P (y|A)log2 dy + P (y| − A)log2 dy
2 ∞ P (y) 2 ∞ P (y)
Note that not always do the equi-probable input symbols nor the symmetric
transition probabilities maximize the average mutual information (or the chan-
nel capacity).

8.4 Shannon’s Channel Capacity

Let us consider a band-limited channel with AWGN. The Shannon’s capacity

is defined as,
1
C = lim max I(X; Y )
T →∞ p(x) T

48
Let us use the samples (or coefficients) to determine I(X; Y ). We want to
find the mutual information between XN = [x1 ...xN ] and YN = [y1 ...yN ], where
N = 2W T and yi = xi + ni .
The capacity of the band-limited AWGN channel with a band-limited and
average power-limited input is shown to be,
µ ¶
Pav
C = W log 2 1 + = W log 2 (1 + SNR)
W N0

where Pav is the average power of x(t). The relationship between SNR and C

Figure 7.1−7
Normalized channel capacity as a function
of SNR for band−limited AWGN channel.

is given in the figure.

• For a fixed SNR, C is proportional to W .

• If W is fixed, C increases with Pav in a channel.

• If Pav is fixed, C can be increased by increasing W in a channel.

Pav = Cεb , εb is energy per bit.

εb 2C/W − 1
=
N0 C/W
µ ¶
εb
lim → ln 2
C
W →0
N0

49
 Figure 7.1−8
Channel capacity as a function
of bandwidth with a fixed
transmitted average power.

Noisy Channel Coding Theorem: There exist channel codes that can
achieve reliable communication with Pe → 0, if the transmission rate R < C.

9 Digitally Modulated Signals

• Modulator is as an interface that maps digital information (i.e., sequence

of binary digits) into analog waveforms.

• Random channel characteristics and random inputs have to be considered

in the (deterministic) mapping of symbols into waveforms.

• Performance measure: bandwidth of waveforms, signal energy and proba-

bility of bit or symbol error.

9.1 Memoryless Modulation

Modulator with memory operates on current and previous digits (or waveforms).
The waveforms may differ in:

Amplitude: binary PAM (or ASK); carrier-modulated PAM: DSB, SSB

50
Phase: BPSK, QPSK; and QAM (PAM-PSK)

Frequency: FSK, MSK

9.1.1 Amplitude Shift Keying (ASK)

ASK signals are also known as Pulse-Amplitude-Modulated (PAM) signals.

sm (t) = Am g(t)cos 2πfc t, m = 1, 2, ..., M, 0≤t≤T

where Am denotes the set of M possible amplitudes corresponding to M (= 2k )

possible k−bit blocks of symbols.

• Am = (2m − 1 − M )d, where 2d is the distance between adjacent signal

amplitudes. ♠M = 4 : (−3d, −d, +d, +3d)

• The waveform g(t) is a real-valued signal pulse whose shape influences the
spectrum of the transmitted signal (flexible in spectrum shaping)

• If input rate to the modulator is R bits/s, then the symbol rate is R/k
symbols/s

51
The energy of signal, sm (t) is
Z T Z
A2m T 2 A2
εm = 2
sm (t)dt = g (t)dt = m εg
0 2 0 2

where εg is the signal energy in the pulse g(t).

Let sm (t) = sm f (t) where f (t) is the unit-energy signal. Note that one-
q
ε
dimensional signal set is enough to expand the waveform. Hence, sm = Am 2g .
The Euclidean distance between amplitudes of signals sm (t) and sn (t) is,
r
p εg p
d(e)
mn = (sm − sn )2 = |Am − An | = d 2εg |m − n|.
2

Therefore, minimum Euclidean distance is

(e) p
dmin = d 2εg

The signal-space diagrams are shown in the figure for different M. They are
also known as constellations.
Antipodal Signal: M = 2, s1 (t) = −s2 (t), ε1 = ε2

Gray Encoding: Mapping of k information bits to M possible amplitudes

is done in such a way that adjacent signal amplitudes differ by one binary digit.
In the case of (most likely) errored detection of the transmitted symbol, only a
single bit error is occurred.
The digital PAM signal can also be used in baseband transmission (i.e., no
modulation). In such a case sm (t) = Am g(t). Figure shows the baseband PAM
and Passband PAM.

52
9.1.2 Phase Shift Keying (PSK)

In digital phase modulation, the M signal waveforms are represented as

· ¸
2π(m − 1)
sm (t) = g(t) cos 2πfc t + , m = 1, ..., M, 0 ≤ t ≤ T.
M

sm (t) = g(t) cos θm cos 2πfc t − g(t) sin θm sin 2πfc t

where θm = 2π(m − 1)/M.

• θm ’s are possible phases of the carrier that convey the transmitted infor-
mation.

εg
• These PSK signals have equal energy of 2 for all the transmitted waveforms
where εg is the signal energy of the pulse g(t).

53
 • These signals are two-dimensional and hence can be represented as a linear
combination of two orthonormal signal waveforms, f1 (t) and f2 (t).

Therefore,
sm (t) = sm1 f1 (t) + sm2 f2 (t)

where, s
2
f1 (t) = g(t) cos 2πfc t
εg
s
2
f2 (t) = − g(t) sin 2πfc t
εg
The 2D vector is given by,
·r r ¸
εg 2π(m − 1) εg 2π(m − 1)
sm = cos sin
2 M 2 M
The minimum Euclidean distance between adjacent phases of the signals is,
s · µ ¶¸
(e) 2π
dmin = εg 1 − cos
M
Signal space diagrams are shown in the figure. M = 2 (BPSK) corresponds
to binary PAM (also antipodal signals). The preferred mapping of symbols to
waveforms is using the Gray coding.

♠Constellation and Waveforms forM = 2, 4, 8.

9.1.3 Quadrature PAM (QAM)

It can be viewed as the combination of ASK and PSK. In QAM, two sym-
bols are efficiently modulated using two quadrature carriers, cos (2πfc t) and
sin (2πfc t). Hence,

sm (t) = Amc g(t) cos (2πfc t) − Ams g(t) sin 2πfc t.

54
 ♠Constellation and Waveforms forM = 8
π
Note the 4 shift to increase the distance between signal points.

9.1.4 Frequency Shift Keying (FSK)

Let us consider M signals that have equal energy of ε with duration of T .

r · ¸
2ε
sm (t) = cos 2π(fc + m4f )t , m = 1, ...M, 0 ≤ t ≤ T.
T

The cross-correlation coefficient between signals sm (t) and sk (t) is

sin [2πT (m − k)4f ]

ρkm = .
2πT (m − k)4f
1
For every 2T , ρkm is zero, i.e., signals are orthogonal. Since |m − k| = 1
1
corresponds to adjacent frequency slots, 4f = 2T represents the minimum

55
frequency separation between orthogonal signals. M (orthogonal) FSK signals
can be represented as vectors
√
s1 = [ ε 0 0... 0 0]
√
sM = [0 0 0... 0 ε]

• The Euclidean distance between pairs of signal is

(e)
√
dkm = 2ε, ∀m, k

• FSK signals are generated by shifting the carrier frequency fc by

1
4fn = 4f In , In = ±1, ±3, ..., ±M − 1.
2
1
• The frequency shift is at least 4f = 2T for orthogonality.

1
• Since 4f = 2T , the smaller pulse (i.e., high speed communication) re-
quires larger frequency separation, and hence the FSK signal bandwidth
is increased.

• ♠fc = 106 Hz, T = 5µs

56
 • ♠Constellation for M = 4, In = ±1, ±3

4f 4f 34f 34f
4f1 = , 4f2 = − , 4f3 = , 4f4 = −
2 2 2 2

9.2 Modulation with Memory

• There exists a dependence between the signals transmitted in the successive

symbol intervals.

• Dependence is introduced to shape the spectrum of the transmitted signals

to match the channel characteristics.

• This is achieved by encoding the data sequence at the modulator input by

means of modulation code.

Example with NRZI

Let us consider a modulation scheme where ”0” and ”1” are represented by a
rectangular pulse of −A and A respectively.

NRZ

NRZI

data 1 0 1 1 0 0 0 1

• Memory is introduced in this signal by the pre-coding operation.

57
 • Transition occurs only when a ”1” is transmitted. This is differential en-
coding and is described mathematically as:

bk = ak ⊕ bk−1

where ak is the binary information sequence into the encoder and bk is the
output sequence of the encoder.

• The output of the encoder (b0k s) is mapped into one of two waveforms.

9.2.1 Continuous Phase FSK (CPFSK)

CPFSK is a continuous phase modulation scheme.

• FSK signals are generated by shifting the carrier frequency by

1
fn = 4f In , In = ±1, ±3, ..., ±M − 1.
2

• Within the symbol duration of T, oscillator has to tune to one of M fre-

quencies. This abrupt switching (i.e., m∆f, m = 1, 2..) results in large
spectral side lobes and hence the FSK signal bandwidth is increased.

Solution: information-bearing signal frequency-modulates a single carrier whose

frequency is changed continuously. The phase of the carrier is also continuous
(CPFSK). It is a modulation scheme with memory.

♠ Rectangle pulse and constant modulation index.

Let d(t) be a PAM signal,

X
d(t) = In g(t − nT ),
n

58
 1
where g(t) is a rectangle pulse of duration T with amplitude 2T . {In } is the
sequence of amplitudes obtained by mapping k-bit blocks of binary digits from
symbols {an } into different levels, M = 2k . Therefore, the carrier-modulated
signal with equal-energy of ε,
r
2ε
s(t) = cos [2πfc t + φ(t; I)],
T
where
Z t Z t X
φ(t; I) = 4πT fd d(τ )dτ = 4πT fd [ In g(τ − nT )]dτ
−∞ −∞ n

where fd is the maximum frequency deviation.

R
Note that d(t) may have discontinuities but d(t) is continuous.
The phase of the carrier in interval, nT ≤ t ≤ (n + 1)T is given by

φ(t; I) = θn + 2πhIn q(t − nT ), nT ≤ t ≤ (n + 1)T

where,
h = 2fd T
n−1
X
θn = πh Ik
k=−∞




 0 t < 0,


q(t) = t/2T 0 ≤ t ≤ T,





1/2 t > T.
• Note the accumulation of memory in θn .
• h is called modulation index.
• In general,
Z t
q(t) = g(τ )dτ.
0

59
 Pulse shapes for CPFSK signals.

Phase Trees: Phase of the PSK signals are described using phase-evolution
tree.
Let In = ±1 be binary symbols and the corresponding phase tree is shown in
the figure. Assume θ0 = 0, g(t) is a rectangle pulse with duration T.
Smoother phase trajectories can be obtained using pulses that do not contain
discontinuities.

9.2.2 Minimum Shift Keying (MSK):

Binary CPFSK with h = 12 . Hence, the maximum frequency deviation, fd = 1

4T

and µ ¶
t − nT
φ(t; I) = θn + πIn , nT ≤ t ≤ (n + 1)T
2T
Therefore, · µ ¶¸
t − nT
s(t) = A cos 2πfc t + θn + πIn
2T

60
 Phase trajectory for binary CPFSK signal with
rectangular pulse.

· µ ¶ ¸
1 π
s(t) = A cos 2π fc + In t − nIn + θn
4T 2
The binary (In = ±1) CPFSK signals can be expressed as a sinusoid having
one of two possible frequencies in the interval nT ≤ t ≤ (n + 1)T . Let

1 1
f1 = fc − f2 = fc +
4T 4T
1 1
Minimum frequency 4f = 2T . Hence, binary CPFSK with h = 2 is called MSK.
Comparison of MSK and QPSK signals:

• MSK signals have continuous phase

• QPSK has either ±π or ±π/2 phase jumps
• OQPSK has ±π/2 phase jumps

See the annexed figure.

61
10 Spectra of Digital Signals

• Spectral content of the digitally modulated signals is important.

• Information sequence is random and hence the modulated signal is a ran-

dom process.

• PSD will be used in the design and analysis of signals.

Let us consider a digital linear modulation with no carrier modulation in which,

∞
X
v(t) = In g(t − nT ),
n=−∞

where {In } is a sequence that is obtained by mapping of symbols {an }. Note

that In is real in PAM and complex in PSK or QAM.
The autocorrelation function of v(t) is
∞ ∞
1 ∗ 1 X X
φvv (t; t+τ ) = E[v(t)v (t+τ )] = E[In∗ Im ]g ∗ (t−nT )g(t−mT +τ ).
2 2 n=−∞ m=−∞

Assume that {In } is wide-sense stationary with mean µi and autocorrelation

φii (m) = 12 E[In∗ In+m ]. Then,
∞
X
E[v(t)] = µi g(t − nT ).
n=−∞

It can also be shown that φvv (τ ) is periodic. That is,

φvv (t + T ; t + τ + T ) = φvv (t; t + τ )

Hence, v(t) is a cyclo-stationary process and PSD is computed by averaging

out within the symbol period T . Therefore,
Z
1 T /2
φ̄vv (τ ) = φvv (t; t + τ )dt
T −T /2

62
The Fourier transform of φ̄vv (τ ) gives average PSD of v(t) as,

1
Φvv (f ) = |G(f )|2 Φii (f )
T

where G(f ) is the Fourier transform of g(t) and Φii (f ) is the PSD of the infor-
mation sequence which is defined as,
∞
X
Φii (f ) = φii (m)e−j2πf mT .
m=−∞

That is, the spectral characteristics of v(t) can be controlled by design of

pulse g(t) as well as correlation properties of the information sequence.

10.1 Spectral Shaping without Precoding

If symbols are mutually uncorrelated, then



σi2 + µ2i , m = 0,
φii (m) =

µ2 ,
i otherwise,

where σi2 denotes the variance of the information sequence {In }.

For uncorrelated real symbols,
∞
σi2 2 µ2i X m
Φvv (f ) = |G(f )| + 2 |G( )|2 δ(f − m/T )
T T m=−∞ T

The first term is continuous spectrum and second term consists of discrete
frequency components spaced 1/T apart in frequency.
What if {In } are equi-probable and symmetrically positioned in the signal
space? µi = 0.

63
 Rectangular pulse and its energy density spectrum

10.1.1 Rectangular pulse

Let g(t) be a rectangular pulse with height A and duration T .

sin (πf T ) −jπf T
G(f ) = AT e
(πf T )
This spectrum contains zeros at multiples of 1/T in frequency and it decays in
1
proportion to f2 . Therefore,
µ ¶2
sin πf T
Φvv (f ) = σi2 A2 T + A2 µ2i δ(f )
πf T

10.1.2 Raised cosine pulse

· ¸
A 2π
g(t) = 1 + cos (t − T /2) , 0 ≤ t ≤ T.
2 T
AT sin πf T
G(f ) = 2 2
e−jπf T
2 πf T (1 − f T )
Power spectrum of the raised cosine decays faster but wider than rectangle
pulse.

64
 Raised cosine pulse and its energy density spectrum

10.2 Spectral Shaping with Precoding

Let
In = bn + bn−1

Assume {bn }’s are uncorrelated random variables each having zero mean and
unit variance. Then 



 2, m = 0,


φii (m) = 1, m = ±1,





0, otherwise.

The PSD of the input sequence is

Φii (f ) = 2(1 + cos (2πf T )) = 4 cos 2 (πf T )

and the corresponding PSD of the (lowpass) modulated signal is

4
Φvv (f ) = |G(f )|2 cos 2 (πf T )
T

65
10.3 PSD of Bandpass Signals

In previous sections, we used baseband signals to analyze the frequency contents

of the digital signals using PSD. Let s(t) be a band-pass signal,

s(t) = Re[v(t)ej2πfc t ],

where v(t) is the equivalent low-pass signal of s(t). The autocorrelation function
of s(t) is,
φss (τ ) = Re[φvv (τ )ej2πfc τ ].

Hence, the PSD is

1
Φss (f ) = [Φvv (f − fc ) + Φ∗vv (−f − fc )].
2

11 Receiver Design for AWGN Channel

When the channel is corrupted by AWGN,

• design of optimum receiver

– demodulator: correlator, matched filter

– symbol detector (MAP, ML and MD criterion)

– sequence detector (ML, Viterbi algorithm)

• performance analysis (Pe )

Consider M-ary transmission with signal waveforms

sm (t), 0 ≤ t ≤ T, m = 1, 2, ..., M.

66
Let the dimension of the M-ary signal set be N , i.e., the orthonormal functions
fn (t), n = 1, 2, ..., N, span the signal space.
Received signal in the symbol interval T is,

r(t) = sm (t) + n(t),

where n(t) is the sample function of the AWGN process with PSD of Φnn (f ) =
N0
2 .

r(t)
S1 (t)

lT (l+1)T

S2 (t)
Binary anti−podal signals:
S2 (t) = − S1 (t)

R
In the figure, intuitively, if r(t)dt > 0, then we can declare that the trans-
mitted pulse is s1 (t)

• Observe r(t) over T and then decide what was transmitted to the best of
knowledge. That is, design a receiver that is optimum in the sense that it
minimizes the probability of making an error.

Receiver consists of a demodulator and a detector.

Demodulator converts the received signal r(t) into N dimensional vector by

projecting r(t) onto N orthonormal functions, r= [r1 r2 ... rN ].

Detector decides which of the M possible waveforms was transmitted based

on received vector r.

67
11.1 Demodulator

Two optimum realizations of the demodulator: correlator and matched filter.

11.1.1 Correlator

• The signal sm (t) and the noise n(t) are expanded into a series of linearly
weighted orthonormal functions.

• Note that {fn (t)} does not span the noise space.

• The received signal r(t) is passed through a parallel bank of N cross cor-
relators that compute the projection of r(t) onto N orthonormal functions
{fn (t)} over T , as shown in the figure.

Consider k th branch between time interval lT and (l + 1)T ,

Z (l+1)T Z (l+1)T
r(t)fk (t)dt = [sm (t) + n(t)]fk (t)dt
lT lT

68
 ⇒ rk = smk + nk , k = 1, 2, ...N,

where
Z (l+1)T Z (l+1)T
smk = sm (t)fk (t)dt and nk = n(t)fk (t)dt.
lT lT

• Transmitted signal sm (t) is represented by sm = [sm1 ... smN ].

• The components {nk } are random variables that arise due to the presence
of additive noise.

• The received signal r(t) can now be written as,

N
X 0
r(t) = rk fk (t) + n (t)
k=1

0
It can be shown that n (t) and rk are statistically independent and consequently,
0
n (t) is irrelevant to the decision as to what was transmitted. Therefore, the
decision can be based on the correlator output (signal and noise) components,
rk = smk + nk .

Statistics of {nk } and {rk }

N noise components {nk } are zero-mean uncorrelated Gaussian RVs with equal
variance of σn2 = 21 N0 . Therefore, the correlator outputs {rk } conditioned on the
mth signal being transmitted are Gaussian RVs with mean and variance given
by,
E(rk ) = E(smk + nk ) = smk
1
σr2 = σn2 = N0
2

69
The correlator outputs (rk ) are statistically independent Gaussian RVs. There-
fore,
N
Y
P (r|sm ) = P (rk |smk ), m = 1, 2, ..., M,
k=1
where, · ¸
1 (rk − smk )2
P (rk |smk ) = √ exp −
πN0 N0
Joint conditional PDFs,
· X N ¸
1 (rk − smk )2
P (r|sm ) = exp −
(πN0 )N/2 k=1
N0

Sufficient Statistics

All the relevant information to make the decision as to which of sm (t) was
transmitted is contained in the correlator outputs {rk } where rk = smk + nk .
♠ M-ary baseband PAM signal
M -ary baseband PAM with basic pulse shape g(t). Am = (2m − 1 − M )d.
PAM signal has dimension N = 1. Let


A, 0 ≤ t ≤ T,
g(t) =

0, otherwise.

Hence, εg = A2 T.

 √
1 1/ T , 0 ≤ t ≤ T,
f (t) = √ g(t) =
A T 
 0, otherwise.

The output of the correlator output is,

Z T Z T Z T
1 1
r= r(t)f (t)dt = √ r(t)dt = √ [sm (t) + n(t)]dt
0 T 0 T 0

70
 ⇒ r = sm + n.

In PAM, sm = Am = (2m − 1 − M )d. As shown earlier, E(n) = 0, E(r) = sm

N0
and σr2 = σn2 = 2 . Hence, the PDF of the sampled output (conditioned on that
sm (t) was transmitted) can be specified as,
· ¸
1 (r − sm )2
P (r|sm ) = √ exp −
πN0 N0

♠ What if M = 2 for anti-podal signals, s1 = −s2 ? What is Pe ?

♠ Design of the correlator?

11.1.2 Matched Filter

Instead of N correlators, N linear (matched) filters can be used in the demod-

ulation.

y(t) = s(t)*h(t)
s(t) h(t) = s(T−t)

A A y(T)

t t t
0 T 0 T 0 T 2T
(a) (b) (c)

Let
hk (t) = fk (T − t), 0 ≤ t ≤ T,

be the impulse response of the N filters where {fk (t)} are the N basis functions
and hk (t) = 0 outside the symbol interval T . The output of the filter is,
Z t Z t
yk (t) = r(τ )hk (t − τ )dτ = r(τ )fk (T − t + τ )dτ, k = 1, 2, ..., N.
0 0

71
If we sample at t = T,
Z T
yk (T ) = r(τ )fk (τ )dτ = rk
0

Therefore, the sampled outputs of the filters at t = T are the exact values
obtained from N correlators.

Computation of Output SNR

If s(t) is corrupted by AWGN, the filter with an impulse response matched

to s(t) maximizes the output SNR. [See the text book for the proof, pp.
237-239.]
Let us compute the SNR using frequency-domain approach.
Z T ·Z T ¸
H(f ) = s(T − t)e−j2πf t dt = s(τ )ej2πf τ dτ e−j2πf T
0 0

H(f ) = S ∗ (f )e−j2πf T

72
 The output spectrum is,

Y (f ) = |S(f )|2 e−j2πf T .

Hence,
Z ∞
ys (t) = |S(f )|2 e−j2πf T ej2πf t df
−∞

By sampling at t = T , we obtain,
Z ∞ Z T
2
ys (T ) = |S(f )| df = s(t)2 dt = εs .
−∞ 0
R∞ 2
Note that Parseval’s theorem states that, the signal energy is, −∞ |s(t)| dt =
R∞ 2
−∞ |S(f )| df.

Hence, the output power (of the signal sampled at t = T ) is Ps = ε2s . The
noise PSD at the output of the matched filter is

N0
Φo (f ) = |H(f )|2
2

Hence, the average noise power is,

Z ∞ Z Z ∞
N0 ∞ N0 1
Pn = Φo (f )df = |H(f )|2 df = |S(f )|2 df = εs N0
−∞ 2 −∞ 2 −∞ 2

Therefore, the output SNR is,

Ps ε2 2εs
SNRo = = 1 s =
Pn 2 εs N0
N0

11.2 Detector

• Either correlator or matched filter produces sampled output vector r =

[r1 r2 ... rN ] which contains all the relevant information in the received
signal waveform.

73
 • Design a detector that makes a decision on sm (t) in each symbol interval
T based on observing r.

• The criterion is to maximize the (probability of) correct decision in the

presence of noise.

11.2.1 MAP, ML and MD Detectors

Maximum Aposteriori Probability (MAP) Criterion:

Let us compute the aposteriori probabilities, P (sm |r), ∀m, and select the max-
imum of the set {P (sm |r)}
Using Bayes’ rule,
P (r|sm )P (sm )
P (sm |r) =
P (r)
where P (sm ) is apriori probability of mth signal being transmitted. Also,
M
X
P (r) = P (r|sm )P (sm ).
m=1

Maximum Likelihood (ML) Criterion:

Let us suppose that the M signals are equiprobable, P (sm ) = 1/M . Note that
P (r) is independent of which signal was transmitted. Therefore, the decision
rule based on finding the signal that maximizes P (sm |r) is equivalent to finding
the signal that maximizes P (r|sm )
Minimum Distance (MD) Criterion:
In the presence of AWGN, the joint conditional probability P (r|sm ) is given by
· X N ¸
1 (rk − smk )2
P (r|sm ) = exp −
(πN0 )N/2 k=1
N0

74
 N
N 1 X
⇒ ln P(r|sm ) = − ln (πN0 ) − (rk − smk )2
2 N0
k=1
Therefore, the maximum of P (r|sm ) over sm is equivalent to finding the signal
P
that minimizes the Euclidean distance, D(r, sm ) = N 2
k=1 (rk − smk ) .

♠ π/4-shifted QPSK with r = [0.5 0.8]. What is D(r, s1 )?

O O
s 1 s 3
r
X

s 2 s 4
O O

Signals as vectors: Optimum for AWGN channel.

N
X N
X N
X
D(r, sm ) = rk2 −2 rk smk + s2mk
k=1 k=1 k=1

⇒k r k2 −2r.sm + k sm k2
ksm k2
Therefore, let us minimize −2r.sm + k sm k2 or maximize r.sm − 2

In the presence of AWGN, the optimum ML detector computes

• a set of M Distance metrics, or

• a set of M Correlation metrics

and selects the signal corresponding to the smallest and the largest metric
respectively.
Note that the above development of optimum detector (ML and MD) as-
sumes equiprobable symbols. Otherwise, the optimum MAP detector bases its

75
decision on P (sm |r), m = 1, 2, ..., M, or equivalently P (r|sm )P (sm ) since P (r)
is irrelevant in the optimization.

11.3 MAP Detector for Non Equiprobable Symbols

√
Binary PAM signals with signal points, s1 = −s2 = εb . Let P (s1 ) = p and
N0
P (s2 ) = 1 − p. Signals are corrupted by AWGN with PSD of 2 .

The received signal vector for binary PAM is, (note that N = 1),

√
r = ± εb + n.

N0
We have seen that n is a zero-mean Gaussian RV with variance σn2 = 2 .

Therefore, the conditional PDFs are

· √ ¸
1 (r − εb )2
P (r|s1 ) = √ exp −
2πσn 2σn2
· √ ¸
1 (r + εb )2
P (r|s2 ) = √ exp −
2πσn 2σn2

76
Let β() ≡ P (r|sm )P (sm ). Based on MAP criterion, the decision rule is

β(r, s1 ) s1
≷ 1.
β(r, s2 ) s2

Since · ¸
√ √
β(r, s1 ) p (r + εb )2 − (r − εb )2
= exp
β(r, s2 ) 1 − p 2σn2
µ ¶
N 0 1 − p
⇒ r ≷ss12 √ ln
4 εb p
√
• Since C(r, s1 ) = r εb , the optimum detector
µ ¶compares the correlation
metric with the threshold χ(p, N0 ) = N40 ln 1−pp and decides as

C(r, s1 ) ≷ss12 χ(p, N0 )

• The decision is influenced by noise PSD, pulse energy and symbol proba-
bility when symbols are not equiprobable.

1
• When p = 2, χ(.) = 0 and in this case, minimum Euclidean distance
criterion can be used.

π
♠ Consider 4 −shifted QPSK and its constellations. What are the decision
regions assuming equiprobable symbols?

11.4 Maximum-Likelihood Sequence Detector (MLSD)

When the signals transmitted in successive symbol intervals are interdependent

(i.e., with memory), the optimum detector bases its decisions on the observation
of a sequence of received signals over successive intervals. MLSD searches for
the minimum Euclidean distance path through the trellis which characterize
the memory.

77
 or
S2 X X S1

or

S3 X S4
X

π
Figure 3: 4
−shifted QPSK Constellations and Decision Regions for Equiprobable Symbols

78
 Let us consider an example with NRZI signal whose memory is shown in the
trellis for binary PAM signaling scheme.
The output of the matched-filter (or correlation modulator) in the lth symbol
interval is
√
rl = ± εb + nl

Let us now consider a sequence of outputs {r1 , r2 , ..., rL }. The noise sequence
{n1 , n2 , ..., nL } is white and Gaussian. Therefore, for any transmitted sequence
s(m) , L marginal PDFs are
L
Y
(m) (m)
P (r1 , r2 , ..., rL |s )= P (rl |sl )
l=1

· X L (m) ¸
1 (rl − sl )2
= exp −
(2πσn2 )L/2 2σn2
l=1
√
with sl = ± εb .
Therefore, with the received sequence {r1 , ..., rL }, the detector determines
(m) (m)
the sequence s(m) = {s1 , ..., sL } that maximizes the above conditional PDF
(maximum-likelihood sequence detector).
Equivalently, the above MLSD selects the sequence s(m) that minimizes the
Euclidean metric,
L
X
(m) (m)
D(r, s )= (rl − sl )2 .
l=1

In general, a total of M L sequences is possible. However, the number of sequence

search in the trellis can be reduced using Viterbi algorithm.

79
Viterbi algorithm - a trellis search algorithm for performing maximum likelihood
sequence detection

At steady state (and thereafter), at t = nT, n ≥ 2, there are two paths entering
and leaving each node.
For example, at t = 2T , two paths entering node s0 corresponding to the
information bits (0, 0) and (1, 1). The Euclidean distances are

√ √
D0 (0, 0) = (r1 + εb )2 + (r2 + εb )2

√ √
D0 (1, 1) = (r1 − εb )2 + (r2 + εb )2

• Survivor paths and metrics are recorded at each node.

• Add-Compare-Select (ACS) operation is done along the trellis.

12 Performance Evaluation in AWGN Channel

12.1 Error Probability

Assume that optimum receiver is used in the following.

80
12.1.1 Binary PAM

Let us consider equiprobable symbols with signals s1 (t) and s2 (t). We have seen
in AWGN channel,
1 √ 2
P (r|s1 ) = √ e−(r− εg ) /N0
πN0
Given that s1 (t) was transmitted,
Z 0 Z 0
1 √ 2
P (e|s1 ) = P (r|s1 )dr = √ e−(r− εg ) /N0 dr
−∞ πN0 −∞
Z −√2εg /N0 Z ∞ µr ¶
1 2 1 2 2εg
P (e|s1 ) = √ e−x /2 dx = √ √ e−x /2 dx = Q
2π −∞ 2π 2εg /N0 N0
R∞ 2
where Q(x) = √12π x et /2 dt.
Similarly, P (e|s2 ) can be computed. Hence, the average probability of error
for equiprobable equal energy binary PAM signals (i.e., antipodal signals) is
µr ¶
antipodal 1 1 2εg p
Pb = P (e|s1 ) + P (e|s2 ) = Q = Q( 2γb )
2 2 N0

where γb is the SNR per bit.

The probability of error does not depend on the shape of the transmitted
signal rather depends on its energy! How about bandwidth of the signal?
√
We noted that for binary PAM, d12 = 2 εg . Hence, in terms of distance
between signals,
µs 2 ¶
d12
Pb = Q
2N0
Let us now compare the performance of antipodal signals with orthogonal sig-
nals using the distance metrics. For binary orthogonal signals with equal energy,
p
d12 = 2εg .

81
 Hence,
µs 2 ¶ µr ¶
orthogonal d12 εg √
Pb =Q =Q = Q( γb )
2N0 N0

The larger the argument the smaller the Pb in Q(.) function. Therefore,
orthogonal signals are 3 dB poorer than antipodal signals. That is, in order to
get the same error performance, the orthogonal signals have to increase their
energy by a factor of 2 compared to antipodal signals.

12.1.2 M-ary Orthogonal Signals

For equal-energy (εs ) orthogonal signals, the optimum detector selects the signal
with the largest cross correlation between the received vector r and each of the
M possible transmitted signal vector {sm }.
M
X
C(r,sm ) = r.sm = rk smk , m = 1, 2, ..., M.
k=1

82
 Let us suppose that the signal s1 was transmitted. Therefore, the received
signal vector is,
√
r = [ εs + n1 n2 n3 ... nM ]

The output of the bank of M correlators are,

√ √
C(r, s1 ) = εs ( εs + n1 ), and
√
C(r, sm ) = εs nm , m = 2, ..., M
√
Therefore, the PDF of the first correlator output (after normalization by εs )
is, · ¸
√
1 (x1 − εs )2
Pr1 (x1 ) = √ exp −
πN0 N0
The PDFs of the other M − 1 correlator outputs are
1 2
Prm (xm ) = √ e−xm /N0
πN0
The average probability of symbol error (PM ) can be computed as follows:
Let the probability of correct decision be Pc . Then, PM = 1 − Pc .
Z ∞
Pc = p(n2 < r1 , ..., nM < r1 )p(r1 )dr1
−∞

Since {nm }’s are statistically independent Gaussian RVs,

Z ∞µ Z r1 √2/N0 ¶M −1
1 −x2 /2
Pc = √ e dx p(r1 )dr1
−∞ 2π −∞
R r1 1
R r1 √2/N0 −x2 /2
where P (nm < r1 |r1 ) = −∞ Prm (xm )dxm = √2π −∞ e dx.

Symbol and Bit Error Probability

• In comparing different modulation schemes, it is desirable to work with

SNR per bit (rather per symbol).

83
 • Since M = 2k , εs = kεg .

• It is desirable to convert probability of symbol error PM , into probability

of bit error, Pb .

For equiprobable orthogonal signals, all symbol errors occur with equal prob-
ability of,
PM PM
= k
M −1 2 −1
PM
♠P (sm |s1 ) = . For M = 2, PM = Pb .
M −1
¡k ¢
There are n ways in which n bits out of k, may be in error. Therefore, the
average number of bit errors per k−bit symbol is,
X k µ ¶
k PM k2k−1
Pb = n k −1
= k PM .
n=1
n 2 2 − 1
2k−1 PM
Hence, the average bit error probability is Pb = P
2k −1 M
≈ 2 , k À 1.

12.1.3 M-ary PAM

q
ε
Consider the carrier-modulated case. Recall: sm = 2 Am and Am = (2m −
g

p
1 − M )d. dmin = d 2εg . Assuming equiprobable symbols, the average energy
is,
M
1 X d2
εav = εm = (M 2 − 1)εg
M m=1 6

• The optimum detector compares the demodulator output r = sm + n with

the set of M − 1 thresholds.

• With equiprobable symbols, the decision thresholds are the mid-points

between two signal points.

84
Figure 4: Probability of bit error for orthogonal signals.

Figure 5: Probability of symbol error for PAM signals.

85
 • The decision is made in favor of the amplitude level closer to r.

• Therefore, average symbol error probability is,

µ r ¶ µ r ¶ µ r ¶
M −2 εg 1 εg 1 εg
PM = P |r − sm | > d + P r > sm + d + P r < sm − d
M 2 M 2 M 2

q
M −1
PM = P (|r − sm | > d εg /2)
M
µs 2 ¶
2(M − 1) d εg
PM = Q
M N0
With k = ln(M ), the symbol error can be expressed in terms of SNR per bit (εg,av /N0 ) as,
µs ¶
2(M − 1) 6 ln(M ) εg,av
PM = Q ,
M (M 2 − 1) N0

6εav
where εav = ln(M )εg,av and εg = (M 2 −1)d2
.

Figure 6: Probability of symbol error for PSK signals.

♠ As M is increased, the effect in symbol error probability for FSK and PSK?

86
12.1.4 M-ary PSK

• Note that the phase-modulated signal waveforms may be expressed as

· ¸
2π
sm (t) = g(t) cos 2πfc t + (m − 1) .
M

• The vector representation is

· ¸
√ 2π √ 2π
sm = εs cos (m − 1) εs sin (m − 1)
M M
εs = εg /2.

• Phase detection problem in PSK.

Since the signal waveforms have equal energy (and assuming equiprobable
symbols), the optimum detector for AWGN channel computes the correlation
metrics, C(r, sm ). The (corrupted) received signal vector is r = [r1 r2 ]. The
phase of r is, θr = tan−1 ( rr21 ). We need to compute the PDF of θr to compute
PM .
Z +π/M
PM = 1 − pθr (Θr )dΘr
−π/M

In general, the integral can only be evaluated numerically except for M = 2, 4.

√
♠ For M=2 (antipodal signals) P2 = Q( 2γb ). For M = 4,
· ¸
p 1 p
P4 = 1 − (1 − P2 )2 = 2Q( 2γb ) 1 − Q( 2γb )
2

12.2 Bandwidth Requirement

One can compare the SNR required to achieve a specified Pe for different modu-
lation schemes. However, it is not just enough or fair. Bandwidth (and achiev-
able data rate) requirement has also to be taken into account.

87
12.2.1 Bandwidth Limited Case

For multi-phase signals (such as PSK, PAM), the bandwidth requirement is the
bandwidth of the equivalent lowpass signal, g(t).
1
Let us suppose that the bandwidth of signal g(t) is W = T (approximately).
k
Since T = R and k = ln (M ),
R
W = .
ln M
The bandwidth efficiency is measured in terms of bit/s/Hz. Hence, as M is
increased, when R is fixed, the bandwidth requirement decreases.

Modulated PAM: SSB and DSB

The bandwidth efficient method to transmit PAM signal is to use SSB modu-
1
lation with bandwidth of 2T . Hence, the efficiency is 2 × ln (M ). This is 3 dB
better than DSB.

12.2.2 Power Limited Case

Orthogonal Signals

Let us construct M orthogonal FSK signals with minimum frequency separation

1
of 2T . Hence, the bandwidth requirement is,
M M M
W = = = R
2T 2(k/R) 2 ln (M )
In this case, as M is increased, when R is fixed, the bandwidth requirement
increases.
The meaningful comparison is based on the normalized data rate (R/W )
versus the SNR per bit (εg /N0 ) required to achieve a given error probability as

88
shown in the figure.

Figure 7: Comparison of several modulation methods at 10−5 symbol error probability.

• In the case of PAM, QAM and PSK, increasing M results in higher ratio
of R/W with the cost of increased SNR per bit (i.e., signal power).

• In M-ary orthogonal signals, increasing M results in lower ratio of R/W

with the benefit of decreased SNR per bit.

89
 • The maximum possible achievable ratio of C/W (where C = R bit/s) is
Shannon capacity limit for AWGN channel. It is an upperbound on the
bandwidth efficiency of any type of modulation (either band-limited or
power-limited case).

12.3 Analog vs Digital Repeaters

We noted that in digital transmission through AWGN channel, performance

in terms of probability of error (either symbol or bit) depends solely on the
εg
received SNR, N0 where the transmitted signal energy is εg .
In addition to additive noise, channel attenuation also affects the perfor-
mance. Hence, the received signal can be,

r(t) = αs(t) + n(t), 0 < α ≤ 1.

2
Therefore,
µ ¶ the received signal energy is α εg . Consequently, the received SNR
2ε
is N0g .
α2

Analog Repeaters

• Each repeater (amplifier) boosts the signal by multiplying α1 , where 1

α ≥ 1.

• Amplifiers boost both signal and noise.

• If K analog repeaters are used, the bit-error probability for binary PAM
transmitted signal is given as
µr ¶
analog 2εg
Pb ≈Q ,
KN0
where α >>
µ 0¶ and the output SNR at the last repeater can be approxi-
2ε
mated as KNg0 .

90
Regenerative Repeaters

• No boosting of signals in digital transmission.

• Instead, demodulator/detector demodulates/detects the transmitted sym-

bol sent by the preceding repeater and then corresponding waveform is
transmitted to the next repeater.

• Therefore, additive noise does not accumulate.

• The only error that occurs is in the detection of the symbol and this error
may propagate up to the last repeater.

• Suppose that the binary PAM is used. Hence, the probability of (bit) error
is, µr ¶
2εg
Pb = Q
N0
2ε
Note that the output SNR of any repeater is ( N0g ).

• Assuming a symbol is incorrectly detected only once during the transmis-

sion through a bank of K repeaters,
µr ¶
digital 2εg
Pb ≈ KQ .
N0

Digital regenerative repeaters offer significant power savings (or equivalently

the number of repeaters is reduced). Note that, for a given probability of error
with K >> 1,
µr ¶ µr ¶
analog 2ε g digital 2εg
Pb ≈Q >> Pb ≈ KQ .
KN0 N0

91
13 Signal Design for Band-limited Channels

• Bandwidth constraint of the channel has to be taken into account in the

design of pulses for transmission of digital information sequences.

• Let us model a channel as a band-limited linear filter having an equivalent

lowpass frequency response of C(f )(= 0, |f | > W ).

• How to design a pulse g(t) to transmit through the (AWGN) channel with
minimum distortion?

In general,
C(f ) = |C(f )|ejθ(f )

where |C(f )| and θ(f ) are amplitude and phase responses respectively.

• If |C(f )| is constant and θ(f ) is linear (within the bandwidth of interest),

it is an ideal (filter) channel.

• Non-ideal channel characteristic causes distortion in amplitude and delay,

resulting in interference of signals from different symbol intervals.

♠ As an example, consider a signal transmitted using the (band-limited) pulse

g(t) as shown in figure (a).

• One can transmit a sequence of pulses every T seconds since g(t) has zeros
at ±T, ±2T, ...

• However, non-ideal channel will cause inter symbol interference (ISI) as

seen in figure (b).

92
 • Compensation for non-ideal frequency response characteristics can be done
using equalizers.

Effect of channel distortion: (a)

channel input; (b) channel output;
(c) equalizer output.

13.1 Inter Symbol Interference (ISI)

A (linearly) modulated signal (such as PAM or PSK) can be represented as

X
v(t) = In g(t − nT ) (1)
n

where In is real and complex for PAM and PSK respectively.

It will be seen that when the channel is ideal, g(t) can be designed to effi-
ciently transmit the symbols (i.e., without ISI). However, with non-ideal chan-
nel, ISI among adjacent symbols occurs at higher rates. Also note that in
addition to ISI, channel noise n(t) will also cause errors in the received signal.

• Let us suppose that g(t) has a band-limited frequency response of G(f )(=
0, |f | > W ). The signal to be transmitted is v(t) as given in (1).

93
• v(t) is transmitted over the band-limited channel whose lowpass equivalent
frequency response is C(f ).

• Hence, the received signal is,

X
r(t) = In h(t − nT ) + n(t), (2)
n
where n(t) represents AWGN and,
Z ∞
h(t) = g(τ )c(t − τ )dτ.
−∞

• Suppose that r(t) is passed through a filter and sampled at a rate 1/T
samples/s.

g(t) c(t) h(t)

*
v(t) c(t), C(f) r(t) h(T−t), H (f) y(t)
(band−limited channel) (matched filter) k

n(t), Sn(f)
(AWGN channel)

h(t) h(T−t) x(t)

n(t) c(t) z(t)

Figure 8: A block diagram.

• In the presence of AWGN, the optimum filter is one that is matched to the
signal at the receiver. What is the signal input to the filter in a symbol
interval?

Therefore, the output of the receive (matched) filter is,

X
y(t) = In x(t − nT ) + z(t),
n

94
 where x(t) and z(t) are responses of the matched filter to h(t) and n(t)
respectively.

Hence, assuming zero transmission delay through the channel, the k th sampled
output is,
X X
y(kT ) ≡ yk = In x(kT − nT ) + z(kT ) = In xk−n + zk
n n

X
yk = Ik x0 + Ik xk−n + zk . (3)
n6=k

The term Ik represents the desired symbol at k th sampling instant and

P
n6=k In xk−n represents ISI. zk is due to noise.

Eye Diagram

An ISI (+ noise) visualization tool.

The amount of ISI (plus noise) can be viewed on an oscilloscope. The effect of
ISI causes the eye to close thereby reducing the margins for the noise to cause
errors.

13.2 Nyquist Pulse Design Criterion for No ISI

What should x(t) (or equivalently g(t)), be such that there is no ISI. That is,
we require (in time domain),


1, k = 0,
x(t = kT ) = xk =

0, k 6= 0.

95
96
Let us assume an ideal channel, i.e., C(f ) = 1 for |f | < W. Then, by looking at
the previous block diagram in Figure 8,

X(f ) = |H(f )|2 = |G(f )C(f )|2 = |G(f )|2 (4)

Hence, from x(t) we can find X(f ) and then G(f ) and finally g(t)!
The necessary and sufficient condition on X(f ) in order for x(t) to satisfy
the above relationship is also known as Nyquist pulse shaping criterion, which
is stated as (in frequency domain),
∞
X
B(f ) = X(f + n/T ) = T
n=−∞

Note that B(f ) is a periodic function in f with period T1 .

See pp 557 in the text book for proof. Note that
Z 1/2T ∞
X
j2πf nT
bn = T B(f )e df with B(f ) = bn e−j2πf nT and bn = T x(−nT )
−1/2T n=−∞

It can be seen from (4), X(f ) = G(f ) = 0, |f | > W, since the channel bandwidth
is W , i.e., C(f ) = 0, |f | > W. The pulse is,
Z ∞p
g(t) = |X(f )|ej2πf t df
−∞

A note of caution: There will still be distortion due to noise n(t); in order
to minimize the effect due to (AWGN) noise, we used the matched filter.

Case 1: T < 1/2W or symbol rate, T1 > 2W .

P
The n X(f +n/T ) consists of non-overlapping X(f ) separated by 1/T . There-
fore, there is no choice for X(f ) to ensure B(f ) = T in this case. Hence, we

97
cannot design a system with no ISI.

1
Case 2: T = 1/2W or symbol rate, T = 2W .
When T = 1/2W, (i.e., at the Nyquist rate that is required for no frequency-
aliasing), the replicas of X(f ) are separated by only 1/T . Hence, there exists
only one X(f ) that results in B(f ) = T. That is,


T, |f | < W,
X(f ) =

0, otherwise.

The corresponding pulse is

µ ¶
πt sin (πt/T )
x(t) = sinc =
T (πt/T )
Rp R √ j2πf t
♠ How about g(t)? g(t) = X(f )ej2πf t df = Te df = √1 sinc(πt/T ).
T

This pulse is • non-causal (and hence non-realizable unless shifted version

of it is used), • converges slower and • timing error in sampling also causes
problems.

98
 1
Case 3: T > 1/2W or symbol rate, T < 2W .

In this case, B(f ) consists of overlapping replicas of X(f ) separated by 1/T .

There are many pulses such that B(f ) = T . In particular, the raised cosine
pulses (in f domain) are used in practice that have desirable spectral charac-
teristics.

99
  µ ¶



 T, 0 ≤ |f | ≤ 1−β

 2T

 ½ · µ ¶¸¾ µ ¶
T πT 1−β 1−β 1+β
Xrc (f ) = 1 + cos |f | − , ≤ |f | ≤
2

β 2T 2T 2T

 µ ¶

 1+β

0 |f | > 2T

where β is called roll-off factor with 0 ≤ β ≤ 1. The excess bandwidth (in

excess of 1/2T ) is expressed as a percentage of the Nyquist bandwidth (i.e.,
2W ).
♠ For example, β = 1 corresponds to the bandwidth of 1/T and the symbol
1
rate is T = W with excess bandwidth of 100%. Time domain signal is,
sin (πt/T ) cos (πβt/T )
x(t) =
(πt/T ) 1 − 4β 2 t2 /T 2
Note that β = 0 corresponds to x(t) = sinc(πt/T ) and the bandwidth of 1/2T.
1
The symbol rate is T = 2W with no excess bandwidth.

Figure 9: Pulses having a raised cosine spectrum.

Note that x(t) satisfies the time domain constraint (i.e., x(kT ) = 0, k 6= 0,)

100
for no ISI.

13.3 Band-limited Signal Design with Controlled ISI

1
• To realize practical filters with zero ISI, T < 2W . That is, the symbol rate
is below the Nyquist rate of 2W samples/s.

• Relax the zero ISI constraint and with controlled ISI, then the symbol rate
comparable to Nyquist rate is achievable.

♠ Example with Duobinary Signal Pulse:



1, n = 0, 1,
x(nT ) =

0, otherwise.

Hence, 

T n = 0, −1,
bn =

0 otherwise.
Then,
B(f ) = T + T e−j2πf T

For T = 1/2W ,

x(t) = sinc(2πW t) + sinc[(2π(W t − 1/2)]

Note that the spectrum of x(t) decays smoothly and as a result, is realizable.
Thus, a symbol rate of 2W samples/s can be achieved.
♠ Example with spectrum shaping: Modified Duobinary Signal Pulse:

101
 



 1, n = −1,


x(nT ) = −1, n = 1,





0, otherwise.

The corresponding pulse x(t) is

· ¸ · ¸
π(t + T ) π(t − T )
x(t) = sinc − sinc
T T

In general, a class of band-limited signal pulses that have the form

X µ n ¶ · µ
n
¶¸
x(t) = x sinc 2πW t −
n
2W 2W

are called partial-response signals (PRS).

In the above examples, controlled amount of ISI is introduced and symbol
transmission of 2W samples/s is achieved within the limited bandwidth of W.

102
103
Assignments

104
 Problem Set: 1

1. From the text book (Proakis)

• Probability Theory: 2.1, 2.6, 2.9

• Random Process: 2.11, 2.12, 2.14, 2.16

• Signals and Systems: 4.3, 4.4

• Signal Space: 4.10, 4.12, 4.16

2. Additional Problems:

Frequency Spectrum: Evaluate the spectra of the signal,

Y t−5
x(t) = ( ) + 8 sin (6πt),
10
Q
where ( t−t
T ) is defined as a unit rectangular pulse centered at t = to
o

with the width of T .

Time Domain Signal: Find the inverse Fourier transform of the spectra
shown in the figure (use properties of the Fourier transform such as
frequency-shifting).

G( ω) G( ω)

1 1
ω ω
−4 −2 2 4 −6 −2 2 6
(a) (b)

105
BSC: Over a binary communication channel, the symbol 0 is transmitted
with probability 0.4 and 1 is transmitted with probability 0.6. It
is known that P (², 0) = 10−6 and P (², 1) = 10−4 , where P (², xi ) is
the probability of detecting the error given that xi is transmitted.
Determine P (²), the error probability of the channel.

106
 Problem Set: 2

1. Encoding of Information: From the text book (Proakis), 3.1, 3.2, 3.7;
and
A source has an alphabet {a1 , a2 , a3 , a4 } with corresponding probabilities
{0.1, 0.2, 0.3, 0.4}.

(a) Find the entropy of the source?

(b) What is the minimum required average codeword length to represent

this source for error-free reconstruction?

(c) Design a Huffman code for the source and compare the average length
of the Huffman code with the entropy of the source?

(d) Design a Huffman code for the second extension of the source (take
two symbols at a time)? What is the average codeword length? What
is the average required binary letters per each source output symbol?

(e) Compare the efficiency of above coding schemes?

2. Communication Channel Capacity: From the text book (Proakis),

7.4, 7.5; and
Determine the capacity of the ternary-input ternary-output channel shown
in the figure?

107
 1-p
x1 y1
p

p
1-p
x2 y2
p
x3 1-p y3

Problem Set: 3

From the text book (Proakis)

• 4.21 (a), (b); 4.22

2
• 4.29 (a) with h = 3

• Additional problems:

1. The signal constellation for 16-QAM is shown in Fig. 1. If the trans-

mitted signal is x(t) = s4 (t) + s2 (t − T ) + s11 (t − 2T ), where T denotes
the symbol duration. Sketch x(t) assuming fc = T2 , where fc denotes
the carrier frequency.

s1 s2 s3 s4
3A

s5 s6 s7 s8
A

−3A −A A 3A
s9 s 10 s 11 s 12
−A

s 13 s 14 s 15 s 16
−3A

Figure 10: Signal Constellation for 16-QAM.

108
 S1 (t) S2 (t) S3 (t) S4 (t)

2A 2A 2A 2A

A A A A

0 0 0 0
T/2 T t T/2 T t T/2 T t T/2 T t
−A −A −A −A

−2A −2A −2A −2A

Figure 11: Baseband 4-ary Signalling.

2. A set of transmitted signals in a baseband 4-ary scheme is shown
in Fig. 1. (i) Find the average energy Ê of the transmitted signals
assuming equiprobable symbols, (ii) Draw the signal constellation for
the above signalling scheme.

109
 Problem Set: 4

From the text book (Proakis)

• 5.2, 5.5, 5.8, 5.18 (a), 5.26 (a)

Note that you have to do only the above problems for credit. However, you
are encouraged to do the following problems for which the solutions will also
be provided.

• 5.19, 5.46, 9.12, 9.16, 9.19

110
Sample Questions

111
1. Probability and Random Process: (15 pts)
Determine the mean and the autocorrelation function of the random process

X(t) = A cos (ωc t + θ),

where θ is an random variable uniformly distributed in the range (0, 2π)

and A is a constant. Is the process wide-sense stationary ?

2. Signal and Vector Space: (15 pts)

Consider the three waveforms fn (t) shown in Fig. 1. Are these waveforms
are orthogonal ?

f (t) f (t) f (t)

1 2 3

1 1 1

t t t
0 2 4 0 4 0 1 2 3 4
−1 −1

(a) (b) (c)

Figure 12: Waveforms, fn (t), n = 1, 2, 3.

3. Autocorrelation and Power Spectral Density: (25 pts)

The white noise process X(t) is the input to the channel whose transfer
function is given by,
1
H(f ) = ,
1 + j2π%f
where % is a constant. The input has the following characteristics: E[X(t)] =
0 and autocorrelation function φxx (τ ) = σ 2 δ(τ ), where δ(.) is a delta func-
tion. The corresponding output process is Y (t). Determine the following:

112
 (a) the output power spectral density Φyy (f ) ?

(b) the autocorrelation function of the output process φyy (τ ) ?

(c) the average output power ?

4. Source Entropy and Block Coding: (25 pts)

A discrete memoryless channel has an alphabet of four letters, xi , i =
1, 2, 3, 4, each occurring with probability 31 , 13 , 61 and 1
6 respectively. Evalu-
ate the efficiency of a fixed-length binary code in which,

(a) each letter is encoded separately into binary sequence ?

(b) two letters are encoded separately into binary sequence ?

(c) from the above results in (a) and (b), what can you conclude about
the coding of symbols ?

5. Noisy Channel Capacity: (20 pts)

A communication channel is characterized as a band-limited (to 3 KHz)
AWGN waveform channel with noise power spectral density of N0 = 8 ×
10−8 Watts/Hz. The average input power to the channel is 24 mWatts.

(a) determine the capacity of the channel ?

(b) is it possible to transmit reliably a signal that has a maximum fre-

quency component at 5 KHz and sampled ar Nyquist rate and encoded
with 8 bits/sample? Justify your answer.

6. Digital Transmission:

(a) [10 pts] The bandwidth of a baseband channel allows a maximum

symbol rate of Rs = 6×103 symbols/second. Since the source produces

113
 a bit rate of Rb = 20 × 103 bits/second, M -ary modulation is used.
Find M .

(b) [15 pts] A video signal is to be stored in a hard drive. The signal
is composed of 24 frames/second with 728 × 512 pixels/frame. The
grayness level of each pixel is represented by R bits. The quantization
process results in signal-to-noise ratio (SNR) of γ(R) = 1.8 + 6R (in
dB). If the output quality requirement dictates an SNR of at least 40
dB, determine how many minutes of video signal can be stored in a 40
GB hard drive?

7. Random Process [20 pts]: Consider a random process

X(t) = Acos(2πf t + θo ),

where f and θo are constants and A is a uniformly distributed random

variable over the interval (α, β) with α 6= 0, β 6= 0.

(a) Find E[X(t)].

(b) Find the autocorrelation of X(t), φX (τ ).

(c) Is it possible that X(t) be wide-sense stationary? If so, under what

conditions and if not, why?

8. Signal Space [10 pts]: Signals s1 (t) and s2 (t) are as shown in Figure 13.
Draw two unit-energy signals s3 (t) and s4 (t) which are orthogonal to both
s1 (t) and s2 (t).

9. Filtering of Random Process [25 pts]: X(t) is a stationary random

process with power spectral density SX (f ). This process is passed through

114
 s (t) s 2(t)
1

A A

T T/2 T

Figure 13: Two signals.

a linear time-invariant system as shown in Figure 14.

(a) Is Y (t) stationary? Justify your answer.

d
(b) What is the power spectral density of Y (t)? Note that, dt ↔ j2πf .

(c) Given that 


K(1 − f /B), |f | < B,
SX (f ) =

0, otherwise,
where B and K are constants. (i) Plot the output power spectral
density and (ii) find the average output power for B = 10 kHz and
6×10−12
K= 8π 2 .

X(t) + d Y(t)
dt

Delay by T

Figure 14: A continuous time LTI system.

10. Source Coding : [20 pts] A discrete memoryless source emits messages
m1 , m2 and m3 with probabilities 0.2, 0.3 and 0.5 respectively.

115
 (a) If two output messages are encoded at a time, determine the optimum
code assignment with fixed code length?

(b) If two output messages are encoded at a time, determine the optimum
code assignment with variable code length?

(c) For both cases above, determine the coding efficiency? What can you
conclude about the coding of symbols from the efficiency?

11. Signals and Vectors [15 pts]: Consider four waveforms as shown in
Fig. 15. (i) Show that {gi (t)}, i = 1, 2, 3, forms an orthogonal set. (ii)
Find the minimal basis functions for the above set. (iii) Express x(t) shown
in Fig. 15 (d) as a vector using the above basis functions.

g (t)
1
g (t)
2
g (t)
3
x(t)
1
1 1 1

t
t t t 0 2 4
0 2 4 0 4 0 1 2 3 4
−1
−1 −1 −1

(d)
(a) (b) (c)

Figure 15: Waveforms.

12. Channel Capacity [15 pts]: A communication channel is characterized

as a band-limited (to 8 kHz) AWGN waveform channel with noise power
spectral density of N0 = 10−7 Watts/Hz. The average input power to the
channel is 0.024 Watts.

(a) Determine the capacity of the channel.

(b) Is it possible to transmitreliably a baseband signal with bandwidth

of 3.5 kHz, sampled at Nyquist rate and encoded with 4 bits/sample?

116
13. Random Process [15 pts]: A random process v(t) is defined as

v(t) = Xcos (2πfc t) − Y sin (2πfc t),

where X and Y are zero-mean orthogonal random variables and fc is

a constant. Determine the condition on the statistics of X and Y un-
der which the process v(t) is wide sense stationary. Aid: cos (A + B) =
cos A cos B − sin A sin B.

14. (a) Source Coding: [20 pts] A binary erase channel (BEC) has the
transition probabilities as shown in Fig. 16. Assuming equiprobable
inputs, determine (i) the source entropy, H(X) and (ii) conditional mu-
tual information, H(X|Y ), (iii) average mutual information I(X; Y ),
in bits/symbol.
q
x1 y1

p
y3
p

x2 q y2

Figure 16: Transition probabilities in BEC.

(b) [15 pts]: A discrete memoryless source emits messages m1 and m2

with probabilities 0.8 and 0.2 respectively. (i) Find the optimum
(Huffman) binary code for this source. (ii) If two output messages
are encoded at a time, determine the optimum code assignment. (iii)
For both cases above, determine the coding efficiency.

15. Power Spectral Density [20 pts]: A noise signal ni (t) with PSD Sni (f ) =
K(a constant), for all f, is first filtered using an ideal low pass filter h(t)

117
 and then is passed through an ideal differentiator
 as shown in Fig. 17. The
1, |f | ≤ B,
filter response is characterized as: H(f ) =
0, otherwise.

(i) Determine the PSD and the power of the noise signal no (t). (ii) Given
3×10−12
that K = 8π 2 , what should be the (ideal) filter bandwidth that will
d
give 1 Watt of output power? Aid: dt ↔ j2πf .

n i (t) d n o(t)
h(t)
dt

Figure 17: A noise processing system.

16. Miscellaneous Questions (30 pts)

(a) Random Process: A random process is described as: x(t) = Acos(2πt/T )
where A is a uniform random variable between 0 and 1. Plot 2 distinct
(non-zero) sample functions of x(t) for 0 ≤ t ≤ 2T on the same graph. (5
pts)
(b) Measure of Information: Explain briefly why the function I(x) =
−log(P (x)) where P (x) is the occurrence probability of event x, can be
considered as an effective measure of information. (5 pts)
(c) MAP Criterion: Consider a binary antipodal modulation scheme
that uses a basic pulse g(t) with energy 10−12 J to communicate through
N0
the AWGN channel with PSD of 2 = 5 × 10−7 W/Hz. Symbols occur
with the following probabilities: P (1) = 0.8 and P (0) = 0.2. Determine
the decision region when maximum aposteori probability (MAP) detector
is employed in the receiver. (6 pts)
(d) Bandwidth: A baseband speech signal has a bandwidth of 4 kHz

118
 and is sampled at the Nyquist rate, encoded into a PCM format using
8 bits/sample, and then transmitted through the AWGN channel using
M −ary baseband PAM. Determine the bandwidth required for distortion-
less transmission when M = 16 assuming that the bandwidth of the basic
1
pulse can be approximated by W = T Hz where T is the symbol duration
in seconds. (6 pts)
sin( πt πt
T ) cos( 2T )
(e) Inter Symbol Interference (ISI): Show that x(t) = πt
( T ) 1− t22
T

satisfies the Nyquist criterion for no ISI. Hint: plot x(t) for t = kT, k=
0, 1, 2, .. (8 pts)

17. M-ary Modulation and Transmission (20 pts)

(a) The signal constellation for 8-QAM is shown in Fig. 18. If a transmit-
ted signal is x(t) = s1 (t) + s8 (t − T ) where T is the symbol duration,
sketch x(t) assuming the carrier frequency of fc = T1 . (10 pts)

sin 2 πfc t
s1 s2
3A

s5 s6
A

−3A −A A 3A cos 2π fct

−A
s7 s8

−3A
s3 s4

Figure 18: Signal constellation for 8-QAM.

119
 (b) A digital source produces binary data with equal probability. Two
types of modulation schemes are considered for transmission of the
above information through a AWGN channel.

Scheme A: 0’s and 1’s are transmitted using +f (t) and −f (t) respec-
tively where 

A 0 ≤ t ≤ T,
f (t) =

0 otherwise.
Scheme B: 0’s and 1’s are transmitted using g(t) and h(t) respec-
tively where

 

Bsin(2πt/T ) 0 ≤ t ≤ T, 
Bcos(2πt/T ) 0 ≤ t ≤ T,
g(t) = h(t) =

0 
0
otherwise, otherwise.

If both schemes are expected to give equal error performance, show

√
that B = 2A. Hint: consider constellation points and signal energy.
(10 pts)

18. Maximum Transmission Rate through Channels (20 pts)

A channel is modelled as a cascaded channel of two binary symmetric
channels (BSCs) as shown in Fig. 19.

(a) Find the overall channel capacity of the cascaded connection in terms
of p assuming that both channels have the same transition probability
as shown in Fig 19. You can also assume that the maximum capacity
is achieved when input symbols are equiprobable, i.e., P (X = 0) =

120
 P (X = 1) = 12 . (10 pts)

input output
BSC BSC

composite channel

1−p 1−p
0 0
0 p
p

p p

1−p 1−p
1 1 1

Figure 19: A cascaded system with BSC channels.

(b) It is known apriori that on the average, 60% of the binary source
outputs are 1’s and the rest are 0’s. A baseband PAM scheme is
employed with rectangular pulse g(t) to transmit through the AWGN
N0
channel with noise PSD of 2 = 5 × 10−12 W/Hz and channel the
bandwidth of 1 kHz. As a system designer, your goal is to achieve
maximum possible reliable transmission rate of 104 bps through the
channel. You are constrained to use only the following waveforms for
transmission: f1 (t) = Ag(t) and f0 (t) = 2Ag(t) where f1 (t) and f0 (t)
are transmitted for 1 and 0 respectively. What should be the value of
A? Hint: use Shannon’s channel capacity limit. (10 pts)

19. Autocorrelation and Power Spectral Density (20 pts)

A random process n(t) has the following characteristics: E[n(t)] = 0 and
autocorrelation function φnn (τ ) = α2 δ(τ ), where δ(.) is a delta function
and α is a constant. This process is passed through the system as shown
in Fig. 20 to generate the output process y(t). The filter response is given

121
 by,
1
H(f ) = ,
1 + j2πβf
where β is a constant.

(a) Show that x(t) a wide sense stationary process? (4 pts)

(b) Determine the power spectral density Φxx (f ) of the process x(t). (4
pts)

(c) Determine the output power spectral density Φyy (f ). (4 pts)

(d) Find the autocorrelation function φyy (τ ) of y(t). (4 pts)

(e) If the required average output power is Pav = 1Watt, show that β =
√
α. (4 pts)

x(t)
n(t) + h(t) y(t)
filter
T
delay

Figure 20: A noise processing system.

20. Spectrum Shaping (20 pts)

In a binary digital transmission system, signal x(t) is transmitted where
P
x(t) = k bk g(t − kT ) with basic pulse g(t) of duration T . The encoding
rule is as follows: bk = −1 when the information bit is 0, and bk = 2
when it is 1. It is known apriori that 0’s and 1’s occur independently with
probabilities P (0) = 3/4 and P (1) = 1/4.

(a) Find E(bk ), E(b2k ) and variance σb2k of the sequence bk . (5 pts)

122
 (b) Find and plot the discrete autocorrelation φbb (m) = E(bk bk+m ). (5
pts)

(c) If g(t) has the raised-cosine spectrum with 100% excess bandwidth,
sketch G(f ). (5 pts)

(d) The power spectral density of x(t) is given as

∞
σb2k 2 [E(bk )]2 X m m
Φxx (f ) = |G(f )| + 2
|G( )|2 δ(f − ).
T T m=−∞
T T
For the above raised-cosine pulse sketch Φxx (f ). (5 pts)

21. Receiver Design and Performance (20 pts)

A baseband binary communication system is shown in Fig. 21(a). The
information bits are equally likely occurring at the source. Rectangular
pulses g(t) are used at the transmitter as shown in Fig. 21(b). The values
of ak = +1 and −1 for symbols 1 and 0 respectively.

The noise n(t) is assumed to be zero-mean AWGN with PSD of N0 /2. Due
0
to distortion in the channel, the received pulses have different shapes g (t)
as shown in Fig. 21(c). The receiver is unaware of the distortion in the
channel and therefore it uses the filter that is matched to the transmit-
ted pulse g(t). Note that an optimal receiver should have matched to the
0
received pulse g (t).

(a) What is the impulse response h(t) of the filter used in the above re-
0
ceiver? Show that g(t) and g (t) have equal energy; i.e., Eg = Eg0 = Eb .
Find Eb . (4 pts)

(b) The one-dimensional decision variable at the output of the sampler is

r = s + n, where s and n are due to signal and noise respectively.

123
 Given that a ”1” was transmitted, find (i) s and (ii) the mean and the
variance of r. (4 pts)

(c) Compute and plot pR (r|1) and pR (r|0) on the same graph where pR (.)
denotes the probability density function. (3 pts)

(d) Find the probability of error Pe in terms of Q(.) function. (6 pts)

(e) What is the performance loss in dB due to distortion in the channel?

(3 pts)

ak g(t−kT) ak g’(t−kT) r = s+n

Binary
Source
g(t) Distorting
channel + h(t)
t = kT
Detector

Transmitter Composite Channel n(t) Receiver

(a)

g(t) g’(t)
8 A
5
A 1 8A
2 5

t 0 t
0 T T/2 T
(b) (c)

Figure 21: (a) A baseband binary communication system through a channel that introduces
0
distortion, (b) transmitted pulse g(t) and (c) channel distorted pulse g (t).

22. Miscellaneous Questions

(a) [4 pts] The bandwidth of a baseband channel allows a maximum sym-

bol rate of Rs = 5 Ksymbols/sec. Since the required bit rate is Rb = 14
Kbits/sec, M -ary PAM is decided to be used. Find M .

(b) [6 pts] For the QAM signal constellation shown in Figure 1, specify
the Gray code.

124
 x
7

5 x

x
3
1 x
−7 −5 −3 −1 1 3 5 7
x x x x x x x x

−1 x

x
−3
x
−5
x
−7

Figure 22: 16-QAM Signal Constellation.

(c) [5 pts] A digital communication system consists of a transmission line

with 100 digital repeaters. Binary antipodal signals are used for trans-
mitting the information. If the overall end-to-end error probability is
10−6 , determine the probability of error for each repeater and the re-
quired Eb /N0 to achieve this performance in AWGN channel. Assume
that each channel between adjacent repeaters has identical error per-
formance. It is given that Q(5.6) = 10−8 .

(d) [5 pts] In a baseband communication system, the equivalent channel

bandwidth is W = 1500 KHz. If raised-cosine pulses with 50% excess
bandwidth are used, determine the maximum symbol rate that would
not cause any ISI.

23. Design of Variable-length Codes and Encoding Efficiency

A 4-level nonuniform quantizer for a Gaussian-distributed signal amplitude
results in the four levels, a1 , a2 , a3 and a4 , with corresponding probabilities
of occurrence, p1 = p2 = 1/3 and p3 = p4 = 1/6. Let J be the number of
symbols in a block.

125
 (a) [3 pts] Determine the entropy of the source.

(b) [4 pts] Design a Huffman code that encodes a single level (J = 1) at

a time and determine the average bit rate.

(c) [6 pts] Design a Huffman code that encodes two output levels (J = 2)
at a time and determine the average bit rate.

(d) [3 pts] Determine the efficiency of the encoding scheme in (b) and
(c). From the efficiency, what can you conclude about the encoding of
blocks ?

(e) [4 pts] What is the lower bound on the bit rate obtained by encoding
J output levels as J → ∞ ?

24. Power Spectral Density and Shaping

P
In a digital signaling scheme, x(t) = k bk g(t−kT ), where g(t) is the basic
pulse with pulse duration of T . The encoding rule is as follows: bk = − A3
2A
when the information bit is 0, and bk = 3 when it is 1. It is known a
priori that 0’s and 1’s are independent with probabilities P (0) = 3/4 and
P (1) = 1/4.

(a) [3 pts] Find E(bk ), E(b2k ) and variance, σb2 , of the sequence bk where
E(.) denotes expectation.

(b) [7 pts] Plot the discrete autocorrelation function, φbb (m) = E(bk bk+m )
for A = 12.

(c) [3 pts] If g(t) is a raised-cosine pulse with 50% excess bandwidth,

sketch G(f ). Hint: See page 561 in the text book or page 7 on the
handouts of Chapter 9.

126
 (d) [7 pts] The power spectral density expression is given as
∞
σb2 2 (E(bk ))2 X m 2 m
Φxx (f ) = |G(f )| + |G( )| δ(f − ).
T T2 m=−∞
T T

For the above raised-cosine pulse and with A = 12, sketch Φxx (f ).

25. Optimum Receiver Design in AWGN

A binary digital communication system employs the signals s(t) = {s0 (t) s1 (t)}
where,
s0 (t) = 0, 0 ≤ t ≤ T,

s1 (t) = A, 0 ≤ t ≤ T,

for transmitting the information. The demodualator cross-correlates the

received signal r(t) with s(t) and samples the output of the correlator t+T .

(a) [8 pts] Determine (i) the optimum detector for an AWGN channel and
(ii) the optimum threshold assuming that the signals are equiprobable.

(b) [8 pts] Given that The noise power spectral density is N0 /2, determine
the probability of symbol error as a function of SNR.

(c) [4 pts] How does this signaling scheme compare with the antipodal
signaling ? Hint: Compare the energy required to achieve the same
error performance.

26. Signal Space Representation and Minimum Distance Detection

Three messages m1 , m2 and m3 are to be transmitted over an AWGN chan-
nel with noise power spectral density N0 /2. The messages are

127
 

A, 0 ≤ t ≤ T,
s1 (t) =

0, otherwise,




 A, 0 ≤ t ≤ T /2,


s2 (t) = −s3 (t) = −A, T /2 < t ≤ T,





0, otherwise,
where A is a constant.

(a) [2 pts] What is the dimensionality of the signal space ?

(b) [3 pts] Find a basis for the signal space. Hint: Can be done by inspec-
tion.

(c) [4 pts] Give a vector representation of the signals and draw the signal
constellation for this problem.

(d) [3 pts] Assuming equiprobable signals, sketch the optimal decision

regions for mi , i = 1, 2, 3.

(e) [8 pts] Prove that the message m1 is more vulnerable to errors. Hint:
Consider P (error|mi transmitted), i = 1, 2, 3.

27. Performance of Baseband 4-ary Signaling

A set of transmitted signals in a baseband 4-ary scheme is shown in Figure
2. All the symbols are equally-likely.

(a) [3 pts] Find the set of basis function(s), {fi }, needed to represent the
above 4 signals.

128
 S (t) S (t) S (t) S (t)
1 2 3 4
2A

0 t 0 t 0 t 0 t
T T T T

−A

Figure 23: Transmitted Signals in a 4-ary Signaling Scheme.

The transmission is through a wireless channel with impulse response is

hc (t) = δ(t), where δ(t) denotes the delta function. Noise is AWGN de-
noted by n(t) with zero-mean and power spectral density of N0 /2. The
received signal x(t) is passed through a filter as shown in Figure 3. In the
figure B denotes a constant. The decision variable shown in the figure has
a component mi due to the transmitted signal and a component n due
to the noise. Function, fY (y) denotes the probability density function of
sampled output, y.

hr(t)
x(t)
B
y = m i+ n Decision
t Device
0 T

Figure 24: Receiver Structure in AWGN Channel.

(b) [7 pts] Show (quantitatively) that the matched-filter shown in Figure

3 is optimal in the sense it minimizes the probability of error. Hint:

129
 First determine x(t) and then consider minimum-distance criterion.

(c) [4 pts] Sketch fY (y|1), fY (y|2), fY (y|3) and fY (y|4), on the same fig-
ure. Here, fY (y|i) denotes the conditional probability for i th signal
transmitted.

(d) [6 pts] Find the probability of symbol error. Hint: Consider for exam-
ple,
P (error|1st signal transmitted) and its relationship to the overall error
probability.

28. Probability and Random Process: (15 pts)

Determine the mean and the autocorrelation function of the random process

X(t) = A cos (ωc t + θ),

where θ is an random variable uniformly distributed in the range (0, 2π)

and A is a constant. Is the process wide-sense stationary ?

29. Signal and Vector Space: (15 pts)

Consider the three waveforms fn (t) shown in Fig. 1. Are these waveforms
are orthogonal ?

30. Autocorrelation and Power Spectral Density: (25 pts)

The white noise process X(t) is the input to the channel whose transfer
function is given by,
1
H(f ) = ,
1 + j2π%f
where % is a constant. The input has the following characteristics: E[X(t)] =
0 and autocorrelation function φxx (τ ) = σ 2 δ(τ ), where δ(.) is a delta func-
tion. The corresponding output process is Y (t). Determine the following:

130
 f (t) f (t) f (t)
1 2 3

1 1 1

t t t
0 2 4 0 4 0 1 2 3 4
−1 −1

(a) (b) (c)

Figure 25: Waveforms, fn (t), n = 1, 2, 3.

(a) the output power spectral density Φyy (f ) ?

(b) the autocorrelation function of the output process φyy (τ ) ?

(c) the average output power ?

31. Source Entropy and Block Coding: (25 pts)

A discrete memoryless channel has an alphabet of four letters, xi , i =
1, 2, 3, 4, each occurring with probability 31 , 13 , 61 and 1
6 respectively. Evalu-
ate the efficiency of a fixed-length binary code in which,

(a) each letter is encoded separately into binary sequence ?

(b) two letters are encoded separately into binary sequence ?

(c) from the above results in (a) and (b), what can you conclude about
the coding of symbols ?

32. Noisy Channel Capacity: (20 pts)

A communication channel is characterized as a band-limited (to 3 KHz)
AWGN waveform channel with noise power spectral density of N0 = 8 ×
10−8 Watts/Hz. The average input power to the channel is 24 mWatts.

131
(a) determine the capacity of the channel ?

(b) is it possible to transmit reliably a signal that has a maximum fre-

quency component at 5 KHz and sampled ar Nyquist rate and encoded
with 8 bits/sample? Justify your answer.

132