Dembo Notes

Probability Theory: STAT310/MATH230
August 27, 2013

Amir Dembo
E-mail address: amir@math.stanford.edu
Department of Mathematics, Stanford University, Stanford, CA 94305.
Contents
Preface
Chapter 1. Probability, measure and integration

1.1. Probability spaces, measures and -algebras
1.2. Random variables and their distribution
1.3. Integration and the (mathematical) expectation
1.4. Independence and product measures
7
7
18
30
54
Chapter 2. Asymptotics: the law of large numbers

2.1. Weak laws of large numbers
2.2. The Borel-Cantelli lemmas
2.3. Strong law of large numbers
71
71
77
85
Chapter 3. Weak convergence, clt and Poisson approximation

3.1. The Central Limit Theorem
3.2. Weak convergence
3.3. Characteristic functions
3.4. Poisson approximation and the Poisson process
3.5. Random vectors and the multivariate clt
95
95
103
117
133
141
Chapter 4. Conditional expectations and probabilities

4.1. Conditional expectation: existence and uniqueness
4.2. Properties of the conditional expectation
4.3. The conditional expectation as an orthogonal projection
4.4. Regular conditional probability distributions
153
153
158
166
171
Chapter 5. Discrete time martingales and stopping times

5.1. Definitions and closure properties
5.2. Martingale representations and inequalities
5.3. The convergence of Martingales
5.4. The optional stopping theorem
5.5. Reversed MGs, likelihood ratios and branching processes
177
177
186
193
207
212
Chapter 6. Markov chains

6.1. Canonical construction and the strong Markov property
6.2. Markov chains with countable state space
6.3. General state space: Doeblin and Harris chains
227
227
235
257
Chapter 7. Continuous, Gaussian and stationary processes

7.1. Definition, canonical construction and law
7.2. Continuous and separable modifications
271
271
276
CONTENTS
7.3. Gaussian and stationary processes
286
Chapter 8. Continuous time martingales and Markov processes

8.1. Continuous time filtrations and stopping times
8.2. Continuous time martingales
8.3. Markov and Strong Markov processes
291
291
296
319
Chapter 9. The Brownian motion

9.1. Brownian transformations, hitting times and maxima
9.2. Weak convergence and invariance principles
9.3. Brownian path: regularity, local maxima and level sets
343
343
351
369
Bibliography
377
Index
379
Preface
These are the lecture notes for a year long, PhD level course in Probability Theory
that I taught at Stanford University in 2004, 2006 and 2009. The goal of this
course is to prepare incoming PhD students in Stanfords mathematics and statistics
departments to do research in probability theory. More broadly, the goal of the text
is to help the reader master the mathematical foundations of probability theory
and the techniques most commonly used in proving theorems in this area. This is
then applied to the rigorous study of the most fundamental classes of stochastic
processes.
Towards this goal, we introduce in Chapter 1 the relevant elements from measure
and integration theory, namely, the probability space and the -algebras of events
in it, random variables viewed as measurable functions, their expectation as the
corresponding Lebesgue integral, and the important concept of independence.
Utilizing these elements, we study in Chapter 2 the various notions of convergence
of random variables and derive the weak and strong laws of large numbers.
Chapter 3 is devoted to the theory of weak convergence, the related concepts
of distribution and characteristic functions and two important special cases: the
Central Limit Theorem (in short clt) and the Poisson approximation.
Drawing upon the framework of Chapter 1, we devote Chapter 4 to the definition,
existence and properties of the conditional expectation and the associated regular
conditional probability distribution.
Chapter 5 deals with filtrations, the mathematical notion of information progression in time, and with the corresponding stopping times. Results about the latter
are obtained as a by product of the study of a collection of stochastic processes
called martingales. Martingale representations are explored, as well as maximal
inequalities, convergence theorems and various applications thereof. Aiming for a
clearer and easier presentation, we focus here on the discrete time settings deferring
the continuous time counterpart to Chapter 8.
Chapter 6 provides a brief introduction to the theory of Markov chains, a vast
subject at the core of probability theory, to which many text books are devoted.
We illustrate some of the interesting mathematical properties of such processes by
examining a few special cases of interest.
Chapter 7 sets the framework for studying right-continuous stochastic processes
indexed by a continuous time parameter, introduces the family of Gaussian processes and rigorously constructs the Brownian motion as a Gaussian process of
continuous sample path and zero-mean, stationary independent increments.
5
PREFACE
Chapter 8 expands our earlier treatment of martingales and strong Markov processes to the continuous time setting, emphasizing the role of right-continuous filtration. The mathematical structure of such processes is then illustrated both in
the context of Brownian motion and that of Markov jump processes.
Building on this, in Chapter 9 we re-construct the Brownian motion via the invariance principle as the limit of certain rescaled random walks. We further delve
into the rich properties of its sample path and the many applications of Brownian
motion to the clt and the Law of the Iterated Logarithm (in short, lil).
The intended audience for this course should have prior exposure to stochastic
processes, at an informal level. While students are assumed to have taken a real
analysis class dealing with Riemann integration, and mastered well this material,
prior knowledge of measure theory is not assumed.
It is quite clear that these notes are much influenced by the text books [Bil95,
Dur10, Wil91, KaS97] I have been using.
I thank my students out of whose work this text materialized and my teaching assistants Su Chen, Kshitij Khare, Guoqiang Hu, Julia Salzman, Kevin Sun and Hua
Zhou for their help in the assembly of the notes of more than eighty students into
a coherent document. I am also much indebted to Kevin Ross, Andrea Montanari
and Oana Mocioalca for their feedback on earlier drafts of these notes, to Kevin
Ross for providing all the figures in this text, and to Andrea Montanari, David
Siegmund and Tze Lai for contributing some of the exercises in these notes.
Amir Dembo
Stanford, California
April 2010
CHAPTER 1
Probability, measure and integration

This chapter is devoted to the mathematical foundations of probability theory.
Section 1.1 introduces the basic measure theory framework, namely, the probability
space and the -algebras of events in it. The next building blocks are random
variables, introduced in Section 1.2 as measurable functions 7 X() and their
distribution.
This allows us to define in Section 1.3 the important concept of expectation as the
corresponding Lebesgue integral, extending the horizon of our discussion beyond
the special functions and variables with density to which elementary probability
theory is limited. Section 1.4 concludes the chapter by considering independence,
the most fundamental aspect that differentiates probability from (general) measure
theory, and the associated product measures.
1.1. Probability spaces, measures and -algebras
We shall define here the probability space (, F , P) using the terminology of measure theory.
The sample space is a set of all possible outcomes of some random experiment. Probabilities are assigned by A 7 P(A) to A in a subset F of all possible
sets of outcomes. The event space F represents both the amount of information
available as a result of the experiment conducted and the collection of all subsets
of possible interest to us, where we denote elements of F as events. A pleasant
mathematical framework results by imposing on F the structural conditions of a
-algebra, as done in Subsection 1.1.1. The most common and useful choices for
this -algebra are then explored in Subsection 1.1.2. Subsection 1.1.3 provides fundamental supplements from measure theory, namely Dynkins and Caratheodorys
theorems and their application to the construction of Lebesgue measure.
1.1.1. The probability space (, F , P). We use 2 to denote the set of all
possible subsets of . The event space is thus a subset F of 2 , consisting of all
allowed events, that is, those subsets of to which we shall assign probabilities.
We next define the structural conditions imposed on F .
Definition 1.1.1. We say that F 2 is a -algebra (or a -field), if
(a) F ,
(b) If A F then Ac F as well (where S
Ac = \ A).
(c) If Ai F for i = 1, 2, 3, . . . then also i Ai F .
T
S
Remark. Using DeMorgans law, we know that ( i Aci )c = i Ai . Thus the
following is equivalent to property (c) of Definition
1.1.1:
T
(c) If Ai F for i = 1, 2, 3, . . . then also i Ai F .
7
1. PROBABILITY, MEASURE AND INTEGRATION
Definition 1.1.2. A pair (, F ) with F a -algebra of subsets of is called a

measurable space. Given a measurable space (, F ), a measure is any countably
additive non-negative set function on this space. That is, : F [0, ], having
the properties:
(a) (A)
= 0 for all A F .
S () P
(b) ( n An ) = n (An ) for any countable collection of disjoint sets An F .
When in addition () = 1, we call the measure a probability measure, and
often label it by P (it is also easy to see that then P(A) 1 for all A F ).
Remark. When (b) of Definition 1.1.2 is relaxed to involve only finite collections
of disjoint sets An , we say that is a finitely additive non-negative set-function.
In measure theory we sometimes consider signed measures, whereby is no longer
non-negative, hence its range is [, ], and say that such measure is finite when
its range is R (i.e. no set in F is assigned an infinite measure).
Definition 1.1.3. A measure space is a triplet (, F , ), with a measure on the
measurable space (, F ). A measure space (, F , P) with P a probability measure
is called a probability space.
The next exercise collects some of the fundamental properties shared by all probability measures.
Exercise 1.1.4. Let (, F , P) be a probability space and A, B, Ai events in F .
Prove the following properties of every probability measure.
(a) Monotonicity. If A B then P(A) P(B).
P
(b) Sub-additivity. If A i Ai then P(A) i P(Ai ).
(c) Continuity from below: If Ai A, that is, A1 A2 . . . and i Ai = A,
then P(Ai ) P(A).
(d) Continuity from above: If Ai A, that is, A1 A2 . . . and i Ai = A,
then P(Ai ) P(A).
Remark. In the more general context of measure theory, note that properties
(a)-(c) of Exercise 1.1.4 hold for any measure , whereas the continuity from above
holds whenever (Ai ) < for all i sufficiently large. Here is more on this:
Exercise 1.1.5. Prove that a finitely additive non-negative set function on a
measurable space (, F ) with the continuity property
Bn F , Bn , (Bn ) <
= (Bn ) 0
must be countably additive if () < . Give an example that it is not necessarily

so when () = .
The -algebra F always contains at least the set and its complement, the empty
set . Necessarily, P() = 1 and P() = 0. So, if we take F0 = {, } as our algebra, then we are left with no degrees of freedom in choice of P. For this reason
we call F0 the trivial -algebra. Fixing , we may expect that the larger the algebra we consider, the more freedom we have in choosing the probability measure.
This indeed holds to some extent, that is, as long as we have no problem satisfying
the requirements in the definition of a probability measure. A natural question is
when should we expect the maximal possible -algebra F = 2 to be useful?
Example 1.1.6. When the sample space is countable we can and typically shall
take F = 2 . Indeed, in such situations we assign a probability p > 0 to each
1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS
P
P
making sure that p = 1. Then, it is easy to see that taking P(A) = A p
for any A results with a probability measure on (, 2 ). For instance, when
1
is finite, we can take p = ||
, the uniform measure on , whereby computing
probabilities is the same as counting. Concrete examples are a single coin toss, for
which we have 1 = {H, T} ( = H if the coin lands on its head and = T if it
lands on its tail), and F1 = {, , H, T}, or when we consider a finite number of
coin tosses, say n, in which case n = {(1 , . . . , n ) : i {H, T}, i = 1, . . . , n}
is the set of all possible n-tuples of coin tosses, while Fn = 2n is the collection
of all possible sets of n-tuples of coin tosses. Another example pertains to the
set of all non-negative integers = {0, 1, 2, . . .} and F = 2 , where we get the
k
Poisson probability measure of parameter > 0 when starting from pk = k! e for
k = 0, 1, 2, . . ..
When is uncountable such a strategy as in Example 1.1.6 will no longer work.
The problem is that if we take p = P({}) > 0 for uncountably many values of
, we shall end up with P() = . Of course we may define everything as before
b of and demand that P(A) = P(A )
b for each A .
on a countable subset
Excluding such trivial cases, to genuinely use an uncountable sample space we
need to restrict our -algebra F to a strict subset of 2 .
Definition 1.1.7. We say that a probability space (, F , P) is non-atomic, or
alternatively call P non-atomic if P(A) > 0 implies the existence of B F , B A
with 0 < P(B) < P(A).
Indeed, in contrast to the case of countable , the generic uncountable sample
space results with a non-atomic probability space (c.f. Exercise 1.1.27). Here is an
interesting property of such spaces (see also [Bil95, Problem 2.19]).
Exercise 1.1.8. Suppose P is non-atomic and A F with P(A) > 0.
(a) Show that for every > 0, we have B A such that 0 < P(B) < .
(b) Prove that if 0 < a < P(A) then there exists B A with P(B) = a.
Hint: Fix n 0 and define inductively numbers xn and sets Gn F with H0 = ,
Hn = k<n Gk , S
xn = sup{P(G) : G A\Hn , P(Hn G) a} and Gn A\Hn
such that P(Hn Gn ) a and P(Gn ) (1 n )xn . Consider B = k Gk .
As you show next, the collection of all measures on a given space is a convex cone.
Exercise 1.1.9. Given any measures {n , n 1} on (, F ), verify that =

P
n=1 cn n is also a measure on this space, for any finite constants cn 0.
Here are few properties of probability measures for which the conclusions of Exercise 1.1.4 are useful.
Exercise 1.1.10. A function d : X X [0, ) is called a semi-metric on
the set X if d(x, x) = 0, d(x, y) = d(y, x) and the triangle inequality d(x, z)
d(x, y) + d(y, z) holds. With AB = (A B c ) (Ac B) denoting the symmetric
difference of subsets A and B of , show that for any probability space (, F , P),
the function d(A, B) = P(AB) is a semi-metric on F .
Exercise 1.1.11. Consider events {An } in a probability space (, F , P) that are
almost disjointP
in the sense that P(An Am ) = 0 for all n 6= m. Show that then
P(
A
)
=
n=1 n
n=1 P(An ).
10
Exercise 1.1.12. Suppose a random outcome N follows the Poisson probability

measure of parameter > 0. Find a simple expression for the probability that N is
an even integer.
1.1.2. Generated and Borel -algebras. Enumerating the sets in the algebra F is not a realistic option for uncountable . Instead, as we see next, the
most common construction of -algebras is then by implicit means. That is, we
demand that certain sets (called the generators) be in our -algebra, and take the
smallest possible collection for which this holds.
Exercise 1.1.13.
(a) Check that the intersection of (possibly uncountably many) -algebras is
also a -algebra.
(b) Verify that for any -algebras H G and any H H, the collection
HH = {A G : A H H} is a -algebra.
(c) Show that H 7 HH is non-increasing with respect to set inclusions, with
H = H and H = G. Deduce that HHH = HH HH for any pair

H, H H.
In view of part (a) of this exercise we have the following definition.
Definition 1.1.14. Given a collection of subsets A (not necessarily countable), we denote the smallest -algebra F such that A F for all either by
({A }) or by (A , ), and call ({A }) the -algebra generated by the sets
A . That is,T
({A }) = {G : G 2 is a algebra, A G }.
Example 1.1.15. Suppose = S is a topological space (that is, S is equipped with

a notion of open subsets, or topology). An example of a generated -algebra is the
Borel -algebra on S defined as ({O S open }) and denoted by BS . Of special
importance is BR which we also denote by B.
Different sets of generators may result with the same -algebra. For example, taking = {1, 2, 3} it is easy to see that ({1}) = ({2, 3}) = {, {1}, {2, 3}, {1, 2, 3}}.
A -algebra F is countably generated if there exists a countable collection of sets
that generates it. Exercise 1.1.17 shows that BR is countably generated, but as you
show next, there exist non countably generated -algebras even on = R.
Exercise 1.1.16. Let F consist of all A such that either A is a countable set
or Ac is a countable set.
(a) Verify that F is a -algebra.
(b) Show that F is countably generated if and only if is a countable set.
Recall that if a collection of sets A is a subset of a -algebra G, then also (A) G.
Consequently, to show that ({A }) = ({B }) for two different sets of generators
{A } and {B }, we only need to show that A ({B }) for each and that
B ({A }) for each . For instance, considering BQ = ({(a, b) : a < b Q}),
we have by this approach that BQ = ({(a, b) : a < b R}), as soon as we
show that any interval (a, b) is in BQ . To see this fact, note that for any real
a < b there are rational numbers qn < rn such that qn a and rn b, hence
(a, b) = n (qn , rn ) BQ . Expanding on this, the next exercise provides useful
alternative definitions of B.
11
Exercise 1.1.17. Verify the alternative definitions of the Borel -algebra B:

({(a, b) : a < b R}) = ({[a, b] : a < b R}) = ({(, b] : b R})
= ({(, b] : b Q}) = ({O R open })
If A R is in B of Example 1.1.15, we say that A is a Borel set. In particular, all

open (closed) subsets of R are Borel sets, as are many other sets. However,
Proposition 1.1.18. There exists a subset of R that is not in B. That is, not all
subsets of R are Borel sets.
Proof. See [Wil91, A.1.1] or [Bil95, page 45].
Example 1.1.19. Another classical example of an uncountable is relevant for

studying the experiment with an infinite number of coin tosses, that is, = N
1
for 1 = {H, T} (indeed, setting H = 1 and T = 0, each infinite sequence
is in correspondence with a unique real number x [0, 1] with being the binary
expansion of x, see Exercise 1.2.13). The -algebra should at least allow us to
consider any possible outcome of a finite number of coin tosses. The natural algebra in this case is the minimal -algebra having this property, or put more
formally Fc = ({A,k , k1 , k = 1, 2, . . .}), where A,k = { : i = i , i =
1 . . . , k} for = (1 , . . . , k ).
The preceding example is a special case of the construction of a product of measurable spaces, which we detail now.
Example 1.1.20. The product of the measurable spaces (i , Fi ), i = 1, . . . , n is
the set = 1 n with the -algebra generated by {A1 An : Ai Fi },
denoted by F1 Fn .
You are now to check that the Borel -algebra of Rd is the product of d-copies of
that of R. As we see later, this helps simplifying the study of random vectors.
Exercise 1.1.21. Show that for any d < ,
BRd = B B = ({(a1 , b1 ) (ad , bd ) : ai < bi R, i = 1, . . . , d})
(you need to prove both identities, with the middle term defined as in Example
1.1.20).
Exercise 1.1.22. Let F = (A , ) where the collection of sets A , is
uncountable (i.e., is uncountable). Prove that for each B F there exists a countable sub-collection {Aj , j = 1, 2, . . .} {A , }, such that B ({Aj , j =
1, 2, . . .}).
Often there is no explicit enumerative description of the -algebra generated by
an infinite collection of subsets, but a notable exception is
Exercise 1.1.23. Show that the sets in G = ({[a, b] : a, b Z}) are all possible
unions of elements from the countable collection {{b}, (b, b + 1), b Z}, and deduce
that B =
6 G.
Probability measures on the Borel -algebra of R are examples of regular measures,
namely:
12
Exercise 1.1.24. Show that if P is a probability measure on (R, B) then for any
A B and > 0, there exists an open set G containing A such that P(A) + >
P(G).
Here is more information about BRd .
Exercise 1.1.25. Show that if is a finitely additive non-negative set function

on (Rd , BRd ) such that (Rd ) = 1 and for any Borel set A,
(A) = sup{(K) : K A, K compact },
then must be a probability measure.

Hint: Argue by contradiction using the conclusion of Exercise 1.1.5. To
Tn this end,
recall the finite intersection property (if compact Ki RTd are such that i=1 Ki are
non-empty for finite n, then the countable intersection i=1 Ki is also non-empty).
1.1.3. Lebesgue measure and Carath

eodorys theorem. Perhaps the
most important measure on (R,
B)
is
the
Lebesgue
measure,S. It is the unique
P
measure that satisfies (F ) = rk=1 (bk ak ) whenever F = rk=1 (ak , bk ] for some
r < and a1 < b1 < a2 < b2 < br . Since (R) = , this is not a probability
measure. However, when we restrict to be the interval (0, 1] we get
Example 1.1.26. The uniform probability measure on (0, 1], is denoted U and
defined as above, now with added restrictions that 0 a1 and br 1. Alternatively,
U is the restriction of the measure to the sub--algebra B(0,1] of B.
Exercise 1.1.27. Show that ((0, 1], B(0,1] , U ) is a non-atomic probability space and
deduce that (R, B, ) is a non-atomic measure space.
Note that any countable union of sets of probability zero has probability zero, but
this is not the case for an uncountable union. For example, U ({x}) = 0 for every
x R, but U (R) = 1.
As we have seen in Example 1.1.26 it is often impossible to explicitly specify the
value of a measure on all sets of the -algebra F . Instead, we wish to specify its
values on a much smaller and better behaved collection of generators A of F and
use Caratheodorys theorem to guarantee the existence of a unique measure on F
that coincides with our specified values. To this end, we require that A be an
algebra, that is,
Definition 1.1.28. A collection A of subsets of is an algebra (or a field) if
(a) A,
(b) If A A then Ac A as well,
(c) If A, B A then also A B A.
Remark. In view of the closure of algebra with respect to complements, we could

have replaced the requirement that A with the (more standard) requirement
that A. As part (c) of Definition 1.1.28 amounts to closure of an algebra
under finite unions (and by DeMorgans law also finite intersections), the difference
between an algebra and a -algebra is that a -algebra is also closed under countable
unions.
We sometimes make use of the fact that unlike generated -algebras, the algebra
generated by a collection of sets A can be explicitly presented.
Exercise 1.1.29. The algebra generated by a given collection of subsets A, denoted

f (A), is the intersection of all algebras of subsets of containing A.
13
(a) Verify that f (A) is indeed an algebra and that f (A) is minimal in the
sense that if G is an algebra and A G, then f (A) G.
(b) Show T
that f (A) is the collection of all finite disjoint unions of sets of the
ni
form j=1
Aij , where for each i and j either Aij or Acij are in A.
We next state Caratheodorys extension theorem, a key result from measure theory, and demonstrate how it applies in the context of Example 1.1.26.
Theorem 1.1.30 (Carath

eodorys extension theorem). If 0 : A 7 [0, ]
is a countably additive set function on an algebra A then there exists a measure
on (, (A)) such that = 0 on A. Furthermore, if 0 () < then such a
measure is unique.
To construct the measure U on B(0,1] let = (0, 1] and
A = {(a1 , b1 ] (ar , br ] : 0 a1 < b1 < < ar < br 1 , r < }
be a collection of subsets of (0, 1]. It is not hard to verify that A is an algebra, and
further that (A) = B(0,1] (c.f. Exercise 1.1.17, for a similar issue, just with (0, 1]
replaced by R). With U0 denoting the non-negative set function on A such that
r
r
[
X
(1.1.1)
U0
(ak , bk ] =
(bk ak ) ,
k=1
k=1
note that U0 ((0, 1]) = 1, hence the existence of a unique probability measure U on
((0, 1], B(0,1]) such that U (A) = U0 (A) for sets A A follows by Caratheodorys
extension theorem, as soon as we verify that
Lemma 1.1.31. The set function U0 is countably additive on A. That is,

P if Ak is a
sequence of disjoint sets in A such that k Ak = A A, then U0 (A) = k U0 (Ak ).
The proof of Lemma 1.1.31 is based on
Sn
Exercise 1.1.32. Show that U0 is finitely additive on A. That is, U0 ( k=1 Ak ) =
P
n
k=1 U0 (Ak ) for any finite collection of disjoint sets A1 , . . . , An A.
Sn
Proof. Let Gn = k=1 Ak and Hn = A \ Gn . Then, Hn and since
Ak , A A which is an algebra it follows that Gn and hence Hn are also in A. By
definition, U0 is finitely additive on A, so
n
X
U0 (A) = U0 (Hn ) + U0 (Gn ) = U0 (Hn ) +
U0 (Ak ) .
k=1
To prove that U0 is countably additive, it suffices to show that U0 (Hn ) 0, for then
n
X
X
U0 (A) = lim U0 (Gn ) = lim
U0 (Ak ) =
U0 (Ak ) .
n
k=1
k=1
To complete the proof, we argue by contradiction, assuming that U0 (Hn ) 2 for

some > 0 and all n, where Hn are elements of A. By the definition of A
and U0 , we can find for each a set J A whose closure J is a subset of H and
U0 (H \ J ) 2 (for example, add to each ak in the representation of H the
minimum of 2 /r and (bk ak )/2). With U0 finitely additive on the algebra A
this implies that for each n,
n
n
X
[
U0
(H \ J )
U0 (H \ J ) .
=1
=1
14
As Hn H for all n, we have that

\
[
[
Hn \
J =
(Hn \ J )
(H \ J ) .
n
Hence, by finite additivity of U0 and our assumption that U0 (Hn ) 2, also

\
\
[
U0 (
J ) = U0 (Hn ) U0 (Hn \
J ) U0 (Hn ) U0 ( (H \ J )) .
n
In particular, for every n, the set n J is non-empty and therefore so are the
T
decreasing sets Kn = n J . Since Kn are compact sets (by Heine-Borel theorem), the set J is then non-empty as well, and since J is a subset of H for all
we arrive at H non-empty, contradicting our assumption that Hn .

Remark. The proof of Lemma 1.1.31 is generic (for finite measures). Namely,
any non-negative finitely additive set function 0 on an algebra A is countably
additive if 0 (Hn ) 0 whenever Hn A and Hn . Further, as this proof shows,
when is a topological space it suffices for countable additivity of 0 to have for
any H A a sequence Jk A such that J k H are compact and 0 (H \ Jk ) 0
as k .
Exercise 1.1.33. Show the necessity of the assumption that A be an algebra in
Caratheodorys extension theorem, by giving an example of two probability measures
6= on a measurable space (, F ) such that (A) = (A) for all A A and
F = (A).
Hint: This can be done with = {1, 2, 3, 4} and F = 2 .
It is often useful to assume that the probability space we have is complete, in the
sense we make precise now.
Definition 1.1.34. We say that a measure space (, F , ) is complete if any
subset N of any B F with (B) = 0 is also in F . If further = P is a probability
measure, we say that the probability space (, F , P) is a complete probability space.
Our next theorem states that any measure space can be completed by adding to
its -algebra all subsets of sets of zero measure (a procedure that depends on the
measure in use).
Theorem 1.1.35. Given a measure space (, F , ), let N = {N : N A for
some A F with (A) = 0} denote the collection of -null sets. Then, there
exists a complete measure space (, F , ), called the completion of the measure
space (, F , ), such that F = {F N : F F , N N } and = on F .
Proof. This is beyond our scope, but see detailed proof in [Dur10, Theorem
A.2.3]. In particular, F = (F , N ) and (A N ) = (A) for any N N and
A F (c.f. [Bil95, Problems 3.10 and 10.5]).

The following collections of sets play an important role in proving the easy part
of Caratheodorys theorem, the uniqueness of the extension .
Definition 1.1.36. A -system is a collection P of sets closed under finite intersections (i.e. if I P and J P then I J P).
A -system is a collection L of sets containing and B\A for any A B A, B L,
15
which is also closed under monotone increasing limits (i.e. if Ai L and Ai A,

then A L as well).
Obviously, an algebra is a -system. Though an algebra may not be a -system,
Proposition 1.1.37. A collection F of sets is a -algebra if and only if it is both
a -system and a -system.
Proof. The fact that a -algebra is a -system is a trivial consequence of
Definition 1.1.1. To prove the converse direction, suppose that F is both a system and a -system. Then is in the -system F and so is Ac = \ A for any
A F . Further, with F also a -system we have that
A B = \ (Ac B c ) F ,
for any A, B F . Consequently,Sif Ai F then so areSalso Gn = A1 An F .
Since F is a -system and Gn i Ai , it follows that i Ai F as well, completing
the verification that F is a -algebra.

The main tool in proving the uniqueness of the extension is Dynkins theorem,
stated next.
Theorem 1.1.38 (Dynkins theorem). If P L with P a -system and
L a -system then (P) L.
Proof. A short though dense exercise in set manipulations shows that the
smallest -system containing P is a -system (for details see [Wil91, Section A.1.3]
or the proof of [Bil95, Theorem 3.2]). By Proposition 1.1.37 it is a -algebra, hence
contains (P). Further, it is contained in the -system L, as L also contains P.
Remark. Proposition 1.1.37 remains valid even if in the definition of -system
we relax the condition that B \ A L for any A B A, B L, to the condition
Ac L whenever A L. However, Dynkins theorem does not hold under the
latter definition.
As we show next, the uniqueness part of Caratheodorys theorem, is an immediate
consequence of the theorem.
Proposition 1.1.39. If two measures 1 and 2 on (, (P)) agree on the system P and are such that 1 () = 2 () < , then 1 = 2 .
Proof. Let L = {A (P) : 1 (A) = 2 (A)}. Our assumptions imply that
P L and that L. Further, (P) is a -system (by Proposition 1.1.37), and
if A B, A, B L, then by additivity of the finite measures 1 and 2 ,
1 (B \ A) = 1 (B) 1 (A) = 2 (B) 2 (A) = 2 (B \ A),
that is, B \ A L. Similarly, if Ai A and Ai L, then by the continuity from
below of 1 and 2 (see remark following Exercise 1.1.4),
1 (A) = lim 1 (An ) = lim 2 (An ) = 2 (A) ,
n
so that A L. We conclude that L is a -system, hence by Dynkins theorem,

(P) L, that is, 1 = 2 .
16
Remark. With a somewhat more involved proof one can relax the condition
1 () = 2 () < to the existence of An P such that An and 1 (An ) <
(c.f. [Bil95, Theorem 10.3] for details). Accordingly, in Caratheodorys extension
theorem we can relax 0 () < to the assumption that 0 is a -finite measure,
that is 0 (An ) < for some An A such that An , as is the case with
Lebesgues measure on R.
We conclude this subsection with an outline the proof of Caratheodorys extension
theorem, noting that since an algebra A is a -system and A, the uniqueness of
the extension to (A) follows from Proposition 1.1.39. Our outline of the existence
of an extension follows [Wil91, Section A.1.8] (or see [Bil95, Theorem 11.3] for
the proof of a somewhat stronger result). This outline centers on the construction
of the appropriate outer measure, a relaxation of the concept of measure, which we
now define.
Definition 1.1.40. An increasing, countably sub-additive, non-negative set function on a measurable space (, F ) is called an outer measure. That is, : F 7
[0, ], having the properties:
(a) ()
(A1 ) (A2 ) for any A1 , A2 F with A1 A2 .
S = 0 andP
(b) ( n An ) n (An ) for any countable collection of sets An F .

In the first step of the proof we define the increasing, non-negative set function
X
[
(E) = inf{
0 (An ) : E
An , An A},
n=1
for E F = 2 , and prove that it is countably sub-additive, hence an outer measure

on F .
By definition, (A)
S 0 (A) for any A A. In the second step we prove that
if in addition A P
n An for An A, then the countable additivity of 0 on A
results with 0 (A) n 0 (An ). Consequently, = 0 on the algebra A.
The third step uses the countable additivity of 0 on A to show that for any A A
the outer measure is additive when splitting subsets of by intersections with A
and Ac . That is, we show that any element of A is a -measurable set, as defined
next.
Definition 1.1.41. Let be a non-negative set function on a measurable space

(, F ), with () = 0. We say that A F is a -measurable set if (F ) =
(F A) + (F Ac ) for all F F .
The fourth step consists of proving the following general lemma.
Lemma 1.1.42 (Carath
eodorys lemma). Let be an outer measure on a
measurable space (, F ). Then the -measurable sets in F form a -algebra G on
which is countably additive, so that (, G, ) is a measure space.
In the current setting, with A contained in the -algebra G, it follows that (A)
G on which is a measure. Thus, the restriction of to (A) is the stated
measure that coincides with 0 on A.
Remark. In the setting of Caratheodorys extension theorem for finite measures,
we have that the -algebra G of all -measurable sets is the completion of (A)
with respect to (c.f. [Bil95, Page 45]). In the context of Lebesgues measure U
17
on B(0,1] , this is the -algebra B (0,1] of all Lebesgue measurable subsets of (0, 1].
Associated with it are the Lebesgue measurable functions f : (0, 1] 7 R for which
f 1 (B) B (0,1] for all B B. However, as noted for example in [Dur10, Theorem
A.2.4], the non Borel set constructed in the proof of Proposition 1.1.18 is also non
Lebesgue measurable.
The following concept of a monotone class of sets is a considerable relaxation of
that of a -system (hence also of a -algebra, see Proposition 1.1.37).
Definition 1.1.43. A monotone class is a collection M of sets closed under both
monotone increasing and monotone decreasing limits (i.e. if Ai M and either
Ai A or Ai A, then A M).
When starting from an algebra instead of a -system, one may save effort by
applying Halmoss monotone class theorem instead of Dynkins theorem.
Theorem 1.1.44 (Halmoss monotone class theorem). If A M with A
an algebra and M a monotone class then (A) M.
Proof. Clearly, any algebra which is a monotone class must be a -algebra.
Another short though dense exercise in set manipulations shows that the intersection m(A) of all monotone classes containing an algebra A is both an algebra and
a monotone class (see the proof of [Bil95, Theorem 3.4]). Consequently, m(A) is
a -algebra. Since A m(A) this implies that (A) m(A) and we complete the
proof upon noting that m(A) M.

Exercise 1.1.45. We say that a subset V of {1, 2, 3, } has Cesaro density (V )
and write V CES if the limit
(V ) = lim n1 |V {1, 2, 3, , n}| ,
n
exists. Give an example of sets V1 CES and V2 CES for which V1 V2

/ CES.
Thus, CES is not an algebra.
Here is an alternative specification of the concept of algebra.
Exercise 1.1.46.
(a) Suppose that A and that A B c A whenever A, B A. Show that
A is an algebra.
(b) Give an example of a collection C of subsets of such that C, if
A C then Ac C and if A, B C are disjoint then also A B C,
while C is not an algebra.
As we already saw, the -algebra structure is preserved under intersections. However, whereas the increasing union of algebras is an algebra, it is not necessarily
the case for -algebras.
Exercise 1.1.47. Suppose that An are classes of sets such that An An+1 .
S
(a) Show that if An are algebras then so is n=1 An . S
(b) Provide an example of -algebras An for which n=1 An is not a algebra.
18
1.2. Random variables and their distribution

Random variables are numerical functions 7 X() of the outcome of our random experiment. However, in order to have a successful mathematical theory, we
limit our interest to the subset of measurable functions (or more generally, measurable mappings), as defined in Subsection 1.2.1 and study the closure properties of
this collection in Subsection 1.2.2. Subsection 1.2.3 is devoted to the characterization of the collection of distribution functions induced by random variables.
1.2.1. Indicators, simple functions and random variables. We start
with the definition of random variables, first in the general case, and then restricted
to R-valued variables.
Definition 1.2.1. A mapping X : 7 S between two measurable spaces (, F )
and (S, S) is called an (S, S)-valued Random Variable (R.V.) if
X 1 (B) := { : X() B} F
B S.
Such a mapping is also called a measurable mapping.

Definition 1.2.2. When we say that X is a random variable, or a measurable
function, we mean an (R, B)-valued random variable which is the most common type
of R.V. we shall encounter. We let mF denote the collection of all (R, B)-valued
measurable mappings, so X is a R.V. if and only if X mF . If in addition is a
topological space and F = ({O open }) is the corresponding Borel -algebra,
we say that X : 7 R is a Borel (measurable) function. More generally, a random
vector is an (Rd , BRd )-valued R.V. for some d < .
The next exercise shows that a random vector is merely a finite collection of R.V.
on the same probability space.
Exercise 1.2.3. Relying on Exercise 1.1.21 and Theorem 1.2.9, show that X :
7 Rd is a random vector if and only if X() = (X1 (), . . . , Xd ()) with each
Xi : 7 R a R.V.
d
T
Hint: Note that X 1 (B1 . . . Bd ) =
Xi1 (Bi ).
i=1
We now provide two important generic examples of random variables.

(
1, A
Example 1.2.4. For any A F the function IA () =
is a R.V.
0,
/A
Indeed, { : IA () B} is for any B R one of the four sets , A, Ac or
(depending on whether 0 B or not and whether 1 B or not), all of whom are
in F . We call such R.V. also an indicator function.
PN
Exercise 1.2.5. By the same reasoning check that X() = n=1 cn IAn () is a
R.V. for any finite N , non-random cn R and sets An F . We call any such X
a simple function, denoted by X SF.
Our next proposition explains why simple functions are quite useful in probability
theory.
Proposition 1.2.6. For every R.V. X() there exists a sequence of simple functions Xn () such that Xn () X() as n , for each fixed .
1.2. RANDOM VARIABLES AND THEIR DISTRIBUTION
19
Proof. Let
fn (x) = n1x>n +
n
n2
1
X
k2n 1(k2n ,(k+1)2n ] (x) ,
k=0
noting that for R.V. X 0, we have that Xn = fn (X) are simple functions. Since
X Xn+1 Xn and X() Xn () 2n whenever X() n, it follows that
Xn () X() as n , for each .
We write a general R.V. as X() = X+ ()X () where X+ () = max(X(), 0)
and X () = min(X(), 0) are non-negative R.V.-s. By the above argument
the simple functions Xn = fn (X+ ) fn (X ) have the convergence property we
claimed.

Note that in case F = 2 , every mapping X : 7 S is measurable (and therefore
is an (S, S)-valued R.V.). The choice of the -algebra F is very important in
determining the class of all (S, S)-valued R.V. For example, there are non-trivial
-algebras G and F on = R such that X() = is a measurable function for
(, F ), but is non-measurable for (, G). Indeed, one such example is when F is the
Borel -algebra B and G = ({[a, b] : a, b Z}) (for example, the set { : }
is not in G whenever
/ Z).
Building on Proposition 1.2.6 we have the following analog of Halmoss monotone
class theorem. It allows us to deduce in the sequel general properties of (bounded)
measurable functions upon verifying them only for indicators of elements of systems.
Theorem 1.2.7 (Monotone class theorem). Suppose H is a collection of
R-valued functions on such that:
(a) The constant function 1 is an element of H.
(b) H is a vector space over R. That is, if h1 , h2 H and c1 , c2 R then
c1 h1 + c2 h2 is in H.
(c) If hn H are non-negative and hn h where h is a (bounded) real-valued
function on , then h H.
If P is a -system and IA H for all A P, then H contains all (bounded)
functions on that are measurable with respect to (P).
Remark. We stated here two versions of the monotone class theorem, with the
less restrictive assumption that (c) holds only for bounded h yielding the weaker
conclusion about bounded elements of m(P). In the sequel we use both versions,
which as we see next, are derived by essentially the same proof. Adapting this
proof you can also show that any collection H of non-negative functions on
satisfying the conditions of Theorem 1.2.7 apart from requiring (b) to hold only
when c1 h1 + c2 h2 0, must contain all non-negative elements of m(P).
Proof. Let L = {A : IA H}. From (a) we have that L, while (b)
implies that B \ A is in L whenever A B are both in L. Further, in view of (c)
the collection L is closed under monotone increasing limits. Consequently, L is a
-system, so by Dynkins - theorem, our assumption that L contains P results
with (P) L. With H a vector space over R, this in turn implies that H contains
all simple functions with respect to the measurable space (, (P)). In the proof of
Proposition 1.2.6 we saw that any (bounded) measurable function is a difference of
20
two (bounded) non-negative functions each of which is a monotone increasing limit

of certain non-negative simple functions. Thus, from (b) and (c) we conclude that
H contains all (bounded) measurable functions with respect to (, (P)).

The concept of almost sure prevails throughout probability theory.
Definition 1.2.8. We say that two (S, S)-valued R.V. X and Y defined on the
same probability space (, F , P) are almost surely the same if P({ : X() 6=
a.s.
Y ()}) = 0. This shall be denoted by X = Y . More generally, same notation
applies to any property of R.V. For example, X() 0 a.s. means that P({ :
a.s.
X() < 0}) = 0. Hereafter, we shall consider X and Y such that X = Y to be the
same S-valued R.V. hence often omit the qualifier a.s. when stating properties
of R.V. We also use the terms almost surely (a.s.), almost everywhere (a.e.), and
with probability 1 (w.p.1) interchangeably.
Since the -algebra S might be huge, it is very important to note that we may
verify that a given mapping is measurable without the need to check that the preimage X 1 (B) is in F for every B S. Indeed, as shown next, it suffices to do
this only for a collection (of our choice) of generators of S.
Theorem 1.2.9. If S = (A) and X : 7 S is such that X 1 (A) F for all
A A, then X is an (S, S)-valued R.V.
Proof. We first check that Sb = {B S : X 1 (B) F } is a -algebra.
Indeed,
a). Sb since X 1 () = .
c
b). If A Sb then X 1 (A) F . With F a -algebra, X 1 (Ac ) = X 1 (A) F .
b
Consequently, Ac S.
c). If An Sb for all n then X 1 (An ) F for all n. With F a -algebra, then also
S
S
S
b
X 1 ( n An ) = n X 1 (An ) F . Consequently, n An S.
b
b as claimed.
Our assumption that A S, then translates to S = (A) S,

The most important -algebras are those generated by ((S, S)-valued) random
variables, as defined next.
Exercise 1.2.10. Adapting the proof of Theorem 1.2.9, show that for any mapping
X : 7 S and any -algebra S of subsets of S, the collection {X 1 (B) : B S} is
a -algebra. Verify that X is an (S, S)-valued R.V. if and only if {X 1 (B) : B
S} F , in which case we denote {X 1 (B) : B S} either by (X) or by F X and
call it the -algebra generated by X.
To practice your understanding of generated -algebras, solve the next exercise,
providing a convenient collection of generators for (X).
Exercise 1.2.11. If X is an (S, S)-valued R.V. and S = (A) then (X) is
generated by the collection of sets X 1 (A) := {X 1 (A) : A A}.
An important example of use of Exercise 1.2.11 corresponds to (R, B)-valued random variables and A = {(, x] : x R} (or even A = {(, x] : x Q}) which
generates B (see Exercise 1.1.17), leading to the following alternative definition of
the -algebra generated by such R.V. X.
21
Definition 1.2.12. Given a function X : 7 R we denote by (X) or by F X

the smallest -algebra F such that X() is a measurable mapping from (, F ) to
(R, B). Alternatively,
(X) = ({ : X() }, R) = ({ : X() q}, q Q) .
More generally, given a random vector X = (X1 , . . . , Xn ), that is, random variables
X1 , . . . , Xn on the same probability space, let (Xk , k n) (or FnX ), denote the
smallest -algebra F such that Xk (), k = 1, . . . , n are measurable on (, F ).
Alternatively,
(Xk , k n) = ({ : Xk () }, R, k n) .
Finally, given a possibly uncountable collection of functions X : 7 R, indexed

by , we denote by (X , ) (or simply by F X ), the smallest -algebra F
such that X (), are measurable on (, F ).
The concept of -algebra is needed in order to produce a rigorous mathematical
theory. It further has the crucial role of quantifying the amount of information
we have. For example, (X) contains exactly those events A for which we can say
whether A or not, based on the value of X(). Interpreting Example 1.1.19 as
corresponding to sequentially tossing coins, the R.V. Xn () = n gives the result
of the n-th coin toss in our experiment of infinitely many such tosses. The algebra Fn = 2n of Example 1.1.6 then contains exactly the information we have
upon observing the outcome of the first n coin tosses, whereas the larger -algebra
Fc allows us to also study the limiting properties of this sequence (and as you show
next, Fc is isomorphic, in the sense of Definition 1.4.24, to B[0,1] ).
Exercise 1.2.13. Let Fc denote the cylindrical -algebra for the set = {0, 1}N
of infinite binary sequences, as in Example 1.1.19.
P
(a) Show that X() = n=1 n 2n is a measurable map from ( , Fc ) to
([0, 1], B[0,1] ).
(b) Conversely, let Y (x) = (1 , . . . , n , . . .) where for each n 1, n (1) = 1
while n (x) = I(2n x is an odd number) when x [0, 1). Show that
Y = X 1 is a measurable map from ([0, 1], B[0,1]) to ( , Fc ).
Here are some alternatives for Definition 1.2.12.
Exercise 1.2.14. Verify the following relations and show that each generating
collection of sets on the right hand side is a -system.
(a) (X) = ({ : X() }, R)
(b) (Xk , k n) = ({ : Xk () k , 1 k n}, 1 , . . . , n R)
(c) (X1 , X2 , . . .) = ({ : Xk () k , 1 k m}, 1 , . . . , m R, m
N)
S
(d) (X1 , X2 , . . .) = ( n (Xk , k n))
As you next show, when approximating a random variable by a simple function,
one may also specify the latter to be based on sets in any generating algebra.
Exercise 1.2.15. Suppose (, F , P) is a probability space, with F = (A) for an

algebra A.
(a) Show that inf{P(AB) : A A} = 0 for any B F (recall that AB =
(A B c ) (Ac B)).
22
(b) Show that for any bounded random variable X and > 0 there exists a
PN
simple function Y = n=1 cn IAn with An A such that P(|X Y | >
) < .
Exercise 1.2.16. Let F = (A , ) and suppose there exist 1 6= 2

such that for any , either {1 , 2 } A or {1 , 2 } Ac .
(a) Show that if mapping X is measurable on (, F ) then X(1 ) = X(2 ).
(b) Provide an explicit -algebra F of subsets of = {1, 2, 3} and a mapping
X : 7 R which is not a random variable on (, F ).
We conclude with a glimpse of the canonical measurable space associated with a

stochastic process (Xt , t T) (for more on this, see Lemma 7.1.7).
Exercise 1.2.17. Fixing a possibly uncountable collection of random variables Xt ,
indexed by t T, let FCX = (Xt , t C) for each C T. Show that
[
FTX =
FCX
C countable
X
and that any R.V. Z on (, FT ) is measurable on FCX for some countable C T.
1.2.2. Closure properties of random variables. For the typical measurable space with uncountable it is impractical to list all possible R.V. Instead,
we state a few useful closure properties that often help us in showing that a given
mapping X() is indeed a R.V.
We start with closure with respect to the composition of a R.V. and a measurable
mapping.
Proposition 1.2.18. If X : 7 S is an (S, S)-valued R.V. and f is a measurable
mapping from (S, S) to (T, T ), then the composition f (X) : 7 T is a (T, T )valued R.V.
Proof. Considering an arbitrary B T , we know that f 1 (B) S since f is
a measurable mapping. Thus, as X is an (S, S)-valued R.V. it follows that
[f (X)]1 (B) = X 1 (f 1 (B)) F .
This holds for any B T , thus concluding the proof.
In view of Exercise 1.2.3 we have the following special case of Proposition 1.2.18,
corresponding to S = Rn and T = R equipped with the respective Borel -algebras.
Corollary 1.2.19. Let Xi , i = 1, . . . , n be R.V. on the same measurable space
(, F ) and f : Rn 7 R a Borel function. Then, f (X1 , . . . , Xn ) is also a R.V. on
the same space.
To appreciate the power of Corollary 1.2.19, consider the following exercise, in
which you show that every continuous function is also a Borel function.
Exercise 1.2.20. Suppose (S, ) is a metric space (for example, S = Rn ). A function g : S 7 [, ] is called lower semi-continuous (l.s.c.) if lim inf (y,x)0 g(y)
g(x), for all x S. A function g is said to be upper semi-continuous(u.s.c.) if g
is l.s.c.
(a) Show that if g is l.s.c. then {x : g(x) b} is closed for each b R.
(b) Conclude that semi-continuous functions are Borel measurable.
(c) Conclude that continuous functions are Borel measurable.
23
A concrete application of Corollary 1.2.19 shows that any linear combination of

finitely many R.V.-s is a R.V.
Example 1.2.21.
PnSuppose Xi are R.V.-s on the same measurable space and ci R.
Then, Wn () = i=1
Pnci Xi () are also R.V.-s. To see this, apply Corollary 1.2.19
for f (x1 , . . . , xn ) = i=1 ci xi a continuous, hence Borel (measurable) function (by
Exercise 1.2.20).
We turn to explore the closure properties of mF with respect to operations of a
limiting nature, starting with the following key theorem.
Theorem 1.2.22. Let R = [, ] equipped with its Borel -algebra
BR = ([, b) : b R) .
If Xi are R-valued R.V.-s on the same measurable space, then

inf Xn ,
n
sup Xn ,
n
lim inf Xn ,
lim sup Xn ,
are also R-valued random variables.

Proof. Pick an arbitrary b R. Then,
[
[
{ : inf Xn () < b} =
{ : Xn () < b} =
Xn1 ([, b)) F .
n
n=1
n=1
Since BR is generated by {[, b) : b R}, it follows by Theorem 1.2.9 that inf n Xn

is an R-valued R.V.
Observing that supn Xn = inf n (Xn ), we deduce from the above and Corollary
1.2.19 (for f (x) = x), that supn Xn is also an R-valued R.V.
Next, recall that
i
h
W = lim inf Xn = sup inf Xl .
n
ln
By the preceding proof we have that Yn = inf ln Xl are R-valued R.V.-s and hence
so is W = supn Yn .
Similarly to the arguments already used, we conclude the proof either by observing
that
i
h
Z = lim sup Xn = inf sup Xl ,
n
ln
or by observing that lim supn Xn = lim inf n (Xn ).
Remark. Since inf n Xn , supn Xn , lim supn Xn and lim inf n Xn may result in values even when every Xn is R-valued, hereafter we let mF also denote the
collection of R-valued R.V.
An important corollary of this theorem deals with the existence of limits of sequences of R.V.
Corollary 1.2.23. For any sequence Xn mF , both
0 = { : lim inf Xn () = lim sup Xn ()}

n
and
1 = { : lim inf Xn () = lim sup Xn () R}
n
are measurable sets, that is, 0 F and 1 F .
24
Proof. By Theorem 1.2.22 we have that Z = lim supn Xn and W = lim inf n Xn
are two R-valued variables on the same space, with Z() W () for all . Hence,
1 = { : Z() W () = 0, Z() R, W () R} is measurable (apply Corollary
1.2.19 for f (z, w) = z w), as is 0 = W 1 ({}) Z 1 ({}) 1 .

The following structural result is yet another consequence of Theorem 1.2.22.
Corollary 1.2.24. For any d < and R.V.-s Y1 , . . . , Yd on the same measurable
space (, F ) the collection H = {h(Y1 , . . . , Yd ); h : Rd 7 R Borel function} is a
vector space over R containing the constant functions, such that if Xn H are
non-negative and Xn X, an R-valued function on , then X H.
Proof. By Example 1.2.21 the collection of all Borel functions is a vector
space over R which evidently contains the constant functions. Consequently, the
same applies for H. Next, suppose Xn = hn (Y1 , . . . , Yd ) for Borel functions hn such
that 0 Xn () X() for all . Then, h(y) = supn hn (y) is by Theorem
1.2.22 an R-valued Borel function on Rd , such that X = h(Y1 , . . . , Yd ). Setting
h(y) = h(y) when h(y) R and h(y) = 0 otherwise, it is easy to check that h is a
real-valued Borel function. Moreover, with X : 7 R (finite valued), necessarily
X = h(Y1 , . . . , Yd ) as well, so X H.

The point-wise convergence of R.V., that is Xn () X(), for every is
often too strong of a requirement, as it may fail to hold as a result of the R.V. being
ill-defined for a negligible set of values of (that is, a set of zero measure). We
thus define the more useful, weaker notion of almost sure convergence of random
variables.
Definition 1.2.25. We say that a sequence of random variables Xn on the same
probability space (, F , P) converges almost surely if P(0 ) = 1. We then set
X = lim supn Xn , and say that Xn converges almost surely to X , or use the
a.s.
notation Xn X .
Remark. Note that in Definition 1.2.25 we allow the limit X () to take the
values with positive probability. So, we say that Xn converges almost surely
to a finite limit if P(1 ) = 1, or alternatively, if X R with probability one.
We proceed with an explicit characterization of the functions measurable with
respect to a -algebra of the form (Yk , k n).
Theorem 1.2.26. Let G = (Yk , k n) for some n < and R.V.-s Y1 , . . . , Yn
on the same measurable space (, F ). Then, mG = {g(Y1 , . . . , Yn ) : g : Rn 7
R is a Borel function}.
Proof. From Corollary 1.2.19 we know that Z = g(Y1 , . . . , Yn ) is in mG for
each Borel function g : Rn 7 R. Turning to prove the converse result, recall
part (b) of Exercise 1.2.14 that the -algebra G is generated by the -system P =
{A : = (1 , . . . , n ) Rn } where IA = h (Y1 , . . . , Yn ) for the Borel function
Qn
h (y1 , . . . , yn ) = k=1 1yk k . Thus, in view of Corollary 1.2.24, we have by the
monotone class theorem that H = {g(Y1 , . . . , Yn ) : g : Rn 7 R is a Borel function}
contains all elements of mG.

We conclude this sub-section with a few exercises, starting with Borel measurability of monotone functions (regardless of their continuity properties).
25
Exercise 1.2.27. Show that any monotone function g : R 7 R is Borel measurable.

Next, Exercise 1.2.20 implies that the set of points at which a given function g is
discontinuous, is a Borel set.
Exercise 1.2.28. Fix an arbitrary function g : S 7 R.
(a) Show that for any > 0 the function g (x, ) = inf{g(y) : (x, y) < } is
u.s.c. and the function g (x, ) = sup{g(y) : (x, y) < } is l.s.c.
(b) Show that Dg = {x : supk g (x, k 1 ) < inf k g (x, k 1 )} is exactly the set
of points at which g is discontinuous.
(c) Deduce that the set Dg of points of discontinuity of g is a Borel set.
Here is an alternative characterization of B that complements Exercise 1.2.20.
Exercise 1.2.29. Show that if F is a -algebra of subsets of R then B F if

and only if every continuous function f : R 7 R is in mF (i.e. B is the smallest
-algebra on R with respect to which all continuous functions are measurable).
Exercise 1.2.30. Suppose Xn and X are real-valued random variables and
P({ : lim sup Xn () X ()}) = 1 .
n
Show that for any > 0, there exists an event A with P(A) < and a non-random
N = N (), sufficiently large such that Xn () < X () + for all n N and every
Ac .
Equipped with Theorem 1.2.22 you can also strengthen Proposition 1.2.6.
Exercise 1.2.31. Show that the class mF of R-valued measurable functions, is

the smallest class containing SF and closed under point-wise limits.
Your next exercise also relies on Theorem 1.2.22.
Exercise 1.2.32. Given a measurable space (, F ) and (not necessarily in
F ), let F = {A : A F }.
(a) Check that (, F ) is a measurable space.
(b) Show that any bounded, F -measurable function (on ), is the restriction
to of some bounded, F -measurable f : R.
Finally, relying on Theorem 1.2.26 it is easy to show that a Borel function can
only reduce the amount of information quantified by the corresponding generated
-algebras, whereas such information content is invariant under invertible Borel
transformations, that is
Exercise 1.2.33. Show that (g(Y1 , . . . , Yn )) (Yk , k n) for any Borel function g : Rn 7 R. Further, if Y1 , . . . , Yn and Z1 , . . . , Zm defined on the same probability space are such that Zk = gk (Y1 , . . . , Yn ), k = 1, . . . , m and Yi = hi (Z1 , . . . , Zm ),
i = 1, . . . , n for some Borel functions gk : Rn 7 R and hi : Rm 7 R, then
(Y1 , . . . , Yn ) = (Z1 , . . . , Zm ).
1.2.3. Distribution, density and law. As defined next, every random variable X induces a probability measure on its range which is called the law of X.
Definition 1.2.34. The law of a real-valued R.V. X, denoted PX , is the probability measure on (R, B) such that PX (B) = P({ : X() B}) for any Borel set
B.
26
Remark. Since X is a R.V., it follows that PX (B) is well defined for all B B.
Further, the non-negativity of P implies that PX is a non-negative set function on
(R, B), and since X 1 (R) = , also PX (R) = 1. Consider next disjoint Borel sets
Bi , observing that X 1 (Bi ) F are disjoint subsets of such that
[
[
X 1 (Bi ) .
X 1 ( Bi ) =
i
Thus, by the countable additivity of P we have that

X
X
[
[
PX (Bi ) .
P(X 1 (Bi )) =
PX ( Bi ) = P( X 1 (Bi )) =
i
This shows that PX is also countably additive, hence a probability measure, as

claimed in Definition 1.2.34.
Note that the law PX of a R.V. X : R, determines the values of the
probability measure P on (X).
D
Definition 1.2.35. We write X = Y and say that X equals Y in law (or in

distribution), if and only if PX = PY .
A good way to practice your understanding of the Definitions 1.2.34 and 1.2.35 is
a.s.
D
by verifying that if X = Y , then also X = Y (that is, any two random variables
we consider to be the same would indeed have the same law).
The next concept we define, the distribution function, is closely associated with
the law PX of the R.V.
Definition 1.2.36. The distribution function FX of a real-valued R.V. X is
FX () = P({ : X() }) = PX ((, ])
Our next result characterizes the set of all functions F : R 7 [0, 1] that are
distribution functions of some R.V.
Theorem 1.2.37. A function F : R 7 [0, 1] is a distribution function of some
R.V. if and only if
(a) F is non-decreasing
(b) limx F (x) = 1 and limx F (x) = 0
(c) F is right-continuous, i.e. limyx F (y) = F (x)
Proof. First, assuming that F = FX is a distribution function, we show that
it must have the stated properties (a)-(c). Indeed, if x y then (, x] (, y],
and by the monotonicity of the probability measure PX (see part (a) of Exercise
1.1.4), we have that FX (x) FX (y), proving that FX is non-decreasing. Further,
(, x] R as x , while (, x] as x , resulting with property (b)
of the theorem by the continuity from below and the continuity from above of the
probability measure PX on R. Similarly, since (, y] (, x] as y x we get
the right continuity of FX by yet another application of continuity from above of
PX .
We proceed to prove the converse result, that is, assuming F has the stated properties (a)-(c), we consider the random variable X () = sup{y : F (y) < } on
the probability space ((0, 1], B(0,1] , U ) and show that FX = F . With F having
27
property (b), we see that for any > 0 the set {y : F (y) < } is non-empty and
further if < 1 then X () < , so X : (0, 1) 7 R is well defined. The identity
(1.2.1)
{ : X () x} = { : F (x)} ,
implies that FX (x) = U ((0, F (x)]) = F (x) for all x R, and further, the sets
(0, F (x)] are all in B(0,1] , implying that X is a measurable function (i.e. a R.V.).
Turning to prove (1.2.1) note that if F (x) then x 6 {y : F (y) < } and so by
definition (and the monotonicity of F ), X () x. Now suppose that > F (x).
Since F is right continuous, this implies that F (x + ) < for some > 0, hence
by definition of X also X () x + > x, completing the proof of (1.2.1) and
with it the proof of the theorem.

Check your understanding of the preceding proof by showing that the collection
of distribution functions for R-valued random variables consist of all F : R 7 [0, 1]
that are non-decreasing and right-continuous.
Remark. The construction of the random variable X () in Theorem 1.2.37 is
called Skorokhods representation. You can, and should, verify that the random
variable X + () = sup{y : F (y) } would have worked equally well for that
purpose, since X + () 6= X () only if X + () > q X () for some rational q,
in which case by definition F (q) , so there are most countably many such
values of (hence P(X + 6= X ) = 0). We shall return to this construction when
dealing with convergence in distribution in Section 3.2. An alternative approach to
Theorem 1.2.37 is to adapt the construction of the probability measure of Example
1.1.26, taking here = RPwith the corresponding change to A and replacing the
r
right side of (1.1.1) with k=1 (F (bk ) F (ak )), yielding a probability measure P
on (R, B) such that P((, ]) = F () for all R (c.f. [Bil95, Theorem 12.4]).
Our next example highlights the possible shape of the distribution function.
Example 1.2.38. Consider Example 1.1.6 of n coin tosses, with -algebra
P Fn =
2n , sample space n = {H, T }n, and the probability measure Pn (A) = A p ,
where p = 2n for each n (that is, = {1 , 2 , , n } for i {H, T }),
corresponding to independent, fair, coin tosses. Let Y () = I{1 =H} measure the
outcome of the first toss. The law of this random variable is,
1
1
PY (B) = 1{0B} + 1{1B}
2
2
and its distribution function is
1, 1
(1.2.2) FY () = PY ((, ]) = Pn (Y () ) = 12 , 0 < 1 .
0, < 0
Note that in general (X) is a strict subset of the -algebra F (in Example 1.2.38
we have that (Y ) determines the probability measure for the first coin toss, but
tells us nothing about the probability measure assigned to the remaining n 1
tosses). Consequently, though the law PX determines the probability measure P
on (X) it usually does not completely determine P.
Example 1.2.38 is somewhat generic. That is, if the R.V. X is a simple function (or
more generally, when the set {X() : } is countable and has no accumulation
points), then its distribution function FX is piecewise constant with jumps at the
28
possible values that X takes and jump sizes that are the corresponding probabilities.
Indeed, note that (, y] (, x) as y x, so by the continuity from below of
PX it follows that
FX (x ) := lim FX (y) = P({ : X() < x}) = FX (x) P({ : X() = x}) ,
yx
for any R.V. X.

A direct corollary of Theorem 1.2.37 shows that any distribution function has a
collection of continuity points that is dense in R.
Exercise 1.2.39. Show that a distribution function F has at most countably many
points of discontinuity and consequently, that for any x R there exist yk and zk
at which F is continuous such that zk x and yk x.
In contrast with Example 1.2.38 the distribution function of a R.V. with a density
is continuous and almost everywhere differentiable, that is,
Definition 1.2.40. We say that a R.V. X() has a probability density function
fX if and only if its distribution function FX can be expressed as
Z
(1.2.3)
FX () =
fX (x)dx ,
R .
By Theorem 1.2.37 a probability density function

R fX must be an integrable, Lebesgue
almost everywhere non-negative function, with R fX (x)dx = 1. Such FX is continX
uous with dF
dx (x) = fX (x) except possibly on a set of values of x of zero Lebesgue
measure.
Remark. To make Definition 1.2.40 precise we temporarily assume that probability density functions fX are Riemann integrable and interpret the integral in (1.2.3)
in this sense. In Section 1.3 we construct Lebesgues integral and extend the scope
of Definition 1.2.40 to Lebesgue integrable density functions fX 0 (in particular,
accommodating Borel functions fX ). This is the setting we assume thereafter, with
the right-hand-side of (1.2.3) interpreted as the integral (fX ; (, ]) of fX with
respect to the restriction on (, ] of the completion of the Lebesgue measure on
R (c.f. Definition 1.3.59 and Example 1.3.60). Further, the function fX is uniquely
defined only as a representative of an equivalence class. That is, in this context we
consider f and g to be the same function when ({x : f (x) 6= g(x)}) = 0.
Building on Example 1.1.26 we next detail a few classical examples of R.V. that
have densities.
Example 1.2.41. The distribution function FU of the R.V. of Example 1.1.26 is
1, > 1
FU () = P(U ) = P(U [0, ]) = , 0 1
(1.2.4)
0, < 0
(
1, 0 u 1
and its density is fU (u) =
.
0, otherwise
The exponential distribution function is
(
0, x 0
F (x) =
,
1 ex , x 0
(
0, x 0
corresponding to the density f (x) =
ex , x > 0
distribution has the density
(x) = (2)1/2 e
x2
2
29
, whereas the standard normal
no closed form expression for the corresponding distribution function (x) =

Rwith
x
(u)du in terms of elementary functions.
Every real-valued R.V. X has a distribution function but not necessarily a density.
For example X = 0 w.p.1 has distribution function FX () = 10 . Since FX is
discontinuous at 0, the R.V. X does not have a density.
Definition 1.2.42. We say that a function F is a Lebesgue singular function if
it has a zero derivative except on a set of zero Lebesgue measure.
Since the distribution function of any R.V. is non-decreasing, from real analysis
we know that it is almost everywhere differentiable. However, perhaps somewhat
surprisingly, there are continuous distribution functions that are Lebesgue singular
functions. Consequently, there are non-discrete random variables that do not have
a density. We next provide one such example.
Example 1.2.43. The Cantor set C is defined by removing (1/3, 2/3) from [0, 1]
and then iteratively removing the middle third of each interval that remains. The
uniform distribution on the (closed) set C corresponds to the distribution function
obtained by setting F (x) = 0 for x 0, F (x) = 1 for x 1, F (x) = 1/2 for
x [1/3, 2/3], then F (x) = 1/4 for x [1/9, 2/9], F (x) = 3/4 for x [7/9, 8/9],
and so on (which as you should check, satisfies the properties (a)-(c) of Theorem
1.2.37). From the definition, we see that dF/dx = 0 for almost every x
/ C and
that the corresponding probability measure has P(C c ) = 0. As the Lebesgue measure
of C is zero, we see that the derivative of F is zero except on a set of zero
R x Lebesgue
measure, and consequently, there is no function f for which F (x) = f (y)dy
holds. Though it is somewhat more involved, you may want to check that F is
everywhere continuous (c.f. [Bil95, Problem 31.2]).
Even discrete distribution functions can be quite complex. As the next example
shows, the points of discontinuity of such a function might form a (countable) dense
subset of R (which in a sense is extreme, per Exercise 1.2.39).
Example 1.2.44. Let q1 , q2 , . . . be an enumeration of the rational numbers and set
X
2i 1[qi ,) (x)
F (x) =
i=1
(where 1[qi ,) (x) = 1 if x qi and zero otherwise). Clearly, such F is nondecreasing, with limits 0 and 1 as x and x , respectively. It is not hard
to check that F is also right continuous, hence a distribution function, whereas by
construction F is discontinuous at each rational number.
As we have that P({ : X() }) = FX () for the generators { : X() }
of (X), we are not at all surprised by the following proposition.
Proposition 1.2.45. The distribution function FX uniquely determines the law
PX of X.
30
Proof. Consider the collection (R) = {(, b] : b R} of subsets of R. It

is easy to see that (R) is a -system, which generates B (see Exercise 1.1.17).
Hence, by Proposition 1.1.39, any two probability measures on (R, B) that coincide
on (R) are the same. Since the distribution function FX specifies the restriction
of such a probability measure PX on (R) it thus uniquely determines the values
of PX (B) for all B B.

Different probability measures P on the measurable space (, F ) may trivialize
different -algebras. That is,
Definition 1.2.46. If a -algebra H F and a probability measure P on (, F )
are such that P(H) {0, 1} for all H H, we call H a P-trivial -algebra.
Similarly, a random variable X is called P-trivial or P-degenerate, if there exists
a non-random constant c such that P(X 6= c) = 0.
Using distribution functions we show next that all random variables on a P-trivial
-algebra are P-trivial.
Proposition 1.2.47. If a random variable X mH for a P-trivial -algebra H,
then X is P-trivial.
Proof. By definition, the sets { : X() } are in H for all R. Since H
is P-trivial this implies that FX () {0, 1} for all R. In view of Theorem 1.2.37
this is possible only if FX () = 1c for some non-random c R (for example, set
c = inf{ : FX () = 1}). That is, P(X 6= c) = 0, as claimed.

We conclude with few exercises about the support of measures on (R, B).
Exercise 1.2.48. Let be a measure on (R, B). A point x is said to be in the
support of if (O) > 0 for every open neighborhood O of x. Prove that the support
is a closed set whose complement is the maximal open set on which vanishes.
Exercise 1.2.49. Given an arbitrary closed set C R, construct a probability
measure on (R, B) whose support is C.
Hint: Try a measure consisting of a countable collection of atoms (i.e. points of
positive probability).
As you are to check next, the discontinuity points of a distribution function are
closely related to the support of the corresponding law.
Exercise 1.2.50. The support of a distribution function F is the set SF = {x R
such that F (x + ) F (x ) > 0 for all > 0}.
(a) Show that all points of discontinuity of F () belong to SF , and that any
isolated point of SF (that is, x SF such that (x , x + ) SF = {x}
for some > 0) must be a point of discontinuity of F ().
(b) Show that the support of the law PX of a random variable X, as defined
in Exercise 1.2.48, is the same as the support of its distribution function
FX .
1.3. Integration and the (mathematical) expectation
A key concept in probability theory is the mathematical expectation of random variables. In Subsection 1.3.1 we provide its definition via the framework
of Lebesgue integration with respect to a measure and study properties such as
1.3. INTEGRATION AND THE (MATHEMATICAL) EXPECTATION
31
monotonicity and linearity. In Subsection 1.3.2 we consider fundamental inequalities associated with the expectation. Subsection 1.3.3 is about the exchange of
integration and limit operations, complemented by uniform integrability and its
consequences in Subsection 1.3.4. Subsection 1.3.5 considers densities relative to
arbitrary measures and relates our treatment of integration and expectation to
Riemanns integral and the classical definition of the expectation for a R.V. with
probability density. We conclude with Subsection 1.3.6 about moments of random
variables, including their values for a few well known distributions.
1.3.1. Lebesgue integral, linearity and monotonicity. Let SF+ denote
the collection of non-negative simple functions with respect to the given measurable
space (S, F ) and mF+ denote the collection of [0, ]-valued measurable functions
on this space. We next define Lebesgues integral with respect to any measure
on (S, F ),
R first for SF+ , then extending it to all f mF+ . With the notation
(f ) := S f (s)d(s) for this integral, we also denote by 0 () the more restrictive
integral, defined only on SF+ , so as to clarify the role each of these plays in some of
our proofs. We call an R-valued measurable function f mF for which (|f |) < ,
a -integrable function, and denote the collection of all -integrable functions by
L1 (S, F , ), extending the definition of the integral (f ) to all f L1 (S, F , ).
Definition 1.3.1. Fix a measure space (S, F , ) and define (f ) by the following
four step procedure:
Step 1. Define 0 (IA ) := (A) for each A F .
Step 2. Any SF+ has a representation =
n
P
l=1
cl IAl for some finite n < ,
non-random cl [0, ] and sets Al F , yielding the definition of the integral via
0 () :=
n
X
cl (Al ) ,
l=1
where we adopt hereafter the convention that 0 = 0 = 0.

Step 3. For f mF+ we define
(f ) := sup{0 () : SF+ , f }.
Step 4. For f mF let f+ = max(f, 0) mF+ and f = min(f, 0) mF+ .
We then set (f ) = (f+ ) (f ) provided either (f+ ) < or (f ) < . In
particular, this applies whenever f L1 (S, F , ), for then (f+ ) + (f ) = (|f |)
is finite, hence (f ) is Rwell defined and finite valued.
We use the notation S f (s)d(s) for (f ) which we call Lebesgue integral of f
with respect to the measure .
The expectation E[X] of aRrandom variable X on a probability space (, F , P) is
merely Lebesgues integral X()dP() of X with respect to P. That is,
Step 1. E [IA ] = P(A) for any A F .
n
P
cl IAl for some non-random
Step 2. Any SF+ has a representation =
l=1
n < , cl 0 and sets Al F , to which corresponds

E[] =
n
X
l=1
cl E[IAl ] =
n
X
l=1
cl P(Al ) .
32
Step 3. For X mF+ define
EX = sup{EY : Y SF+ , Y X}.
Step 4. Represent X mF as X = X+ X , where X+ = max(X, 0) mF+ and

X = min(X, 0) mF+ , with the corresponding definition
EX = EX+ EX ,
provided either EX+ < or EX < .
Remark. Note that we may have EX = while X() < for all . For
instance, take the random variable
X() = for = {1, 2, . . .} and F = 2 . If
P 2
2
P( = k) = ck
c = [ k=1 k ]1 a positive, finite normalization constant,
P with
1
then EX = c k=1 k = .
Similar to the notation of -integrable functions introduced in the last step of

the definition of Lebesgues integral, we have the following definition for random
variables.
Definition 1.3.2. We say that a random variable X is (absolutely) integrable,

or X has finite expectation, if E|X| < , that is, both EX+ < and EX < .
Fixing 1 q < we denote by Lq (, F , P) the collection of random variables X
on (, F ) for which ||X||q = [E|X|q ]1/q < . For example, L1 (, F , P) denotes
the space of all (absolutely) integrable random-variables. We use the short notation
Lq when the probability space (, F , P) is clear from the context.
We next verify that Lebesgues integral of each function f is assigned a unique
value in Definition 1.3.1. To this end, we focus on 0 : SF+ 7 [0, ] of Step 2 of
our definition and derive its structural properties, such as monotonicity, linearity
and invariance to a change of argument on a -negligible set.
Lemma 1.3.3. 0 () assigns a unique value to each SF+ . Further,
a). 0 () = 0 () if , SF+ are such that ({s : (s) 6= (s)}) = 0.
b). 0 is linear, that is
0 ( + ) = 0 () + 0 () ,
0 (c) = c0 () ,
for any , SF+ and c 0.

c). 0 is monotone, that is 0 () 0 () if (s) (s) for all s S.
Proof. Note that a non-negative simple function SF+ has many different
representations as weighted sums of indicator functions. Suppose for example that
n
m
X
X
(1.3.1)
cl IAl (s) =
dk IBk (s) ,
l=1
k=1
for some cl 0, dk 0, Al F , Bk F and all s S. There exists a finite

partition of S to at most 2n+m disjoint sets Ci such that each of the sets Al and
Bk is a union of some Ci , i = 1, . . . , 2n+m . Expressing both sides of (1.3.1) as finite
weighted sums of ICi , we necessarily have for each i the same weight on both sides.
Due to the (finite) additivity of over unions of disjoint sets Ci , we thus get after
some algebra that
n
m
X
X
(1.3.2)
cl (Al ) =
dk (Bk ) .
l=1
k=1
33
Consequently, 0 () is well-defined and independent of the chosen representation

for . Further, the conclusion (1.3.2) applies also when the two sides of (1.3.1)
differ for s C as long as (C) = 0, hence proving the first stated property of the
lemma.
Choosing the representation of + based on the representations of and
immediately results with the stated linearity of 0 . Given this, if (s) (s) for all
s, then = + for some SF+ , implying that 0 () = 0 () + 0 () 0 (),
as claimed.

Remark. The stated monotonicity of 0 implies that () coincides with 0 () on
SF+ . As 0 is uniquely defined for each f SF+ and f = f+ when f mF+ , it
follows that (f ) is uniquely defined for each f mF+ L1 (S, F , ).
All three properties of 0 (hence ) stated in Lemma 1.3.3 for functions in SF+
extend to all of mF+ L1 . Indeed, the facts that (cf ) = c(f ), that (f ) (g)
whenever 0 f g, and that (f ) = (g) whenever ({s : f (s) 6= g(s)}) = 0 are
immediate consequences of our definition (once we have these for f, g SF+ ). Since
f g implies f+ g+ and f g , the monotonicity of () extends to functions
in L1 (by Step 4 of our definition). To prove that (h + g) = (h) + (g) for all
h, g mF+ L1 requires an application of the monotone convergence theorem (in
short MON), which we now state, while deferring its proof to Subsection 1.3.3.
Theorem 1.3.4 (Monotone convergence theorem). If 0 hn (s) h(s) for
all s S and hn mF+ , then (hn ) (h) .
Indeed, recall that while proving Proposition 1.2.6 we constructed the sequence
fn such that for every g mF+ we have fn (g) SF+ and fn (g) g. Specifying
g, h mF+ we have that fn (h) + fn (g) SF+ . So, by Lemma 1.3.3,
(fn (h)+fn (g)) = 0 (fn (h)+fn (g)) = 0 (fn (h))+0 (fn (g)) = (fn (h))+(fn (g)) .
Since fn (h) h and fn (h) + fn (g) h + g, by monotone convergence,
(h + g) = lim (fn (h) + fn (g)) = lim (fn (h)) + lim (fn (g)) = (h) + (g) .
n
To extend this result to g, h mF+ L , note that h + g = f + (h + g) f for

some f mF+ such that h+ +g+ = f +(h+g)+ . Since (h ) < and (g ) < ,
by linearity and monotonicity of () on mF+ necessarily also (f ) < and the
linearity of (h + g) on mF+ L1 follows by elementary algebra. In conclusion, we
have just proved that
Proposition 1.3.5. The integral (f ) assigns a unique value to each f mF+
L1 (S, F , ). Further,
a). (f ) = (g) whenever ({s : f (s) 6= g(s)}) = 0.
b). is linear, that is for any f, h, g mF+ L1 and c 0,
(h + g) = (h) + (g) ,
(cf ) = c(f ) .
c). is monotone, that is (f ) (g) if f (s) g(s) for all s S.

Our proof of the identity (h + g) = (h) + (g) is an example of the following
general approach to proving that certain properties hold for all h L1 .
Definition 1.3.6 (Standard Machine). To prove the validity of a certain property
for all h L1 (S, F , ), break your proof to four easier steps, following those of
Definition 1.3.1.
34
Step 1. Prove the property for h which is an indicator function.

Step 2. Using linearity, extend the property to all SF+ .
Step 3. Using MON extend the property to all h mF+ .
Step 4. Extend the property in question to h L1 by writing h = h+ h and
using linearity.
Here is another application of the standard machine.
Exercise 1.3.7. Suppose that a probability measure P on (R, B) is such that
P(B) = (f IB ) for the Lebesgue measure on R, some non-negative Borel function
f () and all B B. Using the standard machine, prove that then P(h) = (f h) for
any Borel function h such that either h 0 or (f |h|) < .
Hint: See the proof of Proposition 1.3.56.
We shall see more applications of the standard machine later (for example, when
proving Proposition 1.3.56 and Theorem 1.3.61).
We next strengthen the non-negativity and monotonicity properties of Lebesgues
integral () by showing that
Lemma 1.3.8. If (h) = 0 for h mF+ , then ({s : h(s) > 0}) = 0. Consequently, if for f, g L1 (S, F , ) both (f ) = (g) and ({s : f (s) > g(s)}) = 0,
then ({s : f (s) 6= g(s)}) = 0.
Proof. By continuity below of the measure we have that
({s : h(s) > 0}) = lim ({s : h(s) > n1 })
n
(see Exercise 1.1.4). Hence, if ({s : h(s) > 0}) > 0, then for some n < ,
0 < n1 ({s : h(s) > n1 }) = 0 (n1 Ih>n1 ) (h) ,
where the right most inequality is a consequence of the definition of (h) and the
fact that h n1 Ih>n1 SF+ . Thus, our assumption that (h) = 0 must imply
that ({s : h(s) > 0}) = 0.
To prove the second part of the lemma, consider e
h = g f which is non-negative
outside a set N F such that (N ) = 0. Hence, h = (g f )IN c mF+ and
0 = (g) (f ) = (e
h) = (h) by Proposition 1.3.5, implying that ({s : h(s) >
0}) = 0 by the preceding proof. The same applies for e
h and the statement of the
lemma follows.

We conclude this subsection by stating the results of Proposition 1.3.5 and Lemma
1.3.8 in terms of the expectation on a probability space (, F , P).
Theorem 1.3.9. The mathematical expectation E[X] is well defined for every R.V.
X on (, F , P) provided either X 0 almost surely, or X L1 (, F , P). Further,
a.s.
(a) EX = EY whenever X = Y .
(b) The expectation is a linear operation, for if Y and Z are integrable R.V. then
for any constants , the R.V. Y + Z is integrable and E(Y + Z) = (EY ) +
(EZ). The same applies when Y, Z 0 almost surely and , 0.
(c) The expectation is monotone. That is, if Y and Z are either integrable or
non-negative and Y Z almost surely, then EY EZ. Further, if Y and Z are
a.s.
integrable with Y Z a.s. and EY = EZ, then Y = Z.
a.s.
(d) Constants are invariant under the expectation. That is, if X = c for nonrandom c (, ], then EX = c.
35
Remark. Part (d) of the theorem relies on the fact that P is a probability measure, namely P() = 1. Indeed, it is obtained by considering the expectation of
the simple function cI to which X equals with probability one.
The linearity of the expectation (i.e. part (b) of the preceding theorem), is often
extremely helpful when looking for an explicit formula for it. We next provide a
few examples of this.
Exercise 1.3.10. Write (, F , P) for a random experiment whose outcome is a
recording of the results of n independent rolls of a balanced six-sided dice (including
their order). Compute the expectation of the random variable D() which counts
the number of different faces of the dice recorded in these n rolls.
Exercise 1.3.11 (Matching). In a random matching experiment, we apply a
random permutation to the integers {1, 2, . . . , n}, where each of the possible n!
permutations is equally likely. Let Zi = I{(i)=i} be the random variable indicating
whether
i = 1, 2, . . . , n is a fixed point of the random permutation, and Xn =
Pn
Z
count the number of fixed points of the random permutation (i.e. the
i
i=1
number of self-matchings). Show that E[Xn (Xn 1) (Xn k + 1)] = 1 for
k = 1, 2, . . . , n.
Similarly, here is an elementary application of the monotonicity of the expectation
(i.e. part (c) of the preceding theorem).
Exercise 1.3.12. Suppose an integrable random variable X is such that E(XIA ) =
0 for each A (X). Show that necessarily X = 0 almost surely.
1.3.2. Inequalities. The linearity of the expectation often allows us to compute EX even when we cannot compute the distribution function FX . In such cases
the expectation can be used to bound tail probabilities, based on the following classical inequality.
Theorem 1.3.13 (Markovs inequality). Suppose : R 7 [0, ] is a Borel
function and let (A) = inf{(y) : y A} for any A B. Then for any R.V. X,
(A)P(X A) E((X)IXA ) E(X).
Proof. By the definition of (A) and non-negativity of we have that
(A)IxA (x)IxA (x) ,
for all x R. Therefore, (A)IXA (X)IXA (X) for every .
We deduce the stated inequality by the monotonicity of the expectation and the
identity E( (A)IXA ) = (A)P(X A) (due to Step 2 of Definition 1.3.1).
We next specify three common instances of Markovs inequality.
Example 1.3.14. (a). Taking (x) = x+ and A = [a, ) for some a > 0 we have
that (A) = a. Markovs inequality is then
EX+
,
a
which is particularly appealing when X 0, so EX+ = EX.
(b). Taking (x) = |x|q and A = (, a] [a, ) for some a > 0, we get that
(A) = aq . Markovs inequality is then aq P(|X| a) E|X|q . Considering q = 2
P(X a)
36
and X = Y EY for Y L2 , this amounts to
Var(Y )
,
a2
which we call Chebyshevs inequality (c.f. Definition 1.3.67 for the variance and
moments of random variable Y ).
(c). Taking (x) = ex for some > 0 and A = [a, ) for some a R we have
that (A) = ea . Markovs inequality is then
P(|Y EY | a)
P(X a) ea EeX .
This bound provides an exponential decay in a, at the cost of requiring X to have
finite exponential moments.
In general, we cannot compute EX explicitly from the Definition 1.3.1 except
for discrete R.V.s and for R.V.s having a probability density function. We thus
appeal to the properties of the expectation listed in Theorem 1.3.9, or use various
inequalities to bound one expectation by another. To this end, we start with
Jensens inequality, dealing with the effect that a convex function makes on the
expectation.
Proposition 1.3.15 (Jensens inequality). Suppose g() is a convex function
on an open interval G of R, that is,
g(x) + (1 )g(y) g(x + (1 )y)
x, y G,
0 1.
If X is an integrable R.V. with P(X G) = 1 and g(X) is also integrable, then

E(g(X)) g(EX).
Proof. The convexity of g() on G implies that g() is continuous on G (hence
g(X) is a random variable) and the existence for each c G of b = b(c) R such
that
(1.3.3)
g(x) g(c) + b(x c),
x G .
Since G is an open interval of R with P(X G) = 1 and X is integrable, it follows

that EX G. Assuming (1.3.3) holds for c = EX, that X G a.s., and that both
X and g(X) are integrable, we have by Theorem 1.3.9 that
E(g(X)) = E(g(X)IXG ) E[(g(c)+b(X c))IXG ] = g(c)+b(EX c) = g(EX) ,
as stated. To derive (1.3.3) note that if (c h2 , c + h1 ) G for positive h1 and h2 ,
then by convexity of g(),
h1
h2
g(c + h1 ) +
g(c h2 ) g(c) ,
h1 + h2
h1 + h2
which amounts to [g(c + h1 ) g(c)]/h1 [g(c) g(c h2 )]/h2 . Considering the
infimum over h1 > 0 and the supremum over h2 > 0 we deduce that
inf
h>0,c+hG
g(c + h) g(c)
g(c) g(c h)
:= (D+ g)(c) (D g)(c) :=
sup
.
h
h
h>0,chG
With G an open set, obviously (D g)(x) > and (D+ g)(x) < for any x G
(in particular, g() is continuous on G). Now for any b [(D g)(c), (D+ g)(c)] R
we get (1.3.3) out of the definition of D+ g and D g.
37
Remark. Since g() is convex if and only if g() is concave, we may as well state
Jensens inequality for concave functions, just reversing the sign of the inequality in
this case. A trivial instance of Jensens inequality happens when X() = xIA () +
yIAc () for some x, y R and A F such that P(A) = . Then,
EX = xP(A) + yP(Ac ) = x + y(1 ) ,
whereas g(X()) = g(x)IA () + g(y)IAc (). So,
Eg(X) = g(x) + g(y)(1 ) g(x + y(1 )) = g(EX) ,
as g is convex.
Applying Jensens inequality, we show that the spaces Lq (, F , P) of Definition

1.3.2 are nested in terms of the parameter q 1.
Lemma 1.3.16. Fixing Y mF , the mapping q

7 ||Y ||q = [E|Y |q ]1/q is nonq
decreasing for q > 0. Hence, the space L (, F , P) is contained in Lr (, F , P) for
any r q.
Proof. Fix q > r > 0 and consider the sequence of bounded R.V. Xn () =
q/r
{min(|Y ()|, n)}r . Obviously, Xn and Xn are both in L1 . Apply Jensens Inequality for the convex function g(x) = |x|q/r and the non-negative R.V. Xn , to
get that
q
q
(EXn ) r E(Xnr ) = E[{min(|Y |, n)}q ] E(|Y |q ) .
q
For n we have that Xn |Y |r , so by monotone convergence E (|Y |r ) r

(E|Y |q ). Taking the 1/q-th power yields the stated result ||Y ||r ||Y ||q .
We next bound the expectation of the product of two R.V. while assuming nothing
about the relation between them.
lders inequality). Let X, Y be two random variables
Proposition 1.3.17 (Ho
on the same probability space. If p, q > 1 with p1 + 1q = 1, then
(1.3.4)
E|XY | ||X||p ||Y ||q .
Remark. Recall that if XY is integrable then E|XY | is by itself an upper bound

on |[EXY ]|. The special case of p = q = 2 in H
olders inequality
2
E|XY | EX EY 2 ,
is called the Cauchy-Schwarz inequality.
Proof. Fixing p > 1 and q = p/(p 1) let = ||X||p and = ||Y ||q . If = 0
a.s.
a.s.
then |X|p = 0 (see Theorem 1.3.9). Likewise, if = 0 then |Y |q = 0. In either
case, the inequality (1.3.4) trivially holds. As this inequality also trivially holds
when either = or = , we may and shall assume hereafter that both and
are finite and strictly positive. Recall that
yq
xp
+
xy 0,
x, y 0
p
q
(c.f. [Dur10, Page 21] where it is proved by considering the first two derivatives
in x). Taking x = |X|/ and y = |Y |/, we have by linearity and monotonicity of
the expectation that
E|X|p E|Y |q
E|XY |
1 1
+ q
,
1= + =
p q
p p
q
38
yielding the stated inequality (1.3.4).
A direct consequence of H
olders inequality is the triangle inequality for the norm
||X||p in Lp (, F , P), that is,
Proposition 1.3.18 (Minkowskis inequality). If X, Y Lp (, F , P), p 1,
then ||X + Y ||p ||X||p + ||Y ||p .
Proof. With |X + Y | |X| + |Y |, by monotonicity of the expectation we have

the stated inequality in case p = 1. Considering hereafter p > 1, it follows from
H
olders inequality (Proposition 1.3.17) that
E|X + Y |p = E(|X + Y ||X + Y |p1 )
E(|X||X + Y |p1 ) + E(|Y ||X + Y |p1 )

1
(E|X|p ) p (E|X + Y |(p1)q ) q + (E|Y |p ) p (E|X + Y |(p1)q ) q

1
= (||X||p + ||Y ||p ) (E|X + Y |p ) q
(recall that (p 1)q = p). Since X, Y Lp and
|x + y|p (|x| + |y|)p 2p1 (|x|p + |y|p ),

p
x, y R, p > 1,
if follows that ap = E|X + Y | < . There is nothing to prove unless ap > 0, in

which case dividing by (ap )1/q we get that
(E|X + Y |p )
1 q1
giving the stated inequality (since 1
1
q
||X||p + ||Y ||p ,
= p1 ).
Remark. Jensens inequality applies only for probability measures, while both
H
olders inequality (|f g|) (|f |p )1/p (|g|q )1/q and Minkowskis inequality apply for any measure , with exactly the same proof we provided for probability
measures.
To practice your understanding of Markovs inequality, solve the following exercise.
Exercise 1.3.19. Let X be a non-negative random variable with Var(X) 1/2.
Show that then P(1 + EX X 2EX) 1/2.
To practice your understanding of the proof of Jensens inequality, try to prove
its extension to convex functions on Rn .
Exercise 1.3.20. Suppose g : Rn R is a convex function and X1 , X2 , . . . , Xn
are integrable random variables, defined on the same probability space and such that
g(X1 , . . . , Xn ) is integrable. Show that then Eg(X1 , . . . , Xn ) g(EX1 , . . . , EXn ).
Hint: Use convex analysis to show that g() is continuous and further that for any
c Rn there exists b Rn such that g(x) g(c) + hb, x ci for all x Rn (with
h, i denoting the inner product of two vectors in Rn ).
Exercise 1.3.21. Let Y 0 with v = E(Y 2 ) < .
(a) Show that for any 0 a < EY ,
(EY a)2
E(Y 2 )
Hint: Apply the Cauchy-Schwarz inequality to Y IY >a .
(b) Show that (E|Y 2 v|)2 4v(v (EY )2 ).
P(Y > a)
(c) Derive the second Bonferroni inequality,

n
n
X
X
[
P(Ai )
P(Ai Aj ) .
P( Ai )
i=1
i=1
1j<in
How does it compare with the bound of part (a) for Y =
39
Pn
i=1 IAi ?
1.3.3. Convergence, limits and expectation. Asymptotic behavior is a

key issue in probability theory. We thus explore here various notions of convergence
of random variables and the relations among them, focusing on the integrability
conditions needed for exchanging the order of limit and expectation operations.
Unless explicitly stated otherwise, throughout this section we assume that all R.V.
are defined on the same probability space (, F , P).
In Definition 1.2.25 we have encountered the convergence almost surely of R.V. A
weaker notion of convergence is convergence in probability as defined next.
Definition 1.3.22. We say that R.V. Xn converge to a given R.V. X in probp
ability, denoted Xn X , if P({ : |Xn () X ()| > }) 0 as n , for
p
any fixed > 0. This is equivalent to |Xn X | 0, and is a special case of the
convergence in -measure of fn mF to f mF , that is ({s : |fn (s) f (s)| >
}) 0 as n , for any fixed > 0.
Our next exercise and example clarify the relationship between convergence almost
surely and convergence in probability.
Exercise 1.3.23. Verify that convergence almost surely to a finite limit implies
p
a.s.
convergence in probability, that is if Xn X R then Xn X .
Remark 1.3.24. Generalizing Definition 1.3.22, for a separable metric space (S, )
we say that (S, BS )-valued random variables Xn converge to X in probability if and
only if for every > 0, P((Xn , X ) > ) 0 as n (see [Dud89, Section
9.2] for more details). Equipping S = R with a suitable metric (for example,
(x, y) = |(x) (y)| with (x) = x/(1 + |x|) : R 7 [1, 1]), this definition
removes the restriction to X finite in Exercise 1.3.23.
p
a.s.
In general, Xn X does not imply that Xn X .
Example 1.3.25. Consider the probability space ((0, 1], B(0,1] , U ) and Xn () =
1[tn ,tn +sn ] () with sn 0 as n slowly enough and tn [0, 1 sn ] are such
that any (0, 1] is in infinitely many intervals [tn , tn + sn ]. The latter property
applies if tn = (i 1)/k and sn = 1/k when n = k(k 1)/2 + i, i = 1, 2, . . . , k and
p
k = 1, 2, . . . (plot the intervals [tn , tn + sn ] to convince yourself ). Then, Xn 0
(since sn = U (Xn 6= 0) 0), whereas fixing each (0, 1], we have that Xn () =
1 for infinitely many values of n, hence Xn does not converge a.s. to zero.
Associated with each space Lq (, F , P) is the notion of Lq convergence which we
now define.
Lq
Definition 1.3.26. We say that Xn converges in Lq to X , denoted Xn X ,

if Xn , X Lq and ||Xn X ||q 0 as n (i.e., E (|Xn X |q ) 0 as
n .
Remark. For q = 2 we have the explicit formula
||Xn X||22 = E(Xn2 ) 2E(Xn X) + E(X 2 ).
40
Thus, it is often easiest to check convergence in L2 .

The following immediate corollary of Lemma 1.3.16 provides an ordering of Lq
convergence in terms of the parameter q.
Lq
Lr
Corollary 1.3.27. If Xn X and q r, then Xn X .
Next note that the Lq convergence implies the convergence of the expectation of
|Xn |q .
Exercise 1.3.28. Fixing q 1, use Minkowskis inequality (Proposition 1.3.18),
Lq
to show that if Xn X , then E|Xn |q E|X |q and for q = 1, 2, 3, . . . also

q
EXnq EX
.
Further, it follows from Markovs inequality that the convergence in Lq implies

convergence in probability (for any value of q).
Lq
Proposition 1.3.29. If Xn X , then Xn X .

Proof. Fixing q > 0 recall that Markovs inequality results with
P(|Y | > ) q E[|Y |q ] ,
for any R.V. Y and any > 0 (c.f part (b) of Example 1.3.14). The assumed
convergence in Lq means that E[|Xn X |q ] 0 as n , so taking Y = Yn =
Xn X , we necessarily have also P(|Xn X | > ) 0 as n . Since > 0
p

is arbitrary, we see that Xn X as claimed.
The converse of Proposition 1.3.29 does not hold in general. As we next demonstrate, even the stronger almost surely convergence (see Exercise 1.3.23), and having
a non-random constant limit are not enough to guarantee the Lq convergence, for
any q > 0.
Example 1.3.30. Fixing q > 0, consider the probability space ((0, 1], B(0,1], U )
and the R.V. Yn () = n1/q I[0,n1 ] (). Since Yn () = 0 for all n n0 and some
a.s.
finite n0 = n0 (), it follows that Yn () 0 as n . However, E[|Yn |q ] =
nU ([0, n1 ]) = 1 for all n, so Yn does not converge to zero in Lq (see Exercise
1.3.28).
Lq
Considering Example 1.3.25, where Xn 0 while Xn does not converge a.s. to

zero, and Example 1.3.30 which exhibits the converse phenomenon, we conclude
that the convergence in Lq and the a.s. convergence are in general non comparable,
and neither one is a consequence of convergence in probability.
Nevertheless, a sequence Xn can have at most one limit, regardless of which convergence mode is considered.
Lq
a.s.
a.s.
Exercise 1.3.31. Check that if Xn X and Xn Y then X = Y .

Though we have just seen that in general the order of the limit and expectation
operations is non-interchangeable, we examine for the remainder of this subsection
various conditions which do allow for such an interchange. Note in passing that
upon proving any such result under certain point-wise convergence conditions, we
may with no extra effort relax these to the corresponding almost sure convergence
(and the same applies for integrals with respect to measures, see part (a) of Theorem
1.3.9, or that of Proposition 1.3.5).
41
Turning to do just that, we first outline the results which apply in the more
general measure theory setting, starting with the proof of the monotone convergence
theorem.
Proof of Theorem 1.3.4. By part (c) of Proposition 1.3.5, the proof of
which did not use Theorem 1.3.4, we know that (hn ) is a non-decreasing sequence
that is bounded above by (h). It therefore suffices to show that
lim (hn ) = sup{0 () : SF+ , hn }
sup{0 () : SF+ , h} = (h)
(1.3.5)
(see Step 3 of Definition 1.3.1). That is, it suffices to find for each non-negative
simple function h a sequence of non-negative simple functions n hn such
that 0 (n ) 0 () as n . To this end, fixing , we may and shall choose
m
P
cl IAl such that Al F are
without loss of generality a representation =
l=1
disjoint and further cl (Al ) > 0 for l = 1, . . . , m (see proof of Lemma 1.3.3). Using
hereafter the notation f (A) = inf{f (s) : s A} for f mF+ and A F , the
condition (s) h(s) for all s S is equivalent to cl h (Al ) for all l, so
0 ()
m
X
l=1
h (Al )(Al ) = V .
Suppose first that V < , that is 0 < h (Al )(Al ) < for all l. In this case, fixing
< 1, consider for each n the disjoint sets Al,,n = {s Al : hn (s) h (Al )} F
and the corresponding
,n (s) =
m
X
l=1
h (Al )IAl,,n (s) SF+ ,
where ,n (s) hn (s) for all s S. If s Al then h(s) > h (Al ). Thus, hn h
implies that Al,,n Al as n , for each l. Consequently, by definition of (hn )
and the continuity from below of ,
lim (hn ) lim 0 (,n ) = V .
Taking 1 we deduce that limn (hn ) V 0 (). Next suppose that V = ,

so without loss of generality we may and shall assume that h (A1 )(A1 ) = .
Fixing x (0, h (A1 )) let A1,x,n = {s A1 : hn (s) x} F noting that A1,x,n
A1 as n and x,n (s) = xIA1,x,n (s) hn (s) for all n and s S, is a nonnegative simple function. Thus, again by continuity from below of we have that
lim (hn ) lim 0 (x,n ) = x(A1 ) .
Taking x h (A1 ) we deduce that limn (hn ) h (A1 )(A1 ) = , completing the
proof of (1.3.5) and that of the theorem.

Considering probability spaces, Theorem 1.3.4 tells us that we can exchange the
order of the limit and the expectation in case of monotone upward a.s. convergence
of non-negative R.V. (with the limit possibly infinite). That is,
Theorem 1.3.32 (Monotone convergence theorem). If Xn 0 and Xn ()
X () for almost every , then EXn EX .
42
In Example 1.3.30 we have a point-wise convergent sequence of R.V. whose expectations exceed that of their limit. In a sense this is always the case, as stated
next in Fatous lemma (which is a direct consequence of the monotone convergence
theorem).
Lemma 1.3.33 (Fatous lemma). For any measure space (S, F , ) and any fn
mF , if fn (s) g(s) for some -integrable function g, all n and -almost-every
s S, then
lim inf (fn ) (lim inf fn ) .
(1.3.6)
Alternatively, if fn (s) g(s) for all n and a.e. s, then
lim sup (fn ) (lim sup fn ) .
(1.3.7)
Proof. Assume first that fn mF+ and let hn (s) = inf kn fk (s), noting
that hn mF+ is a non-decreasing sequence, whose point-wise limit is h(s) :=
lim inf n fn (s). By the monotone convergence theorem, (hn ) (h). Since
fn (s) hn (s) for all s S, the monotonicity of the integral (see Proposition 1.3.5)
implies that (fn ) (hn ) for all n. Considering the lim inf as n we arrive
at (1.3.6).
Turning to extend this inequality to the more general setting of the lemma, note
a.e.
that our conditions imply that fn = g + (fn g)+ for each n. Considering the
countable union of the -negligible sets in which one of these identities is violated,
we thus have that
a.e.
h := lim inf fn = g + lim inf (fn g)+ .

n
Further, (fn ) = (g) + ((fn g)+ ) by the linearity of the integral in mF+ L1 .
Taking n and applying (1.3.6) for (fn g)+ mF+ we deduce that
lim inf (fn ) (g) + (lim inf (fn g)+ ) = (g) + (h g) = (h)
n
(where for the right most identity we used the linearity of the integral, as well as
the fact that g is -integrable).
Finally, we get (1.3.7) for fn by considering (1.3.6) for fn .

Remark. In terms of the expectation, Fatous lemma is the statement that if
R.V. Xn X, almost surely, for some X L1 and all n, then
(1.3.8)
lim inf E(Xn ) E(lim inf Xn ) ,

n
whereas if Xn X, almost surely, for some X L1 and all n, then
(1.3.9)
lim sup E(Xn ) E(lim sup Xn ) .

n
Some text books call (1.3.9) and (1.3.7) the Reverse Fatou Lemma (e.g. [Wil91,
Section 5.4]).
Using Fatous lemma, we can easily prove Lebesgues dominated convergence theorem (in short DOM).
Theorem 1.3.34 (Dominated convergence theorem). For any measure space
(S, F , ) and any fn mF , if for some -integrable function g and -almost-every
s S both fn (s) f (s) as n , and |fn (s)| g(s) for all n, then f is
-integrable and further (|fn f |) 0 as n .
43
Proof. Up to a -negligible subset of S, our assumption that |fn | g and

fn f , implies that |f | g, hence f is -integrable. Applying Fatous lemma
(1.3.7) for |fn f | 2g such that lim supn |fn f | = 0, we conclude that
0 lim sup (|fn f |) (lim sup |fn f |) = (0) = 0 ,
n
as claimed.
By Minkowskis inequality, (|fn f |) 0 implies that (|fn |) (|f |). The

dominated convergence theorem provides us with a simple sufficient condition for
the converse implication in case also fn f a.e.
Lemma 1.3.35 (Scheff
es lemma). If fn mF converges a.e. to f mF and
(|fn |) (|f |) < then (|fn f |) 0 as n .
a.s.
Remark. In terms of expectation, Scheffes lemma states that if Xn X and

L
E|Xn | E|X | < , then Xn X as well.
Proof. As already noted, we may assume without loss of generality that

fn (s) f (s) for all s S, that is gn (s) = fn (s) f (s) 0 as n ,
for all s S. Further, since (|fn |) (|f |) < , we may and shall assume also
that fn are R-valued and -integrable for all n , hence gn L1 (S, F , ) as well.
Suppose first that fn mF+ for all n . In this case, 0 (gn ) f for all
n and s. As (gn ) (s) 0 for every s S, applying the dominated convergence
theorem we deduce that ((gn ) ) 0. From the assumptions of the lemma (and
the linearity of the integral on L1 ), we get that (gn ) = (fn ) (f ) 0 as
n . Since |x| = x + 2x for any x R, it thus follows by linearity of the
integral on L1 that (|gn |) = (gn ) + 2((gn ) ) 0 for n , as claimed.
In the general case of fn mF , we know that both 0 (fn )+ (s) (f )+ (s) and
0 (fn ) (s) (f ) (s) for every s, so by (1.3.6) of Fatous lemma, we have that
(|f |) = ((f )+ ) + ((f ) ) lim inf ((fn ) ) + lim inf ((fn )+ )
n
lim inf [((fn ) ) + ((fn )+ )] = lim (|fn |) = (|f |) .

n
Hence, necessarily both ((fn )+ ) ((f )+ ) and ((fn ) ) ((f ) ). Since

|x y| |x+ y+ | + |x y | for all x, y R and we already proved the lemma
for the non-negative (fn ) and (fn )+ , we see that
lim (|fn f |) lim (|(fn )+ (f )+ |) + lim (|(fn ) (f ) |) = 0 ,
concluding the proof of the lemma.
We conclude this sub-section with quite a few exercises, starting with an alternative characterization of convergence almost surely.
a.s.
Exercise 1.3.36. Show that Xn 0 if and only if for each > 0 there is n
so that for each random integer M with M () n for all we have that
P({ : |XM() ()| > }) < .
Exercise 1.3.37. Let Yn be (real-valued) random variables on (, F , P), and Nk
positive integer valued random variables on the same probability space.
(a) Show that YNk () = YNk () () are random variables on (, F ).
a.s.
a.s.
a.s
(b) Show that if Yn Y and Nk then YNk Y .
44

p
a.s.
(c) Provide an example of Yn 0 and Nk such that almost surely

YNk = 1 for all k.
a.s.
(d) Show that if Yn Y and P(Nk > r) 1 as k , for every fixed
p
r < , then YNk Y .
In the following four exercises you find some of the many applications of the
monotone convergence theorem.
Exercise 1.3.38. You are now to relax the non-negativity assumption in the monotone convergence theorem.
(a) Show that if E[(X1 ) ] < and Xn () X() for almost every , then
EXn EX.
(b) Show that if in addition supn E[(Xn )+ ] < , then X L1 (, F , P).
Exercise 1.3.39. In this exercise you are to show that for any R.V. X 0,
X
jP({ : j < X() (j + 1)}) .
(1.3.10) EX = lim E X for E X =
0
j=0
First use monotone convergence to show that Ek X converges to EX along the

sequence k = 2k . Then, check that E X E X + for any , > 0 and deduce
from it the identity (1.3.10).
Applying (1.3.10)
verify that if X takes at most countably many values {x1 , x2 , . . .},
P
x
P({
: X() = xi }) (this applies to every R.V. X 0 on a
then EX =
i i
countable ). More generally, verify that such formula applies whenever the series
is absolutely convergent (which amounts to X L1 ).
Exercise 1.3.40. Use monotone convergence to show that for any sequence of
non-negative R.V. Yn ,
X
X
EYn .
Yn ) =
E(
n=1
n=1
Exercise 1.3.41. Suppose Xn , X L1 (, F , P) are such that

(a) Xn 0 almost surely, E[Xn ] = 1, E[Xn log Xn ] 1,
and
(b) E[Xn Y ] E[XY ] as n , for each bounded random variable Y on
(, F ).
Show that then X 0 almost surely, E[X] = 1 and E[X log X] 1.
Hint: Jensens inequality is handy for showing that E[X log X] 1.
Next come few direct applications of the dominated convergence theorem.
Exercise 1.3.42.
(a) Show that for any random variable X, the function t 7 E[e|tX| ] is continuous on R (this function is sometimes called the bilateral exponential
transform).
(b) Suppose X 0 is such that EX q < for some q > 0. Show that then
q 1 (EX q 1) E log X as q 0 and deduce that also q 1 log EX q
E log X as q 0.
Hint: Fixing x 0 deduce from convexity of q 7 xq that q 1 (xq 1) log x as
q 0.
Exercise 1.3.43. Suppose X is an integrable random variable.
45
(a) Show that E(|X|I{X>n} ) 0 as n .

(b) Deduce that for any > 0 there exists > 0 such that
sup{E[|X|IA ] : P(A) } .
(c) Provide an example of X 0 with EX = for which the preceding fails,
that is, P(Ak ) 0 as k while E[XIAk ] is bounded away from zero.
The following generalization of the dominated convergence theorem is also left as
an exercise.
Exercise 1.3.44. Suppose gn () fn () hn () are -integrable functions in the
same measure space (S, F , ) such that for -almost-every s S both gn (s)
g (s), fn (s) f (s) and hn (s) h (s) as n . Show that if further g and
h are -integrable functions such that (gn ) (g ) and (hn ) (h ), then
f is -integrable and (fn ) (f ).
Finally, here is a demonstration of one of the many issues that are particularly
easy to resolve with respect to the L2 (, F , P) norm.
Exercise 1.3.45. Let X = (X(t))tR be a mapping from R into L2 (, F , P).
Show that t 7 X(t) is a continuous mapping (with respect to the norm k k2 on
L2 (, F , P)), if and only if both
(t) = E[X(t)]
and
r(s, t) = E[X(s)X(t)] (s)(t)
are continuous real-valued functions (r(s, t) is continuous as a map from R2 to R).

1.3.4. L1 -convergence and uniform integrability. For probability theory,
a.s.
the dominated convergence theorem states that if random variables Xn X are
such that |Xn | Y for all n and some random variable Y such that EY < , then
L1
X L1 and Xn X . Since constants have finite expectation (see part (d) of

Theorem 1.3.9), we have as its corollary the bounded convergence theorem, that is,
Corollary 1.3.46 (Bounded Convergence). Suppose that a.s. |Xn ()| K for
a.s.
some finite non-random constant K and all n. If Xn X , then X L1 and
L1
Xn X .
We next state a uniform integrability condition that together with convergence in

probability implies the convergence in L1 .
Definition 1.3.47. A possibly uncountable collection of R.V.-s {X , I} is
called uniformly integrable (U.I.) if
(1.3.11)
lim sup E[|X |I|X |>M ] = 0 .
Our next lemma shows that U.I. is a relaxation of the condition of dominated
convergence, and that U.I. still implies the boundedness in L1 of {X , I}.
Lemma 1.3.48. If |X | Y for all and some R.V. Y such that EY < , then
the collection {X } is U.I. In particular, any finite collection of integrable R.V. is
U.I.
Further, if {X } is U.I. then sup E|X | < .
46
Proof. By monotone convergence, E[Y IY M ] EY as M , for any R.V.

Y 0. Hence, if in addition EY < , then by linearity of the expectation,
E[Y IY >M ] 0 as M . Now, if |X | Y then |X |I|X |>M Y IY >M ,
hence E[|X |I|X |>M ] E[Y IY >M ], which does not depend on , and for Y L1
converges to zero when M . We thus proved that if |X | Y for all and
some Y such that EY < , then {X } is a U.I. collection of R.V.-s
For a finite collection of R.V.-s Xi L1 , i = 1, . . . , k, take Y = |X1 | + |X2 | + +
|Xk | L1 such that |Xi | Y for i = 1, . . . , k, to see that any finite collection of
integrable R.V.-s is U.I.
Finally, since
E|X | = E[|X |I|X |M ] + E[|X |I|X |>M ] M + sup E[|X |I|X |>M ] ,
we see that if {X , I} is U.I. then sup E|X | < .
We next state and prove Vitalis convergence theorem for probability measures,
deferring the general case to Exercise 1.3.53.
p
Theorem 1.3.49 (Vitalis convergence theorem). Suppose Xn X . Then,

L
the collection {Xn } is U.I. if and only if Xn X which in turn is equivalent to

Xn being integrable for all n and E|Xn | E|X |.
Remark. In view of Lemma 1.3.48, Vitalis theorem relaxes the assumed a.s.
convergence Xn X of the dominated (or bounded) convergence theorem, and
of Scheffes lemma, to that of convergence in probability.
Proof. Suppose first that |Xn | M for some non-random finite constant M
and all n. For each > 0 let Bn, = { : |Xn () X ()| > }. The assumed
convergence in probability means that P(Bn, ) 0 as n (see Definition
1.3.22). Since P(|X | M + ) P(Bn, ), taking n and considering
= k 0, we get by continuity from below of P that almost surely |X | M .
So, |Xn X | 2M and by linearity and monotonicity of the expectation, for any
n and > 0,
c
E|Xn X | = E[|Xn X |IBn,
] + E[|Xn X |IBn, ]
c ] + E[2M IB
E[IBn,
n, ] + 2M P(Bn, ) .
Since P(Bn, ) 0 as n , it follows that lim supn E|Xn X | . Taking

0 we deduce that E|Xn X | 0 in this case.
Moving to deal now with the general case of a collection {Xn } that is U.I., let
M (x) = max(min(x, M ), M ). As |M (x)M (y)| |xy| for any x, y R, our
p
p
assumption Xn X implies that M (Xn ) M (X ) for any fixed M < .
With |M ()| M , we then have by the preceding proof of bounded convergence
L1
that M (Xn ) M (X ). Further, by Minkowskis inequality, also E|M (Xn )|

E|M (X )|. By Lemma 1.3.48, our assumption that {Xn } are U.I. implies their
L1 boundedness, and since |M (x)| |x| for all x, we deduce that for any M ,
(1.3.12)
> c := sup E|Xn | lim E|M (Xn )| = E|M (X )| .

n
With |M (X )| |X | as M , it follows from monotone convergence that

E|M (X )| E|X |, hence E|X | c < in view of (1.3.12). Fixing >
0, choose M = M () < large enough for E[|X |I|X |>M ] < , and further
47
increasing M if needed, by the U.I. condition also E[|Xn |I|Xn |>M ] < for all n.
Considering the expectation of the inequality |x M (x)| |x|I|x|>M (which holds
for all x R), with x = Xn and x = X , we obtain that
E|Xn X | E|Xn M (Xn )| + E|M (Xn ) M (X )| + E|X M (X )|
2 + E|M (Xn ) M (X )| .
L1
Recall that M (Xn ) M (X ), hence lim supn E|Xn X | 2. Taking 0

completes the proof of L1 convergence of Xn to X .
L1
Suppose now that Xn X . Then, by Jensens inequality (for the convex

function g(x) = |x|),
|E|Xn | E|X || E[| |Xn | |X | |] E|Xn X | 0.
That is, E|Xn | E|X | and Xn , n are integrable.
p
It thus remains only to show that if Xn X , all of which are integrable and
E|Xn | E|X | then the collection {Xn } is U.I. To the end, for any M > 1, let
M (x) = |x|I|x|M1 + (M 1)(M |x|)I(M1,M] (|x|) ,
a piecewise-linear, continuous, bounded function, such that M (x) = |x| for |x|
M 1 and M (x) = 0 for |x| M . Fixing > 0, with X integrable, by dominated
convergence E|X |Em (X ) for some finite m = m(). Further, as |m (x)
p
m (y)| (m 1)|x y| for any x, y R, our assumption Xn X implies that
p
m (Xn ) m (X ). Hence, by the preceding proof of bounded convergence,
followed by Minkowskis inequality, we deduce that Em (Xn ) Em (X ) as
n . Since |x|I|x|>m |x| m (x) for all x R, our assumption E|Xn |
E|X | thus implies that for some n0 = n0 () finite and all n n0 and M m(),
E[|Xn |I|Xn |>M ] E[|Xn |I|Xn |>m ] E|Xn | Em (Xn )
E|X | Em (X ) + 2 .
As each Xn is integrable, E[|Xn |I|Xn |>M ] 2 for some M m finite and all n
(including also n < n0 ()). The fact that such finite M = M () exists for any > 0
amounts to the collection {Xn } being U.I.

The following exercise builds upon the bounded convergence theorem.
Exercise 1.3.50. Show that for any X 0 (do not assume E(1/X) < ), both
(a) lim yE[X 1 IX>y ] = 0 and
y
(b) lim yE[X 1 IX>y ] = 0.

y0
Next is an example of the advantage of Vitalis convergence theorem over the

dominated convergence theorem.
Exercise 1.3.51. On ((0, 1], B(0,1] , U ), let Xn () = (n/ log n)I(0,n1 ) () for n
a.s.
2. Show that the collection {Xn } is U.I. such that Xn 0 and EXn 0, but
there is no random variable Y with finite expectation such that Y Xn for all
n 2 and almost all (0, 1].
By a simple application of Vitalis convergence theorem you can derive a classical
result of analysis, dealing with the convergence of Cesaro averages.
48
Exercise 1.3.52. Let Un denote a random variable whose law is the uniform
probability measure on (0, n], namely, Lebesgue measure restricted to the interval
p
(0, n] and normalized by n1 to a probability measure. Show that g(Un ) 0 as
n , for any Borel function g() such that |g(y)| 0 as y
R n . Further,
assuming that also supy |g(y)| < , deduce that E|g(Un )| = n1 0 |g(y)|dy 0
as n .
Here is Vitalis convergence theorem for a general measure space.
Exercise 1.3.53. Given a measure space (S, F , ), suppose fn , f mF with
(|fn |) finite and (|fn f | > ) 0 as n , for each fixed > 0. Show
that (|fn f |) 0 as n if and only if both supn (|fn |I|fn |>k ) 0 and
supn (|fn |IAk ) 0 for k and some {Ak } F such that (Ack ) < .
We conclude this subsection with a useful sufficient criterion for uniform integrability and few of its consequences.
Exercise 1.3.54. Let f 0 be a Borel function such that f (r)/r as r .
Suppose Ef (|X |) C for some finite non-random constant C and all I. Show
that then {X : I} is a uniformly integrable collection of R.V.
Exercise 1.3.55.
(a) Construct random variables Xn such that supn E(|Xn |) < , but the
collection {Xn } is not uniformly integrable.
(b) Show that if {Xn } is a U.I. collection and {Yn } is a U.I. collection, then
{Xn + Yn } is also U.I.
p
(c) Show that if Xn X and the collection {Xn } is uniformly integrable,
then E(Xn IA ) E(X IA ) as n , for any measurable set A.
1.3.5. Expectation, density and Riemann integral. Applying the standard machine we now show that fixing a measure space (S, F , ), each non-negative
measurable function f induces a measure f on (S, F ), with f being the natural
generalization of the concept of probability density function.
Proposition 1.3.56. Fix a measure space (S, F , ). Every f mF+ induces
a measure f on (S, F ) via (f )(A) = (f IA ) for all A F . These measures
satisfy the composition relation h(f ) = (hf ) for all f, h mF+ . Further, h
L1 (S, F , f ) if and only if f h L1 (S, F , ) and then (f )(h) = (f h).
Proof. Fixing f mF+ , obviously f is a non-negative set function on (S, F )
with (f )() = (f I ) = (0) = 0. To check that f is countably additive, hence
aPmeasure, let A = k Ak for a countable collection of disjoint sets Ak F . Since
n
k=1 f IAk f IA , it follows by monotone convergence and linearity of the integral
that,
n
n
X
X
X
(f IAk ) =
(f IA ) = lim (
f IAk ) = lim
(f IAk )
n
k=1
k=1
Thus, (f )(A) = k (f )(Ak ) verifying that f is a measure.

Fixing f mF+ , we turn to prove that the identity
(1.3.13)
(f )(hIA ) = (f hIA )
A F ,
holds for any h mF+ . Since the left side of (1.3.13) is the value assigned to A
by the measure h(f ) and the right side of this identity is the value assigned to
49
the same set by the measure (hf ), this would verify the stated composition rule
h(f ) = (hf ). The proof of (1.3.13) proceeds by applying the standard machine:
Step 1. If h = IB for B F we have by the definition of the integral of an indicator
function that
(f )(IB IA ) = (f )(IAB ) = (f )(A B) = (f IAB ) = (f IB IA ) ,
which is (1.3.13).
Pn
Step 2. Take h SF+ represented as h = l=1 cl IBl with cl 0 and Bl F .
Then, by Step 1 and the linearity of the integrals with respect to f and with
respect to , we see that
n
n
n
X
X
X
cl IBl IA ) = (f hIA ) ,
cl (f IBl IA ) = (f
cl (f )(IBl IA ) =
(f )(hIA ) =
l=1
l=1
l=1
again yielding (1.3.13).

Step 3. For any h mF+ there exist hn SF+ such that hn h. By Step 2 we
know that (f )(hn IA ) = (f hn IA ) for any A F and all n. Further, hn IA hIA
and f hn IA f hIA , so by monotone convergence (for both integrals with respect to
f and ),
(f )(hIA ) = lim (f )(hn IA ) = lim (f hn IA ) = (f hIA ) ,
n
completing the proof of (1.3.13) for all h mF+ .

Writing h mF as h = h+ h with h+ = max(h, 0) mF+ and h =
min(h, 0) mF+ , it follows from the composition rule that
Z
Z
h d(f ) = (f )(h IS ) = h (f )(S) = ((h f ))(S) = (f h IS ) = f h d .
Observing that f h = (f h) when f mF+ , we thusR deduce thatR h is f integrable if and only if f h is -integrable in which case hd(f ) = f hd, as
stated.
Fixing a measure space (S, F , ), every set D F induces a -algebra FD =

{A F : A D}. Let D denote the restriction of to (D, FD ). As a corollary
of Proposition 1.3.56 we express the integral with respect to D in terms of the
original measure .
Corollary 1.3.57. Fixing D F let hD denote the restriction of h mF to
(D, FD ). Then, D (hD ) = (hID ) for any h mF+ . Further, hD L1 (D, FD , D )
if and only if hID L1 (S, F , ), in which case also D (hD ) = (hID ).
Proof. Note that the measure ID of Proposition 1.3.56 coincides with D
on the -algebra FD and assigns to any set A F the same value it assigns to
A D FD . By Definition 1.3.1 this implies that (ID )(h) = D (hD ) for any
h mF+ . The corollary is thus a re-statement of the composition and integrability
relations of Proposition 1.3.56 for f = ID .

R
Remark 1.3.58. Corollary 1.3.57 justifies Rusing hereafter the notation A f d or
(f ; A) for (f IA ), or writing E(X; A) = A X()dP () for E(XIA ). With this
notation in place, Proposition 1.3.56 states that each Z R 0 such that EZ = 1
induces a probability Rmeasure Q = ZP such that Q(A) = A ZdP for all A F ,
and then EQ (W ) := W dQ = E(ZW ) whenever W 0 or ZW L1 (, F , P)
(the assumption EZ = 1 translates to Q() = 1).
50
Proposition 1.3.56 is closely related to the probability density function of Definition

1.2.40. En-route to showing this, we first define the collection of Lebesgue integrable
functions.
Definition 1.3.59. Consider Lebesgues measure on (R, B) as in Section 1.1.3,
and its completion on (R, B) (see Theorem 1.1.35). A set B B is called
Lebesgue measurable and f : R 7 R is called Lebesgue integrable function if
f mB, and (|f |) < . As we show in Proposition 1.3.64, any non-negative
Riemann integrable function is also
Lebesgue integrable, and the integral values
R
coincide, justifying the notation B f (x)dx for (f ; B), where the function f and
the set B are both Lebesgue measurable.
Example
1.3.60. Suppose f is a non-negative Lebesgue integrable function such
R
that R f (x)dx = 1. Then, P = f of Proposition 1.3.56 is a probability measure
R
on (R, B) such that P(B) = (f ; B) = B f (x)dx for any Lebesgue measurable set
B. By Theorem 1.2.37 it is easy
R to verify that F () = P((, ]) is a distribution
function, such that F () = f (x)dx. That is, P is the law of a R.V. X : R 7
R whose probability density function is f (c.f. Definition 1.2.40 and Proposition
1.2.45).
Our next theorem allows us to compute expectations of functions of a R.V. X
in the space (R, B, PX ), using the law of X (c.f. Definition 1.2.34) and calculus,
instead of working on the original general probability space. One of its immediate
D
consequences is the obvious fact that if X = Y then Eh(X) = Eh(Y ) for any
non-negative Borel function h.
Theorem 1.3.61 (Change of variables formula). Let X : 7 R be a random variable on (, F , P) and h a Borel measurable function such that Eh+ (X) <
or Eh (X) < . Then,
Z
Z
h(x)dPX (x).
h(X())dP() =
(1.3.14)
Proof. Apply the standard machine with respect to h mB:

Step 1. Taking h = IB for B B, note that by the definition of expectation of
indicators
Z
Eh(X) = E[IB (X())] = P({ : X() B}) = PX (B) = h(x)dPX (x).
P
Step 2. Representing h SF+ as h = m
l=1 cl IBl for cl 0, the identity (1.3.14)
follows from Step 1 by the linearity of the expectation in both spaces.
Step 3. For h mB+ , consider hn SF+ such that hn h. Since hn (X())
h(X()) for all , we get by monotone convergence on (, F , P), followed by applying Step 2 for hn , and finally monotone convergence on (R, B, PX ), that
Z
Z
hn (X())dP()
h(X())dP() = lim
n
Z
Z
h(x)dPX (x),
hn (x)dPX (x) =
= lim
n
as claimed.
Step 4. Write a Borel function h(x) as h+ (x) h (x). Then, by Step 3, (1.3.14)
applies for both non-negative functions h+ and h . Further, at least one of these
51
two identities involves finite quantities. So, taking their difference and using the
linearity of the expectation (in both probability spaces), lead to the same result for
h.

Combining Theorem 1.3.61 with Example 1.3.60, we show that the expectation of
a Borel function of a R.V. X having a density fX can be computed by performing
calculus type integration on the real line.
Corollary 1.3.62. Suppose that the distribution function of a R.V. X is of
the form (1.2.3) for some Lebesgue integrable function fX (x). Then, for any
Borel
measurable function h : R 7 R, the R.V.
R
R h(X) is integrable if and only if
|h(x)|fX (x)dx < , in which case Eh(X) = h(x)fX (x)dx. The latter formula
applies also for any non-negative Borel function h().
Proof. Recall Example 1.3.60 that the law PX of X equals to the probability
measure fX . For h 0 we thus deduce from Theorem 1.3.61 that Eh(X) =
fRX (h), which by the composition rule of Proposition 1.3.56 is given by (fX h) =
h(x)fX (x)dx. The decomposition h = h+ h then completes the proof of the
general case.

Our next task is to compare Lebesgues integral (of Definition 1.3.1) with Riemanns integral. To this end recall,
Definition 1.3.63. A function f : (a, b] 7 [0, ] is Riemann integrable
with inteP
gral R(f ) < if for any > 0 there exists = () > 0 such that | l f (xl )(Jl )
R(f )| < , for any xl Jl and {Jl } a finite partition of (a, b] into disjoint subintervals whose length (Jl ) < .
Lebesgues integral of a function f is based on splitting its range to small intervals
and approximating f (s) by a constant on the subset of S for which f () falls into
each such interval. As such, it accommodates an arbitrary domain S of the function,
in contrast to Riemanns integral where the domain of integration is split into small
rectangles hence limited to Rd . As we next show, even for S = (a, b], if f 0
(or more generally, f bounded), is Riemann integrable, then it is also Lebesgue
integrable, with the integrals coinciding in value.
Proposition 1.3.64. If f (x) is a non-negative Riemann integrable function on
an interval (a, b], then it is also Lebesgue integrable on (a, b] and (f ) = R(f ).
Proof. Let f (J) = inf{f (x) : x J} and f (J) = sup{f (x) : x J}.
Varying xl over Jl we see that
X
X
(1.3.15)
R(f )
f (Jl )(Jl )
f (Jl )(Jl ) R(f ) + ,
l
for any finite partition of (a, b] into disjoint subintervals Jl such that P
supl (Jl )
. For any such
partition,
the
non-negative
simple
functions
()
=
l f (Jl )IJl
P
f
(J
)I
are
such
that
()
u(),
whereas
R(f
)
and u() =
l
J
l
l
(()) (u()) R(f ) + , by (1.3.15). Consider the dyadic partitions n
of (a, b] to 2n intervals of length (b a)2n each, such that n+1 is a refinement
of n for each n = 1, 2, . . .. Note that u(n )(x) u(n+1 )(x) for all x (a, b]
and any n, hence u(n ))(x) u (x) a Borel measurable R-valued function (see
52
Exercise 1.2.31). Similarly, (n )(x) (x) for all x (a, b], with also Borel
measurable, and by the monotonicity of Lebesgues integral,
R(f ) lim ((n )) ( ) (u ) lim (u(n )) R(f ) .
n
We deduce that (u ) = ( ) = R(f ) for u f . The set {x (a, b] :

f (x) 6= (x)} is a subset of the Borel set {x (a, b] : u (x) > (x)} whose
Lebesgue measure is zero (see Lemma 1.3.8). Consequently, f is Lebesgue measur
able on (a, b] with (f ) = ( ) = R(f ) as stated.
Here is an alternative, direct proof of the fact that Q in Remark 1.3.58 is a
probability measure.
S
Exercise 1.3.65. Suppose E|X| < and A = n An for some disjoint sets
An F .
(a) Show that then
E(X; An ) = E(X; A) ,
n=0
that is, the sum converges absolutely and has the value on the right.
(b) Deduce from this that for Z 0 with EZ positive and finite, Q(A) :=
EZIA /EZ is a probability measure.
(c) Suppose that X and Y are non-negative random variables on the same
probability space (, F , P) such that EX = EY < . Deduce from the
preceding that if EXIA = EY IA for any A in a -system A such that
a.s.
F = (A), then X = Y .
Exercise 1.3.66. Suppose P is aR probability measure on (R, B) and f 0 is a
Borel function such that P(B) = B f (x)dx for B = (, b], b R. Using the
theorem show that this identity applies for all B B. Building on this result,
use the standard machine to directly prove Corollary 1.3.62 (without Proposition
1.3.56).
1.3.6. Mean, variance and moments. We start with the definition of moments of a random variable.
Definition 1.3.67. If k is a positive integer then EX k is called the kth moment
of X. When it is well defined, the first moment mX = EX is called the mean. If
EX 2 < , then the variance of X is defined to be
(1.3.16)
Var(X) = E(X mX )2 = EX 2 m2X EX 2 .
Since E(aX + b) = aEX + b (linearity of the expectation), it follows from the

definition that
(1.3.17) Var(aX + b) = E(aX + b E(aX + b))2 = a2 E(X mX )2 = a2 Var(X)
We turn to some examples, starting with R.V. having a density.
Example 1.3.68. If X has the exponential distribution then
Z
EX k =
xk ex dx = k!
0
for any k (see Example 1.2.41 for its density). The mean of X is mX = 1 and
its variance is EX 2 (EX)2 = 1. For any > 0, it is easy to see that T = X/
53
has density fT (t) = et 1t>0 , called the exponential density of parameter . By

(1.3.17) it follows that mT = 1/ and Var(T ) = 1/2 .
Similarly, if X has a standard normal distribution, then by symmetry, for k odd,
Z
2
1
k
xk ex /2 dx = 0 ,
EX =
2
whereas by integration by parts, the even moments satisfy the relation
Z
2
1
2
(1.3.18)
EX =
x21 xex /2 dx = (2 1)EX 22 ,
2
for = 1, 2, . . .. In particular,
Var(X) = EX 2 = 1 .
Consider G = X + , where > 0 and R, whose density is
(y)2
1
e 22 .
fG (y) =
2 2
We call the law of G the normal distribution of mean and variance 2 (as EG =
and Var(G) = 2 ).
Next are some examples of R.V. with finite or countable set of possible values.
Example 1.3.69. We say that B has a Bernoulli distribution of parameter p
[0, 1] if P(B = 1) = 1 P(B = 0) = p. Clearly,
EB = p 1 + (1 p) 0 = p .
Further, B 2 = B so EB 2 = EB = p and
Var(B) = EB 2 (EB)2 = p p2 = p(1 p) .
Recall that N has a Poisson distribution with parameter 0 if
P(N = k) =
k
e
k!
for k = 0, 1, 2, . . .
(where in case = 0, P(N = 0) = 1). Observe that for k = 1, 2, . . .,

E(N (N 1) (N k + 1)) =
n=k
n(n 1) (n k + 1)
n
e
n!
X
nk
e = k .
=
(n k)!
k
n=k
Using this formula, it follows that EN = while
Var(N ) = EN 2 (EN )2 = .
The random variable Z is said to have a Geometric distribution of success probability p (0, 1) if
P(Z = k) = p(1 p)k1
for
k = 1, 2, . . .
This is the distribution of the number of independent coin tosses needed till the first
appearance of a Head, or more generally, the number of independent trials till the
54
first occurrence in this sequence of a specific event whose probability is p. Then,

EZ =
EZ(Z 1) =
k=1
X
k=2
kp(1 p)k1 =
1
p
k(k 1)p(1 p)k1 =
2(1 p)
p2
1p
.
p2
Pn
Exercise 1.3.70. Consider a counting random variable Nn = i=1 IAi .
(a) Provide a formula for Var(Nn ) in terms of P(Ai ) and P(Ai Aj ) for
i 6= j.
(b) Using your formula, find the variance of the number Nn of empty boxes
when distributing at random r distinct balls among n distinct boxes, where
each of the possible nr assignments of balls to boxes is equally likely.
Var(Z) = EZ(Z 1) + EZ (EZ)2 =
Exercise 1.3.71. Show that if P(X [a, b]) = 1, then Var(X) (b a)2 /4.
1.4. Independence and product measures
In Subsection 1.4.1 we build-up the notion of independence, from events to random
variables via -algebras, relating it to the structure of the joint distribution function. Subsection 1.4.2 considers finite product measures associated with the joint
law of independent R.V.-s. This is followed by Kolmogorovs extension theorem
which we use in order to construct infinitely many independent R.V.-s. Subsection
1.4.3 is about Fubinis theorem and its applications for computing the expectation
of functions of independent R.V.
1.4.1. Definition and conditions for independence. Recall the classical
definition that two events A, B F are independent if P(A B) = P(A)P(B).
For example, suppose two fair dice are thrown (i.e. = {1, 2, 3, 4, 5, 6}2 with
F = 2 and the uniform probability measure). Let E1 = {Sum of two is 6} and
E2 = {first die is 4} then E1 and E2 are not independent since
P(E1 ) = P({(1, 5) (2, 4) (3, 3) (4, 2) (5, 1)}) =
5
,
36
P(E2 ) = P({ : 1 = 4}) =
1
6
and
1
6= P(E1 )P(E2 ).
36
However one can check that E2 and E3 = {sum of dice is 7} are independent.
P(E1 E2 ) = P({(4, 2)}) =
In analogy with the independence of events we define the independence of two

random vectors and more generally, that of two -algebras.
Definition 1.4.1. Two -algebras H, G F are independent (also denoted Pindependent), if
P(G H) = P(G)P(H),
G G, H H ,
that is, two -algebras are independent if every event in one of them is independent
of every event in the other.
1.4. INDEPENDENCE AND PRODUCT MEASURES
55
The random vectors X = (X1 , . . . , Xn ) and Y = (Y1 , . . . , Ym ) on the same probability space are independent if the corresponding -algebras (X1 , . . . , Xn ) and
(Y1 , . . . , Ym ) are independent.
Remark. Our definition of independence of random variables is consistent with
that of independence of events. For example, if the events A, B F are independent, then so are IA and IB . Indeed, we need to show that (IA ) = {, , A, Ac }
and (IB ) = {, , B, B c} are independent. Since P() = 0 and is invariant under
intersections, whereas P() = 1 and all events are invariant under intersection with
, it suffices to consider G {A, Ac } and H {B, B c }. We check independence
first for G = A and H = B c . Noting that A is the union of the disjoint events
A B and A B c we have that
P(A B c ) = P(A) P(A B) = P(A)[1 P(B)] = P(A)P(B c ) ,
where the middle equality is due to the assumed independence of A and B. The
proof for all other choices of G and H is very similar.
More generally we define the mutual independence of events as follows.
Definition 1.4.2. Events Ai F are P-mutually independent if for any L <
and distinct indices i1 , i2 , . . . , iL ,
P(Ai1 Ai2 AiL ) =
L
Y
P(Aik ).
k=1
We next generalize the definition of mutual independence to -algebras, random

variables and beyond. This definition applies to the mutual independence of both
finite and infinite number of such objects.
Definition 1.4.3. We say that the collections of events A F with I
(possibly an infinite index set) are P-mutually independent if for any L < and
distinct 1 , 2 , . . . , L I,
P(A1 A2 AL ) =
L
Y
k=1
P(Ak ),
Ak Ak , k = 1, . . . , L .
We say that random variables X , I are P-mutually independent if the algebras (X ), I are P-mutually independent.
When the probability measure P in consideration is clear from the context, we say
that random variables, or collections of events, are mutually independent.
Our next theorem gives a sufficient condition for the mutual independence of
a collection of -algebras which as we later show, greatly simplifies the task of
checking independence.
Theorem 1.4.4. Suppose Gi = (Ai ) F for i = 1, 2, , n where Ai are systems. Then, a sufficient condition for the mutual independence of Gi is that Ai ,
i = 1, . . . , n are mutually independent.
Proof. Let H = Ai1 Ai2 AiL , where i1 , i2 , . . . , iL are distinct elements
from {1, 2, . . . , n 1} and Ai Ai for i = 1, . . . , n 1. Consider the two finite
measures 1 (A) = P(A H) and 2 (A) = P(H)P(A) on the measurable space
(, Gn ). Note that
1 () = P( H) = P(H) = P(H)P() = 2 () .
56
If A An , then by the mutual independence of Ai , i = 1, . . . , n, it follows that

1 (A) = P(Ai1 Ai2 Ai3 AiL A) = (
L
Y
P(Aik ))P(A)
k=1
= P(Ai1 Ai2 AiL )P(A) = 2 (A) .
Since the finite measures 1 () and 2 () agree on the -system An and on , it

follows that 1 = 2 on Gn = (An ) (see Proposition 1.1.39). That is, P(G H) =
P(G)P(H) for any G Gn .
Since this applies for arbitrary Ai Ai , i = 1, . . . , n 1, in view of Definition
1.4.3 we have just proved that if A1 , A2 , . . . , An are mutually independent, then
A1 , A2 , . . . , Gn are mutually independent.
Applying the latter relation for Gn , A1 , . . . , An1 (which are mutually independent
since Definition 1.4.3 is invariant to a permutation of the order of the collections) we
get that Gn , A1 , . . . , An2 , Gn1 are mutually independent. After n such iterations
we have the stated result.

Because the mutual independence of the collections of events A , I amounts
to the mutual independence of any finite number of these collections, we have the
immediate consequence:
Corollary 1.4.5. If -systems of events A , I, are mutually independent,
then (A ), I, are also mutually independent.
Another immediate consequence deals with the closure of mutual independence
under projections.
Corollary 1.4.6. If the -systems of events H, , (, ) J are mutually
independent, then the -algebras G = ( H, ), are also mutually independent.
Proof. Let A be the collection of sets of the form A = m

j=1 Hj where Hj
H,j for some m < and distinct 1 , . . . , m . Since H, are -systems, it follows
that so is A for each . Since a finite intersection of sets Ak Ak , k = 1, . . . , L is
merely a finite intersection of sets from distinct collections Hk ,j (k) , the assumed
mutual independence of H, implies the mutual independence of A . By Corollary
1.4.5, this in turn implies the mutual independence of (A ). To complete the proof,
simply note that for any , each H H, is also an element of A , implying that
G (A ).

Relying on the preceding corollary you can now establish the following characterization of independence (which is key to proving Kolmogorovs 0-1 law).
Exercise 1.4.7. Show that if for each n 1 the -algebras FnX = (X1 , . . . , Xn )
and (Xn+1 ) are P-mutually independent then the random variables X1 , X2 , X3 , . . .
are P-mutually independent. Conversely, show that if X1 , X2 , X3 , . . . are independent, then for each n 1 the -algebras FnX and TnX = (Xr , r > n) are independent.
It is easy to check that a P-trivial -algebra H is P-independent of any other
-algebra G F . Conversely, as we show next, independence is a great tool for
proving that a -algebra is P-trivial.
Lemma
S 1.4.8. If each of the -algebras Gk Gk+1 is P-independent of a -algebra
H ( k1 Gk ) then H is P-trivial.
57
Remark. In particular, if H is P-independent of itself, then H is P-trivial.
S Proof. Since Gk Gk+1 for all k and Gk are -algebras, it follows that A =
k1 Gk is a -system. The assumed P-independence of H and Gk for each k
yields the P-independence of H and A. Thus, by Theorem 1.4.4 we have that
H and (A) are P-independent. Since H (A) it follows that in particular
P(H) = P(H H) = P(H) P(H) for each H H. So, necessarily P(H) {0, 1}
for all H H. That is, H is P-trivial.

We next define the tail -algebra of a stochastic process.
Definition 1.4.9. For a stochastic process {Xk } we set TnX = (Xr , r > n) and
call T X = n TnX the tail -algebra of the process {Xk }.
As we next see, the P-triviality of the tail -algebra of independent random variables is an immediate consequence of Lemma 1.4.8. This result, due to Kolmogorov,
is just one of the many 0-1 laws that exist in probability theory.
Corollary 1.4.10 (Kolmogorovs 0-1 law). If {Xk } are P-mutually independent then the corresponding tail -algebra T X is P-trivial.
S
X
Proof. Note that FkX Fk+1
and T X F X = (Xk , k 1) = ( k1 FkX )
(see Exercise 1.2.14 for the latter identity). Further, recall Exercise 1.4.7 that for
any n 1, the -algebras TnX and FnX are P-mutually independent. Hence, each of
the -algebras FkX is also P-mutually independent of the tail -algebra T X , which
by Lemma 1.4.8 is thus P-trivial.

Out of Corollary 1.4.6 we deduce that functions of disjoint collections of mutually
independent random variables are mutually independent.
Corollary 1.4.11. If R.V. Xk,j , 1 k m, 1 j l(k) are mutually independent and fk : Rl(k) 7 R are Borel functions, then Yk = fk (Xk,1 , . . . , Xk,l(k) ) are
mutually independent random variables for k = 1, . . . , m.
Proof. We apply Corollary 1.4.6 for the index set J = {(k, j) : 1 k
m, 1 j l(k)}, and mutually independent -systems Hk,j = (Xk,j ), to deduce
the mutual independence of Gk = (j Hk,j ). Recall that Gk = (Xk,j , 1 j l(k))
and (Yk ) Gk (see Definition 1.2.12 and Exercise 1.2.33). We complete the proof
by noting that Yk are mutually independent if and only if (Yk ) are mutually
independent.

Our next result is an application of Theorem 1.4.4 to the independence of random
variables.
Corollary 1.4.12. Real-valued random variables X1 , X2 , . . . , Xm on the same
probability space (, F , P) are mutually independent if and only if
m
Y
P(Xi xi ) , x1 , . . . , xm R.
(1.4.1)
P(X1 x1 , . . . , Xm xm ) =
i=1
Proof. Let Ai denote the collection of subsets of of the form Xi1 ((, b])
for b R. Recall that Ai generates (Xi ) (see Exercise 1.2.11), whereas (1.4.1)
states that the -systems Ai are mutually independent (by continuity from below
of P, taking xi for i 6= i1 , i 6= i2 , . . . , i 6= iL , has the same effect as taking a
subset of distinct indices i1 , . . . , iL from {1, . . . , m}). So, just apply Theorem 1.4.4
to conclude the proof.
58
The condition (1.4.1) for mutual independence of R.V.-s is further simplified when
these variables are either discrete valued, or having a density.
Exercise 1.4.13. Suppose (X1 , . . . , Xm ) are random variables and (S1 , . . . , Sm )
are countable sets such that P(Xi Si ) = 1 for i = 1, . . . , m. Show that if
m
Y
P(Xi = xi )
P(X1 = x1 , . . . , Xm = xm ) =
i=1
whenever xi Si , i = 1, . . . , m, then X1 , . . . , Xm are mutually independent.
Exercise 1.4.14. Suppose the random vector X = (X1 , . . . , Xm ) has a joint probability density function fX (x) = g1 (x1 ) gm (xm ). That is,
Z
P((X1 , . . . , Xm ) A) =
g1 (x1 ) gm (xm )dx1 . . . dxm ,
A BRm ,
A
where gi are non-negative, Lebesgue integrable functions. Show that then X1 , . . . , Xm

are mutually independent.
Beware that pairwise independence (of each pair Ak , Aj for k 6= j), does not imply
mutual independence of all the events in question and the same applies to three or
more random variables. Here is an illustrating example.
Exercise 1.4.15. Consider the sample space = {0, 1, 2}2 with probability measure on (, 2 ) that assigns equal probability (i.e. 1/9) to each possible value of
= (1 , 2 ) . Then, X() = 1 and Y () = 2 are independent R.V.
each taking the values {0, 1, 2} with equal (i.e. 1/3) probability. Define Z0 = X,
Z1 = (X + Y )mod3 and Z2 = (X + 2Y )mod3.
(a) Show that Z0 is independent of Z1 , Z0 is independent of Z2 , Z1 is independent of Z2 , but if we know the value of Z0 and Z1 , then we also know
Z2 .
(b) Construct four {1, 1}-valued random variables such that any three of
them are independent but all four are not.
Hint: Consider products of independent random variables.
Here is a somewhat counter intuitive example about tail -algebras, followed by
an elaboration on the theme of Corollary 1.4.11.
Exercise 1.4.16. Let (A, A ) denote the smallest -algebra G such that any
function measurable on A or on A is also measurable on G. Let W0 , W1 , W2 , . . .
be independent random variables with P(Wn = +1) = P(Wn = 1) = 1/2 for all
n. For each n 1, define Xn := W0 W1 . . .Wn .
(a) Prove that the variables X1 , X2 , . . . are independent.
(b) Show that S = (T0W , T X ) is a strict subset of the -algebra F = n (T0W , TnX ).
Hint: Show that W0 mF is independent of S.
Exercise 1.4.17. Consider random variables (Xi,j , 1 i, j n) on the same
probability space. Suppose that the -algebras R1 , . . . , Rn are P-mutually independent, where Ri = (Xi,j , 1 j n) for i = 1, . . . , n. Suppose further that the
-algebras C1 , . . . , Cn are P-mutually independent, where Cj = (Xi,j , 1 i n).
Prove that the random variables (Xi,j , 1 i, j n) must then be P-mutually independent.
We conclude this subsection with an application in number theory.
59
Exercise
P 1.4.18. Recall Eulers zeta-function which for real s > 1 is given by
(s) = k=1 k s . Fixing such s, let X and Y be independent random variables
with P(X = k) = P(Y = k) = k s /(s) for k = 1, 2, . . ..
(a) Show that the events Dp = {X is divisible by p}, with p a prime number,
are P-mutually independent.
(b) By considering the event {X
Q = 1}, provide a probabilistic explanation of
Eulers formula 1/(s) = p (1 1/ps ).
(c) Show that the probability that no perfect square other than 1 divides X is
precisely 1/(2s).
(d) Show that P(G = k) = k 2s /(2s), where G is the greatest common
divisor of X and Y .
1.4.2. Product measures and Kolmogorovs theorem. Recall Example

1.1.20 that given two measurable spaces (1 , F1 ) and (2 , F2 ) the product (measurable) space (, F ) consists of = 1 2 and F = F1 F2 , which is the same
as F = (A) for
A=
U
m
n]
j=1
o
Aj Bj : Aj F1 , Bj F2 , m < ,
where throughout, denotes the union of disjoint subsets of .

We now construct product measures on such product spaces, first for two, then
for finitely many, probability (or even -finite) measures. As we show thereafter,
these product measures are associated with the joint law of independent R.V.-s.
Theorem 1.4.19. Given two -finite measures i on (i , Fi ), i = 1, 2, there exists
a unique -finite measure 2 on the product space (, F ) such that
2 (
m
]
j=1
Aj Bj ) =
m
X
Aj F1 , Bj F2 , m < .
1 (Aj )2 (Bj ),
j=1
We denote 2 = 1 2 and call it the product of the measures 1 and 2 .

Proof. By Caratheodorys extension theorem, it suffices to show that A is an
algebra on which 2 is countably additive (see Theorem 1.1.30 for the case of finite
measures). To this end, note that = 1 2 A. Further, A is closed under
intersections, since
m
]
j=1
Aj Bj )
n
]
\ ]
( Ci Di ) = [(Aj Bj ) (Ci Di )]
i=1
i,j
]
= (Aj Ci ) (Bj Di ) .
i,j
It is also closed under complementation, for

(
m
]
j=1
Aj Bj )c =
m
\
j=1
[(Acj Bj ) (Aj Bjc ) (Acj Bjc )] .
By DeMorgans law, A is an algebra.

Note that countable unions of disjoint elements of A are also countable unions of
disjoint elements of the collection R = {A B : A F1 , B F2 } of measurable
60
rectangles. Hence, if we show that

m
X
(1.4.2)
1 (Aj )2 (Bj ) =
1 (Ci )2 (Di ) ,
j=1
Um
U
whenever j=1 Aj Bj = i (Ci Di ) for some m < , Aj , Ci F1 and Bj , Di
F2 , then we deduce that the value of 2 (E) is independent of the representation
we choose for E A in terms of measurable rectangles, and further that 2 is
countably additive on A. To this end, note that the preceding set identity amounts
to
m
X
X
x 1 , y 2 .
ICi (x)IDi (y)
IAj (x)IBj (y) =
i
j=1
P
Hence, fixing x 1 , we have that (y) = m
j=1 IAj (x)IBj (y) SF+ is the monoPn
tone increasing limit of n (y) = i=1 ICi (x)IDi (y) SF+ as n . Thus, by
linearity of the integral with respect to 2 and monotone convergence,
g(x) :=
m
X
2 (Bj )IAj (x) = 2 () = lim 2 (n ) = lim

n
j=1
n
X
ICi (x)2 (Di ) .
i=1
We deduce that the non-negative g(x) mF1 isPthe monotone increasing limit of
the non-negative measurable functions hn (x) = ni=1 2 (Di )ICi (x). Hence, by the
same reasoning,
m
X
2 (Bj )1 (Aj ) = 1 (g) = lim 1 (hn ) =

n
j=1
2 (Di )1 (Ci ) ,
proving (1.4.2) and the theorem.
It follows from Theorem 1.4.19 by induction on n that given any finite collection
of -finite measure spaces (i , Fi , i ), i = 1, . . . , n, there exists a unique product
measure n = 1 n on the product space (, F ) (i.e., = 1 n
and F = (A1 An ; Ai Fi , i = 1, . . . , n)), such that
(1.4.3)
n (A1 An ) =
n
Y
i=1
i (Ai )
Ai Fi , i = 1, . . . , n.
Remark 1.4.20. A notable special case of this construction is when i = R with

the Borel -algebra and Lebesgue measure of Section 1.1.3. The product space
is then Rn with its Borel -algebra and the product measure is n , the Lebesgue
measure on Rn .
The notion of the law PX of a real-valued random variable X as in Definition
1.2.34, naturally extends to the joint law PX of a random vector X = (X1 , . . . , Xn )
which is the probability measure PX = P X 1 on (Rn , BRn ).
We next characterize the joint law of independent random variables X1 , . . . , Xn
as the product of the laws of Xi for i = 1, . . . , n.
Proposition 1.4.21. Random variables X1 , . . . , Xn on the same probability space,
having laws i = PXi , are mutually independent if and only if their joint law is
n = 1 n .
61
Proof. By Definition 1.4.3 and the identity (1.4.3), if X1 , . . . , Xn are mutually

independent then for Bi B,
PX (B1 Bn ) = P(X1 B1 , . . . , Xn Bn )
=
n
Y
i=1
P(Xi Bi ) =
n
Y
i=1
i (Bi ) = 1 n (B1 Bn ) .
This shows that the law of (X1 , . . . , Xn ) and the product measure n agree on the
collection of all measurable rectangles B1 Bn , a -system that generates BRn
(see Exercise 1.1.21). Consequently, these two probability measures agree on BRn
(c.f. Proposition 1.1.39).
Conversely, if PX = 1 n , then by same reasoning, for Borel sets Bi ,
P(
n
\
i=1
{ : Xi () Bi }) = PX (B1 Bn ) = 1 n (B1 Bn )
=
n
Y
i (Bi ) =
i=1
n
Y
i=1
P({ : Xi () Bi }) ,
which amounts to the mutual independence of X1 , . . . , Xn .
We wish to extend the construction of product measures to that of an infinite collection of independent random variables. To this end, let N = {1, 2, . . .} denote the
set of natural numbers and RN = {x = (x1 , x2 , . . .) : xi R} denote the collection
of all infinite sequences of real numbers. We equip RN with the product -algebra
Bc = (R) generated by the collection R of all finite dimensional measurable rectangles (also called cylinder sets), that is sets of the form {x : x1 B1 , . . . , xn Bn },
where Bi B, i = 1, . . . , n N (e.g. see Example 1.1.19).
Kolmogorovs extension theorem provides the existence of a unique probability
measure P on (RN , Bc ) whose projections coincide with a given consistent sequence
of probability measures n on (Rn , BRn ).
Theorem 1.4.22 (Kolmogorovs extension theorem). Suppose we are given
probability measures n on (Rn , BRn ) that are consistent, that is,
n+1 (B1 Bn R) = n (B1 Bn )
Bi B,
i = 1, . . . , n <
Then, there is a unique probability measure P on (RN , Bc ) such that
(1.4.4) P({ : i Bi , i = 1, . . . , n}) = n (B1 Bn ) Bi B, i n <
Proof. (sketch only) We take a similar approach as in the proof of Theorem

1.4.19. That is, we use (1.4.4) to define the non-negative set function P0 on the
collection R of all finite dimensional measurable rectangles, where by the consistency of {n } the value of P0 is independent of the specific representation chosen
for a set in R. Then, we extend P0 to a finitely additive set function on the algebra
m
o
n]
Ej : Ej R, m < ,
A=
j=1
in the same linear manner we used when proving Theorem 1.4.19. Since A generates
Bc and P0 (RN ) = n (Rn ) = 1, by Caratheodorys extension theorem it suffices to
check that P0 is countably additive on A. The countable additivity of P0 is verified
by the method we already employed when dealing with Lebesgues measure. That
62
is, by the remark after Lemma 1.1.31, it suffices to prove that P0 (Hn ) 0 whenever
Hn A and Hn . The proof by contradiction of the latter, adapting the
argument of Lemma 1.1.31, is based on approximating each H A by a finite
union Jk H of compact rectangles, such that P0 (H \ Jk ) 0 as k . This is
done for example in [Bil95, Page 490].

Example 1.4.23. To systematically construct an infinite sequence of independent
random variables {Xi } of prescribed laws PXi = i , we apply Kolmogorovs extension theorem for the product measures n = 1 n constructed following
Theorem 1.4.19 (where it is by definition that the sequence n is consistent). Alternatively, for infinite product measures one can take arbitrary probability spaces
(i , Fi , i ) and directly show by contradiction that P0 (Hn ) 0 whenever Hn A
and Hn (for more details, see [Str93, Exercise 1.1.14]).
Remark. As we shall find in Sections 6.1 and 7.1, Kolmogorovs extension theorem is the key to the study of stochastic processes, where it relates the law of
the process to its finite dimensional distributions. Certain properties of R are key
to the proof of Kolmogorovs extension theorem which indeed is false if (R, B) is
replaced with an arbitrary measurable space (S, S) (see the discussions in [Dur10,
Subsection 2.1.4] and [Dud89, notes for Section 12.1]). Nevertheless, as you show
next, the conclusion of this theorem applies for any B-isomorphic measurable space
(S, S).
Definition 1.4.24. Two measurable spaces (S, S) and (T, T ) are isomorphic if
there exists a one to one and onto measurable mapping between them whose inverse
is also a measurable mapping. A measurable space (S, S) is B-isomorphic if it is
isomorphic to a Borel subset T of R equipped with the induced Borel -algebra
T = {B T : B B}.
Here is our generalized version of Kolmogorovs extension theorem.
Corollary 1.4.25. Given a measurable space (S, S) let SN denote the collection
of all infinite sequences of elements in S equipped the product -algebra Sc generated
by the collection of all cylinder sets of the form {s : s1 A1 , . . . , sn An }, where
Ai S for i = 1, . . . , n. If (S, S) is B-isomorphic then for any consistent sequence
of probability measures n on (Sn , S n ) (that is, n+1 (A1 An S) = n (A1
An ) for all n and Ai S), there exists a unique probability measure Q on
(SN , Sc ) such that for all n and Ai S,
(1.4.5)
Q({s : si Ai , i = 1, . . . , n}) = n (A1 An ) .
Next comes a guided proof of Corollary 1.4.25 out of Theorem 1.4.22.

Exercise 1.4.26.
(a) Verify that our proof of Theorem 1.4.22 applies in case (R, B) is replaced
by T B equipped with the induced Borel -algebra T (with RN and Bc
replaced by TN and Tc , respectively).
(b) Fixing such (T, T ) and (S, S) isomorphic to it, let g : S 7 T be one to
one and onto such that both g and g 1 are measurable. Check that the
one to one and onto mappings gn (s) = (g(s1 ), . . . , g(sn )) are measurable
and deduce that n (B) = n (gn1 (B)) are consistent probability measures
on (Tn , T n ).
63
(c) Consider the one to one and onto mapping g (s) = (g(s1 ), . . . , g(sn ), . . .)
from SN to TN and the unique probability measure P on (TN , Tc ) for
which (1.4.4) holds. Verify that Sc is contained in the -algebra of subsets
A of SN for which g (A) is in Tc and deduce that Q(A) = P(g (A)) is
a probability measure on (SN , Sc ).
(d) Conclude your proof of Corollary 1.4.25 by showing that this Q is the
unique probability measure for which (1.4.5) holds.
Remark. Recall that Caratheodorys extension theorem applies for any -finite
measure. It follows that, by the same proof as in the preceding exercise, any
consistent sequence of -finite measures n uniquely determines a -finite measure
Q on (SN , Sc ) for which (1.4.5) holds, a fact which we use in later parts of this text
(for example, in the study of Markov chains in Section 6.1).
Our next proposition shows that in most applications one encounters B-isomorphic
measurable spaces (for which Kolmogorovs theorem applies).
Proposition 1.4.27. If S BM for a complete separable metric space M and S
is the restriction of BM to S then (S, S) is B-isomorphic.
Remark. While we do not provide the proof of this proposition, we note in passing
that it is an immediate consequence of [Dud89, Theorem 13.1.1].
1.4.3. Fubinis theorem and its application. Returning to (, F , ) which
is the product of two -finite measure spaces, as in Theorem 1.4.19, we now prove
that:
Theorem 1.4.28 (Fubinis theorem). Suppose = 1 2 is the product of
the -finite measures
R 1 on (X, X) and 2 on (Y, Y). If h mF for F = X Y is
such that h 0 or |h| d < , then,

Z
Z Z
h d =
h(x, y) d2 (y) d1 (x)
(1.4.6)
XY
X
Y

Z Z
h(x, y) d1 (x) d2 (y)
=
Y
Remark. The iterated

integrals on the right side of (1.4.6) are finite and well
R
defined whenever |h|d < . However, for h
/ mF+ the inner integrals might
be well defined only in the almost everywhere sense.
Proof of Fubinis theorem. Clearly, it suffices to prove the first identity
of (1.4.6), as the second immediately follows by exchanging the roles of the two
measure spaces. We thus prove Fubinis theorem by showing that
(1.4.7)
(1.4.8)
y 7 h(x, y) mY, x X,
Z
h(x, y) d2 (y) mX,
x 7 fh (x) :=
Y
so the double integral on the right side of (1.4.6) is well defined and
Z
Z
fh (x)d1 (x) .
h d =
(1.4.9)
XY
We do so in three steps, first proving (1.4.7)-(1.4.9) for finite measures and bounded
h, proceeding to extend these results to non-negative h Rand -finite measures, and
then showing that (1.4.6) holds whenever h mF and |h|d is finite.
64
Step 1. Let H denote the collection of bounded functions on XY for which (1.4.7)
(1.4.9) hold. Assuming that both 1 (X) and 2 (Y) are finite, we deduce that H
contains all bounded h mF by verifying the assumptions of the monotone class
theorem (i.e. Theorem 1.2.7) for H and the -system R = {A B : A X, B Y}
of measurable rectangles (which generates F ).
To this end, note that if h = IE and E = AB R, then either h(x, ) = IB () (in
case x A), or h(x, ) is identically zero (when x 6 A). With IB mY we thus have
(1.4.7) for any such h. Further, in this case the simple function fh (x) = 2 (B)IA (x)
on (X, X) is in mX and
Z
Z
IE d = 1 2 (E) = 2 (B)1 (A) =
fh (x)d1 (x) .
XY
Consequently, IE H for all E R; in particular, the constant functions are in H.

Next, with both mY and mX vector spaces over R, by the linearity of h 7 fh
over the vector space of bounded functions satisfying (1.4.7) and the linearity of
fh 7 1 (fh ) and h 7 (h) over the vector spaces of bounded measurable fh and
h, respectively, we deduce that H is also a vector space over R.
Finally, if non-negative hn H are such that hn h, then for each x X the
mapping y 7 h(x, y) = supn hn (x, y) is in mY+ (by Theorem 1.2.22). Further,
fhn mX+ and by monotone convergence fhn fh (for all x X), so by the same
reasoning fh mX+ . Applying monotone convergence twice more, it thus follows
that
(h) = sup (hn ) = sup 1 (fhn ) = 1 (fh ) ,
n
so h satisfies (1.4.7)(1.4.9). In particular, if h is bounded then also h H .

Step 2. Suppose now that h mF+ . If 1 and 2 are finite measures, then
we have shown in Step 1 that (1.4.7)(1.4.9) hold for the bounded non-negative
functions hn = h n. With hn h we have further seen that (1.4.7)-(1.4.9) hold
also for the possibly unbounded h. Further, the closure of (1.4.8) and (1.4.9) with
respect to monotone increasing limits of non-negative functions has been shown
by monotone convergence, and as such it extends to -finite measures 1 and 2 .
Turning now to -finite 1 and 2 , recall that there exist En = An Bn R
such that An X, Bn Y, 1 (An ) < and 2 (Bn ) < . As h is the monotone
increasing limit of hn = hI
R En mF+ it thus suffices to verify that for each n
the non-negative fn (x) = Y hn (x, y)d2 (y) is measurable with (hn ) = 1 (fn ).
Fixing n and simplifying our notations to E = En , A = An and B = Bn , recall
Corollary 1.3.57 that (hn ) = E (hE ) for the restrictions hE and E of h and to
the measurable space (E, FE ). Also, as E = A B we have that FE = XA YB
and E = (1 )A R (2 )B for the finite measures (1 )A and (2 )B . Finally, as
fn (x) = fhE (x) := B hE (x, y)d(2 )B (y) when x A and zero otherwise, it follows
that 1 (fn ) = (1 )A (fhE ). We have thus reduced our problem (for hn ), to the case
of finite measures E = (1 )A (2 )B which we have already successfully resolved.
Step 3. Write h mF as h = h+ h , with h mF+ . By Step 2 we know that
y 7 h (x, y) mY for each x X, hence
the same applies for y 7 h(x, y). Let
R
X0 denote the subset of X for which Y |h(x, y)|d2 (y) < . By linearity of the
integral with respect to 2 we have that for all x X0
(1.4.10)
fh (x) = fh+ (x) fh (x)
65
is finite. By Step 2 we know that fh mX, hence X0 = {x : fh+ (x) + fh (x) <
} is in X. From
R Step 2 we further have that 1 (fh ) = (h ) whereby our
assumption that |h| d = 1 (fh+ + fh ) < implies that 1 (Xc0 ) = 0. Let
/ X0 . Clearly, feh mX
feh (x) = fh+ (x) fh (x) on X0 and feh (x) = 0 for all x
is 1 -almost-everywhere the same as the inner integral on the right side of (1.4.6).
Moreover, in view of (1.4.10) and linearity of the integrals with respect to 1 and
we deduce that
(h) = (h+ ) (h ) = 1 (fh+ ) 1 (fh ) = 1 (feh ) ,
which is exactly the identity (1.4.6).
Equipped with Fubinis theorem, we have the following simpler formula for the
expectation of a Borel function h of two independent R.V.
Theorem 1.4.29. Suppose that X and Y are independent random variables of
laws 1 = PX and 2 = PY . If h : R2 7 R is a Borel measurable function such
that h 0 or E|h(X, Y )| < , then,
Z hZ
i
(1.4.11)
Eh(X, Y ) =
h(x, y) d1 (x) d2 (y)
In particular, for Borel functions f, g : R 7 R such that f, g 0 or E|f (X)| <
and E|g(Y )| < ,
(1.4.12)
E(f (X)g(Y )) = Ef (X) Eg(Y )
Proof. Subject to minor changes of notations, the proof of Theorem 1.3.61

applies to any (S, S)-valued R.V. Considering this theorem for the random vector
(X, Y ) whose joint law is 1 2 (c.f. Proposition 1.4.21), together with Fubinis
theorem, we see that
Z hZ
Z
i
h(x, y) d(1 2 )(x, y) =
h(x, y) d1 (x) d2 (y) ,
Eh(X, Y ) =
R2
which is (1.4.11). Take now h(x, y) = f (x)g(y) for non-negative Borel functions
f (x) and g(y). In this case, the iterated integral on the right side of (1.4.11) can
be further simplified to,
Z hZ
Z
Z
i
E(f (X)g(Y )) =
f (x)g(y) d1 (x) d2 (y) = g(y)[ f (x) d1 (x)] d2 (y)
Z
= [Ef (X)]g(y) d2 (y) = Ef (X) Eg(Y )
(with Theorem 1.3.61 applied twice here), which is the stated identity (1.4.12).
To deal with Borel functions f and g that are not necessarily non-negative, first
apply (1.4.12) for the non-negative functions |f | and |g| to get that E(|f (X)g(Y )|) =
E|f (X)|E|g(Y )| < . Thus, the assumed integrability of f (X) and of g(Y ) allows
us to apply again (1.4.11) for h(x, y) = f (x)g(y). Now repeat the argument we
used for deriving (1.4.12) in case of non-negative Borel functions.

Another consequence of Fubinis theorem is the following integration by parts formula.
66
Rx
Lemma 1.4.30 (integration by parts). Suppose H(x) = h(y)dy for a
non-negative Borel function h and all x R. Then, for any random variable X,
Z
h(y)P(X > y)dy .
(1.4.13)
EH(X) =
R
Proof. Combining the change of variables formula (Theorem 1.3.61), with our
assumption about H(), we have that
Z hZ
Z
i
h(y)Ix>y d(y) dPX (x) ,
H(x)dPX (x) =
EH(X) =
R
where denotes Lebesgues measure on (R, B). For each y R, the expectation of
the simple function x 7 h(x, y) = h(y)Ix>y with respect to (R, B, PX ) is merely
h(y)P(X > y). Thus, applying Fubinis theorem for the non-negative measurable
function h(x, y) on the product space R R equipped with its Borel -algebra BR2 ,
and the -finite measures 1 = PX and 2 = , we have that
Z
Z hZ
i
h(y)P(X > y)dy ,
h(y)Ix>y dPX (x) d(y) =
EH(X) =
R
as claimed.
Indeed, as we see next, by combining the integration by parts formula with H

olders
inequality we can convert bounds on tail probabilities to bounds on the moments
of the corresponding random variables.
Lemma 1.4.31.
(a) For any r > p > 0 and any random variable Y 0,
Z
Z
py p1 P(Y y)dy
py p1 P(Y > y)dy =
EY p =
0
0
Z
p
= (1 )
py p1 E[min(Y /y, 1)r ]dy .
r 0
(b) If X, Y 0 are such that P(Y y) y 1 E[XIY y ] for all y > 0, then
kY kp qkXkp for any p > 1 and q = p/(p 1).
(c) Under the same hypothesis also EY 1 + E[X(log Y )+ ].
Proof. (a) The first identity is merely the integration by parts formula for
hp (y) = py p1 1y>0 and Hp (x) = xp 1x0 and the second identity follows by the
fact that P(Y = y) = 0 up to a (countable)
set of zero Lebesgue measure. Finally,
R
it is easy to check that Hp (x) = R hp,r (x, y)dy for the non-negative Borel function
hp,r (x, y) = (1 p/r)py p1 min(x/y, 1)r 1x0 1y>0 and any r > p > 0. Hence,
replacing h(y)I
x>y throughout the proof of Lemma 1.4.30 by hp,r (x, y) we find that
R
E[Hp (X)] = 0 E[hp,r (X, y)]dy, which is exactly our third identity.
(b) In a similar manner it follows from Fubinis theorem that for p > 1 and any
non-negative random variables X and Y
Z
Z
E[XY p1 ] = E[XHp1 (Y )] = E[ hp1 (y)XIY y dy] =
hp1 (y)E[XIY y ]dy .
R
Thus, with y
EY p =
hp (y) = qhp1 (y) our hypothesis implies that

Z
qhp1 (y)E[XIY y ]dy = qE[XY p1 ] .
hp (y)P(Y y)dy
67
Applying H
olders inequality we deduce that
EY p qE[XY p1 ] qkXkp kY p1 kq = qkXkp [EY p ]1/q
where the right-most equality is due to the fact that (p 1)q = p. In case Y
is bounded, dividing both sides of the preceding bound by [EY p ]1/q implies that
kY kp qkXkp . To deal with the general case, let Yn = Y n, n = 1, 2, . . . and
note that either {Yn y} is empty (for n < y) or {Yn y} = {Y y}. Thus, our
assumption implies that P(Yn y) y 1 E[XIYn y ] for all y > 0 and n 1. By
the preceding argument kYn kp qkXkp for any n. Taking n it follows by
monotone convergence that kY kp qkXkp .
(c) Considering part (a) with p = 1, we bound P(Y y) by one for y [0, 1] and
by y 1 E[XIY y ] for y > 1, to get by Fubinis theorem that
Z
Z
EY =
P(Y y)dy 1 +
y 1 E[XIY y ]dy
0
1
Z
y 1 IY y dy] = 1 + E[X(log Y )+ ] .
= 1 + E[X
1

We further have the following corollary of (1.4.12), dealing with the expectation
of a product of mutually independent R.V.
Corollary 1.4.32. Suppose that X1 , . . . , Xn are P-mutually independent random
variables such that either Xi 0 for all i, or E|Xi | < for all i. Then,
(1.4.14)
n
Y
i=1
n
Y
EXi ,
Xi =
i=1
that is, the expectation on the left exists and has the value given on the right.
Proof. By Corollary 1.4.11 we know that X = X1 and Y = X2 Xn are
independent. Taking f (x) = |x| and g(y) = |y| in Theorem 1.4.29, we thus have
that E|X1 Xn | = E|X1 |E|X2 Xn | for any n 2. Applying this identity
iteratively for Xl , . . . , Xn , starting with l = m, then l = m + 1, m + 2, . . . , n 1
leads to
(1.4.15)
E|Xm Xn | =
n
Y
k=m
E|Xk | ,
holding for any 1 m n. If Xi 0 for all i, then |Xi | = Xi and we have (1.4.14)
as the special case m = 1.
To deal with the proof in case Xi L1 for all i, note that for m = 2 the identity
(1.4.15) tells us that E|Y | = E|X2 Xn | < , so using Theorem 1.4.29 with
f (x) = x and g(y) = y we have that E(X1 Xn ) = (EX1 )E(X2 Xn ). Iterating
this identity for Xl , . . . , Xn , starting with l = 1, then l = 2, 3, . . . , n 1 leads to
the desired result (1.4.14).

Another application of Theorem 1.4.29 provides us with the familiar formula for
the probability density function of the sum X + Y of independent random variables
X and Y , having densities fX and fY respectively.
68
Corollary 1.4.33. Suppose that R.V. X with a Borel measurable probability

density function fX and R.V. Y with a Borel measurable probability density function
fY are independent. Then, the random variable Z = X + Y has the probability
density function
Z
fZ (z) =
fX (z y)fY (y)dy .
Proof. Fixing z R, apply Theorem 1.4.29 for h(x, y) = 1(x+yz) , to get

that
Z hZ
i
FZ (z) = P(X + Y z) = Eh(X, Y ) =
h(x, y)dPX (x) dPY (y) .
R
Considering the inner integral for a fixed value of y, we have that

Z
Z
Z
h(x, y)dPX (x) =
I(,zy] (x)dPX (x) = PX ((, z y]) =
R
zy
fX (x)dx ,
where the right most equality is by the existence of a density fX (x) for X (c.f.
R zy
Rz
Definition 1.2.40). Clearly, fX (x)dx = fX (x y)dx. Thus, applying
Fubinis theorem for the Borel measurable function g(x, y) = fX (x y) 0 and
the product of the -finite Lebesgues measure on (, z] and the probability
measure PY , we see that
Z z hZ
Z hZ z
i
i
fX (x y)dPY (y) dx
fX (x y)dx dPY (y) =
FZ (z) =
R
(in this application of Fubinis theorem we replace one iterated integral by another,
exchanging the order of integrations). Since this applies for any z R, it follows
by definition that Z has the probability density
Z
fX (z y)dPY (y) = EfX (z Y ) .
fZ (z) =
R
With Y having density fY , the stated formula for fZ is a consequence of Corollary

1.3.62.

R
Definition 1.4.34. The expression f (z y)g(y)dy is called the convolution of
the non-negative Borel functions f and g, denoted by f g(z). The convolution of
measures
and on (R, B) is the measure on (R, B) such that (B) =
R
(B x)d(x) for any B B (where B x = {y : x + y B}).
Corollary 1.4.33 states that if two independent random variables X and Y have
densities, then so does Z = X + Y , whose density is the convolution of the densities
of X and Y . Without assuming the existence of densities, one can show by a similar
argument that the law of X + Y is the convolution of the law of X and the law of
Y (c.f. [Dur10, Theorem 2.1.10] or [Bil95, Page 266]).
Convolution is often used in analysis to provide a more regular approximation to
a given function. Here are few of the reasons for doing so.
Exercise R1.4.35. Suppose Borel functions f, g are such that g is a probability
density and |f (x)|dx is finite. Consider the scaled densities gn () = ng(n), n 1.
R
R
(a) Show that f g(y) is a Borel function with |f g(y)|dy |f (x)|dx
and if g is uniformly continuous, then so is f g.
(b) Show that if g(x) = 0 whenever |x| 1, then f gn (y) f (y) as n ,
for any continuous f and each y R.
69
Next you find two of the many applications of Fubinis theorem in real analysis.
Exercise 1.4.36. Show that the set Gf = {(x, y) R2 : 0 y f (x)} of points
under the graph of a non-negative Borel function
f : R 7 [0, ) is in BR2 and
R
deduce the well-known formula (Gf ) = f (x)d(x), for its area.
Exercise 1.4.37. For n 2, consider the unit sphere S n1 = {x Rn : kxk = 1}

equipped with the topology induced by Rn . Let the surface measure of A BS n1 be
(A) = nn (C0,1 (A)), for Ca,b (A) = {rx : r (a, b], x A} and the n-fold product
Lebesgue measure n (as in Remark 1.4.20).
(a) Check that Ca,b (A) BRn and deduce that () is a finite measure on
S n1 (which is further invariant under orthogonal transformations).
n
n
(b) Verify that n (Ca,b (A)) = b a
(A) and deduce that for any B BRn
n
Z hZ
i
IrxB d(x) rn1 d(r) .
n (B) =
0
S n1
Hint: Recall that (B) = n n (B) for any 0 and B BRn .
Combining (1.4.12) with Theorem 1.2.26 leads to the following characterization of

the independence between two random vectors (compare with Definition 1.4.1).
Exercise 1.4.38. Show that the Rn -valued random variable (X1 , . . . , Xn ) and the
Rm -valued random variable (Y1 , . . . , Ym ) are independent if and only if
E(h(X1 , . . . , Xn )g(Y1 , . . . , Ym )) = E(h(X1 , . . . , Xn ))E(g(Y1 , . . . , Ym )),
for all bounded, Borel measurable functions g : Rm 7 R and h : Rn 7 R.
Then show that the assumption of h() and g() bounded can be relaxed to both
h(X1 , . . . , Xn ) and g(Y1 , . . . , Ym ) being in L1 (, F , P).
Here is another application of (1.4.12):
Exercise 1.4.39. Show that E(f (X)g(X)) (Ef (X))(Eg(X)) for every random
variable X and any bounded non-decreasing functions f, g : R 7 R.
In the following exercise you bound the exponential moments of certain random
variables.
Exercise 1.4.40. Suppose Y is an integrable random variable such that E[eY ] is
finite and E[Y ] = 0.
(a) Show that if |Y | then
log E[eY ] 2 (e 1)E[Y 2 ] .
Hint: Use the Taylor expansion of eY Y 1.

(b) Show that if E[Y 2 euY ] 2 E[euY ] for all u [0, 1], then
log E[eY ] log cosh() .
Hint: Note that (u) = log E[euY ] is convex, non-negative and finite on
[0, 1] with (0) = 0 and (0) = 0. Verify that (u) + (u)2 2 while
(u) = log cosh(u) satisfies the differential equation (u)+ (u)2 = 2 .
As demonstrated next, Fubinis theorem is also handy in proving the impossibility
of certain constructions.
70
Exercise 1.4.41. Explain why it is impossible to have P-mutually independent

random variables Ut (), t [0, 1], on the same probability space (, F , P), having
each the uniform probability measure on [1/2, 1/2], such that t 7 Ut () is a Borel
function for almost every
.
Rr
Hint: Show that E[( 0 Ut ()dt)2 ] = 0 for all r [0, 1].
Random variables X and Y such that E(X 2 ) < and E(Y 2 ) < are called
uncorrelated if E(XY ) = E(X)E(Y ). It follows from (1.4.12) that independent
random variables X, Y with finite second moment are uncorrelated. While the
converse is not necessarily true, it does apply for pairs of random variables that
take only two different values each.
Exercise 1.4.42. Suppose X and Y are uncorrelated random variables.
(a) Show that if X = IA and Y = IB for some A, B F then X and Y are
also independent.
(b) Using this, show that if {a, b}-valued R.V. X and {c, d}-valued R.V. Y
are uncorrelated, then they are also independent.
(c) Give an example of a pair of R.V. X and Y that are uncorrelated but not
independent.
Next come a pair of exercises utilizing Corollary 1.4.32.
Exercise 1.4.43. Suppose X and Y are random variables on the same probability
space, X has a Poisson distribution with parameter > 0, and Y has a Poisson
distribution with parameter > (see Example 1.3.69).
(a) Show that if X and Y are independent then P(X Y ) exp((

2
) ).
(b) Taking = for > 1, find I() > 0 such that P(X Y )
2 exp(I()) even when X and Y are not independent.
Exercise 1.4.44. Suppose X and Y are independent random variables of identical
distribution such that X > 0 and E[X] < .
(a) Show that E[X 1 Y ] > 1 unless X() = c for some non-random c and
almost every .
(b) Provide an example in which E[X 1 Y ] = .
We conclude this section with a concrete application of Corollary 1.4.33, computing the density of the sum of mutually independent R.V., each having the same
exponential density. To this end, recall
Definition 1.4.45. The gamma density with parameters > 0 and > 0 is given
by
f (s) = ()1 s1 es 1s>0 ,
R 1 s
where () = 0 s
e ds is finite and positive. In particular, = 1 corresponds
to the exponential density fT of Example 1.3.68.
Exercise 1.4.46. Suppose X has a gamma density of parameters 1 and and Y

has a gamma density of parameters 2 and . Show that if X and Y are independent then X + Y has a gamma density of parameters 1 + 2 and . Deduce that
if T1 , . . . , Tn are mutuallyP
independent R.V. each having the exponential density of
parameter , then Wn = ni=1 Ti has the gamma density of parameters = n and
.
CHAPTER 2
Asymptotics: the law of large numbers

Building upon the foundations of Chapter 1 we turn to deal with asymptotic
theory. To this end, this chapter is devoted to degenerate limit laws, that is,
situations in which a sequence of random variables converges to a non-random
(constant) limit. Though not exclusively
Pn dealing with it, our focus here is on the
sequence of empirical averages n1 i=1 Xi as n .
Section 2.1 deals with the weak law of large numbers, where convergence in probability (or in Lq for some q > 1) is considered. This is strengthened in Section 2.3
to a strong law of large numbers, namely, to convergence almost surely. The key
tools for this improvement are the Borel-Cantelli lemmas, to which Section 2.2 is
devoted.
2.1. Weak laws of large numbers
A weak law of large numbers corresponds to the situation where the normalized
sums of large number of random variables converge in probability to a non-random
constant. Usually, the derivation of a weak low involves the computation of variances, on which we focus in Subsection 2.1.1. However, the L2 convergence we
obtain there is of a somewhat limited scope of applicability. To remedy this, we
introduce the method of truncation in Subsection 2.1.2 and illustrate its power in
a few representative examples.
2.1.1. L2 limits for sums of uncorrelated variables. The key to our
derivation of weak laws of large numbers is the computation of variances. As a
preliminary step we define the covariance of two R.V. and extend the notion of a
pair of uncorrelated random variables, to a (possibly infinite) family of R.V.
Definition 2.1.1. The covariance of two random variables X, Y L2 (, F , P) is
Cov(X, Y ) = E[(X EX)(Y EY )] = EXY EXEY ,
so in particular, Cov(X, X) = Var(X).

We say that random variables X L2 (, F , P) are uncorrelated if
E(X X ) = E(X )E(X )
or equivalently, if
Cov(X , X ) = 0
6= ,
6= .
As we next show, the variance of the sum of finitely many uncorrelated random
variables is the sum of the variances of the variables.
Lemma 2.1.2. Suppose X1 , . . . , Xn are uncorrelated random variables (which necessarily are defined on the same probability space). Then,
(2.1.1)
Var(X1 + + Xn ) = Var(X1 ) + + Var(Xn ) .

71
72
2. ASYMPTOTICS: THE LAW OF LARGE NUMBERS
Proof. Let Sn =
n
P
Xi . By Definition 1.3.67 of the variance and linearity of
i=1
the expectation we have that

Var(Sn ) = E([Sn ESn ]2 ) = E [
n
X
i=1
Xi
n
X
i=1
n
X

EXi ]2 = E [ (Xi EXi )]2 .
i=1
Writing the square of the sum as the sum of all possible cross-products, we get that
n
X
E[(Xi EXi )(Xj EXj )]
Var(Sn ) =
=
i,j=1
n
X
Cov(Xi , Xj ) =
n
X
Cov(Xi , Xi ) =
i=1
i,j=1
n
X
Var(Xi ) ,
i=1
where we use the fact that Cov(Xi , Xj ) = 0 for each i 6= j since Xi and Xj are
uncorrelated.

Equipped with this lemma we have our
Theorem 2.1.3 (L2 weak law of large numbers). Consider Sn =
n
P
Xi
i=1
for uncorrelated random variables X1 , . . . , Xn , . . .. Suppose that Var(Xi ) C and

L2
EXi = x for some finite constants C, x, and all i = 1, 2, . . .. Then, n1 Sn x as

p
n , and hence also n1 Sn x.
Proof. Our assumptions imply that E(n1 Sn ) = x, and further by Lemma

2.1.2 we have the bound Var(Sn ) nC. Recall the scaling property (1.3.17) of the
variance, implying that
h
i

1
C
E (n1 Sn x)2 = Var n1 Sn = 2 Var(Sn )
0
n
n
L2
as n . Thus, n1 Sn x (recall Definition 1.3.26). By Proposition 1.3.29 this

p

implies that also n1 Sn x.
The most important special case of Theorem 2.1.3 is,
Example 2.1.4. Suppose that X1 , . . . , Xn are independent and identically distributed (or in short, i.i.d.), with EX12 < . Then, EXi2 = C and EXi = mX are
both finite and independent of i. So, the L2 weak law of large numbers tells us that
L2
n1 Sn mX , and hence also n1 Sn mX .
Remark. As we shall see, the weaker condition E|Xi | < suffices for the convergence in probability of n1 Sn to mX . In Section 2.3 we show that it even suffices
for the convergence almost surely of n1 Sn to mX , a statement called the strong
law of large numbers.
Exercise 2.1.5. Show that the conclusion of the L2 weak law of large numbers
holds even for correlated Xi , provided EXi = x and Cov(Xi , Xj ) r(|i j|) for all
i, j, and some bounded sequence r(k) 0 as k .
With an eye on generalizing the L2 weak law of large numbers we observe that
Lemma 2.1.6. If the random variables Zn L2 (, F , P) and the non-random bn

L2
1
are such that b2
n Var(Zn ) 0 as n , then bn (Zn EZn ) 0.
2.1. WEAK LAWS OF LARGE NUMBERS
73
2
2
Proof. We have E[(b1

n (Zn EZn )) ] = bn Var(Zn ) 0.
Pn
Example 2.1.7. Let Zn = k=1 Xk for uncorrelated random variables {Xk }. If
Var(Xk )/k 0 as k , then Lemma 2.1.6 applies for Zn and bn = n, hence
n1 (Zn EZn ) 0 in L2 (and in probability). Alternatively, if also Var(Xk ) 0,
then Lemma 2.1.6 applies even for Zn and bn = n1/2 .
Many limit theorems involve random variables of the form Sn =
n
P
Xn,k , that is,
k=1
the row sums of triangular arrays of random variables {Xn,k : k = 1, . . . , n}. Here
are two such examples, both relying on Lemma 2.1.6.
Example 2.1.8 (Coupon collectors problem). Consider i.i.d. random variables U1 , U2 , . . ., each distributed uniformly on {1, 2, . . . , n}. Let |{U1 , . . . , Ul }| denote the number of distinct elements among the first l variables, and kn = inf{l :
|{U1 , . . . , Ul }| = k} be the first time one has k distinct values. We are interested
in the asymptotic behavior as n of Tn = nn , the time it takes to have at least
one representative of each of the n possible values.
To motivate the name assigned to this example, think of collecting a set of n
different coupons, where independently of all previous choices, each item is chosen
at random in such a way that each of the possible n outcomes is equally likely.
Then, Tn is the number of items one has to collect till having the complete set.
n
Setting 0n = 0, let Xn,k = kn k1
denote the additional time it takes to get
an item different from the first k 1 distinct items collected. Note that Xn,k has
1
a geometric distribution of success probability qn,k = 1 k1
n , hence EXn,k = qn,k
2
and Var(Xn,k ) qn,k
(see Example 1.3.69). Since
Tn =
nn
0n
n
X
k=1
(kn
n
k1
)
n
X
Xn,k ,
k=1
we have by linearity of the expectation that

n
n
X
X
k 1 1
1
ETn =
=n
1 = n(log n + n ) ,
n
k=1
where n =
n
P
=1
Rn
1
=1
x1 dx is between zero and one (by monotonicity of x 7
x1 ). Further, Xn,k is independent of each earlier waiting time Xn,j , j = 1, . . . , k

1, hence we have by Lemma 2.1.2 that
n
n
X
X
X
k 1 2
n2
2 = Cn2 ,
Var(Tn ) =
Var(Xn,k )
1
n
k=1
k=1
=1
for some C < . Applying Lemma 2.1.6 with bn = n log n, we deduce that
Tn n(log n + n ) L2
0.
n log n
Since n / log n 0, it follows that
Tn L2
1,
n log n
and Tn /(n log n) 1 in probability as well.
74
One possible extension of Example 2.1.8 concerns infinitely many possible coupons.
That is,
Exercise 2.1.9. Suppose {k } are i.i.d. positive integer valued random variables,
with P(1 = i) = pi > 0 for i = 1, 2, . . .. Let Dl = |{1 , . . . , l }| denote the number
of distinct elements among the first l variables.
a.s.
(a) Show that Dn as n .
p
(b) Show that n1 EDn 0 as n and deduce that n1 Dn 0.
Hint: Recall that (1 p)n 1 np for any p [0, 1] and n 0.
Example 2.1.10 (An occupancy problem). Suppose we distribute at random
r distinct balls among n distinct boxes, where each of the possible nr assignments
of balls to boxes is equally likely. We are interested in the asymptotic behavior of
the number Nn of empty boxes when r/n [0, ], while n . To this
n
P
IAi . Since
end, let Ai denote the event that the i-th box is empty, so Nn =
i=1
P(Ai ) = (1 1/n)r for each i, it follows that E(n1 Nn ) = (1 1/n)r e .

n
P
P(Ai Aj ) and P(Ai Aj ) = (1 2/n)r for each i 6= j.
Further, ENn2 =
i,j=1
Hence, splitting the sum according to i = j or i 6= j, we see that

1
1
1
1
2
1
1
Var(n1 Nn ) = 2 ENn2 (1 )2r = (1 )r + (1 )(1 )r (1 )2r .
n
n
n
n
n
n
n
As n , the first term on the right side goes to zero, and with r/n , each of
the other two terms converges to e2 . Consequently, Var(n1 Nn ) 0, so applying
Lemma 2.1.6 for bn = n we deduce that
Nn
e
n
in L2 and in probability.
2.1.2. Weak laws and truncation. Our next order of business is to extend
the weak law of large numbers for row sums Sn in triangular arrays of independent
Xn,k which lack a finite second moment. Of course, with Sn no longer in L2 , there
is no way to establish convergence in L2 . So, we aim to retain only the convergence
in probability, using truncation. That is, we consider the row sums S n for the
truncated array X n,k = Xn,k I|Xn,k |bn , with bn slowly enough to control the
variance of S n and fast enough for P(Sn 6= S n ) 0. As we next show, this gives
the convergence in probability for S n which translates to same convergence result
for Sn .
Theorem 2.1.11 (Weak law for triangular arrays). Suppose that for each
n, the random variables Xn,k , k = 1, . . . , n are pairwise independent. Let X n,k =
Xn,k I|Xn,k |bn for non-random bn > 0 such that as n both
n
P
P(|Xn,k | > bn ) 0,
(a)
k=1
and
n
P
(b) b2
Var(X n,k ) 0.
n
k=1
Then, b1
n (Sn an ) 0 as n , where Sn =
n
P
k=1
Xn,k and an =
n
P
k=1
EX n,k .
2.1. WEAK LAWS OF LARGE NUMBERS
75
Pn
Proof. Let S n = k=1 X n,k . Clearly, for any > 0,

n

o [ S a
Sn a n
n
n

> Sn 6= S n
> .
bn
bn
Consequently,
(2.1.2)
S a
S a

n
n
n
n
P
> P(Sn 6= S n ) + P
> .
bn
bn
To bound the first term, note that our condition (a) implies that as n ,
n
[

{Xn,k 6= X n,k }
P(Sn 6= S n ) P
k=1
n
X
k=1
P(Xn,k 6= X n,k ) =
n
X
k=1
P(|Xn,k | > bn ) 0 .
Turning to bound the second term in (2.1.2), recall that pairwise independence
is preserved under truncation, hence X n,k , k = 1, . . . , n are uncorrelated random
variables (to convince yourself, apply (1.4.12) for the appropriate functions). Thus,
an application of Lemma 2.1.2 yields that as n ,
2
Var(b1
n S n ) = bn
n
X
k=1
Var(X n,k ) 0 ,
by our condition (b). Since an = ES n , from Chebyshevs inequality we deduce that

for any fixed > 0,
S a

n
n
P
> 2 Var(b1
n Sn) 0 ,
bn
as n . In view of (2.1.2), this completes the proof of the theorem.
Specializing the weak law of Theorem 2.1.11 to a single sequence yields the following.
Proposition 2.1.12 (Weak law of large numbers). Consider i.i.d. random
p
variables {Xi }, such that xP(|X1 | > x) 0 as x . Then, n1 Sn n 0,
n
P
Xi and n = E[X1 I{|X1 |n} ].
where Sn =
i=1
Proof. We get the result as an application of Theorem 2.1.11 for Xn,k = Xk

and bn = n, in which case an = nn . Turning to verify condition (a) of this
theorem, note that
n
X
k=1
P(|Xn,k | > n) = nP(|X1 | > n) 0
as n , by our assumption. Thus, all that remains to do is to verify that

condition (b) of Theorem 2.1.11 holds here. This amounts to showing that as
n ,
n
X
n = n2
Var(X n,k ) = n1 Var(X n,1 ) 0 .
k=1
76
Recall that for any R.V. Z,

Var(Z) = EZ 2 (EZ)2 E|Z|2 =
2yP(|Z| > y)dy
(see part (a) of Lemma 1.4.31 for the right identity). Considering Z = X n,1 =
X1 I{|X1 |n} for which P(|Z| > y) = P(|X1 | > y) P(|X1 | > n) P(|X1 | > y)
when 0 < y < n and P(|Z| > y) = 0 when y n, we deduce that
Z n
1
1
n = n Var(Z) n
g(y)dy ,
0
where by our assumption, g(y) = 2yP(|X1 | > y) 0 for y . Further, the

non-negative
Borel function g(y) 2y is then uniformly bounded on [0, ), hence
Rn
n1 0 g(y)dy 0 as n (c.f. Exercise 1.3.52). Verifying that n 0, we
established condition (b) of Theorem 2.1.11 and thus completed the proof of the
proposition.

Remark. The condition xP(|X1 | > x) 0 for x is indeed necessary for the
p
existence of non-random n such that n1 Sn n 0 (c.f. [Fel71, Page 234-236]
for a proof).
Exercise 2.1.13. Let {Xi } be i.i.d. with P(X1 = P
(1)k k) = 1/(ck 2 log k) for
2
integers k 2 and a normalization constant c =
k 1/(k log k). Show that
p
1
E|X1 | = , but there is a non-random < such that n Sn .
p
As a corollary to Proposition 2.1.12 we next show that n1 Sn mX as soon as

the i.i.d. random variables Xi are in L1 .
n
P
Xk for i.i.d. random variables {Xi } such
Corollary 2.1.14. Consider Sn =
p
k=1
that E|X1 | < . Then, n1 Sn EX1 as n .
Proof. In view of Proposition 2.1.12, it suffices to show that if E|X1 | < ,

then both nP(|X1 | > n) 0 and EX1 n = E[X1 I{|X1 |>n} ] 0 as n .
To this end, recall that E|X1 | < implies that P(|X1 | < ) = 1 and hence
the sequence X1 I{|X1 |>n} converges to zero a.s. and is bounded by the integrable
|X1 |. Thus, by dominated convergence E[X1 I{|X1 |>n} ] 0 as n . Applying
dominated convergence for the sequence nI{|X1 |>n} (which also converges a.s. to
zero and is bounded by the integrable |X1 |), we deduce that nP(|X1 | > n) =
E[nI{|X1 |>n} ] 0 when n , thus completing the proof of the corollary.

We conclude this section by considering an example for which E|X1 | = and
Proposition 2.1.12 does not apply, but nevertheless, Theorem 2.1.11 allows us to
p
deduce that c1
n Sn 1 for some cn such that cn /n .
Example 2.1.15. Let {Xi } be i.i.d. random variables such that P(X1 = 2j ) =
2j for j = 1, 2, . . .. This has the interpretation of a game, where in each of its
independent rounds you win 2j dollars if it takes exactly j tosses of a fair coin
to get the first Head. This example is called the St. Petersburg paradox, since
though EX1 = , you clearly would not pay an infinite amount just in order to
play this game. Applying Theorem 2.1.11 we find that one should be willing to pay
p
roughly n log2 n dollars for playing n rounds of this game, since Sn /(n log2 n) 1
as n . Indeed, the conditions of Theorem 2.1.11 apply for bn = 2mn provided
2.2. THE BOREL-CANTELLI LEMMAS
77
the integers mn are such that mn log2 n . Taking mn log2 n + log2 (log2 n)
implies that bn n log2 n and an /(n log2 n) = mn / log2 n 1 as n , with the
p
consequence of Sn /(n log2 n) 1 (for details see [Dur10, Example 2.2.7]).
2.2. The Borel-Cantelli lemmas
When dealing with asymptotic theory, we often wish to understand the relation
between countably many events An in the same probability space. The two BorelCantelli lemmas of Subsection 2.2.1 provide information on the probability of the set
of outcomes that are in infinitely many of these events, based only on P(An ). There
are numerous applications to these lemmas, few of which are given in Subsection
2.2.2 while many more appear in later sections of these notes.
2.2.1. Limit superior and the Borel-Cantelli lemmas. We are often interested in the limits superior and limits inferior of a sequence of events An on
the same measurable space (, F ).
Definition 2.2.1. For a sequence of subsets An , define
[
\
A := lim sup An =
A
m=1 =m
= { : An for infinitely many ns }
= { : An infinitely often } = {An i.o. }
Similarly,
lim inf An =
m=1 =m
= { : An for all but finitely many ns }

= { : An eventually } = {An ev. }
Remark. Note that if An F are measurable, then so are lim sup An and
lim inf An . By DeMorgans law, we have that {An ev. } = {Acn i.o. }c , that is,
An for all n large enough if and only if Acn for finitely many ns.
Also, if An eventually, then certainly An infinitely often, that is
lim inf An lim sup An .
The notations lim sup An and lim inf An are due to the intimate connection of
these sets to the lim sup and lim inf of the indicator functions on the sets An . For
example,
lim sup IAn () = Ilim sup An (),
n
since for a given , the lim sup on the left side equals 1 if and only if the
sequence n
7 IAn () contains an infinite subsequence of ones. In other words, if
and only if the given is in infinitely many of the sets An . Similarly,
lim inf IAn () = Ilim inf An (),
n
since for a given , the lim inf on the left side equals 1 if and only if there
are only finitely many zeros in the sequence n 7 IAn () (for otherwise, their limit
inferior is zero). In other words, if and only if the given is in An for all n large
enough.
78
In view of the preceding remark, Fatous lemma yields the following relations.
Exercise 2.2.2. Prove that for any sequence An F ,
P(lim sup An ) lim sup P(An ) lim inf P(An ) P(lim inf An ) .
n
Show that the right most inequality holds even when the probability measure is replaced
S by an arbitrary measure (), but the left most inequality may then fail unless
( kn Ak ) < for some n.
Practice your understanding of the concepts of lim sup and lim inf of sets by solving
the following exercise.
Exercise 2.2.3. Assume that P(lim sup An ) = 1 and P(lim inf Bn ) = 1. Prove
that P(lim sup(An Bn )) = 1. What happens if the condition on {Bn } is weakened
to P(lim sup Bn ) = 1?
Our next result, called the first Borel-Cantelli lemma, states that if the probabilities P(An ) of the individual events An converge to zero fast enough, then almost
surely, An occurs for only finitely many values of n, that is, P(An i.o.) = 0. This
lemma is extremely useful, as the possibly complex relation between the different
events An is irrelevant for its conclusion.
Lemma 2.2.4 (Borel-Cantelli I). Suppose An F and
P(An i.o.) = 0.
Proof. Define N () =
and our assumption,
k=1 IAk ().
E[N ()] = E
hX
k=1
n=1
P(An ) < . Then,
By the monotone convergence theorem
i X
IAk () =
P(Ak ) < .
k=1
Since the expectation of N is finite, certainly P({ : N () = }) = 0. Noting

that the set { : N () = } is merely { : An i.o.}, the conclusion P(An i.o.) = 0
of the lemma follows.

Our next result, left for the reader to prove, relaxes somewhat the conditions of
Lemma 2.2.4.
P
Exercise 2.2.5. Suppose An F are such that
P(An Acn+1 ) < and
n=1
P(An ) 0. Show that then P(An i.o.) = 0.
P
then
The first Borel-Cantelli lemma states that if the series n P(An ) converges
P
almost every is in finitely many sets An . If P(An ) 0, but the series n P(An )
diverges, then the event {An i.o.} might or might not have positive probability. In
this sense, the Borel-Cantelli I is not tight, as the following example demonstrates.
Example 2.2.6. Consider the uniform probability measure U on ((0, 1], B(0,1] ),
andPthe events An = (0, 1/n]. Then An , so {An i.o.} = , but U (An ) = 1/n,
so n U (An ) = and the Borel-Cantelli I does not apply.
Recall also Example 1.3.25 showing the existence of An = (tn , tn + 1/n] such that
U (An ) = 1/n while {An i.o.} = (0, 1]. Thus, in general the probability of {An i.o.}
depends on the relation between the different events An .
79
P
As seen in the preceding example, the divergence of the series n P(An ) is not
sufficient for the occurrence of a set of positive probability of values, each of
which is in infinitely many events An . However, upon adding the assumption that
the events An are mutually independent (flagrantly not the case in Example 2.2.6),
we conclude that almost all must be in infinitely many of the events An :
Lemma 2.2.7 (Borel-Cantelli II). Suppose An F are mutually independent
P
and
P(An ) = . Then, necessarily P(An i.o.) = 1.
n=1
Proof. Fix 0 < m < n < . Use the mutual independence of the events A
and the inequality 1 x ex for x 0, to deduce that
n
n
n
\

Y
Y
P
P(Ac ) =
(1 P(A ))
Ac =
=m
As n , the set
n
T
=m
=m
n
Y
=m
eP(A ) = exp(
=m
n
X
P(A )) .
=m
Ac shrinks. With the series in the exponent diverging, by
continuity from above of the probability measure P() we see that for any m,

\
X
c
A exp(
P(A )) = 0 .
P
=m
=m
S
Take the complement to see that P(Bm ) = 1 for Bm =
=m A and all m. Since
Bm {An i.o. } when m , it follows by continuity from above of P() that
P(An i.o.) = lim P(Bm ) = 1 ,
m
as stated.
As an immediate corollary of the two Borel-Cantelli lemmas, we observe yet another 0-1 law.
Corollary 2.2.8. If An F are P-mutually independent then P(An i.o.) is
either 0 or 1. In other words, for any given sequence of mutually independent
events, either almost all outcomes are in infinitely many of these events, or almost
all outcomes are in finitely many of them.
The Kochen-Stone lemma, left as an exercise, generalizes Borel-Cantelli II to situations lacking independence.
Exercise 2.2.9. Suppose Ak are events on the same probability space such that
P
k P(Ak ) = and
lim sup
n
n
X
k=1
2
P(Ak ) /
1j,kn

P(Aj Ak ) = > 0 .
Prove that then P(An i.o. ) .

P
Hint: Consider part (a) of Exercise 1.3.21 for Yn = kn IAk and an = EYn .
80
2.2.2. Applications. In the sequel we explore various applications of the two

Borel-Cantelli lemmas. In doing so, unless explicitly stated otherwise, all events
and random variables are defined on the same probability space.
We know that the convergence a.s. of Xn to X implies the convergence in probability of Xn to X , but not vice verse (see Exercise 1.3.23 and Example 1.3.25).
As our first application of Borel-Cantelli I, we refine the relation between these
two modes of convergence, showing that convergence in probability is equivalent to
convergence almost surely along sub-sequences.
p
Theorem 2.2.10. Xn X if and only if for every subsequence m 7 Xn(m)

a.s.
there exists a further sub-subsequence Xn(mk ) such that Xn(mk ) X as k .
We start the proof of this theorem with a simple analysis lemma.
Lemma 2.2.11. Let yn be a sequence in a topological space. If every subsequence
yn(m) has a further sub-subsequence yn(mk ) that converges to y, then yn y.
Proof. If yn does not converge to y, then there exists an open set G containing
y and a subsequence yn(m) such that yn(m)
/ G for all m. But clearly, then we
cannot find a further subsequence of yn(m) that converges to y.

L1
Remark. Applying Lemma 2.2.11 to yn = E|Xn X | we deduce that Xn X

if and only if any subsequence n(m) has a further sub-subsequence n(mk ) such that
L1
Xn(mk ) X as k .
p
Proof of Theorem 2.2.10. First, we show sufficiency, assuming Xn X .

Fix a subsequence n(m) and k 0. By the definition of convergence in probability,

there exists a sub-subsequence n(mk )
such that P |Xn(mk ) X | > k

2k . Call P
this sequence of events Ak = : |Xn(mk ) () X ()| > k . Then
the series k P(Ak ) converges. Therefore, by Borel-Cantelli I, P(lim sup Ak ) =
0. For any
/ lim sup Ak there are only finitely many values of k such that
|Xn(mk ) X | > k , or alternatively, |Xn(mk ) X | k for all k large enough.
Since k 0, it follows that Xn(mk ) () X () when
/ lim sup Ak , that is,
with probability one.
Conversely, fix > 0. Let yn = P(|Xn X | > ). By assumption, for every subsequence n(m) there exists a further subsequence n(mk ) so that Xn(mk ) converges
to X almost surely, hence in probability, and in particular, yn(mk ) 0. Applying
Lemma 2.2.11 we deduce that yn 0, and since > 0 is arbitrary it follows that
p

Xn X .
It is not hard to check that convergence almost surely is invariant under application
of an a.s. continuous mapping.
Exercise 2.2.12. Let g : R 7 R be a Borel function and denote by Dg its set of
a.s.
discontinuities. Show that if Xn X finite valued, and P(X Dg ) = 0, then
a.s.
g(Xn ) g(X ) as well (recall Exercise 1.2.28 that Dg B). This applies for a
continuous function g in which case Dg = .
A direct consequence of Theorem 2.2.10 is that convergence in probability is also
preserved under an a.s. continuous mapping (and if the mapping is also bounded,
we even get L1 convergence).
81
Corollary 2.2.13. Suppose Xn X , g is a Borel function and P(X Dg ) =

L1
0. Then, g(Xn ) g(X ). If in addition g is bounded, then g(Xn ) g(X ) (and

Eg(Xn ) Eg(X )).
Proof. Fix a subsequence Xn(m) . By Theorem 2.2.10 there exists a subsequence Xn(mk ) such that P(A) = 1 for A = { : Xn(mk ) () X () as k }.
Let B = { : X ()
/ Dg }, noting that by assumption P(B) = 1. For any
A B we have g(Xn(mk ) ()) g(X ()) by the continuity of g outside Dg .
a.s.
Therefore, g(Xn(mk ) ) g(X ). Now apply Theorem 2.2.10 in the reverse direction: For any subsequence, we have just constructed a further subsequence with
p
convergence a.s., hence g(Xn ) g(X ).
Finally, if g is bounded, then the collection {g(Xn )} is U.I. yielding, by Vitalis
convergence theorem, its convergence in L1 (and hence that Eg(Xn ) Eg(X )).

You are next to extend the scope of Theorem 2.2.10 and the continuous mapping
of Corollary 2.2.13 to random variables taking values in a separable metric space.
Exercise 2.2.14. Recall the definition of convergence in probability in a separable
metric space (S, ) as in Remark 1.3.24.
(a) Extend the proof of Theorem 2.2.10 to apply for any (S, BS )-valued random variables {Xn , n } (and in particular for R-valued variables).
(b) Denote by Dg the set of discontinuities of a Borel measurable g : S 7
R (defined similarly to Exercise 1.2.28, where real-valued functions are
p
considered). Suppose Xn X and P(X Dg ) = 0. Show that then
L1
g(Xn ) g(X ) and if in addition g is bounded, then also g(Xn )

g(X ).
The following result in analysis is obtained by combining the continuous mapping

of Corollary 2.2.13 with the weak law of large numbers.
Exercise 2.2.15 (Inverting Laplace transforms). The LaplaceR transform of
a bounded, continuous function h(x) on [0, ) is the function Lh (s) = 0 esx h(x)dx
on (0, ).
(a) Show that for any s > 0 and positive integer k,
Z
k (k1)
(s)
sk xk1
k1 s Lh
(1)
=
esx
h(x)dx = E[h(Wk )] ,
(k 1)!
(k 1)!
0
(k1)
where Lh
() denotes the (k 1)-th derivative of the function Lh () and
Wk has the gamma density with parameters k and s.
(b) Recall Exercise
Pn 1.4.46 that for s = n/y the law of Wn coincides with the
law of n1 i=1 Ti where Ti 0 are i.i.d. random variables, each having
the exponential distribution of parameter 1/y (with ET1 = y and finite
moments of all order, c.f. Example 1.3.68). Deduce that the inversion
formula
h(y) = lim (1)n1
n
holds for any y > 0.
(n/y)n (n1)
L
(n/y) ,
(n 1)! h
82
The next application of Borel-Cantelli I provides our first strong law of large
numbers.
Proposition 2.2.16. Suppose E[Zn2 ] C for some C < and all n. Then,
a.s.
n1 Zn 0 as n .
Proof. Fixing > 0 let Ak = { : |k 1 Zk ()| > } for k = 1, 2, . . .. Then, by
Chebyshevs inequality and our assumption,
P(Ak ) = P({ : |Zk ()| k})
C
E(Zk2 )
2 k 2 .
2
(k)
P
Since k k 2 < , it follows by Borel Cantelli I that P(A ) = 0, where A =
{ : |k 1 Zk ()| > for infinitely many values of k}. Hence, for any fixed
> 0, with probability one k 1 |Zk ()| for all large enough k, that is,
lim supn n1 |Zn ()| a.s. Considering a sequence m 0 we conclude that
n1 Zn 0 for n and a.e. .

Exercise 2.2.17. Let Sn =
EX1 = 0 and EX14 < .
n
P
l=1
Xl , where {Xi } are i.i.d. random variables with
a.s.
(a) Show that n1 Sn 0.

Hint: Verify that Proposition 2.2.16 applies for Zn = n1 Sn2 .
a.s.
(b) Show that n1 Dn 0 where Dn denotes the number of distinct integers
among {k , k n} P
and {k } are i.i.d. integer valued random variables.
n
Hint: Dn 2M + k=1 I|k |M .
In contrast, here is an example where the empirical averages of integrable, zero

mean independent variables do not converge to zero. Of course, the trick is to have
non-identical distributions, with the bulk of the probability drifting to negative one.
Exercise 2.2.18. Suppose Xi are mutually independent random variables such
that P(Xn = n2 1) = 1 P(Xn = 1) = n2 for n = 1, 2, . . .. Show that
Pn
a.s.
EXn = 0, for all n, while n1 i=1 Xi 1 for n .
Next we have few other applications of Borel-Cantelli I, starting with some additional properties of convergence a.s.
Exercise 2.2.19. Show that for any R.V. Xn
a.s.
(a) Xn 0 if and only if P(|Xn | > i.o. ) = 0 for each > 0.

a.s.
(b) There exist non-random constants bn such that Xn /bn 0.
Exercise 2.2.20. Show that if Wn > 0 and EWn 1 for every n, then almost
surely,
lim sup n1 log Wn 0 .
n
Our next example demonstrates how Borel-Cantelli I is typically applied in the

study of the asymptotic growth of running maxima of random variables.
Example 2.2.21 (Head runs). Let {Xk , k Z} be a two-sided sequence of i.i.d.
{0, 1}-valued random variables, with P(X1 = 1) = P(X1 = 0) = 1/2. With m =
max{i : Xmi+1 = = Xm = 1} denoting the length of the run of 1s going
83
backwards from time m, we are interested in the asymptotics of the longest such
run during 1, 2, . . . , n, that is,
Ln = max{m : m = 1, . . . , n}
= max{m k : Xk+1 = = Xm = 1 for some m = 1, . . . , n} .
Noting that m + 1 has a geometric distribution of success probability p = 1/2, we
deduce by an application of Borel-Cantelli I that for each > 0, with probability
one, n (1 + ) log2 n for all n large enough. Hence, on the same set of probability
one, we have N = N () finite such that Ln max(LN , (1+) log2 n) for all n N .
Dividing by log2 n and considering n followed by k 0, this implies that
lim sup
n
Ln a.s.
1.
log2 n
For each fixed > 0 let An = {Ln < kn } for kn = [(1 ) log2 n]. Noting that
An
m
\n
Bic ,
i=1
for mn = [n/kn ] and the independent events Bi = {X(i1)kn +1 = = Xikn = 1},

yields a bound of the form P(An ) exp(nP
/(2 log2 n)) for all n large enough (c.f.
[Dur10, Example 2.3.3] for details). Since n P(An ) < , we have that
lim inf
n
Ln a.s.
1
log2 n
by yet another application of Borel-Cantelli I, followed by k 0. We thus conclude

that
Ln a.s.
1.
log2 n
The next exercise combines both Borel-Cantelli lemmas to provide the 0-1 law for
another problem about head runs.
Exercise 2.2.22. Let {Xk } be a sequence of i.i.d. {0, 1}-valued random variables,
with P(X1 = 1) = p and P(X1 = 0) = 1 p. Let Ak be the event that Xm = =
Xm+k1 = 1 for some 2k m 2k+1 k. Show that P(Ak i.o. ) = 1 if p 1/2
and P(Ak i.o. ) = 0 if p < 1/2.
Hint: When p 1/2 consider only m = 2k + (i 1)k for i = 0, . . . , [2k /k].
Here are a few direct applications of the second Borel-Cantelli lemma.
Exercise 2.2.23. Suppose that {Zk } are i.i.d. random variables such that P(Z1 =
z) < 1 for any z R.
(a) Show that P(Zk converges for k ) = 0.

(b) Determine the values of lim supn (Zn / log n) and lim inf n (Zn / log n)
in case Zk has the exponential distribution (of parameter = 1).
After deriving the classical bounds on the tail of the normal distribution, you
use both Borel-Cantelli lemmas in bounding the fluctuations of the sums of i.i.d.
standard normal variables.
Exercise 2.2.24. Let {Gi } be i.i.d. standard normal random variables.
84
(a) Show that for any x > 0,

1
(x
)e
x2 /2
ey
/2
dy x1 ex
/2
Many texts prove these estimates, for example see [Dur10, Theorem
1.2.3].
(b) Show that, with probability one,
Gn
lim sup
= 1.
2 log n
n
(c) Let Sn = G1 + + Gn . Recall that n1/2 Sn has the standard normal
distribution. Show that
p
P(|Sn | < 2 n log n, ev. ) = 1 .
Remark. Ignoring the dependence between the elements of the sequence Sk , the
bound in part (c) of the preceding exercise is not tight. The definite result here is
the law of the iterated logarithm (in short lil), which states that when the i.i.d.
summands are of zero mean and variance one,
(2.2.1)
Sn
= 1) = 1 .
P(lim sup
2n log log n
n
We defer the derivation of (2.2.1) to Theorem 9.2.29, building on a similar lil for
the Brownian motion (but, see [Bil95, Theorem 9.5] for a direct proof of (2.2.1),
using both Borel-Cantelli lemmas).
The next exercise relates explicit integrability conditions for i.i.d. random variables to the asymptotics of their running maxima.
Exercise 2.2.25. Consider possibly R-valued, i.i.d. random variables {Yi } and
their running maxima Mn = maxkn Yk .
(a) Using (2.3.4) if needed, show that P(|Yn | > n i.o. ) = 0 if and only if
E[|Y1 |] < .
a.s.
(b) Show that n1 Yn 0 if and only if E[|Y1 |] < .
a.s.
1
(c) Show that n Mn 0 if and only if E[(Y1 )+ ] < and P(Y1 > ) >
0.
p
(d) Show that n1 Mn 0 if and only if nP(Y1 > n) 0 and P(Y1 >
) > 0.
p
(e) Show that n1 Yn 0 if and only if P(|Y1 | < ) = 1.
In the following exercise, you combine Borel Cantelli I and the variance computation of Lemma 2.1.2 to improve upon Borel Cantelli II.
P
Exercise 2.2.26.
n=1 P(An ) = for pairwise independent events
Pn Suppose
{Ai }. Let Sn = i=1 IAi be the number of events occurring among the first n.
p
(a) Prove that Var(Sn ) E(Sn ) and deduce from it that Sn /E(Sn ) 1.
a.s.
(b) Applying Borel-Cantelli I show that Snk /E(Snk ) 1 as k , where
nk = inf{n : E(Sn ) k 2 }.
(c) Show that E(Snk+1 )/E(Snk ) 1 and since n 7 Sn is non-decreasing,
a.s.
deduce that Sn /E(Sn ) 1.
2.3. STRONG LAW OF LARGE NUMBERS
85
Remark. Borel-Cantelli II is the a.s. convergence Sn for n , which is

a consequence of part (c) of the preceding exercise (since ESn ).
We conclude this section with an example in which the asymptotic rate of growth
of random variables of interest is obtained by an application of Exercise 2.2.26.
Example 2.2.27 (Record values). Let {Xi } be a sequence of i.i.d. random
variables with a continuous distribution function FX (x). The event Ak = {Xk >
Xj , j = 1, . . . , k 1} represents the occurrence of a record at the k instance (for
example, think of Xk as an athletes kth distance jump). We are interested in the
n
P
IAi of record events during the first n instances.
asymptotics of the count Rn =
i=1
Because of the continuity of FX we know that a.s. the values of Xi , i = 1, 2, . . . are

distinct. Further, rearranging the random variables X1 , X2 , . . . , Xn in a decreasing
order induces a random permutation n on {1, 2, . . . , n}, where all n! possible permutations are equally likely. From this it follows that P(Ak ) = P(k (k) = 1) = 1/k,
and though definitely not obvious at first sight, the events Ak are mutually independent (see [Dur10, Example 2.3.2] for details). So, ERn = log n + n where n is
a.s.
between zero and one, and from Exercise 2.2.26 we deduce that (log n)1 Rn 1
as n . Note that this result is independent of the law of X, as long as the
distribution function FX is continuous.
2.3. Strong law of large numbers
In Corollary 2.1.14 we got the classical weak law of large
Pn numbers, namely, the
convergence in probability of the empirical averages n1 i=1 Xi of i.i.d. integrable
random variables Xi to the mean EX1 . Assuming in addition that EX14 < , you
used Borel-Cantelli I in Exercise 2.2.17 en-route to the corresponding strong law of
large numbers, that is, replacing the convergence in probability with the stronger
notion of convergence almost surely.
We provide here two approaches to the strong law of large numbers, both of which
get rid of the unnecessary finite moment assumptions. Subsection 2.3.1 follows
Etemadis (1981) direct proof of this result via the subsequence method. Subsection
2.3.2 deals in a more systematic way with the convergence of random series, yielding
the strong law of large numbers as one of its consequences.
2.3.1. The subsequence method. Etemadis key observation is that it essentially suffices to consider non-negative Xi , for which upon proving the a.s. convergence along a not too sparse subsequence nl , the
P interpolation to the whole
sequence can be done by the monotonicity of n 7 n Xi . This is an example of
a general approach to a.s. convergence, called the subsequence method, which you
have already encountered in Exercise 2.2.26.
We thus start with the strong law for integrable, non-negative variables.
Pn
Proposition 2.3.1. Let Sn =
i=1 Xi for non-negative, pairwise independent
a.s.
and identically distributed, integrable random variables {Xi }. Then, n1 Sn
EX1 as n .
Proof. The proof progresses along the themes of Section 2.1,
Pnstarting with
the truncation X k = Xk I|Xk |k and its corresponding sums S n = i=1 X i .
86
Since {Xi } are identically distributed and x 7 P(|X1 | > x) is non-increasing, we

have that
Z
X
X
P(Xk 6= X k ) =
P(|X1 | > k)
P(|X1 | > x)dx = E|X1 | <
k=1
k=1
(see part (a) of Lemma 1.4.31 for the rightmost identity and recall our assumption
that X1 is integrable). Thus, by Borel-Cantelli I, with probability one, Xk () =
X k () for all but finitely many ks, in which case necessarily supn |Sn () S n ()|
a.s.
is finite. This shows that n1 (Sn S n ) 0, whereby it suffices to prove that
a.s.
n1 S n EX1 .
To this end, we next show that it suffices to prove the following lemma about
almost sure convergence of S n along suitably chosen subsequences.
Lemma 2.3.2. Fixing > 1 let nl = [l ]. Under the conditions of the proposition,
a.s.
n1
l (S nl ES nl ) 0 as l .
By dominated convergence, E[X1 I|X1 |k ] EX1 as k , and consequently, as
n ,
n
n
1
1X
1X
ES n =
EX k =
E[X1 I|X1 |k ] EX1
n
n
n
k=1
k=1
(we have used here the consistency of Cesaro averages, c.f. Exercise 1.3.52 for an
a.s.
integral version). Thus, assuming that Lemma 2.3.2 holds, we have that n1
l S nl
EX1 when l , for each > 1.
We complete the proof of the proposition by interpolating from the subsequences
nl = [l ] to the whole sequence. To this end, fix > 1. Since n 7 S n is nondecreasing, we have for all and any n [nl , nl+1 ],
S n ()
nl S nl ()
nl+1 S nl+1 ()
nl+1 nl
n
nl
nl+1
With nl /nl+1 1/ for l , the a.s. convergence of m1 S m along the subsequence m = nl implies that the event
1
S n ()
S n ()
EX1 lim inf
lim sup
EX1 } ,
n
n
n
n
has
we deduce that the event B :=
T probability one. Consequently, taking m 1,
1
also
has
probability
one,
and
further,
n
A
S n () EX1 for each B.
m
m
a.s.
We thus deduce that n1 S n EX1 , as needed to complete the proof of the
proposition.

A := { :
Remark. The monotonicity of certain random variables (here n 7 S n ), is crucial

to the successful application of the subsequence method. The subsequence nl for
which we need a direct proof of convergence is completely determined by the scaling
function bn1 applied to this monotone sequence (here bn = n); we need bnl+1 /bnl
, which should be arbitrarily close to 1. For example, same subsequences nl = [l ]
are to be used whenever bn is roughly of a polynomial growth in n, while even
nl = (l!)c would work in case bn = log n.
Likewise, the truncation level is determined by the highest moment of the basic
variables which is assumed to be finite. For example, we can take X k = Xk I|Xk |kp
for any p > 0 such that E|X1 |1/p < .
87
Proof of Lemma 2.3.2. Note that E[X k ] is non-decreasing in k. Further,

X k are pairwise independent, hence uncorrelated, so by Lemma 2.1.2,
n
n
X
X
2
2
Var(S n ) =
Var(X k )
E[X k ] nE[X n ] = nE[X12 I|X1 |n ] .
k=1
k=1
Combining this with Chebychevs inequality yield the bound
P(|S n ES n | n) (n)2 Var(S n ) 2 n1 E[X12 I|X1 |n ] ,

for any > 0. Applying Borel-Cantelli I for the events Al = {|S nl ES nl | nl },
followed by m 0, we get the a.s. convergence to zero of n1 |S n ES n | along any
subsequence nl for which
X
X
2
2
n1
E[X
I
]
=
E[X
n1
|X
|n
1
1
1
l
l
l I|X1 |nl ] <
l=1
l=1
(the latter identity is a special case of Exercise 1.3.40). Since E|X1 | < , it thus
suffices to show that for nl = [l ] and any x > 0,
X
1
(2.3.1)
u(x) :=
n1
,
l Ixnl cx
l=1
where c = 2/( 1) < . To establish (2.3.1) fix > 1 and x > 0, setting
L = min{l 1 : nl x}. Then, L x, and since [y] y/2 for all y 1,
X
X
u(x) =
n1
2
l = cL cx1 .
l
l=L
l=L
So, we have established (2.3.1) and hence completed the proof of the lemma.
As already promised, it is not hard to extend the scope of the strong law of large
numbers beyond integrable and non-negative random variables.
Pn
Theorem 2.3.3 (Strong law of large numbers). Let Sn =
i=1 Xi for
pairwise independent and identically distributed random variables {Xi }, such that
a.s.
either E[(X1 )+ ] is finite or E[(X1 ) ] is finite. Then, n1 Sn EX1 as n .
Proof. First consider non-negative Xi . The case of EX1 < has already
Pn
(m)
(m)
been dealt with in Proposition 2.3.1. In case EX1 = , consider Sn = i=1 Xi
for the bounded, non-negative, pairwise independent and identically distributed
(m)
= min(Xi , m) Xi . Since Proposition 2.3.1 applies for
random variables Xi
(m)
{Xi }, it follows that a.s. for any fixed m < ,
(2.3.2)
(m)
lim inf n1 Sn lim inf n1 Sn(m) = EX1

n
= E min(X1 , m) .
Taking m , by monotone convergence E min(X1 , m) EX1 = , so (2.3.2)

results with n1 Sn a.s.
Turning to the general case, we have the decomposition Xi = (Xi )+ (Xi ) of
each random variable to its positive and negative parts, with
n
n
X
X
(Xi )
(Xi )+ n1
(2.3.3)
n1 Sn = n1
i=1
i=1
Since (Xi )+ are non-negative, pairwise independent and identically distributed,

Pn
a.s.
it follows that n1 i=1 (Xi )+ E[(X1 )+ ] as n . For the same reason,
88
Pn
a.s.
also n1 i=1 (Xi ) E[(X1 ) ]. Our assumption that either E[(X1 )+ ] < or
E[(X1 ) ] < implies that EX1 = E[(X1 )+ ] E[(X1 ) ] is well defined, and in
view of (2.3.3) we have the stated a.s. convergence of n1 Sn to EX1 .

Exercise 2.3.4. You are to prove now a converse to the strong law of large numbers (for a more general result, due to Feller (1946), see [Dur10, Theorem 2.5.9]).
(a) Let YPdenote the integer part of a random variable Z 0. Show that
Y =
n=1 I{Zn} , and deduce that
(2.3.4)
n=1
P(Z n) EZ 1 +
n=1
P(Z n) .
(b) Suppose {Xi } are i.i.d R.V.s with E[|X1 | ] = for some > 0. Show
that for any k > 0,
n=1
P(|Xn | > kn1/ ) = ,
and deduce that a.s. lim supn n1/ |Xn | = .

(c) Conclude that if Sn = X1 + X2 + + Xn , then
lim sup n1/ |Sn | = ,
a.s.
We provide next two classical applications of the strong law of large numbers, the
first of which deals with the large sample asymptotics of the empirical distribution
function.
Example 2.3.5 (Empirical distribution function). Let
Fn (x) = n1
n
X
I(,x] (Xi ) ,
i=1
denote the observed fraction of values among the first n variables of the sequence
{Xi } which do not exceed x. The functions Fn () are thus called the empirical
distribution functions of this sequence.
For i.i.d. {Xi } with distribution function FX our next result improves the strong
law of large numbers by showing that Fn converges uniformly to FX as n .
Theorem 2.3.6 (Glivenko-Cantelli). For i.i.d. {Xi } with arbitrary distribution function FX , as n ,
a.s.
Dn = sup |Fn (x) FX (x)| 0 .

xR
Remark. While outside our scope, we note in passing the Dvoretzky-KieferWolfowitz inequality that P(Dn > ) 2 exp(2n2 ) for any n and all > 0,
quantifying the rate of convergence of Dn to zero (see [DKW56], or [Mas90] for
the optimal pre-exponential constant).
Proof. By the right continuity of both x 7 Fn (x) and x 7 FX (x) (c.f.
Theorem 1.2.37), the value of Dn is unchanged when the supremum over x R is
replaced by the one over x Q (the rational numbers). In particular, this shows
that each Dn is a random variable (c.f. Theorem 1.2.22).
89
Applying the strong law of large numbers for the i.i.d. non-negative I(,x] (Xi )
a.s.
whose expectation is FX (x), we deduce that Fn (x) FX (x) for each fixed nonrandom x R. Similarly, considering the strong law of large numbers for the i.i.d.
a.s.
non-negative I(,x) (Xi ) whose expectation is FX (x ), we have that Fn (x )
FX (x ) for each fixed non-random x R. Consequently, for any fixed l < and
x1,l , . . . , xl,l we have that
l
k=1
k=1
a.s.
Dn,l = max(max |Fn (xk,l ) FX (xk,l )|, max |Fn (x

k,l ) FX (xk,l )|) 0 ,
as n . Choosing xk,l = inf{x : FX (x) k/(l + 1)}, we get out of the

monotonicity of x 7 Fn (x) and x 7 FX (x) that Dn Dn,l + l1 (c.f. [Bil95,
Proof of Theorem 20.6]). Therefore, taking n followed by l completes
the proof of the theorem.

We turn to our second example, which is about counting processes.
Example 2.3.7 (Renewal
theory). Let {i } be i.i.d. positive, finite random
Pn
variables and Tn = k=1 k . Here Tn is interpreted as the time of the n-th occurrence of a given event, with k representing the length of the time interval between
the (k 1) occurrence and that of the k-th such occurrence. Associated with Tn is
the dual process Nt = sup{n : Tn t} counting the number of occurrences during
the time interval [0, t]. In the next exercise you are to derive the strong law for the
large t asymptotics of t1 Nt .
Exercise 2.3.8. Consider the setting of Example 2.3.7.
a.s.
(a) By the strong law of large numbers argue that n1 Tn E1 . Then,
a.s.
1
= 0, deduce that t1 Nt 1/E1 for t .
adopting the convention
Hint: From the definition of Nt it follows that TNt t < TNt +1 for all
t 0.
a.s.
(b) Show that t1 Nt 1/E2 as t , even if the law of 1 is different
from that of the i.i.d. {i , i 2}.
Here is a strengthening of the preceding result to convergence in L1 .
Exercise 2.3.9. In the context of Example 2.3.7 fix > 0 such that P(1 > ) >
Pn
ek for the i.i.d. random variables ei = I{i >} . Note that
and let Ten =
k=1
et = sup{n : Ten t}.
Ten Tn and consequently Nt N
et2 < .
(a) Show that lim supt t2 EN
1
(b) Deduce that {t Nt : t 1} is uniformly integrable (see Exercise 1.3.54),
and conclude that t1 ENt 1/E1 when t .
The next exercise deals with an elaboration over Example 2.3.7.
Exercise 2.3.10. For i = 1, 2, . . . the ith light bulb burns for an amount of time i
and then remains burned out for time si before being replaced by the (i + 1)th bulb.
Let Rt denote the fraction of time during [0, t] in which we have a working light.
Assuming that the two sequences {i } and {si } are independent, each consisting of
a.s.
i.i.d. positive and integrable random variables, show that Rt E1 /(E1 + Es1 ).
Here is another exercise, dealing with sampling at times of heads in independent
fair coin tosses, from a non-random bounded sequence of weights v(l), the averages
of which converge.
90
Exercise 2.3.11. For a sequence {Bi } of i.i.d. Bernoulli random variables of

parameter p = 1/2, let Tn be the time that the corresponding partial sums reach
Pk
level n. That is, Tn = inf{k : i=1 Bi n}, for n = 1, 2, . . ..
a.s.
(a) Show that n1 Tn 2 as n .

Pk
a.s.
(b) Given non-negative, non-random {v(k)} show that k 1 i=1 v(Ti ) s
P
a.s.
n
as k , for some non-random s, if and only if n1 l=1 v(l)Bl s/2
as n .
Pn
Pn
(c) Deduce that if n1 l=1 v(l)2 is bounded and n1 l=1 v(l) s as n
Pk
a.s.
, then k 1 i=1 v(Ti ) s as k .
Pn
Hint: For part (c) consider first the limit of n1 l=1 v(l)(Bl 0.5) as n .
We proceed with a few additional applications of the strong law of large numbers,
first to a problem of universal hypothesis testing, then an application involving
stochastic geometry, and finally one motivated by investment science.
Exercise 2.3.12. Consider i.i.d. [0, 1]-valued random variables {Xk }.
(a) Find Borel measurable functions fn : [0, 1]n 7 {0, 1}, which are indea.s.
pendent of the law of Xk , such that fn (X1 , X2 , . . . , Xn ) 0 whenever
a.s.
EX1 < 1/2 and fn (X1 , X2 , . . . , Xn ) 1 whenever EX1 > 1/2.
a.s.
(b) Modify your answer to assure that fn (X1 , X2 , . . . , Xn ) 1 also in case
EX1 = 1/2.
Exercise 2.3.13. Let {Un } be i.i.d. random vectors, each uniformly distributed
on the unit ball {u R2 : |u| 1}. Consider the R2 -valued random vectors
Xn = |Xn1 |Un , n = 1, 2, . . . starting at a non-random, non-zero vector X0 (that is,
each point is uniformly chosen in a ball centered at the origin and whose radius is the
a.s.
distance from the origin to the previously chosen point). Show that n1 log |Xn |
1/2 as n .
Exercise 2.3.14. Let {Vn } be i.i.d. non-negative random variables. Fixing r > 0
and q (0, 1], consider the sequence W0 = 1 and Wn = (qr + (1 q)Vn )Wn1 ,
n = 1, 2, . . .. A motivating example is of Wn recording the relative growth of a
portfolio where a constant fraction q of ones wealth is re-invested each year in a
risk-less asset that grows by r per year, with the remainder re-invested in a risky
asset whose annual growth factors are the random Vn .
a.s.
(a) Show that n1 log Wn w(q), for w(q) = E log(qr + (1 q)V1 ).

(b) Show that q 7 w(q) is concave on (0, 1].
(c) Using Jensens inequality show that w(q) w(1) in case EV1 r. Further, show that if EV11 r1 , then the almost sure convergence applies
also for q = 0 and that w(q) w(0).
(d) Assuming that EV12 < and EV12 < show that sup{w(q) : q [0, 1]}
is finite, and further that the maximum of w(q) is obtained at some q
(0, 1) when EV1 > r > 1/EV11 . Interpret your results in terms of the
preceding investment example.
Hint: Consider small q > 0 and small 1q > 0 and recall that log(1+x) xx2 /2
for any x 0.
We conclude this subsection with another example where an almost sure convergence is derived by the subsequence method.
91
Exercise 2.3.15. Show that almost surely lim supn log Zn / log EZn 1 for
a.s.
any positive, non-decreasing random variables Zn such that Zn .
2.3.2. Convergence of random series. A second approach to the strong
law of large numbers is based on studying the convergence of random series. The
key tool in this approach is Kolmogorovs maximal inequality, which we prove next.
Proposition 2.3.16 (Kolmogorovs maximal inequality). The random variables Y1 , . . . , Yn are mutually independent, with EYl2 < and EYl = 0 for l =
1, . . . , n. Then, for Zk = Y1 + + Yk and any z > 0,
z 2 P( max |Zk | z) Var(Zn ) .
(2.3.5)
1kn
Remark. Chebyshevs inequality gives only z 2 P(|Zn | z) Var(Zn ) which is

significantly weaker and insufficient for our current goals.
Proof. Fixing z > 0 we decompose the event A = {max1kn |Zk | z}
according to the minimal index k for which |Zk | z. That is, A is the union of the
disjoint events Ak = {|Zk | z > |Zj |, j = 1, . . . , k 1} over 1 k n. Obviously,
z 2 P(A) =
(2.3.6)
n
X
k=1
since
Zk2
z 2 P(Ak )
n
X
E[Zk2 ; Ak ] ,
k=1
z on Ak . Further, EZn = 0 and Ak are disjoint, so

Var(Zn ) = EZn2
(2.3.7)
n
X
E[Zn2 ; Ak ] .
k=1
It suffices to show that E[(Zn Zk )Zk ; Ak ] = 0 for any 1 k n, since then

E[Zn2 ; Ak ] E[Zk2 ; Ak ] = E[(Zn Zk )2 ; Ak ] + 2E[(Zn Zk )Zk ; Ak ]
= E[(Zn Zk )2 ; Ak ] 0 ,
and (2.3.5) follows by comparing (2.3.6) and (2.3.7). Since Zk IAk can be represented
as a non-random Borel function of (Y1 , . . . , Yk ), it follows that Zk IAk is measurable
on (Y1 , . . . , Yk ). Consequently, for fixed k and l > k the variables Yl and Zk IAk
are independent, hence uncorrelated. Further EYl = 0, so
n
n
X
X
E[(Zn Zk )Zk ; Ak ] =
E[Yl Zk IAk ] =
E(Yl )E(Zk IAk ) = 0 ,
l=k+1
l=k+1
completing the proof of Kolmogorovs inequality.
Equipped with Kolmogorovs inequality, we provide an easy to check sufficient

condition for the convergence of random series of independent R.V.
Theorem 2.3.17. Suppose
{Xi } are independent random variables with P
Var(Xi ) <
P
w.p.1.
the
random
series
and EXi = 0. If n Var(Xn ) < then
n Xn ()
P
converges (that is, the sequence Sn () = nk=1 Xk () has a finite limit S ()).
Proof. Applying Kolmogorovs maximal inequality for the independent variables Yl = Xl+r , we have that for any > 0 and positive integers r and n,
P( max
rkr+n
|Sk Sr | )
Var(Sr+n Sr ) =
r+n
X
l=r+1
Var(Xl ) .
92
Taking n , we get by continuity from below of P that
X
P(sup |Sk Sr | ) 2
Var(Xl )
kr
l=r+1
P
By our assumption that n Var(Xn ) is finite, it follows that l>r Var(Xl ) 0 as
r . Hence, if we let Tr = supn,mr |Sn Sm |, then for any > 0,
P
P(Tr 2) P(sup |Sk Sr | ) 0

kr
as r . Further, r 7 Tr () is non-increasing, hence,
P(lim sup TM 2) = P(inf TM 2) P(Tr 2) 0 .

M
a.s.
That is, TM () 0 for M . By definition, the convergence to zero of TM ()

is the statement that Sn () is a Cauchy sequence. Since every Cauchy sequence in
R converges to a finite limit, we have the stated a.s. convergence of Sn ().

We next provide some applications of Theorem 2.3.17.
P
Example 2.3.18. Considering non-random an such that n a2n < and independent Bernoulli variables Bn of parameter p = 1/2, Theorem 2.3.17 tellsPus that
P
Bn
an converges with probability one. That is, when the signs in n an
n (1)
are chosen onP
the toss of a fair coin, the series almost always converges (though
quite possibly n |an | = ).
Exercise 2.3.19. Consider the record events Ak of Example 2.2.27.

(a) Verify that the events Ak are mutually
independent with P(Ak ) = 1/k.
P
(b) Show that the random series n2 (IAn 1/n)/ log n converges almost
a.s.
surely and deduce that (log n)1 Rn 1 as n .
(c) Provide a counterexample to the preceding in case the distribution function FX (x) is not continuous.
The link between convergence of random series and the strong law of large numbers
is the following classical analysis lemma.
Lemma 2.3.20 (Kroneckers lemma). Consider
two sequences of real numbers
P
{xn } and {bn } where bn > 0 and bn . If n xn /bn converges, then sn /bn 0
for sn = x1 + + xn .
Pn
Proof. Let un =
k=1 (xk /bk ) which by assumption converges to a finite
limit denoted u . Setting u0 = b0 = 0, summation by parts yields the identity,
n
n
X
X
sn =
bk (uk uk1 ) = bn un
(bk bk1 )uk1 .
k=1
k=1
Since un u and bn , the Cesaro averages b1

n
converge to u . Consequently, sn /bn u u = 0.
Pn
k=1 (bk
bk1 )uk1 also

Theorem 2.3.17 provides an alternative proof for the strong law of large numbers
of Theorem 2.3.3 in case {Xi } are i.i.d. (that is, replacing pairwise independence
by mutual independence). Indeed, applying the same truncation scheme as in the
proof of Proposition 2.3.1, it suffices to prove the following alternative to Lemma
2.3.2.
Lemma 2.3.21. For integrable i.i.d. random variables {Xk }, let S m =

a.s.
and X k = Xk I|Xk |k . Then, n1 (S n ES n ) 0 as n .
93
Pm
k=1
Xk
Lemma 2.3.21, in contrast to Lemma 2.3.2, does not require the restriction to a
subsequence nl . Consequently, in this proof of the strong law there is no need for
an interpolation argument so it is carried directly for Xk , with no need to split each
variable to its positive and negative parts.
Proof of Lemma 2.3.21. We will shortly show that
X
(2.3.8)
k 2 Var(X k ) 2E|X1 | .
k=1
With X1 integrable, applying Theorem 2.3.17 for the independent variables Yk =

1
k
P (X k EX k ) this implies that for some A with P(A) = 1, the random series
Using Kroneckers lemma for bn = n and
n Yn () converges for all A. P
n
xn = X n () EX n we get that n1 k=1 (X k EX k ) 0 as n , for every
A, as stated.
The proof of (2.3.8) is similar to the computation employed in the proof of Lemma
2
2.3.2. That is, Var(X k ) EX k = EX12 I|X1 |k and k 2 2/(k(k + 1)), yielding
that
X
X
2
EX12 I|X1 |k = EX12 v(|X1 |) ,
k 2 Var(X k )
k(k + 1)
k=1
k=1
where for any x > 0,
X
v(x) = 2
k=x
X
1
1
2
1
=
=2
2x1 .
k(k + 1)
k
k+1
x
k=x
Consequently, EX12 v(|X1 |) 2E|X1 |, and (2.3.8) follows.
Many of the ingredients of this proof of the strong law of large numbers are also
relevant for solving the following exercise.
Exercise 2.3.22. Let cn be a bounded sequence of non-random constants, and
Pn
a.s.
{Xi } i.i.d. integrable R.V.-s of zero mean. Show that n1 k=1 ck Xk 0 for
n .
Next you find few exercises that illustrate how useful Kroneckers lemma is when
proving the strong law of large numbers in case of independent but not identically
distributed summands.
Pn
Exercise 2.3.23. Let Sn = k=1 Yk for independent random variables {Yi } such
a.s.
that Var(Yk ) 0 is fixed (this falls short of the law of the iterated logarithm of
(2.2.1), but each Yk is allowed here to have a different distribution).
Exercise 2.3.24. Suppose the independent random variables {Xi } are such that
Var(Xk ) pk < and EXk = 0 for k = 1, 2, . . ..
P
Pn
a.s.
1
(a) Show that if k pk <
k=1 kXk 0.
P then n
(b) Conversely, assuming k pk = , give an example of independent random variables {Xi }, such that Var(Xk ) pk , EXk = 0, for which almost
surely lim supn Xn () = 1.
94
(c) Show that the P

example you just gave is such that with probability one, the
n
sequence n1 k=1 kXk () does not converge to a finite limit.
Exercise 2.3.25. Consider independent, non-negative random variables Xn .

(a) Show that if
X
(2.3.9)
[P(Xn 1) + E(Xn IXn <1 )] <
n=1
P
w.p.1.
then the random series n Xn () converges
P
X
()
converges w.p.1. then
(b) Prove the converse, namely, that if
n n
(2.3.9) holds.
(c) Suppose Gn are mutually independent random variables, with Gn having
the normal distribution N (n , vn ). Show
P that2 w.p.1. the random series
P
2
G
()
converges
if
and
only
if
e
=
n (n + vn ) is finite.
n n
(d) Suppose n are mutually independent random variables, with n having
the exponentialP
distribution of parameter n > 0.PShow that w.p.1. the
random series n n () converges if and only if n 1/n is finite.
P
is finite if and
Hint: For
Pseries n an
Q part (b) recall that for any an [0, 1), the
only if n (1 an ) > 0. For part (c) let f (y) = n min((n + vn y)2 , 1) and
observe that if e = then f (y) + f (y) = for all y 6= 0.
You can now also show that for such strong law of large numbers (that is, with
independent but not identically distributed summands), it suffices to strengthen
the corresponding weak law (only) along the subsequence nr = 2r .
Pk
Exercise 2.3.26. Let Zk = j=1 Yj where Yj are mutually independent R.V.-s.
P
a.s.
(a) Fixing > 0 show that if 2r Z2r 0 then r P(|Z2r+1 Z2r | > 2r )
p
is finite and if m1 Zm 0 then maxm<k2m P(|Z2m Zk | m) 0.
(b) Adapting the proof of Kolmogorovs maximal inequality show that for any
n and z > 0,
P( max |Zk | 2z) min P(|Zn Zk | < z) P(|Zn | > z) .
1kn
1kn
a.s.
a.s.
(c) Deduce that if both m1 Zm 0 and 2r Z2r 0 then also n1 Zn 0.

Hint: For part (c) combine parts (a) and (b)P
with z = n, n = 2r and the mutually
independent Yj+n , 1 j n, to show that r P(2r Dr 2) is finite for Dr =
max2r <k2r+1 |Zk Z2r | and any fixed > 0.
Finally, here is an interesting property of non-negative random variables, regardless of their level of dependence.
P
p
Exercise 2.3.27. Suppose random variables Yk 0 are such that n1 nk= Yk
P
p
p
n
1. Show that then n1 maxnk=1 Yk 0, and conclude that nr k=1 Ykr 0, for
any fixed r > 1.
CHAPTER 3
Weak convergence, clt and Poisson approximation

After dealing in Chapter 2 with examples in which random variables converge
to non-random constants, we focus here on the more general theory of weak convergence, that is situations in which the laws of random variables converge to a
limiting law, typically of a non-constant random variable. To motivate this theory,
we start with Section 3.1 where we derive the celebrated Central Limit Theorem (in
short clt), the most widely used example of weak convergence. This is followed by
the exposition of the theory, to which Section 3.2 is devoted. Section 3.3 is about
the key tool of characteristic functions and their role in establishing convergence
results such as the clt. This tool is used in Section 3.4 to derive the Poisson approximation and provide an introduction to the Poisson process. In Section 3.5 we
generalize the characteristic function to the setting of random vectors and study
their properties while deriving the multivariate clt.
3.1. The Central Limit Theorem
We start this section with the property of the normal distribution that makes it
the likely limit for properly scaled sums of independent random variables. This is
followed by a bare-hands proof of the clt for triangular arrays in Subsection 3.1.1.
We then present in Subsection 3.1.2 some of the many examples and applications
of the clt.
Recall the normal distribution of mean R and variance v > 0, denoted hereafter N (, v), the density of which is
(3.1.1)
f (y) =
(y )2
1
exp(
).
2v
2v
As we show next, the normal distribution is preserved when the sum of independent
variables is considered (which is the main reason for its role as the limiting law for
the clt).
Lemma 3.1.1. Let Yn,k be mutually independent random
Pn variables, each having
the normal distribution N (n,k , vn,k
).
Then,
G
=
Yn,k has the normal
n
Pn
Pk=1
n
distribution N (n , vn ), with n = k=1 n,k and vn = k=1 vn,k .
Proof. Recall that Y has a N (, v) distribution if and only if Y has the

N (0, v) distribution. Therefore, we may and shall assume without loss of generality
that n,k = 0 for all k and n. Further, it suffices to prove the lemma for n = 2, as
the general case immediately follows by an induction argument. With n = 2 fixed,
we simplify our notations by omitting it everywhere. Next recall the formula of
Corollary 1.4.33 for the probability density function of G = Y1 + Y2 , which for Yi
95
96
3. WEAK CONVERGENCE, clt AND POISSON APPROXIMATION
of N (0, vi ) distribution, i = 1, 2, is
Z
1
1
(z y)2
y2
)
)dy .
fG (z) =
exp(
exp(
2v1
2v2
2v1
2v2
Comparing this with the formula of (3.1.1) for v = v1 + v2 , it just remains to show
that for any z R,
Z
y2
z2
(z y)2
1
(3.1.2)
1=
)dy ,
exp(
2v
2v1
2v2
2u
where u = v1 v2 /(v1 + v2 ). It is not hard to check that the argument of the exponential function in (3.1.2) is (y cz)2 /(2u) for c = v2 /(v1 + v2 ). Consequently,
(3.1.2) is merely the obvious fact that the N (cz, u) density function integrates to
one (as any density function should), no matter what the value of z is.

Considering Lemma 3.1.1 for Yn,k = (nv)1/2 (Yk ) and i.i.d. random variables
Yk , each having a normal distribution of mean and variance v, we see that n,k =
Pn
0 and vn,k = 1/n, so Gn = (nv)1/2 ( k=1 Yk n) has the standard N (0, 1)
distribution, regardless of n.
3.1.1. Lindebergs clt for triangular arrays. Our next proposition, the
P
celebrated clt, states that the distribution of Sbn = (nv)1/2 ( nk=1 Xk n) approaches the standard normal distribution in the limit n , even though Xk
may well be non-normal random variables.
Proposition 3.1.2 (Central Limit Theorem). Let
n

1 X
Xk n ,
Sbn =
nv
k=1
where {Xk } are i.i.d with v = Var(X1 ) (0, ) and = E(X1 ). Then,
Z b
y2
1
exp( )dy
for every b R .
(3.1.3)
lim P(Sbn b) =
n
2
2
As we have seen in the context of the weak law of large numbers, it pays to extend
the scope of consideration to triangular arrays in which the random variables Xn,k
are independent within each row, but not necessarily of identical distribution. This
is the context of Lindebergs clt, which we state next.
Pn
Theorem 3.1.3 (Lindebergs clt). Let Sbn =
k=1 Xn,k for P-mutually independent random variables Xn,k , k = 1, . . . , n, such that EXn,k = 0 for all k
and
n
X
2
vn =
EXn,k
1 as n .
k=1
Then, the conclusion (3.1.3) applies if for each > 0,
(3.1.4)
gn () =
n
X
k=1
2
E[Xn,k
; |Xn,k | ] 0
as n .
Note that the variables in different rows need not be independent of each other
and could even be defined on different probability spaces.
3.1. THE CENTRAL LIMIT THEOREM
97
Remark 3.1.4. Under the assumptions of Proposition 3.1.2 the variables Xn,k =
(nv)1/2 (Xk ) are mutually independent and such that
EXn,k = (nv)1/2 (EXk ) = 0,
vn =
n
X
2
EXn,k
=
k=1
n
1 X
Var(Xk ) = 1 .
nv
k=1
Further, per fixed n these Xn,k are identically distributed, so

2
gn () = nE[Xn,1
; |Xn,1 | ] = v 1 E[(X1 )2 I|X1 |nv ] .
For each > 0 the sequence (X1 )2 I|X1 |nv converges a.s. to zero for
n and is dominated by the integrable random variable (X1 )2 . Thus, by
dominated convergence, gn () 0 as n . We conclude that all assumptions
of Theorem 3.1.3 are satisfied for this choice of Xn,k , hence Proposition 3.1.2 is a
special instance of Lindebergs clt, to which we turn our attention next.
2
Let rn = max{ vn,k : k = 1, . . . , n} for vn,k = EXn,k
. Since for every n, k and
> 0,
2
2
2
vn,k = EXn,k
= E[Xn,k
; |Xn,k | < ] + E[Xn,k
; |Xn,k | ] 2 + gn () ,
it follows that
rn2 2 + gn ()
n, > 0 ,
hence Lindebergs condition (3.1.4) implies that rn 0 as n .

Remark. Lindeberg proved Theorem 3.1.3, introducing the condition (3.1.4).
Later, Feller proved that (3.1.3) plus rn 0 implies that Lindebergs condition
holds. Together, these two results are known as the Feller-Lindeberg Theorem.
We see that the variables Xn,k are of uniformly small variance for large n. So,
considering independent random variables Yn,k that are also independent of the
Xn,k and such that each Yn,k has a N (0, vn,k ) distribution, for a smooth function
h() one may control |Eh(Sbn ) Eh(Gn )| by a Taylor expansion upon successively
replacing the Xn,k by Yn,k . This indeed is the outline of Lindebergs proof, whose
core is the following lemma.
Lemma 3.1.5. For h : R 7 R of continuous and uniformly bounded second and
third derivatives, Gn having the N (0, vn ) law, every n and > 0, we have that
r
n
vn kh k + gn ()kh k ,
+
|Eh(Sbn ) Eh(Gn )|
6
2
with kf k = supxR |f (x)| denoting the supremum norm.
D
Remark. Recall that Gn = n G for n = vn . So, assuming vn 1 and
Lindebergs condition which implies that rn 0 for n , it follows from the
lemma that |Eh(Sbn ) Eh(n G)| 0 as n . Further, |h(n x) h(x)|
|n 1||x|kh k , so taking the expectation with respect to the standard normal
law we see that |Eh(n G) Eh(G)| 0 if the first derivative of h is also uniformly
bounded. Hence,
(3.1.5)
lim Eh(Sbn ) = Eh(G) ,
98
for any continuous function h() of continuous and uniformly bounded first three
derivatives. This is actually all we need from Lemma 3.1.5 in order to prove Lindebergs clt. Further, as we show in Section 3.2, convergence in distribution as in
(3.1.3) is equivalent to (3.1.5) holding for all continuous, bounded functions h().
P
Proof of Lemma 3.1.5. Let Gn = nk=1 Yn,k for mutually independent Yn,k ,
distributed according to N (0, vn,k ), that are independent of {Xn,k }. Fixing n and
h, we simplify the notations by eliminating n, that is, we write Yk for Yn,k , and Xk
for Xn,k . To facilitate the proof define the mixed sums
Ul =
l1
X
Xk +
k=1
n
X
Yk ,
l = 1, . . . , n
k=l+1
Note the following identities

Gn = U1 + Y1 ,
Un + Xn = Sbn ,
Ul + Xl = Ul+1 + Yl+1 , l = 1, . . . , n 1,
which imply that,

(3.1.6)
|Eh(Gn ) Eh(Sbn )| = |Eh(U1 + Y1 ) Eh(Un + Xn )|
n
X
l ,
l=1
where l = |E[h(Ul + Yl ) h(Ul + Xl )]|, for l = 1, . . . , n. For any l and R,

consider the remainder term
Rl () = h(Ul + ) h(Ul ) h (Ul )
2
h (Ul )
2
in second order Taylors expansion of h() at Ul . By Taylors theorem, we have that

||3
,
6
2
|Rl ()| kh k || ,
|Rl ()| kh k
(from third order expansion)

(from second order expansion)
whence,
(3.1.7)
||3
|Rl ()| min kh k
, kh k ||2
6
Considering the expectation of the difference between the two identities,

Xl2
h (Ul ) + Rl (Xl ) ,
2
Y2
h(Ul + Yl ) = h(Ul ) + Yl h (Ul ) + l h (Ul ) + Rl (Yl ) ,
2
h(Ul + Xl ) = h(Ul ) + Xl h (Ul ) +
we get that

X2

Y2

l E[(Xl Yl )h (Ul )] + E ( l l )h (Ul ) + |E[Rl (Xl ) Rl (Yl )]| .
2
2
Recall that Xl and Yl are independent of Ul and chosen such that EXl = EYl and
EXl2 = EYl2 . As the first two terms in the bound on l vanish we have that
(3.1.8)
l E|Rl (Xl )| + E|Rl (Yl )| .
99
Further, utilizing (3.1.7),

E|Rl (Xl )| kh k E

|Xl |3
; |Xl | + kh k E[|Xl |2 ; |Xl | ]
6

kh k E[|Xl |2 ] + kh k E[Xl2 ; |Xl | ] .
6
P
Summing these bounds over l = 1, . . . , n, by our assumption that nl=1 EXl2 = vn
and the definition of gn (), we get that
n
X
(3.1.9)
l=1
E|Rl (Xl )|
vn kh k + gn ()kh k .
6
Recall that Yl / vn,l is a standard normal random variable, whose fourth moment
is 3 (see (1.3.18)). By monotonicity in q of the Lq -norms (c.f. Lemma 1.3.16), it
3/2
follows that E[|Yl / vn,l |3 ] 3, hence E|Yl |3 3vn,l 3rn vn,l . Utilizing once
Pn
more (3.1.7) and the fact that vn = l=1 vn,l , we arrive at
(3.1.10)
n
X
l=1
E|Rl (Yl )|
n
kh k X
rn
vn kh k .
E|Yl |3
6
2
l=1
Plugging (3.1.8)(3.1.10) into (3.1.6) completes the proof of the lemma.
In view of (3.1.5), Lindebergs clt builds on the following elementary lemma,

whereby we approximate the indicator function on (, b] by continuous, bounded
functions hk : R 7 R for each of which Lemma 3.1.5 applies.
Lemma 3.1.6. There exist h

k (x) of continuous and uniformly bounded first three
+
derivatives, such that 0 h
k (x) I(,b) (x) and 1 hk (x) I(,b] (x) as
k .
Proof. There are many ways to prove this. Here is one which is from first principles, hence requires no analysis knowledge. The function : [0, 1] 7 [0, 1] given
R1
by (x) = 140 x u3 (1 u)3 du is monotone decreasing, with continuous derivatives
of all order, such that (0) = 1, (1) = 0 and whose first three derivatives at 0 and
at 1 are all zero. Its extension (x) = (min(x, 1)+ ) to a function on R that is one
for x 0 and zero for x 1 is thus non-increasing, with continuous and uniformly
bounded first three derivatives. It is easy to check that the translated and scaled
functions h+
k (x) = (k(x b)) and hk (x) = (k(x b) + 1) have all the claimed
properties.

Proof of Theorem 3.1.3. Applying (3.1.5) for h
k (), then taking k
we have by monotone convergence that
b
lim inf P(Sbn < b) lim E[h
k (Sn )] = E[hk (G)] FG (b ) .
n
h+
k (),
Similarly, considering
then taking k we have by bounded convergence
that
+
b
lim sup P(Sbn b) lim E[h+
k (Sn )] = E[hk (G)] FG (b) .
n
Since FG () is a continuous function we conclude that P(Sbn b) converges to

FG (b) = FG (b ), as n . This holds for every b R as claimed.
100
3.1.2. Applications of the clt. We start with the simpler, i.i.d. case. In
D
doing so, we use the notation Zn G when the analog of (3.1.3) holds for the
sequence {Zn }, that is P(Zn b) P(G b) as n for all b R (where G is
a standard normal variable).
Example 3.1.7 (Normal approximation of the Binomial). Consider i.i.d.
random variables {Bi }, each of whom is Bernoulli of parameter 0 < p < 1 (i.e.
P (B1 = 1) = 1 P (B1 = 0) = p). The sum Sn = B1 + + Bn has the Binomial
distribution of parameters (n, p), that is,

n k
P(Sn = k) =
p (1 p)nk , k = 0, . . . , n .
k
For example, if Bi indicates that the ith independent toss of the same coin lands on
a Head then Sn counts the total numbers of Heads in the first n tosses of the coin.
Recall that EB = p and Var(B) = p(1 p) (see Example 1.3.69), so the clt states
p
D
that (Sn np)/ np(1 p) G. It allows us to approximate, for all large enough
n, the typically non-computable weighted sums of binomial terms by integrals with
respect to the standard normal density.
Here is another example that is similar and almost as widely used.
Example 3.1.8 (Normal approximation of the Poisson distribution). It
is not hard to verify that the sum of two independent Poisson random variables has
the Poisson distribution, with a parameter which is the sum of the parameters of
the summands. Thus, by induction, if {Xi } are i.i.d. each of Poisson distribution
of parameter 1, then Nn = X1 + . . . + Xn has a Poisson distribution of parameter n. Since E(N1 ) = Var(N1 ) = 1 (see Example 1.3.69), the clt applies for
(Nn n)/n1/2 . This provides an approximation for the distribution function of the
Poisson variable N of parameter that is a large integer. To deal with non-integer
values = n + for some (0, 1), consider the mutually independent Poisson
D
D
variables Nn , N and N1 . Since N = Nn +N and Nn+1 = Nn +N +N1 , this
provides a monotone coupling , that is, a construction of the random variables Nn ,
N and Nn+1 on the same probability space, such that Nn N Nn+1 .
Because
of this monotonicity, for any
> 0 and all n n0 (b, ) the event
{(N )/ b}
is between {(Nn+1 (n + 1))/ n + 1 b } and {(Nn n)/ n b + }. Considering the limit as n followed by 0, it thus follows that the convergence
D
D
(Nn n)/n1/2 G implies also that (N )/1/2 G as . In words,
the normal distribution is a good approximation of a Poisson with large parameter.
In Theorem 2.3.3 we established the strong law of large numbers when the summands Xi are only pairwise independent. Unfortunately, as the next example shows,
pairwise independence is not good enough for the clt.
Example 3.1.9. Consider i.i.d. {i } such that P(i = 1) = P(i = 1) = 1/2
for all i. Set X1 = 1 and successively let X2k +j = Xj k+2 for j = 1, . . . , 2k and
k = 0, 1, . . .. Note that each Xl is a {1, 1}-valued variable, specifically, a product
of a different finite subset of i -s that corresponds to the positions of ones in the
binary representation of 2l 1 (with 1 for its least significant digit, 2 for the next
digit, etc.). Consequently, each Xl is of zero mean and if l 6= r then in EXl Xr
at least one of the i -s will appear exactly once, resulting with EXl Xr = 0, hence
with {Xl } being uncorrelated variables. Recall part (b) of Exercise 1.4.42, that such
101
variables are pairwise independent. Further, EXl = 0 and Xl {1, 1} mean that
P(Xl = 1) = P(X
P l = 1) = 1/2 are identically distributed. As for the zero mean
variables Sn = nj=1 Xj , we have arranged things such that S1 = 1 and for any
k0
k
S2k+1 =
2
X
(Xj + X2k +j ) =
2
X
Xj (1 + k+2 ) = S2k (1 + k+2 ) ,
j=1
j=1
Qk+1
hence S2k = 1 i=2 (1 + i ) for all k 1. In particular, S2k = 0 unless 2 = 3 =
. . . = k+1 = 1, an event of probability 2k . Thus, P(S2k 6= 0) = 2k and certainly
the clt result (3.1.3) does not hold along the subsequence n = 2k .
We turn next to applications of Lindebergs triangular array clt, starting with
the asymptotic of the count of record events till time n 1.
Exercise 3.1.10. Consider the count Rn of record events during the first n instances of i.i.d. R.V. with a continuous distribution function, as in Example 2.2.27.
Recall that Rn = B1 + +Bn for mutually independent Bernoulli random variables
{Bk } such that P(Bk = 1) = 1 P(Bk = 0) = k 1 .
(a) Check that bn / log n 1 where bn = Var(Rn ).
(b) Show that Lindebergs clt applies for Xn,k = (log n)1/2 (Bk k 1 ).
D
(c) Recall that |ERn log n| 1, and conclude that (Rn log n)/ log n
G.
Remark. Let Sn denote the symmetric group of permutations on {1, . . . , n}. For
s Sn and i {1, . . . , n}, denoting by Li (s) the smallest j n such that sj (i) = i,
we call {sj (i) : 1 j Li (s)} the cycle of s containing i. If each s Sn is equally
likely, then the law of the number Tn (s) of different cycles in s is the same as that
of Rn of Example 2.2.27 (for a proof see [Dur10, Example 2.2.4]). Consequently,
D
Exercise 3.1.10 also shows that in this setting (Tn log n)/ log n G.
Part (a) of the following exercise is a special case of Lindebergs clt, known also
as Lyapunovs theorem.
P
Exercise 3.1.11 (Lyapunovs theorem). Let Sn = nk=1 Xk for {Xk } mutually
independent such that vn = Var(Sn ) < .
(a) Show that if there exists q > 2 such that
lim vnq/2
n
1/2
vn (Sn
n
X
k=1
E(|Xk EXk |q ) = 0 ,
ESn ) G.
then
(b) Show that part (a) applies in case vn and E(|Xk EXk |q )
C(Var Xk )r for r = 1, some q > 2, C < and k = 1, 2, . . ..
(c) Provide an example where the conditions of part (b) hold with r = q/2
1/2
but vn (Sn ESn ) does not converge in distribution.
The next application of Lindebergs clt involves the use of truncation (which we
have already introduced in the context of the weak law of large numbers), to derive
the clt for normalized sums of certain i.i.d. random variables of infinite variance.
102
Proposition 3.1.12. Suppose {Xk } are i.i.d of symmetric distribution, that is

D
X1 = X1 (or P(X1 > x) = P(X1 < x) for all x) such that P(|X1 | > x) = x2
P
D
for x 1. Then, n 1log n nk=1 Xk G as n .
R
Remark 3.1.13. Note that Var(X1 ) = EX12 = 0 2xP(|X1 | > x)dx = (c.f.
part (a) of Lemma 1.4.31), so the usual clt of Proposition 3.1.2 does not apply here.
Indeed, the infinite P
variance of the summands results in a different normalization
of the sums Sn = nk=1 Xk that is tailored to the specific tail behavior of x 7
P(|X1 | > x).
Caution should be exercised here, since when P(|X1 | > x) = x for x > 1 and
some 0 < < 2, there is no way to approximate the distribution of (Sn an )/bn
by the standard normal distribution. Indeed, in this case bn = n1/ and the
approximation is by an -stable law (c.f. Definition 3.3.32 and Exercise 3.3.34).
Proof. We plan to apply Lindebergs
clt for the truncated random variables
n
log
n
and cn 1 are such that both cn /bn
where
b
=
Xn,k = b1
X
I
n
n k |Xk |cn
0 and cn / n . Indeed, for each n the variables Xn,k , k = 1, . . . , n, are i.i.d.
of bounded and symmetric distribution (since both the distribution of Xk and the
truncation function are symmetric). Consequently, EXn,k = 0 for all n and k.
Further, we have chosen bn such that
Z
n cn
n
2
2x[P(|X1 | > x) P(|X1 | > cn )]dx
vn = nEXn,1
= 2 EX12 I|X1 |cn = 2
bn
bn 0
Z cn
Z cn
Z
nh 1
2
2x i 2n log cn
= 2
2xdx +
dx
dx =
1
bn 0
x
c2n
b2n
1
0
as n . Finally, note that |Xn,k | cn /bn 0 as n , implying that
gn () = 0 for any > 0 and all n large enough, hence Lindebergs condition
D
trivially holds. We thus deduce from Lindebergs clt that n 1log n S n G as
P
n , where S n = nk=1 Xk I|Xk |cn is the sum of the truncated variables. We
have chosen the truncation level cn large enough to assure that
n
X
P(Sn 6= S n )
P(|Xk | > cn ) = nP(|X1 | > cn ) = nc2
n 0
k=1
for n , hence we may now conclude that
1
S
n log n n
G as claimed.
We conclude this section with Kolmogorovs three series theorem, the most definitive result on the convergence of random series.
Theorem 3.1.14 (Kolmogorovs three series theorem). Suppose {Xk } are
(c)
independent random variables. For non-random c > 0 let Xn = Xn I|Xn |c be the
corresponding truncated variables and consider the three series
X
X
X
(3.1.11)
P(|Xn | > c),
EXn(c) ,
Var(Xn(c) ).
n
P
Then, the random series n Xn converges a.s. if and only if for some c > 0 all
three series of (3.1.11) converge.
Remark. By convergence of a series we mean the existence of a finite limit to the

sum of its first m entries when m . Note that the theorem implies that if all
3.2. WEAK CONVERGENCE
103
three series of (3.1.11) converge for some c > 0, then they necessarily converge for
every c > 0.
Proof. We prove the sufficiency first, that is, assume that for some c > 0
all three series of (3.1.11) converge. By Theorem 2.3.17 and the finiteness of
P
P
(c)
(c)
(c)
n (Xn EXn ) converges a.s.
n Var(Xn ) it follows that the random series
P
P
(c)
(c)
Then, by our assumption that n EXn converges, also n Xn converges a.s.
(c)
Further, by assumption the sequence of probabilities P(Xn 6= Xn ) = P(|Xn | > c)
(c)
is summable, hence by Borel-Cantelli I, we have that a.s. Xn 6= Xn for at most
P
(c)
finitely manyP
ns. The convergence a.s. of n Xn thus results with the convergence a.s. of n Xn , as claimed.
We turn to prove
P the necessity of convergence of the three series in (3.1.11) to the
convergence ofP n Xn , which is where we use the clt. To this end, assume the
random series n Xn converges
P a.s. (to a finite limit) and fix an arbitrary constant
c > 0. The convergence of n Xn implies that |Xn | 0, hence a.s. |Xn | > c
for only finitely many ns. In view of the independence of these events and BorelCantelli II, necessarily the sequence P(|Xn | > c) is summable,
Pthat is, the series
P
P(|X
|
>
c)
converges.
Further,
the
convergence
a.s.
of
n
n Xn then results
n
P
(c)
with the a.s. convergence of n Xn .
P
(c)
Suppose now that the non-decreasing sequence vn = nk=1 Var(Xk ) is unbounded,
1/2 Pn
(c)
in which case the latter convergence implies that a.s. Tn = vn
k=1 Xk 0
when n . We further claim that in this case Lindebergs clt applies for
P
Sbn = nk=1 Xn,k , where
(c)
(c)
Xn,k = vn1/2 (Xk mk ),
(c)
(c)
and mk = EXk .
Indeed, per fixed n the variables Xn,k are mutually independent of zero mean
Pn
(c)
2
and such that k=1 EXn,k
= 1. Further, since |Xk | c and we assumed that
vn it follows that |Xn,k | 2c/ vn 0 as n , resulting with Lindebergs
condition holding (as gn () = 0 when > 2c/ vn , i.e. for all n large enough).
a.s.
D
Combining Lindebergs clt conclusion that Sbn G and Tn 0, we deduce that
D
1/2 Pn
(c)
(Sbn Tn ) G (c.f. Exercise 3.2.8). However, since Sbn Tn = vn
k=1 mk
are non-random, the sequence P(Sbn Tn 0) is composed of zeros and ones,
hence cannot converge to P(G 0) = 1/2. We arrive at a contradiction to our
(c)
assumption that vn , and so conclude that the sequence Var(Xn ) is summable,
P
(c)
that is, the series n Var(Xn ) converges.
P
(c)
(c)
By Theorem 2.3.17, the summability of Var(Xn ) implies that the series n (Xn
P
(c)
(c)
mn ) converges a.s. We have already seen that n Xn converges a.s. so it follows
P
(c)
that their difference n mn , which is the middle term of (3.1.11), converges as
well.

3.2. Weak convergence
Focusing here on the theory of weak convergence, we first consider in Subsection
3.2.1 the convergence in distribution in a more general setting than that of the clt.
This is followed by the study in Subsection 3.2.2 of weak convergence of probability
measures and the theory associated with it. Most notably its relation to other modes
104
of convergence, such as convergence in total variation or point-wise convergence of

probability density functions. We conclude by introducing in Subsection 3.2.3 the
key concept of uniform tightness which is instrumental to the derivation of weak
convergence statements, as demonstrated in later sections of this chapter.
3.2.1. Convergence in distribution. Motivated by the clt, we explore here
the convergence in distribution, its relation to convergence in probability, some
additional properties and examples in which the limiting law is not the normal law.
To start off, here is the definition of convergence in distribution.
Definition 3.2.1. We say that R.V.-s Xn converge in distribution to a R.V. X ,
D
denoted by Xn X , if FXn () FX () as n for each fixed which is
a continuity point of FX .
Similarly, we say that distribution functions Fn converge weakly to F , denoted
w
by Fn F , if Fn () F () as n for each fixed which is a continuity
point of F .
Remark. If the limit R.V. X has a probability density function, or more generally whenever FX is a continuous function, the convergence in distribution of Xn
to X is equivalent to the point-wise convergence of the corresponding distribution functions. Such is the case of the clt, since the normal R.V. G has a density.
Further,
w
Exercise 3.2.2. Show that if Fn F and F () is a continuous function then

also supx |Fn (x) F (x)| 0.
The clt is not the only example of convergence in distribution we have already
met. Recall the Glivenko-Cantelli theorem (see Theorem 2.3.6), whereby a.s. the
empirical distribution functions Fn of an i.i.d. sequence of variables {Xi } converge
uniformly, hence point-wise to the true distribution function FX .
Here is an explicit necessary and sufficient condition for the convergence in distribution of integer valued random variables
D
Exercise 3.2.3. Let Xn , 1 n be integer valued R.V.-s. Show that Xn

X if and only if P(Xn = k) n P(X = k) for each k Z.
In contrast with all of the preceding examples, we demonstrate next why the
D
convergence Xn X has been chosen to be strictly weaker than the pointwise convergence of the corresponding distribution functions. We also see that
Eh(Xn ) Eh(X ) or not, depending upon the choice of h(), and even within the
collection of continuous functions with image in [1, 1], the rate of this convergence
is not uniform in h.
Example 3.2.4. The random variables Xn = 1/n converge in distribution to
X = 0. Indeed, it is easy to check that FXn () = I[1/n,) () converge to
FX () = I[0,) () at each 6= 0. However, there is no convergence at the
discontinuity point = 0 of FX as FX (0) = 1 while FXn (0) = 0 for all n.
Further, Eh(Xn ) = h( n1 ) h(0) = Eh(X ) if and only if h(x) is continuous at
x = 0, and the rate of convergence varies with the modulus of continuity of h(x) at
x = 0.
More generally, if Xn = X + 1/n then FXn () = FX ( 1/n) FX ( ) as
n . So, in order for X + 1/n to converge in distribution to X as n , we
105
have to restrict such convergence to the continuity points of the limiting distribution
function FX , as done in Definition 3.2.1.
We have seen in Examples 3.1.7 and 3.1.8 that the normal distribution is a good
approximation for the Binomial and the Poisson distributions (when the corresponding parameter is large). Our next example is of the same type, now with the
approximation of the Geometric distribution by the Exponential one.
Example 3.2.5 (Exponential approximation of the Geometric). Let Zp
be a random variable with a Geometric distribution of parameter p (0, 1), that is,
P(Zp k) = (1 p)k1 for any positive integer k. As p 0, we see that
P(pZp > t) = (1 p)t/p et
for all
t0
That is, pZp T , with T having a standard exponential distribution. As Zp

corresponds to the number of independent trials till the first occurrence of a specific event whose probability is p, this approximation corresponds to waiting for the
occurrence of rare events.
At this point, you are to check that convergence in probability implies the convergence in distribution, which is hence weaker than all notions of convergence
explored in Section 1.3.3 (and is perhaps a reason for naming it weak convergence).
The converse cannot hold, for example because convergence in distribution does not
require Xn and X to be even defined on the same probability space. However,
convergence in distribution is equivalent to convergence in probability when the
limiting random variable is a non-random constant.
p
Exercise 3.2.6. Show that if Xn X , then Xn X . Conversely, if

p
D
Xn X and X is almost surely a non-random constant, then Xn X .
w
Further, as the next theorem shows, given Fn F , it is possible to construct

a.s.
random variables Yn , n such that FYn = Fn and Yn Y . The catch
of course is to construct the appropriate coupling, that is, to specify the relation
between the different Yn s.
Theorem 3.2.7. Let Fn be a sequence of distribution functions that converges
weakly to F . Then there exist random variables Yn , 1 n on the probability
a.s.
space ((0, 1], B(0,1] , U ) such that FYn = Fn for 1 n and Yn Y .
Proof. We use Skorokhods representation as in the proof of Theorem 1.2.37.
That is, for (0, 1] and 1 n let Yn+ () Yn () be
Yn+ () = sup{y : Fn (y) },
Yn () = sup{y : Fn (y) < } .
While proving Theorem 1.2.37 we saw that FYn = Fn for any n , and as
remarked there Yn () = Yn+ () for all but at most countably many values of ,
hence P(Yn = Yn+ ) = 1. It thus suffices to show that for all (0, 1),
+
Y
() lim sup Yn+ () lim sup Yn ()
n
lim inf Yn () Y
() .
(3.2.1)
Y () for any
setting Yn = Yn+ for 1
Yn ()
Indeed, then
P(A) = 1. Hence,
theorem.
A = { : Y
() = Y
()} where
n would complete the proof of the
106
Turning to prove (3.2.1) note that the two middle inequalities are trivial. Fixing
(0, 1) we proceed to show that
+
Y
() lim sup Yn+ () .
(3.2.2)
Since the continuity points of F form a dense subset of R (see Exercise 1.2.39),
+
it suffices for (3.2.2) to show that if z > Y
() is a continuity point of F , then
+
+
necessarily z Yn () for all n large enough. To this end, note that z > Y
()
implies by definition that F (z) > . Since z is a continuity point of F and
w
Fn F we know that Fn (z) F (z). Hence, Fn (z) > for all sufficiently large
n. By definition of Yn+ and monotonicity of Fn , this implies that z Yn+ (), as
needed. The proof of
() ,
lim inf Yn () Y
(3.2.3)
Y
()
is analogous. For y <

we know by monotonicity of F that F (y) < .
Assuming further that y is a continuity point of F , this implies that Fn (y) <
for all sufficiently large n, which in turn results with y Yn (). Taking continuity
points yk of F such that yk Y

() will yield (3.2.3) and complete the proof.
The next exercise provides useful ways to get convergence in distribution for one
sequence out of that of another sequence. Its result is also called the converging
together lemma or Slutskys lemma.
D
Exercise 3.2.8. Suppose that Xn X and Yn Y , where Y is nonrandom and for each n the variables Xn and Yn are defined on the same probability
space.
D
(a) Show that then Xn + Yn X + Y .

Hint: Recall that the collection of continuity points of FX is dense.
D
D
D
(b) Deduce that if Zn Xn 0 then Xn X if and only if Zn X.
D
(c) Show that Yn Xn Y X .
For example, here is an application of Exercise 3.2.8 en-route to a clt connected
to renewal theory.
Exercise 3.2.9.
(a) Suppose {Nm } are non-negative integer-valued random variables and bm
p
Pnare non-random integers such that Nm /bm 1. Show that if Sn =

k=1 Xk for i.i.d. random variables {Xk } with v = Var(X1 ) (0, )
D
and E(X1 ) = 0, then SNm / vbm G as m .
p
Hint: Use Kolmogorovs inequality to show that SNm / vbm Sbm / vbm
0.
Pn
(b) Let Nt = sup{n : Sn t} for Sn = k=1 Yk and i.i.d. random variables
Yk > 0 such that v = Var(Y1 ) (0, ) and E(Y1 ) = 1. Show that
D
(Nt t)/ vt G as t .
Theorem 3.2.7 is key to solving the following:
D
Exercise 3.2.10. Suppose that Zn Z . Show that then bn (f (c + Zn /bn )

D
f (c))/f (c) Z for every positive constants bn and every Borel function
107
f : R R (not necessarily continuous) that is differentiable at c R, with a

derivative f (c) 6= 0.
Consider the following exercise as a cautionary note about your interpretation of

Theorem 3.2.7.
Pn Qk
Pn Qn
Exercise 3.2.11. Let Mn = k=1 i=1 Ui and Wn = k=1 i=k Ui , where {Ui }
are i.i.d. uniformly on [0, c] and c > 0.
a.s.
(a) Show that Mn M as n , with M taking values in [0, ].

(b) Prove that M is a.s. finite if and only if c < e (but EM is finite only
for c < 2).
D
(c) In case c < e prove that Wn M as n while Wn can not have
an almost sure limit. Explain why this does not contradict Theorem 3.2.7.
The next exercise relates the decay (in n) of sups |FX (s) FXn (s)| to that of
sup |Eh(Xn ) Eh(X )| over all functions h : R 7 [M, M ] with supx |h (x)| L.
Exercise 3.2.12. Let n = sups |FX (s) FXn (s)|.
(a) Show that if supx |h(x)| M and supx |h (x)| L, then for any b > a,
C = 4M + L(b a) and all n
|Eh(Xn ) Eh(X )| Cn + 4M P(X
/ [a, b]) .
(b) Show that if X [a, b] and fX (x) > 0 for all x [a, b], then
|Qn () Q ()| 1 n for any (n , 1 n ), where Qn () =
sup{x : FXn (x) < } denotes -quantile for the law of Xn . Using this,
D
construct Yn = Xn such that P(|Yn Y | > 1 n ) 2n and deduce
the bound of part (a), albeit the larger value 4M + L/ of C.
Here is another example of convergence in distribution, this time in the context
of extreme value theory.
Exercise 3.2.13. Let Mn = max1in {Ti }, where Ti , i = 1, 2, . . . are i.i.d. random variables of distribution function FT (t). Noting that FMn (x) = FT (x)n , show
D
that b1
n (Mn an ) M when:
(a) FT (t) = 1 et for t 0 (i.e. Ti are Exponential of parameter one).
Here, an = log n, bn = 1 and FM (y) = exp(ey ) for y R.
(b) FT (t) = 1 t for t 1 and > 0. Here, an = 0, bn = n1/ and
FM (y) = exp(y ) for y > 0.
(c) FT (t) = 1 |t| for 1 t 0 and > 0. Here, an = 0, bn = n1/
and FM (y) = exp(|y| ) for y 0.
Remark. Up to the linear transformation y 7 (y )/, the three distributions

of M provided in Exercise 3.2.13 are the only possible limits of maxima of i.i.d.
random variables. They are thus called the extreme value distributions of Type 1
(or Gumbel-type), in case (a), Type 2 (or Frechet-type), in case (b), and Type 3
(or Weibull-type), in case (c). Indeed,
Exercise 3.2.14.
(a) Building upon part (a) of Exercise 2.2.24, show that if G has the standard
normal distribution, then for any y R
1 FG (t + y/t)
= ey .
lim
t
1 FG (t)
108
(b) Let Mn = max1in {Gi } for i.i.d. standard normal random variables
D
Gi . Show that bn (Mn bn ) M where FM (y) = exp(ey ) and bn

is such that 1 FG (bn ) = n1 .
p
(c) Show that bn / 2 log n 1 as n and deduce that Mn / 2 log n 1.
(d) More generally, suppose Tt = inf{x 0 : Mx t}, where x 7 Mx
is some monotone non-decreasing family of random variables such that
D
M0 = 0. Show that if et Tt T as t with T having the
D
standard exponential distribution then (Mx log x) M as x ,
y
where FM (y) = exp(e ).
Our next example is of a more combinatorial flavor.
Exercise 3.2.15 (The birthday problem). Suppose {Xi } are i.i.d. with each
Xi uniformly distributed on {1, . . . , n}. Let Tn = min{k : Xk = Xl , for some l < k}
mark the first coincidence among the entries of the sequence X1 , X2 , . . ., so
r
Y
k1
),
(1
P(Tn > r) =
n
k=2
is the probability that among r items chosen uniformly and independently from
a set of n different objects, no two are the same (the name birthday problem
corresponds to n = 365 with the items interpreted as the birthdays for a group of
size r). Show that P(n1/2 Tn > s) exp(s2 /2) as n , for any fixed s 0.
Hint: Recall that x x2 log(1 x) x for x [0, 1/2].
The symmetric,
Pnsimple random walk on the integers is the sequence of random
variables Sn = k=1 k where k are i.i.d. such that P(k = 1) = P(k = 1) = 12 .
D
From the clt we already know that n1/2 Sn G. The next exercise provides
the asymptotics of the first and last visits to zero by this random sequence, namely
R = inf{ 1 : S = 0} and Ln = sup{ n : S = 0}. Much more is known about
this random sequence (c.f. [Dur10, Section 4.3] or [Fel68, Chapter 3]).
Exercise 3.2.16. Let qn,r = P(S1 > 0, . . . , Sn1 > 0, Sn = r) and

n n
k = (n + r)/2 .
pn,r = P(Sn = r) = 2
k
(a) Counting paths of the walk, prove the discrete reflection principle that
Px (R < n, Sn = y) = Px (Sn = y) = pn,x+y for any positive integers
x, y, where Px () denote probabilities for the walk starting at S0 = x.
(b) Verify that qn,r = 21 (pn1,r1 pn1,r+1 ) for any n, r 1.
Hint: Paths of the walk contributing to qn,r must have S1 = 1. Hence,
use part (a) with x = 1 and y = r.
(c) Deduce that P(R > n) = pn1,0 + pn1,1 and that P(L2n = 2k) =
p2k,0 p2n2k,0 for k = 0, 1, . . . , n.
(d) Using Stirlings formula (that 2n(n/e)n /n! 1 as n ), show
D
that nP(R > 2n) 1 and that (2n)1 L2n X, where X has the
arc-sine probability density function fX (x) = 1
on [0, 1].
x(1x)
(e) Let H2n count the number of 1 k 2n such that Sk 0 and Sk1 0.
D
D
Show that H2n = L2n , hence (2n)1 H2n X.
109
3.2.2. Weak convergence of probability measures. We first extend the

definition of weak convergence from distribution functions to measures on Borel
-algebras.
Definition 3.2.17. For a topological space S, let Cb (S) denote the collection of all
continuous bounded functions on S. We say that a sequence of probability measures
n on a topological space S equipped with its Borel -algebra (see Example 1.1.15),
w
converges weakly to a probability measure , denoted n , if n (h) (h)
for each h Cb (S).
As we show next, Definition 3.2.17 is an alternative definition of convergence in
distribution, which, in contrast to Definition 3.2.1, applies to more general R.V.
(for example to the Rd -valued random variables we consider in Section 3.5).
Proposition 3.2.18. The weak convergence of distribution functions is equivalent
to the weak convergence of the corresponding laws as probability measures on (R, B).
D
Consequently, Xn X if and only if for each h Cb (R), we have Eh(Xn )

Eh(X ) as n .
w
Proof. Suppose first that Fn F and let Yn , 1 n be the random

a.s.
variables given by Theorem 3.2.7 such that Yn Y . For h Cb (R) we have by
a.s.
continuity of h that h(Yn ) h(Y ), and by bounded convergence also
Pn (h) = E(h(Yn )) E(h(Y )) = P (h) .
w
Conversely, suppose that Pn P per Definition 3.2.17. Fixing R, let the
+
non-negative h
k Cb (R) be such that hk (x) I(,) (x) and hk (x) I(,] (x)
as k (c.f. Lemma 3.1.6 for a construction of such functions). We have by the
weak convergence of the laws when n , followed by monotone convergence as
k , that
lim inf Pn ((, )) lim Pn (h

k ) = P (hk ) P ((, )) = F ( ) .
n
Similarly, considering
that
h+
k ()
and then k , we have by bounded convergence
+
lim sup Pn ((, ]) lim Pn (h+
k ) = P (hk ) P ((, ]) = F () .
n
For any continuity point of F we conclude that Fn () = Pn ((, ]) converges

as n to F () = F ( ), thus completing the proof.

By yet another application of Theorem 3.2.7 we find that convergence in distribution is preserved under a.s. continuous mappings (see Corollary 2.2.13 for the
analogous statement for convergence in probability).
Proposition 3.2.19 (Continuous mapping). For a Borel function g let Dg
D
denote its set of points of discontinuity. If Xn X and P(X Dg ) = 0,

D
then g(Xn ) g(X ). If in addition g is bounded then Eg(Xn ) Eg(X ).

D
Proof. Given Xn X , by Theorem 3.2.7 there exists Yn = Xn , such that

Yn Y . Fixing h Cb (R), clearly Dhg Dg , so
a.s.
P(Y Dhg ) P(Y Dg ) = 0.
110

a.s.
Therefore, by Exercise 2.2.12, it follows that h(g(Yn )) h(g(Y )). Since h g is

D
bounded and Yn = Xn for all n, it follows by bounded convergence that
Eh(g(Xn )) = Eh(g(Yn )) E(h(g(Y )) = Eh(g(X )) .
This holds for any h Cb (R), so by Proposition 3.2.18, we conclude that g(Xn )
g(X ).

Our next theorem collects several equivalent characterizations of weak convergence
of probability measures on (R, B). To this end we need the following definition.
Definition 3.2.20. For a subset A of a topological space S, we denote by A the
boundary of A, that is A = A \ Ao is the closed set of points in the closure of A
but not in the interior of A. For a measure on (S, BS ) we say that A BS is a
-continuity set if (A) = 0.
Theorem 3.2.21 (portmanteau theorem). The following four statements are
equivalent for any probability measures n , 1 n on (R, B).
w
(a) n
(b) For every closed set F , one has lim sup n (F ) (F )
n
(c) For every open set G, one has lim inf n (G) (G)
n
(d) For every -continuity set A, one has lim n (A) = (A)
n
Remark. As shown in Subsection 3.5.1, this theorem holds with (R, B) replaced
by any metric space S and its Borel -algebra BS .
For n = PXn we get the formulation of the Portmanteau theorem for random
variables Xn , 1 n , where the following four statements are then equivalent
D
to Xn X :
(a) Eh(Xn ) Eh(X ) for each bounded continuous h
(b) For every closed set F one has lim sup P(Xn F ) P(X F )
n
(c) For every open set G one has lim inf P(Xn G) P(X G)
n
(d) For every Borel set A such that P(X A) = 0, one has
lim P(Xn A) = P(X A)
n
Proof. It suffices to show that (a) (b) (c) (d) (a), which we
shall establish in that order. To this end, with Fn (x) = n ((, x]) denoting the
w
corresponding distribution functions, we replace n of (a) by the equivalent
w
condition Fn F (see Proposition 3.2.18).
w
(a) (b). Assuming Fn F , we have the random variables Yn , 1 n of
a.s.
Theorem 3.2.7, such that PYn = n and Yn Y . Since F is closed, the function
IF is upper semi-continuous bounded by one, so it follows that a.s.
lim sup IF (Yn ) IF (Y ) ,
n
and by Fatous lemma,

lim sup n (F ) = lim sup EIF (Yn ) E lim sup IF (Yn ) EIF (Y ) = (F ) ,
n
as stated in (b).
111
(b) (c). The complement F = Gc of an open set G is a closed set, so by (b) we

have that
1 lim inf n (G) = lim sup n (Gc ) (Gc ) = 1 (G) ,
n
implying that (c) holds. In an analogous manner we can show that (c) (b), so
(b) and (c) are equivalent.
(c) (d). Since (b) and (c) are equivalent, we assume now that both (b) and (c)
hold. Then, applying (c) for the open set G = Ao and (b) for the closed set F = A
we have that
(A) lim sup n (A) lim sup n (A)
n
lim inf n (A) lim inf n (Ao ) (Ao ) .
(3.2.4)
Further, A = A A so (A) = 0 implies that (A) = (Ao ) = (A)

(with the last equality due to the fact that Ao A A). Consequently, for such a
set A all the inequalities in (3.2.4) are equalities, yielding (d).
(d) (a). Consider the set A = (, ] where is a continuity point of F .
Then, A = {} and ({}) = F () F ( ) = 0. Applying (d) for this
choice of A, we have that
lim Fn () = lim n ((, ]) = ((, ]) = F () ,
which is our version of (a).
We turn to relate the weak convergence to the convergence point-wise of probability density functions. To this end, we first define a new concept of convergence
of measures, the convergence in total-variation.
Definition 3.2.22. The total variation norm of a finite signed measure on the
measurable space (S, F ) is
kktv = sup{(h) : h mF , sup |h(s)| 1}.
sS
We say that a sequence of probability measures n converges in total variation to a

t.v.
probability measure , denoted n , if kn ktv 0.
Remark. Note that kktv = 1 for any probability measure (since (h)
(|h|) khk (1) 1 for the functions h considered, with equality for h = 1). By
a similar reasoning, k ktv 2 for any two probability measures , on (S, F ).
Convergence in total-variation obviously implies weak convergence of the same
probability measures, but the converse fails, as demonstrated for example by n =
1/n , the probability measure on (R, B) assigning probability one to the point 1/n,
which converge weakly to = 0 (see Example 3.2.4), whereas kn k = 2
for all n. The difference of course has to do with the non-uniformity of the weak
convergence with respect to the continuous function h.
To gain a better understanding of the convergence in total-variation, we consider
an important special case.
Proposition 3.2.23. Suppose P = f and Q = g for some measure on (S, F )
and f, g mF+ such that (f ) = (g) = 1. Then,
Z
(3.2.5)
kP Qktv = |f (s) g(s)|d(s) .
S
112
Further, suppose n = fn with fn mF+ such that (fn ) = 1 for all n .

t.v.
Then, n if fn (s) f (s) for -almost-every s S.
Proof. For any measurable function h : S 7 [1, 1] we have that
(f )(h) (g)(h) = (f h) (gh) = ((f g)h) (|f g|) ,
with equality when h(s) = sgn((f (s)g(s)) (see Proposition 1.3.56 for the left-most
identity and note that f h and gh are in L1 (S, F , )). Consequently, kP Qktv =
sup{(f )(h) (g)(h) : h as above } = (|f g|), as claimed.
For n = fn , we thus have that kn ktv = (|fn f |), so the convergence in
total-variation is equivalent to fn f in L1 (S, F , ). Since fn 0 and (fn ) = 1
for any n , it follows from Scheffes lemma (see Lemma 1.3.35) that the latter
convergence is a consequence of fn (s) f (s) for a.e. s S.

Two specific instances of Proposition 3.2.23 are of particular value in applications.
Example 3.2.24. Let n = PXn denote the laws of random variables Xn that
have probability density functions fn , n = 1, 2, . . . , . Recall Exercise 1.3.66 that
then n = fn for Lebesgues measure on (R, B). Hence, by the preceding proposition, the convergence point-wise of fn (x) to f (x) implies the convergence in
D
total-variation of PXn to PX , and in particular implies that Xn X .
Example 3.2.25. Similarly, if Xn are integer valued for n = 1, 2 . . ., then n =
e for fn (k) = P(Xn = k) and the counting measure
e on (Z, 2Z ) such that
fn
e
({k})
= 1 for each k Z. So, by the preceding proposition, the point-wise convergence of Exercise 3.2.3 is not only necessary and sufficient for weak convergence
but also for convergence in total-variation of the laws of Xn to that of X .
In the next exercise, you are to rephrase Example 3.2.25 in terms of the topological
space of all probability measures on Z.
Exercise 3.2.26. Show that d(, ) = k ktv is a metric on the collection of all
probability measures on Z, and that in this space the convergence in total variation
is equivalent to the weak convergence which in turn is equivalent to the point-wise
convergence at each x Z.
Hence, under the framework of Example 3.2.25, the Glivenko-Cantelli theorem
tells us that the empirical measures of integer valued i.i.d. R.V.-s {Xi } converge in
total-variation to the true law of X1 .
Here is an example from statistics that corresponds to the framework of Example
3.2.24.
Exercise 3.2.27. Let Vn+1 denote the central value on a list of 2n + 1 values (that
is, the (n + 1)th largest value on the list). Suppose the list consists of mutually
independent R.V., each chosen uniformly in [0, 1).
n
n
(a) Show that Vn+1 has probability density function (2n + 1) 2n
n v (1 v)
at each v [0, 1).
(b) Verify that the density fn (v) of Vbn = 2n(2Vn+1 1) is of the form
n
fn (v) = cn (1 v 2 /(2n))
for some normalization constant cn that is
independent of |v| 2n.

(c) Deduce that for n the densities fn (v) converge point-wise to the
D
standard normal density, and conclude that Vbn G.
113
Here is an interesting interpretation of the clt in terms of weak convergence of

probability measures.
Exercise
3.2.28. Let MR denote the set of probability measures on (R, B) for
R
which x2 d(x) = 1 and xd(x) = 0, and M denote the standard normal
distribution.
Consider the mapping T : M 7 M where T is the law of (X1 +
X2 )/ 2 for X1 and X2 i.i.d. of law each. Explain why the clt implies that
w
T m as m , for any M. Show that T = (see Lemma 3.1.1), and
explain why is the unique, globally attracting fixed point of T in M.
Your next exercise is the basis behind the celebrated method of moments for weak
convergence.
Exercise 3.2.29. Suppose that X and Y are [0, 1]-valued random variables such
that E(X n ) = E(Y n ) for n = 0, 1, 2, . . ..
(a) Show that Ep(X) = Ep(Y ) for any polynomial p().
(b) Show that Eh(X) = Eh(Y ) for any continuous function h : [0, 1]
7 R
D
and deduce that X = Y .

Hint: Recall Weierstrass approximation theorem, that if h is continuous on [0, 1]
then there exist polynomials pn such that supx[0,1] |h(x) pn (x)| 0 as n .
We conclude with the following example about weak convergence of measures in
the space of infinite binary sequences.
Exercise 3.2.30. Consider the topology of coordinate wise convergence on S =
{0, 1}N and the Borel probability measures {n } on S, where n is the uniform
measure over the 2n
binary sequences of precisely n ones among the first 2n
n
w
coordinates, followed by zeros from position 2n + 1 onwards. Show that n
where denotes the law of i.i.d. Bernoulli random variables of parameter p = 1/2.
Hint: Any open subset of S is a countable union of disjoint sets of the form A,k =
{ S : i = i , i = 1, . . . , k} for some = (1 , . . . , k ) {0, 1}k and k N.
3.2.3. Uniform tightness and vague convergence. So far we have studied
the properties of weak convergence. We turn to deal with general ways to establish
such convergence, a subject to which we return in Subsection 3.3.2. To this end,
the most important concept is that of uniform tightness, which we now define.
Definition 3.2.31. We say that a probability measure on (S, BS ) is tight if for
each > 0 there exists a compact set K S such that (Kc ) < . A collection
{ } of probability measures on (S, BS ) is called uniformly tight if for each > 0
there exists one compact set K such that (Kc ) < for all .
Since bounded closed intervals are compact and [M, M ]c as M , by
continuity from above we deduce that each probability measure on (R, B) is
tight. The same argument applies for a finite collection of probability measures on
(R, B) (just choose the maximal value among the finitely many values of M = M
that are needed for the different measures). Further, in the case of S = R which we
study here one can take without loss of generality the compact K as a symmetric
bounded interval [M , M ], or even consider instead (M , M ] (whose closure
is compact) in order to simplify notations. Thus, expressing uniform tightness
in terms of the corresponding distribution functions leads in this setting to the
following alternative definition.
114
Definition 3.2.32. A sequence of distribution functions Fn is called uniformly

tight, if for every > 0 there exists M = M such that
lim sup[1 Fn (M ) + Fn (M )] < .
n
Remark. As most texts use in the context of Definition 3.2.32 tight (or tight
sequence) instead of uniformly tight, we shall adopt the same convention here.
Uniform tightness of distribution functions has some structural resemblance to the
U.I. condition (1.3.11). As such we have the following simple sufficient condition
for uniform tightness (which is the analog of Exercise 1.3.54).
Exercise 3.2.33. A sequence of probability measures n on (R, B) is uniformly
tight if supn n (f (|x|)) is finite for some non-negative Borel function such that
f (r) as r . Alternatively, if supn Ef (|Xn |) < then the distribution
functions FXn form a tight sequence.
The importance of uniform tightness is that it guarantees the existence of limit
points for weak convergence.
Theorem 3.2.34 (Prohorov theorem). A collection of probability measures
on a complete, separable metric space S equipped with its Borel -algebra BS , is
uniformly tight if and only if for any sequence m there exists a subsequence
mk that converges weakly to some probability measure on (S, BS ) (where is
not necessarily in and may depend on the subsequence mk ).
Remark. For a proof of Prohorovs theorem, which is beyond the scope of these
notes, see [Dud89, Theorem 11.5.4].
Instead of Prohorovs theorem, we prove here a bare-hands substitute for the
special case S = R. When doing so, it is convenient to have the following notion of
convergence of distribution functions.
Definition 3.2.35. When a sequence Fn of distribution functions converges to a
right continuous, non-decreasing function F at all continuous points of F , we
v
say that Fn converges vaguely to F , denoted Fn F .
In contrast with weak convergence, the vague convergence allows for the limit
F (x) = ((, x]) to correspond to a measure such that (R) < 1.
Example 3.2.36. Suppose Fn = pI[n,) + qI[n,) + (1 p q)F for some p, q 0

such that p+ q 1 and a distribution function F that is independent of n. It is easy
v
to check that Fn F as n , where F = q + (1 p q)F is the distribution
function of an R-valued random variable, with probability mass p at + and mass
q at . If p + q > 0 then F is not a distribution function of any measure on R
and Fn does not converge weakly.
The preceding example is generic, that is, the space R is compact, so the only loss
of mass when dealing with weak convergence on R has to do with its escape to .
It is thus not surprising that every sequence of distribution functions have vague
limit points, as stated by the following theorem.
Theorem 3.2.37 (Hellys selection theorem). For every sequence Fn of distribution functions, there is a subsequence Fnk and a non-decreasing right continuous function F such that Fnk (y) F (y) as k at all continuity points y
v
of F , that is Fnk F .
115
Deferring the proof of Hellys theorem to the end of this section, uniform tightness
is exactly what prevents probability mass from escaping to , thus assuring the
existence of limit points for weak convergence.
Lemma 3.2.38. The sequence of distribution functions {Fn } is uniformly tight if
and only if each vague limit point of this sequence is a distribution function. That
v
is, if and only if when Fnk F , necessarily 1 F (x) + F (x) 0 as x .
v
Proof. Suppose first that {Fn } is uniformly tight and Fnk F . Fixing > 0,
there exist r1 < M and r2 > M that are both continuity points of F . Then, by
the definition of vague convergence and the monotonicity of Fn ,
1 F (r2 ) + F (r1 ) = lim (1 Fnk (r2 ) + Fnk (r1 ))
k
lim sup(1 Fn (M ) + Fn (M )) < .

n
It follows that lim supx (1 F (x) + F (x)) and since > 0 is arbitrarily
small, F must be a distribution function of some probability measure on (R, B).
Conversely, suppose {Fn } is not uniformly tight, in which case by Definition 3.2.32,
for some > 0 and nk
(3.2.6)
1 Fnk (k) + Fnk (k)
for all k.
By Hellys theorem, there exists a vague limit point F to Fnk as k . That

v
is, for some kl as l we have that Fnkl F . For any two continuity
points r1 < 0 < r2 of F , we thus have by the definition of vague convergence, the
monotonicity of Fnkl , and (3.2.6), that
1 F (r2 ) + F (r1 ) = lim (1 Fnkl (r2 ) + Fnkl (r1 ))
l
lim inf (1 Fnkl (kl ) + Fnkl (kl )) .

l
Considering now r = min(r1 , r2 ) , this shows that inf r (1F (r)+F (r)) ,
hence the vague limit point F cannot be a distribution function of a probability
measure on (R, B).

Remark. Comparing Definitions 3.2.31 and 3.2.32 we see that if a collection
of probability measures on (R, B) is uniformly tight, then for any sequence m
the corresponding sequence Fm of distribution functions is uniformly tight. In view
of Lemma 3.2.38 and Hellys theorem, this implies the existence of a subsequence
w
mk and a distribution function F such that Fmk F . By Proposition 3.2.18
w
we deduce that mk , a probability measure on (R, B), thus proving the only
direction of Prohorovs theorem that we ever use.
Proof of Theorem 3.2.37. Fix a sequence of distribution function Fn . The
key to the proof is to observe that there exists a sub-sequence nk and a nondecreasing function H : Q 7 [0, 1] such that Fnk (q) H(q) for any q Q.
This is done by a standard analysis argument called the principle of diagonal
selection. That is, let q1 , q2 , . . ., be an enumeration of the set Q of all rational
numbers. There exists then a limit point H(q1 ) to the sequence Fn (q1 ) [0, 1],
(1)
that is a sub-sequence nk such that Fn(1) (q1 ) H(q1 ). Since Fn(1) (q2 ) [0, 1],
(2)
(1)
there exists a further sub-sequences nk of nk such that

Fn(i) (qi ) H(qi ) for i = 1, 2.
k
116

(i)
(i1)
In the same manner we get a collection of nested sub-sequences nk nk

that
Fn(i) (qj ) H(qj ),
for all j i.
such
The diagonal
(k)
nk
then has the property that

Fn(k) (qj ) H(qj ),
for all j,
(k)
so nk = nk is our desired sub-sequence, and since each Fn is non-decreasing, the

limit function H must also be non-decreasing on Q.
Let F (x) := inf{H(q) : q Q, q > x}, noting that F [0, 1] is non-decreasing.
Further, F is right continuous, since
lim F (xn ) = inf{H(q) : q Q, q > xn for some n}
xn x
= inf{H(q) : q Q, q > x} = F (x).
Suppose that x is a continuity point of the non-decreasing function F . Then, for

any > 0 there exists y < x such that F (x) < F (y) and rational numbers
y < r1 < x < r2 such that H(r2 ) < F (x) + . It follows that
(3.2.7)
F (x) < F (y) H(r1 ) H(r2 ) < F (x) + .
Recall that Fnk (x) [Fnk (r1 ), Fnk (r2 )] and Fnk (ri ) H(ri ) as k , for i = 1, 2.
Thus, by (3.2.7) for all k large enough
F (x) < Fnk (r1 ) Fnk (x) Fnk (r2 ) < F (x) + ,
which since > 0 is arbitrary implies Fnk (x) F (x) as k .
Exercise 3.2.39. Suppose that the sequence of distribution functions {FXk } is

uniformly tight and EXk2 < are such that EXn2 as n . Show that then
also Var(Xn ) as n .
L2
Hint: If |EXnl |2 then supl Var(Xnl ) < yields Xnl /EXnl 1, whereas the
p
uniform tightness of {FXnl } implies that Xnl /EXnl 0.
Using Lemma 3.2.38 and Hellys theorem, you next explore the possibility of establishing weak convergence for non-negative random variables out of the convergence
of the corresponding Laplace transforms.
Exercise 3.2.40.
(a) Based on Exercise 3.2.29 show that if Z 0 and W 0 are such that
D
E(esZ ) = E(esW ) for each s > 0, then Z = W .
(b) Further, show that for any Z 0, the function LZ (s) = E(esZ ) is
infinitely differentiable at all s > 0 and for any positive integer k,
E[Z k ] = (1)k lim
s0
dk
LZ (s) ,
dsk
even when (both sides are) infinite.

(c) Suppose that Xn 0 are such that L(s) = limn E(esXn ) exists for all
s > 0 and L(s) 1 for s 0. Show that then the sequence of distribution
functions {FXn } is uniformly tight and that there exists a random variable
D
X 0 such that Xn X and L(s) = E(esX ) for all s > 0.
3.3. CHARACTERISTIC FUNCTIONS
117
Hint: To show that Xn X try reading and adapting the proof of

Theorem 3.3.18.
P
(d) Let Xn = n1 nk=1 kIk for Ik {0, 1} independent random variables,
D
with P(Ik = 1) = k 1 . Show that there exists X 0 such that Xn

R1
X and E(esX ) = exp( 0 t1 (est 1)dt) for all s > 0.
Remark. The idea of using transforms to establish weak convergence shall be

further developed in Section 3.3, with the Fourier transform instead of the Laplace
transform.
3.3. Characteristic functions
This section is about the fundamental concept of characteristic function, its relevance for the theory of weak convergence, and in particular for the clt.
In Subsection 3.3.1 we define the characteristic function, providing illustrating examples and certain general properties such as the relation between finite moments
of a random variable and the degree of smoothness of its characteristic function. In
Subsection 3.3.2 we recover the distribution of a random variable from its characteristic function, and building upon it, relate tightness and weak convergence with
the point-wise convergence of the associated characteristic functions. We conclude
with Subsection 3.3.3 in which we re-prove the clt of Section 3.1 as an application of the theory of characteristic functions we have thus developed. The same
approach will serve us well in other settings which we consider in the sequel (c.f.
Sections 3.4 and 3.5).
3.3.1. Definition, examples, moments and derivatives. We start off
with the definition of the characteristic function of a random variable. To this
end, recall that a C-valued random variable is a function Z : 7 C such that the
real and imaginary parts of Z are measurable,
and for Z = X + iY with X, Y R
integrable random variables (and i = 1), let E(Z) = E(X) + iE(Y ) C.

Definition 3.3.1. The characteristic function X of a random variable X is the
map R 7 C given by
X () = E[eiX ] = E[cos(X)] + iE[sin(X)]
where R and obviously both cos(X) and sin(X) are integrable R.V.-s.
We also denote by () the characteristic function associated with a probability
measure on (R, B). That is, () = (eix ) is the characteristic function of a
R.V. X whose law PX is .
Here are some of the properties of characteristic functions, where the complex
conjugate x iy ofp
z = x + iy C is denoted throughout by z and the modulus of
z = x + iy is |z| = x2 + y 2 .
Proposition 3.3.2. Let X be a R.V. and X its characteristic function, then
(a) X (0) = 1
(b) X () = X ()
(c) |X ()| 1
(d) 7 X () is a uniformly continuous function on R
(e) aX+b () = eib X (a)
118
Proof. For (a), X (0) = E[ei0X ] = E[1] = 1. For (b), note that
X () = E cos(X) + iE sin(X)
= E cos(X) iE sin(X) = X () .
p
For (c), note that the function |z| = x2 + y 2 : R2 7 R is convex, hence by
Jensens inequality (c.f. Exercise 1.3.20),
|X ()| = |EeiX | E|eiX | = 1
(since the modulus |eix | = 1 for any real x and ).

For (d), since X (+h)X () = EeiX (eihX 1), it follows by Jensens inequality
for the modulus function that
|X ( + h) X ()| E[|eiX ||eihX 1|] = E|eihX 1| = (h)
(using the fact that |zv| = |z||v|). Since 2 |eihX 1| 0 as h 0, by bounded

convergence (h) 0. As the bound (h) on the modulus of continuity of X ()
is independent of , we have uniform continuity of X () on R.
For (e) simply note that aX+b () = Eei(aX+b) = eib Eei(a)X = eib X (a).
We also have the following relation between finite moments of the random variable
and the derivatives of its characteristic function.
Lemma 3.3.3. If E|X|n < , then the characteristic function X () of X has
continuous derivatives up to the n-th order, given by
dk
X () = E[(iX)k eiX ] ,
for k = 1, . . . , n
dk
Proof. Note that for any x, h R
Z h
eiux du .
eihx 1 = ix
(3.3.1)
Consequently, for any h 6= 0, R and positive integer k we have the identity

(3.3.2)
k,h (x) = h1 (ix)k1 ei(+h)x (ix)k1 eix (ix)k eix
Z h
k ix 1
= (ix) e h
(eiux 1)du ,
0
from which we deduce that |k,h (x)| 2|x|k for all and h 6= 0, and further that
|k,h (x)| 0 as h 0. Thus, for k = 1, . . . , n we have by dominated convergence
(and Jensens inequality for the modulus function) that
|Ek,h (X)| E|k,h (X)| 0
Taking k = 1, we have from (3.3.2) that
for
h 0.
E1,h (X) = h1 (X ( + h) X ()) E[iXeiX ] ,
so its convergence to zero as h 0 amounts to the identity (3.3.1) holding for

k = 1. In view of this, considering now (3.3.2) for k = 2, we have that
E2,h (X) = h1 (X ( + h) X ()) E[(iX)2 eiX ] ,
and its convergence to zero as h 0 amounts to (3.3.1) holding for k = 2. We

continue in this manner for k = 3, . . . , n to complete the proof of (3.3.1). The
continuity of the derivatives follows by dominated convergence from the convergence
to zero of |(ix)k ei(+h)x (ix)k eix | 2|x|k as h 0 (with k = 1, . . . , n).
119
The converse of Lemma 3.3.3 does not hold. That is, there exist random variables
with E|X| = for which X () is differentiable at = 0 (c.f. Exercise 3.3.24).
However, as we see next, the existence of a finite second derivative of X () at
= 0 implies that EX 2 < .
Lemma 3.3.4. If lim inf 0 2 (2X (0)X ()X ()) < , then EX 2 < .
Proof. Note that 2 (2X (0) X () X ()) = Eg (X), where
g (x) = 2 (2 eix eix ) = 22 [1 cos(x)] x2
Since g (x) 0 for all and x, it follows by Fatous lemma that
for 0 .
lim inf Eg (X) E[lim inf g (X)] = EX 2 ,

0
thus completing the proof of the lemma.
We continue with a few explicit computations of the characteristic function.

Example 3.3.5. Consider a Bernoulli random variable B of parameter p, that is,
P(B = 1) = p and P(B = 0) = 1 p. Its characteristic function is by definition
B () = E[eiB ] = pei + (1 p)ei0 = pei + 1 p .
The same type of explicit formula applies to any discrete valued R.V. For example,
if N has the Poisson distribution of parameter then
X
(ei )k
e = exp((ei 1)) .
(3.3.3)
N () = E[eiN ] =
k!
k=0
The characteristic function has an explicit form also when the R.V. X has a
probability density function fX as in Definition 1.2.40. Indeed, then by Corollary
1.3.62 we have that
Z
eix fX (x)dx ,
(3.3.4)
X () =
R
which is merely the Fourier transform of the density fX (and is well defined since
cos(x)fX (x) and sin(x)fX (x) are both integrable with respect to Lebesgues measure).
Example 3.3.6. If G has the N (, v) distribution, namely, the probability density
function fG (y) is given by (3.1.1), then its characteristic function is
G () = eiv
/2
Indeed, recall Example 1.3.68 that G = X + for = v and X of a standard

normal distribution N (0, 1). Hence, considering part (e) of Proposition 3.3.2 for
2
a = v and b = , it suffices to show that X () = e /2 . To this end, as X is
integrable, we have from Lemma 3.3.3 that
Z
x sin(x)fX (x)dx
X () = E(iXeiX ) =
R
(since x cos(x)fX (x) is an integrable odd function, whose integral is thus zero).
The standard normal density is such that fX

(x) = xfX (x), hence integrating by
parts we find that
Z
Z
sin(x)fX (x)dx = cos(x)fX (x)dx = X ()

X () =
R
120
(since sin(x)fX (x) is an integrable odd function). We know that X (0) = 1 and
2
since () = e /2 is the unique solution of the ordinary differential equation
() = () with (0) = 1, it follows that X () = ().

Example 3.3.7. In another example, applying the formula (3.3.4) we see that
the random variable U = U (a, b) whose probability density function is fU (x) =
(b a)1 1a<x<b , has the characteristic function
U () =
eib eia
i(b a)
Rb
(recall that a ezx dx = (ezb eza )/z for any z C). For a = b the characteristic
function simplifies to sin(b)/(b). Or, in case b = 1 and a = 0 we have U () =
(ei 1)/(i) for the random variable U of Example 1.1.26.
For a = 0 and z = + i, > 0, the same integration identity applies also
when b (since the real part of z is negative). Consequently, by (3.3.4), the
exponential distribution of parameter > 0 whose density is fT (t) = et 1t>0
(see Example 1.3.68), has the characteristic function T () = /( i).
Finally, for the density fS (s) = 0.5e|s| it is not hard to check that S () =
0.5/(1 i) + 0.5/(1 + i) = 1/(1 + 2 ) (just break the integration over s R in
(3.3.4) according to the sign of s).
We next express the characteristic function of the sum of independent random
variables in terms of the characteristic functions of the summands. This relation
makes the characteristic function a useful tool for proving weak convergence statements involving sums of independent variables.
Lemma 3.3.8. If X and Y are two independent random variables, then
X+Y () = X ()Y ()
Proof. By the definition of the characteristic function
X+Y () = Eei(X+Y ) = E[eiX eiY ] = E[eiX ]E[eiY ] ,
where the right-most equality is obtained by the independence of X and Y (i.e.
applying (1.4.12) for the integrable f (x) = g(x) = eix ). Observing that the rightmost expression is X ()Y () completes the proof.

Here are three simple applications of this lemma.
Example 3.3.9. If X and Y are independent and uniform on (1/2, 1/2) then
by Corollary 1.4.33 the random variable = X + Y has the triangular density,
f (x) = (1 |x|)1|x|1 . Thus, by Example 3.3.7, Lemma 3.3.8, and the trigonometric identity cos = 1 2 sin2 (/2) we have that its characteristic function is
2 sin(/2) 2
2(1 cos )
() = [X ()]2 =
.
=
2
e be i.i.d. random variables.
Exercise 3.3.10. Let X, X
e is a non-negative,
(a) Show that the characteristic function of Z = X X
real-valued function.
e
(b) Prove that there do not exist a < b and i.i.d. random variables X, X
e is the uniform random variable on (a, b).
such that X X
121
In the next exercise you construct a random variable X whose law has no atoms
while its characteristic function does not converge to zero for .
P
k
Exercise 3.3.11. Let X = 2
Bk for {Bk } i.i.d. Bernoulli random varik=1 3
ables such that P(Bk = 1) = P(Bk = 0) = 1/2.
(a) Show that X (3k ) = X () 6= 0 for k = 1, 2, . . ..

(b) Recall that X has the uniform distribution on the Cantor set C, as specified in Example 1.2.43. Verify that x 7 FX (x) is everywhere continuous,
hence the law PX has no atoms (i.e. points of positive probability).
We conclude with an application of characteristic functions for proving an interesting identity in law.
Exercise 3.3.12. For integer 1 n and i.i.d. P
Tk , k = 1, 2, . . ., each of which
n
has the standard exponential distribution, let Sn := k=1 k 1 (Tk 1).
(a) Show that with probability one, S is finite valued and Sn S as

n .
Pn
(b) Show that Sn + k=1 k 1 has the same distribution as Mn := maxnk=1 {Tk }.
y
(c) Deduce that P(S + y) = ee , for all y R, where is Eulers
constant, namely the limit as n of
n
X
1
log n .
n :=
k
k=1
3.3.2. Inversion, continuity and convergence. Is it possible to recover the

distribution function from the characteristic function? Then answer is essentially
yes.
Theorem 3.3.13 (L
evys inversion theorem). Suppose X is the characteristic function of random variable X whose distribution function is FX . For any real
numbers a < b and , let
Z b
1
eia eib
(3.3.5)
a,b () =
.
eiu du =
2 a
i2
Then,
(3.3.6)
lim
Furthermore, if
density function
(3.3.7)
a,b ()X ()d =

R
1
1
[FX (b) + FX (b )] [FX (a) + FX (a )] .
2
2
|X ()|d < , then X has the bounded continuous probability

fX (x) =
1
2
eix X ()d .
Remark. The identity (3.3.7) is a special case of the

R Fourier transform inversion
formula, and as such is in duality with X () = R eix fX (x)dx of (3.3.4). The
formula (3.3.6) should be considered its integrated version, which thereby holds
even in the absence of a density for X.
Here is a simple application of the duality between (3.3.7) and (3.3.4).
122
Example 3.3.14. The Cauchy density is fX (x) = 1/[(1 + x2 )]. Recall Example
3.3.7 that the density fS (s) = 0.5e|s| has the positive, integrable characteristic
function 1/(1 + 2 ). Thus, by (3.3.7),
Z
1
1
eits dt .
0.5e|s| =
2 R 1 + t2
Multiplying both sides by two, then changing t to x and s to , we get (3.3.4) for
the Cauchy density, resulting with its characteristic function X () = e||.
When using characteristic functions for proving limit theorems we do not need
the explicit formulas of Levys inversion theorem, but rather only the fact that the
characteristic function determines the law, that is:
Corollary 3.3.15. If the characteristic functions of two random variables X and
D
Y are the same, that is X () = Y () for all , then X = Y .
Remark. While the real-valued moment generating function MX (s) = E[esX ] is
perhaps a simpler object than the characteristic function, it has a somewhat limited
scope of applicability. For example, the law of a random variable X is uniquely
determined by MX () provided MX (s) is finite for all s [, ], some > 0 (c.f.
[Bil95, Theorem 30.1]). More generally, assuming all moments of X are finite, the
Hamburger moment problem is about uniquely determining the law of X from a
given sequence of moments EX k . You saw in Exercise 3.2.29 that this is always
possible when X has bounded support, but unfortunately, this is not always the case
when X has unbounded support. For more on this issue, see [Dur10, Subsection
3.3.5].
Proof of Corollary 3.3.15. Since X = Y , comparing the right side of
(3.3.6) for X and Y shows that
[FX (b) + FX (b )] [FX (a) + FX (a )] = [FY (b) + FY (b )] [FY (a) + FY (a )] .
As FX is a distribution function, both FX (a) 0 and FX (a ) 0 when a .
For this reason also FY (a) 0 and FY (a ) 0. Consequently,
FX (b) + FX (b ) = FY (b) + FY (b )
for all b R .
In particular, this implies that FX = FY on the collection C of continuity points

of both FX and FY . Recall that FX and FY have each at most a countable set of
points of discontinuity (see Exercise 1.2.39), so the complement of C is countable,
and consequently C is a dense subset of R. Thus, as distribution functions are nondecreasing and right-continuous we know that FX (b) = inf{FX (x) : x > b, x C}
and FY (b) = inf{FY (x) : x > b, x C}. Since FX (x) = FY (x) for all x C, this
D
identity extends to all b R, resulting with X = Y .
Remark. In Lemma 3.1.1, it was shown directly that the sum of independent
random variables of P
normal distributions
P N (k , vk ) has the normal distribution
N (, v) where = k k and v = k vk . The proof easily reduces to dealing
with two independent random variables, X of distribution N (1 , v1 ) and Y of
distribution N (2 , v2 ) and showing that X + Y has the normal distribution N (1 +
2 , v1 +v2 ). Here is an easy proof of this result via characteristic functions. First by
123
the independence of X and Y (see Lemma 3.3.8), and their normality (see Example
3.3.6),
X+Y () = X ()Y () = exp(i1 v1 2 /2) exp(i2 v2 2 /2)
1
= exp(i(1 + 2 ) (v1 + v2 )2 )
2
We recognize this expression as the characteristic function corresponding to the
N (1 + 2 , v1 + v2 ) distribution, which by Corollary 3.3.15 must indeed be the
distribution of X + Y .
Proof of L
evys inversion theorem. Consider the product of the law
PX of X which is a probability measure on R and Lebesgues measure of [T, T ],
noting that is a finite measure on R [T, T ] of total mass 2T .
Fixing a < b R let ha,b (x, ) = a,b ()eix , where by (3.3.5) and Jensens
inequality for the modulus function (and the uniform measure on [a, b]),
Z b
1
ba
|ha,b (x, )| = |a,b ()|
.
|eiu |du =
2 a
2
R
Consequently, |ha,b |d < , and applying Fubinis theorem, we conclude that
JT (a, b) :=
=
a,b ()X ()d =
T
T hZ
hZ
i
a,b ()
eix dPX (x) d
Z hZ
i
ha,b (x, )dPX (x) d =
R
R
T
i
ha,b (x, )d dPX (x) .
Since ha,b (x, ) is the difference between the function eiu /(i2) at u = x a and
the same function at u = x b, it follows that
Z T
ha,b (x, )d = R(x a, T ) R(x b, T ) .
T
Further, as the cosine function is even and the sine function is odd,
Z T iu
Z T
e
sgn(u)
sin(u)
R(u, T ) =
d =
d =
S(|u|T ) ,
i2
T
0
Rr
with S(r) = 0 x1 sin x dx for r > 0.
R
Even though the Lebesgue integral 0 x1 sin x dx does not exist, because both
the integral of the positive part and the integral of the negative part are infinite,
we still have that S(r) is uniformly bounded on (0, ) and
lim S(r) =
(c.f. Exercise 3.3.16). Consequently,
0 if x < a or x > b
lim [R(x a, T ) R(x b, T )] = ga,b (x) = 21 if x = a or x = b
T
1 if a < x < b
124
Since S() is uniformly bounded, so is |R(x a, T ) R(x b, T )| and by bounded

convergence,
Z
Z
lim JT (a, b) = lim [R(x a, T ) R(x b, T )]dPX (x) =
ga,b (x)dPX (x)
T
1
1
= PX ({a}) + PX ((a, b)) + PX ({b}) .
2
2
With PX ({a}) = FX (a) FX (a ), PX ((a, b)) = FX (b ) FX (a) and PX ({b}) =
FX (b) FX (b ), we arrive at the assertion (3.3.6).
R
Suppose now that R |X ()|d = C < . This implies that both the real and the
imaginary parts of eix X () are integrable with respect to Lebesgues measure on
R, hence fX (x) of (3.3.7) is well defined. Further, |fX (x)| C is uniformly bounded
and by dominated convergence with respect to Lebesgues measure on R,
Z
1
lim |fX (x + h) fX (x)| lim
|eix ||X ()||eih 1|d = 0,
h0
h0 2 R
implying that fX () is also continuous. Turning to prove that fX () is the density
of X, note that
ba
|a,b ()X ()|
|X ()| ,
2
so by dominated convergence we have that
Z
a,b ()X ()d .
(3.3.8)
lim JT (a, b) = J (a, b) =
T
Further, in view of (3.3.5), upon applying Fubinis theorem for the integrable function eiu I[a,b] (u)X () with respect to Lebesgues measure on R2 , we see that
Z b
Z hZ b
i
1
iu
e
du X ()d =
fX (u)du ,
J (a, b) =
2 R a
a
for the bounded continuous function fX () of (3.3.7). In particular, J (a, b) must

be continuous in both a and b. Comparing (3.3.8) with (3.3.6) we see that
1
1
J (a, b) = [FX (b) + FX (b )] [FX (a) + FX (a )] ,
2
2
so the continuity of J (, ) implies that FX () must also be continuous everywhere,
with
Z
b
FX (b) FX (a) = J (a, b) =
fX (u)du ,
for all a < b. This shows that necessarily fX (x) is a non-negative real-valued
function, which is the density of X.

R 1 iz
Exercise 3.3.16. Integrating z e dz around the contour formed by the upper semi-circles
of radii and r and the intervals [r, ] and [r, ], deduce that
Rr
S(r) = 0 x1 sin xdx is uniformly bounded on (0, ) with S(r) /2 as r .
Our strategy for handling the clt and similar limit results is to establish the
convergence of characteristic functions and deduce from it the corresponding convergence in distribution. One ingredient for this is of course the fact that the
characteristic function uniquely determines the corresponding law. Our next result
provides an important second ingredient, that is, an explicit sufficient condition for
uniform tightness in terms of the limit of the characteristic functions.
125
Lemma 3.3.17. Suppose {n } are probability measures on (R, B) and n () =

n (eix ) the corresponding characteristic functions. If n () () as n ,
for each R and further () is continuous at = 0, then the sequence {n } is
uniformly tight.
Remark. To see why continuity of the limit () at 0 is required, consider the
sequence n of normal distributions N (0, n2 ). From Example 3.3.6 we see that
the point-wise limit () = I=0 of n () = exp(n2 2 /2) exists but is discontinuous at = 0. However, for any M < we know that n ([M, M ]) =
1 ([M/n, M/n]) 0 as n , so clearly the sequence {n } is not uniformly
tight. Indeed, the corresponding distribution functions Fn (x) = F1 (x/n) converge
vaguely to F (x) = F1 (0) = 1/2 which is not a distribution function (reflecting
escape of all the probability mass to ).
Proof. We start the proof by deriving the key inequality
Z
1 r
(3.3.9)
(1 ())d ([2/r, 2/r]c) ,
r r
which holds for every probability measure on (R, B) and any r > 0, relating the
smoothness of the characteristic function at 0 with the tail decay of the corresponding probability measure at . To this end, fixing r > 0, note that
Z r
Z r
2 sin rx
.
(cos x + i sin x)d = 2r
(1 eix )d = 2r
J(x) :=
x
r
r
So J(x) is non-negative (since | sin u| |u| for all u), and bounded below by 2r
2/|x| (since | sin u| 1). Consequently,
J(x) max(2r
(3.3.10)
2
, 0) rI{|x|>2/r} .
|x|
Now, applying Fubinis theorem for the function 1 eix whose modulus is bounded
by 2 and the product of the probability measure and Lebesgues measure on
[r, r], which is a finite measure of total mass 2r, we get the identity
Z r
Z r hZ
Z
i
(1 ())d =
(1 eix )d(x) d =
J(x)d(x) .
r
Thus, the lower bound (3.3.10) and monotonicity of the integral imply that
Z
Z
Z
1 r
1
(1 ())d =
J(x)d(x)
I{|x|>2/r} d(x) = ([2/r, 2/r]c ) ,
r r
r R
R
hence establishing (3.3.9).

We turn to the application of this inequality for proving the uniform tightness.
Since n (0) = 1 for all n and n (0) (0), it follows that (0) = 1. Further,
() is continuous at = 0, so for any > 0, there exists r = r() > 0 such that
|1 ()|
4
and hence also
1
2
r
for all [r, r],

r
|1 ()|d .
126
The point-wise convergence of n to implies that |1 n ()| |1 ()|. By

bounded convergence with respect to Uniform measure of on [r, r], it follows
that for some finite n0 = n0 () and all n n0 ,
Z
1 r
|1 n ()|d ,
r r
which in view of (3.3.9) results with
Z
1 r
[1 n ()]d n ([2/r, 2/r]c ) .

r r
Since > 0 is arbitrary and M = 2/r is independent of n, by Definition 3.2.32 this

amounts to the uniform tightness of the sequence {n }.
Building upon Corollary 3.3.15 and Lemma 3.3.17 we can finally relate the pointwise convergence of characteristic functions to the weak convergence of the corresponding measures.
vys continuity theorem). Let n , 1 n be probaTheorem 3.3.18 (Le
bility measures on (R, B).
w
(a) If n , then n () () for each R.

(b) Conversely, if n () converges point-wise to a limit () that is continw
uous at = 0, then {n } is a uniformly tight sequence and n such
that = .
Proof. For part (a), since both x 7 cos(x) and x 7 sin(x) are bounded
continuous functions, the assumed weak convergence of n to implies that
n () = n (eix ) (eix ) = () (c.f. Definition 3.2.17).
Turning to deal with part (b), recall that by Lemma 3.3.17 we know that the
collection = {n } is uniformly tight. Hence, by Prohorovs theorem (see the
remark preceding the proof of Lemma 3.2.38), for every subsequence n(m) there is a
further sub-subsequence n(mk ) that converges weakly to some probability measure
. Though in general might depend on the specific choice of n(m), we deduce
from part (a) of the theorem that necessarily = . Since the characteristic
function uniquely determines the law (see Corollary 3.3.15), here the same limit
= applies for all choices of n(m). In particular, fixing h Cb (R), the sequence
yn = n (h) is such that every subsequence yn(m) has a further sub-subsequence
yn(mk ) that converges to y = (h). Consequently, yn = n (h) y = (h) (see
w
Lemma 2.2.11), and since this applies for all h Cb (R), we conclude that n
such that = .

Here is a direct consequence of Levys continuity theorem.
D
Exercise 3.3.19. Show that if Xn X , Yn Y and Yn is independent of

D
Xn for 1 n , then Xn + Yn X + Y .
Combining Exercise 3.3.19 with the Portmanteau theorem and the clt, you can
now show thatPa finite second moment is necessary for the convergence in distribun
tion of n1/2 k=1 Xk for i.i.d. {Xk }.
D
ek } are i.i.d. and n1/2 Pn Xk
Exercise 3.3.20. Suppose {Xk , X
Z (with
k=1
the limit Z R).
127
D
e with Z and
ek and show that n1/2 Pn Yk
Z Z,
(a) Set Yk = Xk X
k=1
e
Z i.i.d.
(b) Let Uk = Yk I|Yk |b and Vk = Yk I|Yk |>b . Show that for any u < and
all n,
n
n
n
n
X
X
X
1 X
Uk u n,
Vk 0) P(
Uk u n) .
P(
Yk u n) P(
2
k=1
k=1
k=1
k=1
(c) Apply the Portmanteau theorem and the clt for the bounded i.i.d. {Uk }
to get that for any u, b < ,
q
e u) 1 P(G u/ EU 2 ) .
P(Z Z
1
2
Considering the limit b followed by u deduce that EY12 < .
Pn
D
(d) Conclude that if n1/2 k=1 Xk Z, then necessarily EX12 < .
ek whose law is
Remark. The trick of replacing Xk by the variables Yk = Xk X
D
symmetric (i.e. Yk = Yk ), is very useful in many problems. It is often called the
symmetrization trick.
Exercise 3.3.21. Provide an example of
R a random variable X with a bounded
probability density function but for which R |X ()|d = , and another example
of a random variable X whose characteristic function X () is not differentiable at
= 0.
As you find out next, Levys inversion theorem can help when computing densities.
Exercise 3.3.22. Suppose the random variables Uk are i.i.d. where the law of each
Uk is the uniform probability measure on (1, 1). ConsideringPExample 3.3.7, show
n
that for each n 2, the probability density function of Sn = k=1 Uk is
Z
1
fSn (s) =
cos(s)(sin /)n d ,
0
R
and deduce that 0 cos(s)(sin /)n d = 0 for all s > n 2.
Exercise 3.3.23. Deduce from

3.3.14 that if {Xk } are i.i.d. each having
PExample
n
the Cauchy density, then n1 k=1 Xk has the same distribution as X1 , for any
value of n.
We next relate differentiability of X () with the weak law of large numbers and
show that it does not imply that E|X| is finite.
Pn
Exercise 3.3.24. Let Sn = k=1 Xk where the i.i.d. random variables {Xk } have
each the characteristic function X ().
p
1
X
Sn a
(a) Show that if d
d (0) = z C, then z = ia for some a R and n
as n .
p
(b) Show that if n1 Sn a, then X (hk )nk eia for any hk 0, > 0
X
and nk = [/hk ], and deduce that d
d (0) = ia.
p
(c) Conclude that the weak law of large numbers holds (i.e. n1 Sn a for
some non-random a), if and only if X () is differentiable at = 0 (this
result is due to E.J.G. Pitman, see [Pit56]).
(d) Use Exercise 2.1.13 to provide a random variable X for which X () is
differentiable at = 0 but E|X| = .
128

D
As you show next, Xn X yields convergence of Xn () to X (), uniformly

over compact subsets of R.
D
Exercise 3.3.25. Show that if Xn X then for any r finite,

lim sup |Xn () X ()| = 0 .
n ||r
a.s.
Hint: By Theorem 3.2.7 you may further assume that Xn X .

Characteristic functions of modulus one correspond to lattice or degenerate laws,
as you show in the following refinement of part (c) of Proposition 3.3.2.
Exercise 3.3.26. Suppose |Y ()| = 1 for some 6= 0.
(a) Show that Y is a (2/)-lattice random variable, namely, that Y mod (2/)
is P-degenerate.
Hint: Check conditions for equality when applying p
Jensens inequality for
(cos Y, sin Y ) and the convex function g(x, y) = x2 + y 2 .
(b) Deduce that if in addition |Y ()| = 1 for some
/ Q then Y must be
P-degenerate, in which case Y () = exp(ic) for some c R.
Building on the preceding two exercises, you are to prove next the following convergence of types result.
D
D
Exercise 3.3.27. Suppose Zn Y and n Zn + n Yb for some Yb , non-Pdegenerate Y , and non-random n 0, n .
(a) Show that n 0 finite.

Hint: Start with the finiteness of limit points of {n }.
(b) Deduce that n finite.
D
(c) Conclude that Yb = Y + .
Hint: Recall Slutskys lemma.
Remark. This convergence of types fails for P-degenerate Y . For example, if

D
D
D
D
Zn = N (0, n3 ), then both Zn 0 and nZn 0. Similarly, if Zn = N (0, 1)
D
then n Zn = N (0, 1) for the non-converging sequence n = (1)n (of alternating
signs).
Mimicking the proof of Levys inversion theorem, for random variables of bounded
support you get the following alternative inversion formula, based on the theory of
Fourier series.
Exercise 3.3.28. Suppose R.V. X supported on (0, t) has the characteristic function X and the distribution function FX . Let 0 = 2/t and a,b () be as in
(3.3.5), with a,b (0) = ba
2 .
(a) Show that for any 0 < a < b < t
lim
T
X
k=T
0 (1
1
1
|k|
)a,b (k0 )X (k0 ) = [FX (b) + FX (b )] [FX (a) + FX (a )] .
T
2
2
PT
Hint: Recall that ST (r) = k=1 (1 k/T ) sinkkr is uniformly bounded for
r (0, 2) and integer T 1, and ST (r) r
2 as T .
129
P
(b) Show that if
k |X (k0 )| < then X has the bounded continuous
probability density function, given for x (0, t) by
0 X ik0 x
fX (x) =
e
X (k0 ) .
2
kZ
(c) Deduce that if R.V.s X and Y supported on (0, t) are such that X (k0 ) =
D
Y (k0 ) for all k Z, then X = Y .

Here is an application of the preceding exercise for the random walk on the circle
S 1 of radius one (c.f. Definition 5.1.6 for the random walk on R).
Exercise 3.3.29. Let t = 2 and denote the unit circle S 1 parametrized by
the angular coordinate to yield the identification = [0, t] where both end-points
are considered the same point. We equip with the topology induced by [0, t] and
the surface measure similarly induced by Lebesgues measure (as in Exercise
1.4.37). In particular, R.V.-s on (, B ) correspond to Borel periodic functions on
1
R, of period
Pn t. In this context we call U of law t a uniform R.V. and call
Sn = ( k=1 k )mod t, with i.i.d , k , a random walk.
(a) Verify that Exercise 3.3.28 applies for 0 = 1 and R.V.-s on .
(b) Show that if probability measures n on (, B ) are such that n (k)
w
(k) for n and fixed k Z, then n and (k) = (k) for
all k Z.
Hint: Since is compact the sequence {n } is uniformly tight.
(c) Show that U (k) = 1k=0 and Sn (k) = (k)n . Deduce from these facts
D
that if has a density with respect to then Sn U as n .

Hint: Recall part (a) of Exercise 3.3.26.
(d) Check that if = is non-random for some /t
/ Q, then Sn does
D
not converge in distribution, but SKn U for Kn which are uniformly
chosen in {1, 2, . . . , n}, independently of the sequence {k }.
3.3.3. Revisiting the clt. Applying the theory of Subsection 3.3.2 we provide an alternative proof of the clt, based on characteristic functions. One can
prove many other weak convergence results for sums of random variables by properly adapting this approach, which is exactly what we will do when demonstrating
the convergence to stable laws (see Exercise 3.3.34), and in proving the Poisson
approximation theorem (in Subsection 3.4.1), and the multivariate clt (in Section
3.5).
To this end, we start by deriving the analog of the bound (3.1.7) for the characteristic function.
Lemma 3.3.30. If a random variable X has E(X) = 0 and E(X 2 ) = v < , then
for all R,

1

X () 1 v2 2 E min(|X|2 , |||X|3 /6).
2
Proof. Let R2 (x) = eix 1 ix (ix)2 /2. Then, rearranging terms, recalling
E(X) = 0 and using Jensens inequality for the modulus function, we see that

i2
1

X () 1 v2 = E eiX 1 iX 2 X 2 = ER2 (X) E|R2 (X)|.
2
2
130
Since |R2 (x)| min(|x|2 , |x|3 /6) for any x R (see also Exercise 3.3.35), by monotonicity of the expectation we get that E|R2 (X)| E min(|X|2 , |X|3 /6), completing the proof of the lemma.

The following simple complex analysis estimate is needed for relating the approximation of the characteristic function of summands to that of their sum.
Pn
Lemma 3.3.31. Suppose zn,k C are such that zn = k=1 zn,k z and n =
P
n
2
k=1 |zn,k | 0 when n . Then,
n :=
n
Y
k=1
(1 + zn,k ) exp(z )
for n .
Proof. Recall that the power series expansion

log(1 + z) =
X
(1)k1 z k
k=1
converges for |z| < 1. In particular, for |z| 1/2 it follows that
| log(1 + z) z|
X
|z|k
k=2
|z|
X
2(k2)
k=2
|z|
k=2
2(k1) = |z|2 .
Let n = max{|zn,k | : k = 1, . . . , n}. Note that n2 n , so our assumption that

n 0 implies that n 1/2 for all n sufficiently large, in which case
| log n zn | = | log
n
Y
k=1
(1 + zn,k )
n
X
k=1
zn,k |
n
X
k=1
| log(1 + zn,k ) zn,k | n .
With zn z and n 0, it follows that log n z . Consequently, n

exp(z ) as claimed.

We will give now an alternative proof of the clt of Theorem 3.1.2.
2
Proof of Theorem 3.1.2. From Example 3.3.6 we know that G () = e 2

is the characteristic function of the standard normal distribution. So, by Levys
continuity theorem it suffices to show that Sbn () exp(2 /2) as n , for
Pn
each R. Recall that Sbn =

k=1 Xn,k , with Xn,k = (Xk )/ vn i.i.d.
random variables, so by independence (see Lemma 3.3.8) and scaling (see part (e)
of Proposition 3.3.2), we have that
n := Sbn () =
n
Y
Xn,k () = Y (n1/2 )n = (1 + zn /n)n ,
k=1
where Y = (X1 )/ v and zn = zn () := n[Y (n1/2 ) 1]. Applying Lemma

3.3.31 for zn,k = zn /n it remains only to show that zn 2 /2 (for then n =
|zn |2 /n 0). Indeed, since E(Y ) = 0 and E(Y 2 ) = 1, we have from Lemma 3.3.30
that

|zn + 2 /2| = n[Y (n1/2 ) 1] + 2 /2 EVn ,
a.s.
for Vn = min(|Y |2 , n1/2 |Y |3 /6). With Vn 0 as n and Vn ||2 |Y |2

which is integrable, it follows by dominated convergence that EVn 0 as n ,
hence zn 2 /2 completing the proof of Theorem 3.1.2.
131
We proceed with a brief introduction of stable laws, their domain of attraction

and the corresponding limit theorems (which are a natural generalization of the
clt).
Definition 3.3.32. Random variable Y has a stable law if it is non-degenerate
D
and for any m 1 there exist constants dm > 0 and cm , such that Y1 + . . . + Ym =
dm Y + cm , where {Yi } are i.i.d. copies of Y . Such variable has a symmetric stable
D
law if in addition Y = Y . We further say that random variable X is in the
domain of attraction of non-degenerate Y if there exist constants bn > 0 and an
Pn
D
such that Zn (X) = (Sn an )/bn Y for Sn = k=1 Xk and i.i.d. copies Xk of
X.
By
definition, the collection of stable laws is closed under the affine map Y 7
vY + for R and v > 0 (which correspond to the centering and scale of the
law, but not necessarily its mean and variance). Clearly, each stable law is in its
own domain of attraction and as we see next, only stable laws have a non-empty
domain of attraction.
Proposition 3.3.33. If X is in the domain of attraction of some non-degenerate
variable Y , then Y must have a stable law.
Proof. Fix m 1, and setting n = km let n = bn /bk > 0 and n =
(an mak )/bk . We then have the representation
m
X
(i)
Zk ,
n Zn (X) + n =
i=1
where
(i)
Zk
= (X(i1)k+1 + . . . + Xik ak )/bk are i.i.d. copies of Zk (X). From our

D
assumption that Zk (X) Y we thus deduce (by at most m 1 applications of

D
Exercise 3.3.19), that n Zn (X)+n Yb , where Yb = Y1 +. . .+Ym for i.i.d. copies
D
{Yi } of Y . Moreover, by assumption Zn (X) Y , hence by the convergence of
D
types Yb = dm Y + cm for some finite non-random dm 0 and cm (c.f. Exercise
3.3.27). Recall Lemma 3.3.8 that Yb () = [Y ()]m . So, with Y assumed nondegenerate the same applies to Yb (see Exercise 3.3.26), and in particular dm > 0.
Since this holds for any m 1, by definition Y has a stable law.
We have already seen two examples of symmetric stable laws, namely those associated with the zero-mean normal density and with the Cauchy density of Example
3.3.14. Indeed, as you show next, for each (0, 2) there corresponds the symmetric -stable variable Y whose characteristic function is Y () = exp(|| )
(so the Cauchy distribution corresponds to the symmetric stable of index = 1
and the normal distribution corresponds to index = 2).
D
Exercise 3.3.34. Fixing (0, 2), suppose X = X and P(|X| > x) = x for
all x 1.
R
(a) Check that X () = 1(||)|| where (r) = r (1cos u)u(+1) du
converges as r 0 to (0) finite and positive.
Pn
(b) Setting ,0 () = exp(|| ), bn = ((0)n)1/ and Sbn = b1
n
k=1 Xk
for i.i.d. copies Xk of X, deduce that Sbn () ,0 () as n , for
any fixed R.
132
(c) Conclude that X is in the domain of attraction of a symmetric stable

variable Y , whose characteristic function is ,0 ().
(d) Fix = 1 and show that with probability one lim supn Sbn = and
lim inf n Sbn = .
Hint: Recall Kolmogorovs 0-1 law. The same proof applies for any > 0
once we verify that Y has unbounded
Pn support.
1
(e) Show that if = 1 then n log
k=1 |Xk | 1 in probability but not
n
almost surely (in contrast, X is integrable when > 1, in which case the
strong law of large numbers applies).
Remark. While outside the scope of these notes, one can show that (up to scaling)
any symmetric stable variable must be of the form Y for some (0, 2]. Further,
D
for any (0, 2) the necessary and sufficient condition for X = X to be in the
domain of attraction of Y is that the function L(x) = x P(|X| > x) is slowly

varying at (that is, L(ux)/L(x) 1 for x and fixed u > 0). Indeed,
as
shown for example in [Bre92, Theorem 9.32], up to the mapping Y 7 vY + ,

the collection of all stable laws forms a two parameter family Y, , parametrized
by the index (0, 2] and skewness [1, 1]. The corresponding characteristic
functions are
(3.3.11)
, () = exp(|| (1 + isgn()g ())) ,
where g1 (r) = (2/) log |r| and g = tan(/2) is constant for all 6= 1 (in
particular, g2 = 0 so the parameter is irrelevant when = 2). Further, in case
< 2 the domain of attraction of Y, consists precisely of the random variables
X for which L(x) = x P(|X| > x) is slowly varying at and (P(X > x) P(X <
x))/P(|X| > x) as x (for example, see [Bre92, Theorem 9.34]). To
complete this picture, we recall [Fel71, Theorem XVII.5.1], that X is in the domain
of attraction of the normal variable Y2 if and only if L(x) = E[X 2 I|X|x ] is slowly
varying (as is of course the case whenever EX 2 is finite).
As shown in the following exercise, controlling the modulus of the remainder term
for the n-th order Taylor approximation of eix one can generalize the bound on
X () beyond the case n = 2 of Lemma 3.3.30.
Exercise 3.3.35. For any x R and non-negative integer n, let
Rn (x) = e
ix
n
X
(ix)k
k=0
k!
Rx
(a) Show that Rn (x) = 0 iRn1 (y)dy for all n 1 and deduce by induction
on n that
2|x|n |x|n+1
for all x R, n = 0, 1, 2, . . . .
|Rn (x)| min
,
n! (n + 1)!
(b) Conclude that if E|X|n < then
n
h
X

(i)k EX k
2|X|n |||X|n+1 i
X ()
.
||n E min
,
k!
n!
(n + 1)!
k=0
By solving the next exercise you generalize the proof of Theorem 3.1.2 via characteristic functions to the setting of Lindebergs clt.
3.4. POISSON APPROXIMATION AND THE POISSON PROCESS
133
Pn
Exercise 3.3.36. Consider Sbn = k=1 Xn,k for mutually independent random
variables Xn,k , k = 1, . . . , n, of zero mean and variance vn,k , such that vn =
P
n
k=1 vn,k 1 as n .
(a) Fixing R show that
n
Y
n = Sbn () =
(1 + zn,k ) ,
k=1
where zn,k = Xn,k () 1.

(b) With z = 2 /2, use Lemma 3.3.30 to verify that |zn,k | 22 vn,k and
further, for any > 0,
n
X
||3
vn ,
|zn,k vn,k z | 2 gn () +
|zn vn z |
6
k=1
P
where zn = nk=1 zn,k and gn () is given by (3.1.4).
(c) Recall that Lindebergs condition gn () 0 implies that P
rn2 = maxk vn,k
n
0 as n . Deduce that then zn z and n = k=1 |zn,k |2 0
when n .
D
(d) Applying Lemma 3.3.31, conclude that Sbn G.
We conclude this section with an exercise that reviews various techniques one may
use for establishing convergence in distribution for sums of independent random
variables.
P
Exercise 3.3.37. Throughout this problem Sn = nk=1 Xk for mutually independent random variables {Xk }.
(a) Suppose that P(Xk = k ) = P(Xk = k ) = 1/(2k ) and P(Xk = 0) =
1 k . Show that for any fixed R and > 1, the series Sn ()
converges almost surely as n .
(b) Consider the setting of part (a) when 0 < 1 and = 2 + 1 is
D
positive. Find non-random bn such that b1
n Sn Z and 0 < FZ (z) < 1
for some z R. Provide also the characteristic function Z () of Z.
(c) Repeat part (b) in case = 1 and > 0 (see Exercise 3.1.11 for = 0).
(d) Suppose now that P(Xk = 2k) = P(Xk = 2k) = 1/(2k 2 ) and P(Xk =
D
1) = P(Xk = 1) = 0.5(1 k 2 ). Show that Sn / n G.
3.4. Poisson approximation and the Poisson process
Subsection 3.4.1 deals with the Poisson approximation theorem and few of its applications. It leads naturally to the introduction of the Poisson process in Subsection
3.4.2, where we also explore its relation to sums of i.i.d. Exponential variables and
to order statistics of i.i.d. uniform random variables.
3.4.1. Poisson approximation. The Poisson approximation theorem is about

the law of the sum Sn of a large number (= n) of independent random variables.
In contrast to the clt that also deals with such objects, here all variables are nonnegative integer valued and the variance of Sn remains bounded, allowing for the
approximation in law of Sn by an integer valued variable. The Poisson distribution
results when the number of terms in the sum grows while the probability that each
of them is non-zero decays. As such, the Poisson approximation is about counting
the number of occurrences among many independent rare events.
134
Theorem 3.4.1 (Poisson approximation). Let Sn =
n
X
Zn,k , where for each
k=1
n the random variables Zn,k for 1 k n, are mutually independent, each taking
value in the set of non-negative integers. Suppose that pn,k = P(Zn,k = 1) and
n,k = P(Zn,k 2) are such that as n ,
n
X
(a)
pn,k < ,
k=1
(b)
(c)
max {pn,k } 0,
k=1, ,n
n
X
k=1
D
n,k 0.
Then, Sn N of a Poisson distribution with parameter , as n .

Proof. The first step of the proof is to apply truncation by comparing Sn
with
n
X
Sn =
Z n,k ,
k=1
where Z n,k = Zn,k IZn,k 1 for k = 1, . . . , n. Indeed, observe that,

P(S n 6= Sn )
=
n
X
k=1
n
X
k=1
p
P(Z n,k 6= Zn,k ) =

n,k 0
n
X
k=1
P(Zn,k 2)
for n ,
by assumption (c) .
D
Hence, (S n Sn ) 0. Consequently, the convergence S n N of the sums of

D
truncated variables imply that also Sn N (c.f. Exercise 3.2.8).

As seen in the context of the clt, characteristic functions are a powerful tool
for the convergence in distribution of sums of independent random variables (see
Subsection 3.3.3). This is also evident in our proof of the Poisson approximation
D
theorem. That is, to prove that S n N , if suffices by Levys continuity theorem

to show the convergence of the characteristic functions S n () N () for each
R.
To this end, recall that Z n,k are independent Bernoulli variables of parameters
pn,k , k = 1, . . . , n. Hence, by Lemma 3.3.8 and Example 3.3.5 we have that for
zn,k = pn,k (ei 1),
S n () =
n
Y
n
Y
Z n,k () =
k=1
(1 pn,k + pn,k ei ) =
k=1
Our assumption (a) implies that for n

zn :=
n
X
k=1
n
Y
(1 + zn,k ) .
k=1
n
X
zn,k = (
pn,k )(ei 1) (ei 1) := z .
k=1
Further, since |zn,k | 2pn,k , our assumptions (a) and (b) imply that for n ,
n =
n
X
k=1
|zn,k |2 4
n
X
k=1
n
X
p2n,k 4( max {pn,k })(
pn,k ) 0 .
k=1,...,n
k=1
135
Applying Lemma 3.3.31 we conclude that when n ,
S n () exp(z ) = exp((ei 1)) = N ()
(see (3.3.3) for the last identity), thus completing the proof.
Remark. Recall Example 3.2.25 that the weak convergence of the laws of the
integer valued Sn to that of N also implies their convergence in total
Pn variation.
In the setting of the Poisson approximation theorem, taking n = k=1 pn,k , the
more quantitative result
||PS n PNn ||tv =
k=0
|P(S n = k) P(Nn = k)| 2 min(1

n , 1)
n
X
p2n,k
k=1
due to Stein (1987) also holds (see also [Dur10, (3.6.1)] for a simpler argument,
due to Hodges and Le Cam (1960), which is just missing the factor min(n1 , 1)).
For the remainder of this subsection we list applications of the Poisson approximation theorem, starting with
Example 3.4.2 (Poisson approximation for the Binomial). Take independent variables Zn,k {0, 1}, so n,k = 0, with pn,k = pn that does not depend on
k. Then, the variable Sn = S n has the Binomial distribution of parameters (n, pn ).
By Steins result, the Binomial distribution of parameters (n, pn ) is approximated
well by the Poisson distribution of parameter n = npn , provided pn 0. In case
n = npn < , Theorem 3.4.1 yields that the Binomial (n, pn ) laws converge
weakly as n to the Poisson distribution of parameter . This is in agreement
with Example 3.1.7 where we approximate the Binomial distribution of parameters
(n, p) by the normal distribution, for in Example 3.1.8 we saw that, upon the same
scaling, Nn is also approximated well by the normal distribution when n .
Recall the occupancy problem where we distribute at random r distinct balls
among n distinct boxes and each of the possible nr assignments of balls to boxes is
equally likely. In Example 2.1.10 we considered the asymptotic fraction of empty
boxes when r/n and n . Noting that the number of balls Mn,k in the
k-th box follows the Binomial distribution of parameters (r, n1 ), we deduce from
D
Example 3.4.2 that Mn,k N . Thus, P(Mn,k = 0) P(N = 0) = e .
That is, for large n each box is empty with probability about e , which may
explain (though not prove) the result of Example 2.1.10. Here we use the Poisson
approximation theorem to tackle a different regime, in which r = rn is of order
n log n, and consequently, there are fewer empty boxes.
Proposition 3.4.3. Let Sn denote the number of empty boxes. Assuming r = rn
D
is such that ner/n [0, ), we have that Sn N as n .
Proof. Let Zn,k = IMn,k =0 for k = 1, . . . , n, that is Zn,k = 1 if the k-th box
Pn
is empty and Zn,k = 0 otherwise. Note that Sn =
k=1 Zn,k , with each Zn,k
having the Bernoulli distribution of parameter pn = (1 n1 )r . Our assumption
about rn guarantees that npn . If the occupancy Zn,k of the various boxes were
mutually independent, then the stated convergence of Sn to N would have followed
from Theorem 3.4.1. Unfortunately, this is not the case, so we present a barehands approach showing that the dependence is weak enough to retain the same
136
conclusion. To this end, first observe that for any l = 1, 2, . . . , n, the probability
that given boxes k1 < k2 < . . . < kl are all empty is,
l r
) .
n
Let pl = pl (r, n) = P(Sn = l) denote the probability that exactly l boxes are empty
out of the n boxes into which the r balls are placed at random. Then, considering
all possible choices of the locations of these l 1 empty boxes we get the identities
pl (r, n) = bl (r, n)p0 (r, n l) for

n
l r
.
(3.4.1)
bl (r, n) =
1
n
l
P(Zn,k1 = Zn,k2 = = Zn,kl = 1) = (1
Further, p0 (r, n) = 1P( at least one empty box), so that by the inclusion-exclusion
formula,
(3.4.2)
p0 (r, n) =
n
X
(1)l bl (r, n) .
l=0
According to part (b) of Exercise 3.4.4, p0 (r, n) e . Further, for fixed l we

have that (n l)er/(nl) , so as before we conclude that p0 (r, n l) e .
By part (a) of Exercise 3.4.4 we know that bl (r, n) l /l! for fixed l, hence
pl (r, n) e l /l!. As pl = P(Sn = l), the proof of the proposition is thus
complete.

The following exercise provides the estimates one needs during the proof of Proposition 3.4.3 (for more details, see [Dur10, Theorem 3.6.5]).
Exercise 3.4.4. Assuming ner/n , show that
(a) bl (r, n) of (3.4.1) converges to l /l! for each fixed l.
(b) p0 (r, n) of (3.4.2) converges to e .
Finally, here is an application of Proposition 3.4.3 to the coupon collectors problem of Example 2.1.8, where Tn denotes the number of independent trials, it takes
to have at least one representative of each of the n possible values (and each trial
produces a value Ui that is distributed uniformly on the set of n possible values).
Example 3.4.5 (Revisiting the coupon collectors problem). For any
x R, we have that
(3.4.3)
lim P(Tn n log n nx) = exp(ex ),
which is an improvement over our weak law result that Tn /n log n 1. Indeed, to
derive (3.4.3) view the first r trials of the coupon collector as the random placement
of r balls into n distinct boxes that correspond to the n possible values. From this
point of view, the event {Tn r} corresponds to filling all n boxes with the r
balls, that is, having none empty. Taking r = rn = [n log n + nx] we have that
ner/n = ex , and so it follows from Proposition 3.4.3 that P(Tn rn )
P(N = 0) = e , as stated
Pn in (3.4.3).
Note that though Tn = k=1 Xn,k with Xn,k independent, the convergence in distribution of Tn , given by (3.4.3), is to a non-normal limit. This should not surprise
you, for the terms Xn,k with k near n are large and do not satisfy Lindebergs
condition.
137
Exercise 3.4.6. Recall that n denotes the first time one has distinct values
when collecting coupons that are uniformly distributed on {1, 2, . . . , n}. Using the
Poisson approximation theorem show that if n and = (n) is such that
D
n1/2 [0, ), then n N with N a Poisson random variable of
parameter 2 /2.
3.4.2. Poisson Process. The Poisson process is a continuous time stochastic
process 7 Nt (), t 0 which belongs to the following class of counting processes.
Definition 3.4.7. A counting process is a mapping 7 Nt (), where Nt ()

is a piecewise constant, non-decreasing, right continuous function of t 0, with
N0 () = 0 and (countably) infinitely many jump discontinuities, each of whom is
of size one.
Associated with each sample path Nt () of such a process are the jump times
0 = T0 < T1 < < Tn < such that Tk = inf{t 0 : Nt k} for each k, or
equivalently
Nt = sup{k 0 : Tk t}.
In applications we find such Nt as counting the number of discrete events occurring
in the interval [0, t] for each t 0, with Tk denoting the arrival or occurrence time
of the k-th such event.
Remark. It is possible to extend the notion of counting processes to discrete

events indexed on Rd , d 2. This is done by assigning random integer counts
NA to Borel subsets A of Rd in an additive manner, that is, NAB = NA + NB
whenever A and B are disjoint. Such processes are called point processes. See also
Exercise 7.1.13 for more about Poisson point process and inhomogeneous Poisson
processes of non-constant rate.
Among all counting processes we characterize the Poisson process by the joint
distribution of its jump (arrival) times {Tk }.
Definition 3.4.8. The Poisson process of rate > 0 is the unique counting
process with the gaps between jump times k = Tk Tk1 , k = 1, 2, . . . being i.i.d.
random variables, each having the exponential distribution of parameter .
Thus, from Exercise 1.4.46 we deduce that the k-th arrival time Tk of the Poisson
process of rate has the gamma density of parameters = k and ,
k uk1 u
e
1u>0 .
(k 1)!
As we have seen in Example 2.3.7, counting processes appear in the context of
renewal theory. In particular, as shown in Exercise 2.3.8, the Poisson process of
a.s.
rate satisfies the strong law of large numbers t1 Nt .
Recall that a random variable N has the Poisson() law if
n
e , n = 0, 1, 2, . . . .
P(N = n) =
n!
fTk (u) =
Our next proposition, which is often used as an alternative definition of the Poisson
process, also explains its name.
Proposition 3.4.9. For any and any 0 = t0 < t1 < < t , the increments
Nt1 , Nt2 Nt1 , . . . , Nt Nt1 , are independent random variables and for some
> 0 and all t > s 0, the increment Nt Ns has the Poisson((t s)) law.
138
Thus, the Poisson process has independent increments, each having a Poisson law,
where the parameter of the count Nt Ns is proportional to the length of the
corresponding interval [s, t].
The proof of Proposition 3.4.9 relies on the lack of memory of the exponential
distribution. That is, if the law of a random variable T is exponential (of some
parameter > 0), then for all t, s 0,
(3.4.4)
P(T > t + s|T > t) =
e(t+s)
P(T > t + s)
= es = P(T > s) .
=
P(T > t)
et
Indeed, the key to the proof of Proposition 3.4.9 is the following lemma.
Lemma 3.4.10. Fixing t > 0, the variables {j } with 1 = TNt +1 t, and j =
TNt +j TNt +j1 , j 2 are i.i.d. each having the exponential distribution of parameter . Further, the collection {j } is independent of Nt which has the Poisson
distribution of parameter t.
Remark. Note that in particular, Et = TNt +1 t which counts the time till
next arrival occurs, hence called the excess life time at t, follows the exponential
distribution of parameter .
Proof.
R x Fixing t > 0 and n 1 let Hn (x) = P(t Tn > t x). With
Hn (x) = 0 fTn (t y)dy and Tn independent of n+1 , we get by Fubinis theorem
(for ItTn >tn+1 ), and the integration by parts of Lemma 1.4.30 that
(3.4.5)
P(Nt = n) = P(t Tn > t n+1 ) = E[Hn (n+1 )]

Z t
=
fTn (t y)P(n+1 > y)dy
0
Z t n
(t y)n1 (ty) y
(t)n
=
e
e
dy = et
.
(n 1)!
n!
0
As this applies for any n 1, it follows that Nt has the Poisson distribution of
parameter t. Similarly, observe that for any s1 0 and n 1,
P(Nt = n, 1 > s1 ) = P(t Tn > t n+1 + s1 )
Z t
=
fTn (t y)P(n+1 > s1 + y)dy
0
= es1 P(Nt = n) = P(1 > s1 )P(Nt = n) .
Since T0 = 0, P(Nt = 0) = et and T1 = 1 , in view of (3.4.4) this conclusion

extends to n = 0, proving that 1 is independent of Nt and has the same exponential
law as 1 .
Next, fix arbitrary integer k 2 and non-negative sj 0 for j = 1, . . . , k. Then,
for any n 0, since {n+j , j 2} are i.i.d. and independent of (Tn , n+1 ),
P(Nt = n, j > sj , j = 1, . . . , k)
= P(t Tn > t n+1 + s1 , Tn+j Tn+j1 > sj , j = 2, . . . , k)

= P(t Tn > t n+1 + s1 )
k
Y
j=2
P(n+j > sj ) = P(Nt = n)
k
Y
j=1
P(j > sj ).
139
Since sj 0 and n 0 are arbitrary, this shows that the random variables Nt and
j , j = 1, . . . , k are mutually independent (c.f. Corollary 1.4.12), with each j having an exponential distribution of parameter . As k is arbitrary, the independence
of Nt and the countable collection {j } follows by Definition 1.4.3.

Proof of Proposition 3.4.9. Fix t, sj 0, j = 1, . . . , k, and non-negative
integers n and mj , 1 j k. The event {Nsj = mj , 1 j k} is of the form
{(1 , . . . , r ) H} for r = mk + 1 and
H=
k
\
j=1
{x [0, )r : x1 + + xmj sj < x1 + + xmj +1 } .
Since the event {(1 , . . . , r ) H} is merely {Nt+sj Nt = mj , 1 j k}, it

follows form Lemma 3.4.10 that
P(Nt = n, Nt+sj Nt = mj , 1 j k) = P(Nt = n, (1 , . . . , r ) H)
= P(Nt = n)P((1 , . . . , r ) H) = P(Nt = n)P(Nsj = mj , 1 j k) .
By induction on this identity implies that if 0 = t0 < t1 < t2 < < t , then
(3.4.6)
P(Nti Nti1 = ni , 1 i ) =
i=1
P(Nti ti1 = ni )
(the case = 1 is trivial, and to advance the induction to + 1 set k = , t = t1 ,

Pj+1
n = n1 and sj = tj+1 t1 , mj = i=2 ni ).
Considering (3.4.6) for = 2, t2 = t > s = t1 , and summing over the values of n1
we see that P(Nt Ns = n2 ) = P(Nts = n2 ), hence by (3.4.5) we conclude that
Nt Ns has the Poisson distribution of parameter (t s), as claimed.

The Poisson process is also related to the order statistics {Vn,k } for the uniform
measure, as stated in the next two exercises.
Exercise 3.4.11. Let U1 , U2 , . . . , Un be i.i.d. with each Ui having the uniform
measure on (0, 1]. Denote by Vn,k the k-th smallest number in {U1 , . . . , Un }.
(a) Show that (Vn,1 , . . . , Vn,n ) has the same law as (T1 /Tn+1 , . . . , Tn /Tn+1 ),
where {Tk } are the jump (arrival) times for a Poisson process of rate
(see Subsection 1.4.2 for the definition of the law PX of a random vector
X).
D
(b) Taking = 1, deduce that nVn,k Tk as n while k is fixed, where
Tk has the gamma density of parameters = k and s = 1.
Exercise 3.4.12. Fixing any positive integer n and 0 t1 t2 tn t,
show that
Z tn
Z Z
n! t1 t2
dxn )dxn1 dx1 .
(
P(Tk tk , k = 1, . . . , n|Nt = n) = n
t 0
xn1
x1
That is, conditional on the event Nt = n, the first n jump times {Tk : k = 1, . . . , n}
have the same law as the order statistics {Vn,k : k = 1, . . . , n} of a sample of n i.i.d
random variables U1 , . . . , Un , each of which is uniformly distributed in [0, t].
Here is an application of Exercise 3.4.12.
Exercise 3.4.13. Consider a Poisson process Nt of rate and jump times {Tk }.
140
(a) Compute the values of g(n) = E(INt =n
n
X
Tk ).
k=1
Nt
X
(b) Compute the value of v = E( (t Tk )).
k=1
(c) Suppose that Tk is the arrival time to the train station of the k-th passenger on a train that departs the station at time t. What is the meaning
of Nt and of v in this case?
The representation of the order statistics {Vn,k } in terms of the jump times of
a Poisson process is very useful when studying the large n asymptotics of their
spacings {Rn,k }. For example,
Exercise 3.4.14. Let Rn,k = Vn,k Vn,k1 , k = 1, . . . , n, denote the spacings
between Vn,k of Exercise 3.4.11 (with Vn,0 = 0). Show that as n ,
n
p
max Rn,k 1 ,
(3.4.7)
log n k=1,...,n
and further for each fixed x 0,
(3.4.8)
Gn (x) := n1
n
X
k=1
(3.4.9)
Bn (x) := P( min
k=1,...,n
I{Rn,k >x/n} ex ,
Rn,k > x/n2 ) ex .
As we show next, the Poisson approximation theorem provides a characterization

of the Poisson process that is very attractive for modeling real-world phenomena.
Corollary 3.4.15. If Nt is a Poisson process of rate > 0, then for any fixed
k, 0 < t1 < t2 < < tk and non-negative integers n1 , n2 , , nk ,
P(Ntk +h Ntk = 1|Ntj = nj , j k) = h + o(h),
P(Ntk +h Ntk 2|Ntj = nj , j k) = o(h),
where o(h) denotes a function f (h) such that h1 f (h) 0 as h 0.

Proof. Fixing k, the tj and the nj , denote by A the event {Ntj = nj , j k}.
For a Poisson process of rate the random variable Ntk +h Ntk is independent of
A with P(Ntk +h Ntk = 1) = eh h and P(Ntk +h Ntk 2) = 1 eh(1 + h).
Since eh = 1h+o(h) we see that the Poisson process satisfies this corollary.
Our next exercise explores the phenomenon of thinning, that is, the partitioning
of Poisson variables as sums of mutually independent Poisson variables of smaller
parameter.
Exercise 3.4.16. Suppose {Xi } are i.i.d. with P(Xi = j) = pj for j = 0, 1, . . . , k
and N a Poisson random variable of parameter that is independent of {Xk }. Let
Nj =
N
X
IXi =j
j = 0, . . . , k .
i=1
(a) Show that the variables Nj , j = 0, 1, . . . , k are mutually independent with

Nj having a Poisson distribution of parameter pj .
3.5. RANDOM VECTORS AND THE MULTIVARIATE clt
141
(b) Show that the sub-sequence of jump times {Tek } obtained by independently
keeping with probability p each of the jump times {Tk } of a Poisson proet of rate p.
cess Nt of rate , yields in turn a Poisson process N
We conclude this section noting the superposition property, namely that the sum
of two independent Poisson processes is yet another Poisson process.
(1)
(2)
(1)
(2)
Exercise 3.4.17. Suppose Nt = Nt + Nt where Nt and Nt are two independent Poisson processes of rates 1 > 0 and 2 > 0, respectively. Show that Nt
is a Poisson process of rate 1 + 2 .
3.5. Random vectors and the multivariate clt
The goal of this section is to extend the clt to random vectors, that is, Rd -valued
random variables. Towards this end, we revisit in Subsection 3.5.1 the theory
of weak convergence, this time in the more general setting of Rd -valued random
variables. Subsection 3.5.2 is devoted to the extension of characteristic functions
and Levys theorems to the multivariate setting, culminating with the Cramerwold reduction of convergence in distribution of random vectors to that of their
one dimensional linear projections. Finally, in Subsection 3.5.3 we introduce the
important concept of Gaussian random vectors and prove the multivariate clt.
3.5.1. Weak convergence revisited. Recall Definition 3.2.17 of weak convergence for a sequence of probability measures on a topological space S, which
suggests the following definition for convergence in distribution of S-valued random
variables.
Definition 3.5.1. We say that (S, BS )-valued random variables Xn converge in
D
distribution to a (S, BS )-valued random variable X , denoted by Xn X , if
w
PXn PX .
As already remarked, the Portmanteau theorem about equivalent characterizations
of the weak convergence holds also when the probability measures n are on a Borel
measurable space (S, BS ) with (S, ) any metric space (and in particular for S = Rd ).
Theorem 3.5.2 (portmanteau theorem). The following five statements are
equivalent for any probability measures n , 1 n on (S, BS ), with (S, ) any
metric space.
w
(a) n
(b) For every closed set F , one has lim sup n (F ) (F )
n
(c) For every open set G, one has lim inf n (G) (G)
n
(d) For every -continuity set A, one has lim n (A) = (A)
n
(e) If the Borel function g : S 7 R is such that (Dg ) = 0, then n g 1

g 1 and if in addition g is bounded then n (g) (g).
Remark. For S = R, the equivalence of (a)(d) is the content of Theorem 3.2.21
while Proposition 3.2.19 derives (e) out of (a) (in the context of convergence in
D
D
distribution, that is, Xn X and P(X Dg ) = 0 implying that g(Xn )
g(X )). In addition to proving the converse of the continuous mapping property,
we extend the validity of this equivalence to any metric space (S, ), for we shall
142
apply it again in Subsection 9.2, considering there S = C([0, )), the metric space
of all continuous functions on [0, ).
Proof. The derivation of (b) (c) (d) in Theorem 3.2.21 applies for any
topological space. The direction (e) (a) is also obvious since h Cb (S) has
Dh = and Cb (S) is a subset of the bounded Borel functions on the same space
(c.f. Exercise 1.2.20). So taking g Cb (S) in (e) results with (a). It thus remains
only to show that (a) (b) and that (d) (e), which we proceed to show next.
(a) (b). Fixing A BS let A (x) = inf yA (x, y) : S 7 [0, ). Since |A (x)
A (x )| (x, x ) for any x, x , it follows that x 7 A (x) is a continuous function
on (S, ). Consequently, hr (x) = (1 rA (x))+ Cb (S) for all r 0. Further,
A (x) = 0 for all x A, implying that hr IA for all r. Thus, applying part (a)
of the Portmanteau theorem for hr we have that
lim sup n (A) lim n (hr ) = (hr ) .
n
As A (x) = 0 if and only if x A it follows that hr IA as r , resulting with

lim sup n (A) (A) .
n
Taking A = A = F a closed set, we arrive at part (b) of the theorem.

(d) (e). Fix a Borel function g : S 7 R with K = supx |g(x)| < such that
(Dg ) = 0. Clearly, { R : g 1 ({}) > 0} is a countable set. Thus, fixing
> 0 we can pick < and 0 < 1 < < such that g 1 ({i }) = 0
for 0 i , 0 < K < K < and i i1 < for 1 i . Let
Ai = {x : i1 < g(x) i } for i = 1, . . . , , noting that Ai {x : g(x) = i1 ,
or g(x) = i } Dg . Consequently, by our assumptions about g() and {i } we
have that (Ai ) = 0 for each i = 1, . . . , . It thus follows from part (d) of the
Portmanteau theorem that
X
i=1
i n (Ai )
X
i=1
i (Ai )
P
as n . Our choice of i and Ai is such that g i=1 i IAi g + , resulting
with
X
i n (Ai ) n (g) +
n (g)
i=1
for n = 1, 2, . . . , . Considering first n followed by 0, we establish that

n (g) (g). More generally, recall that Dhg Dg for any g : S 7 R and h
Cb (R). Thus, by the preceding proof n (h g) (h g) as soon as (Dg ) = 0.
w
This applies for every h Cb (R), so in this case n g 1 g 1 .

We next show that the relation of Exercise 3.2.6 between convergences in probability and in distribution also extends to any metric space (S, ), a fact we will later
use in Subsection 9.2, when considering the metric space of all continuous functions
on [0, ).
Corollary 3.5.3. If random variables Xn , 1 n on the same probability
p
space and taking value in a metric space (S, ) are such that (Xn , X ) 0, then
D
Xn X .
143
Proof. Fixing h Cb (S) and > 0, we have by continuity of h() that Gr S,

where
Gr = {y S : |h(x) h(y)| whenever (x, y) r1 } .
By definition, if X Gr and (Xn , X ) r1 then |h(Xn )h(X )| . Hence,
for any n, r 1,
E[|h(Xn ) h(X )|] + 2khk (P(X
/ Gr ) + P((Xn , X ) > r1 )) ,
where khk = supxS |h(x)| is finite (by the boundedness of h). Considering n
followed by r we deduce from the convergence in probability of (Xn , X )
to zero, that
lim sup E[|h(Xn ) h(X )|] + 2khk lim P(X
/ Gr ) = .
r
Since this applies for any > 0, it follows by the triangle inequality that Eh(Xn )
D
Eh(X ) for all h Cb (S), i.e. Xn X .
Remark. The notion of distribution function for an R -valued random vector

X = (X1 , . . . , Xd ) is
FX (x) = P(X1 x1 , . . . , Xd xd ) .
Inducing a partial order on Rd by x y if and only if x y has only non-negative

coordinates, each distribution function FX (x) has the three properties listed in
Theorem 1.2.37. Unfortunately, these three properties are not sufficient for a given
function F : Rd 7 [0, 1] to be a distribution function. For example, since the meaQd
sure of each rectangle A = i=1 (ai , bi ] should be positive, the additional constraint
P2d
of the form A F = j=1 F (xj ) 0 should hold if F () is to be a distribution
function. Here xj enumerates the 2d corners of the rectangle A and each corner
is taken with a positive sign if and only if it has an even number of coordinates
from the collection {a1 , . . . , ad }. Adding the fourth property that A F 0 for
each rectangle A Rd , we get the necessary and sufficient conditions for F () to be
a distribution function of some Rd -valued random variable (c.f. [Bil95, Theorem
12.5] for a detailed proof).
Recall Definition 3.2.31 of uniform tightness, where for S = Rd we can take K =
[M , M ]d with no loss of generality. Though Prohorovs theorem about uniform
tightness (i.e. Theorem 3.2.34) is beyond the scope of these notes, we shall only
need in the sequel the fact that a uniformly tight sequence of probability measures
has at least one limit point. This can be proved for S = Rd in a manner similar
to what we have done in Theorem 3.2.37 and Lemma 3.2.38 for S = R1 , using the
corresponding concept of distribution function FX () (see [Dur10, Theorem 3.9.2]
for more details).
3.5.2. Characteristic function. We start by extending the useful notion of
characteristic function to the context of Rd -valued random variables (which we also
call hereafter random vectors).
Pd
Definition 3.5.4. Adopting the notation (x, y) = i=1 xi yi for x, y Rd , a random vector X = (X1 , X2 , , Xd ) with values in Rd has the characteristic function
X () = E[ei(,X) ] ,
where = (1 , 2 , , d ) Rd and i = 1.
144
Remark. The characteristic function X : Rd 7 C exists for any X since

ei(,X) = cos(, X) + i sin(, X) ,
(3.5.1)
with both real and imaginary parts being bounded (hence integrable) random variables. Actually, it is easy to check that all five properties of Proposition 3.3.2 hold,
where part (e) is modified to At X+b () = exp(i(b, ))X (A), for any non-random
d d-dimensional matrix A and b Rd (with At denoting the transpose of the
matrix A).
Here is the extension of the notion of probability density function (as in Definition
1.2.40) to a random vector.
RDefinition 3.5.5. Suppose fX is a non-negative Borel measurable function with
f (x)dx = 1. We say that a random vector X = (X1 , . . . , Xd ) has a probability
Rd X
density function fX () if for every b = (b1 , . . . , bd ),
Z bd
Z b1
FX (b) =
fX (x1 , . . . , xd )dxd dx1
(such fX is sometimes called the joint density of X1 , . . . , Xd ). This is the same as

saying that the law of X is of the form fX d with d the d-fold product Lebesgue
measure on Rd (i.e. the d > 1 extension of Example 1.3.60).
Example 3.5.6. We have the following extension of the Fourier transform formula
(3.3.4) to random vectors X with density,
Z
ei(,x) fX (x)dx
X () =
Rd
(this is merely a special case of the extension of Corollary 1.3.62 to h : Rd 7 R).
We next state and prove the corresponding extension of Levys inversion theorem.
vys inversion theorem). Suppose X () is the characterTheorem 3.5.7 (Le
istic function of random vector X = (X1 , . . . , Xd ) whose law is PX , a probability
measure on (Rd , BRd ). If A = [a1 , b1 ] [ad , bd ] with PX (A) = 0, then
Z
d
Y
(3.5.2)
PX (A) = lim
aj ,bj (j )X ()d
T
[T,T ]d j=1
for a,b () of (3.3.5). Further, the characteristic function determines the law of a
random vector. That is, if X () = Y () for all then X has the same law as Y .
Proof. We derive (3.5.2) by adapting the proof of Theorem 3.3.13. First apply
Fubinis theorem with respect to the product of Lebesgues measure on [T, T ]d
and the law of X (both of which are finite measures on Rd ) to get the identity
Z
Z hY
d
d Z T
i
Y
JT (a, b) :=
aj ,bj (j )X ()d =
haj ,bj (xj , j )dj dPX (x)
[T,T ]d j=1
Rd
j=1
(where ha,b (x, ) = a,b ()eix ). In the course of proving Theorem 3.3.13 we have
seen that for j = 1, . . . , d the integral over j is uniformly bounded in T and that
it converges to gaj ,bj (xj ) as T . Thus, by bounded convergence it follows that
Z
ga,b (x)dPX (x) ,
lim JT (a, b) =
T
Rd
where
ga,b (x) =
d
Y
145
gaj ,bj (xj ) ,
j=1
is zero on Ac and one on Ao (see the explicit formula for ga,b (x) provided there).
So, our assumption that PX (A) = 0 implies that the limit of JT (a, b) as T is
merely PX (A), thus establishing (3.5.2).
Suppose now that X () = Y () for all . Adapting the proof of Corollary 3.3.15
to the current setting, let J = { R : P(Xj = ) > 0 or P(Yj = ) > 0 for some
j = 1, . . . , d} noting that if all the coordinates {aj , bj , j = 1, . . . , d} of a rectangle
A are from the complement of J then both PX (A) = 0 and PY (A) = 0. Thus,
by (3.5.2) we have that PX (A) = PY (A) for any A in the collection C of rectangles
with coordinates in the complement of J . Recall that J is countable, so for any
rectangle A there exists An C such that An A, and by continuity from above of
both PX and PY it follows that PX (A) = PY (A) for every rectangle A. In view of
Proposition 1.1.39 and Exercise 1.1.21 this implies that the probability measures

PX and PY agree on all Borel subsets of Rd .
We next provide the ingredients needed when using characteristic functions enroute to the derivation of a convergence in distribution result for random vectors.
To this end, we start with the following analog of Lemma 3.3.17.
Lemma 3.5.8. Suppose the random vectors X n , 1 n on Rd are such that
X n () X () as n for each Rd . Then, the corresponding sequence
of laws {PX n } is uniformly tight.
Proof. Fixing Rd consider the sequence of random variables Yn = (, X n ).
Since Yn () = X n () for 1 n , we have that Yn () Y () for all
R. The uniform tightness of the laws of Yn then follows by Lemma 3.3.17.
Considering 1 , . . . , d which are the unit vectors in the d different coordinates,
we have the uniform tightness of the laws of Xn,j for the sequence of random
vectors X n = (Xn,1 , Xn,2 , . . . , Xn,d ) and each fixed coordinate j = 1, . . . , d. For
the compact sets K = [M , M ]d and all n,
/ K )
P(X n
d
X
j=1
P(|Xn,j | > M ) .
As d is finite, this leads from the uniform tightness of the laws of Xn,j for each
j = 1, . . . , d to the uniform tightness of the laws of X n .

Equipped with Lemma 3.5.8 we are ready to state and prove Levys continuity
theorem.
Theorem 3.5.9 (L
evys continuity theorem). Let X n , 1 n be random
D
vectors with characteristic functions X n (). Then, X n X if and only if

X n () X () as n for each fixed Rd .
Proof. This is a re-run of the proof of Theorem 3.3.18, adapted to Rd -valued

random variables. First, both x 7 cos((, x)) and x 7 sin((, x)) are bounded
D
continuous functions, so if X n X , then clearly as n ,
i
h
i

X n () = E ei(,X n ) E ei(,X ) = X ().
146
For the converse direction, assuming that X n X point-wise, we know from

Lemma 3.5.8 that the collection {PX n } is uniformly tight. Hence, by Prohorovs
theorem, for every subsequence n(m) there is a further sub-subsequence n(mk ) such
that PX n(m ) converges weakly to some probability measure PY , possibly dependent
k
upon the choice of n(m). As X n(mk ) Y , we have by the preceding part of the
proof that X n(m ) Y , and necessarily Y = X . The characteristic function
k
determines the law (see Theorem 3.5.7), so Y = X is independent of the choice

of n(m). Thus, fixing h Cb (Rd ), the sequence yn = Eh(X n ) is such that every
subsequence yn(m) has a further sub-subsequence yn(mk ) that converges to y .
Consequently, yn y (see Lemma 2.2.11). This applies for all h Cb (Rd ), so we
D
conclude that X n X , as stated.

Remark. As in the case of Theorem 3.3.18, it is not hard to show that if X n ()
() as n and () is continuous at = 0 then is necessarily the characD
teristic function of some random vector X and consequently X n X .
The proof of the multivariate clt is just one of the results that rely on the following
immediate corollary of Levys continuity theorem.
D
Corollary 3.5.10 (Cram
er-Wold device). A sufficient condition for X n
D
X is that (, X n ) (, X ) for each Rd .
D
Proof. Since (, X n ) (, X ) it follows by Levys continuity theorem (for

d = 1, that is, Theorem 3.3.18), that
i
h
i
h
lim E ei(,X n ) = E ei(,X ) .
n
As this applies for any Rd , we get that X n X by applying Levys

continuity theorem in Rd (i.e., Theorem 3.5.9), now in the converse direction.
Remark. Beware that it is not enough to consider only finitely many values of
in the Cramer-Wold device. For example, consider the random vectors X n =
(Xn , Yn ) with {Xn , Y2n } i.i.d. and Y2n+1 = X2n+1 . Convince yourself that in this
D
D
case Xn X1 and Yn Y1 but the random vectors X n do not converge in
distribution (to any limit).
The computation of the characteristic function is much simplified in the presence
of independence.
Exercise 3.5.11. Show that if Y = (Y1 , . . . , Yd ) with Yk mutually independent
R.V., then for all = (1 , . . . , d ) Rd ,
(3.5.3)
Y () =
d
Y
Yk (k )
k=1
Conversely, show that if (3.5.3) holds for all Rd , the random variables Yk ,
k = 1, . . . , d are mutually independent of each other.
147
3.5.3. Gaussian random vectors and the multivariate clt. Recall the
following linear algebra concept.
Definition 3.5.12. An d d matrix A with entries Ajk is called non-negative
definite (or positive semidefinite) if Ajk = Akj for all j, k, and for any Rd
(, A) =
d X
d
X
j=1 k=1
j Ajk k 0.
We are ready to define the class of multivariate normal distributions via the corresponding characteristic functions.
Definition 3.5.13. We say that a random vector X = (X1 , X2 , , Xd ) is Gaussian, or alternatively that it has a multivariate normal distribution if
(3.5.4)
X () = e 2 (,V) ei(,) ,
for some non-negative definite d d matrix V, some = (1 , . . . , d ) Rd and all

= (1 , . . . , d ) Rd . We denote such a law by N (, V).
Remark. For d = 1 this definition coincides with Example 3.3.6.
Our next proposition proves that the multivariate N (, V) distribution is well
defined and further links the vector and the matrix V to the first two moments
of this distribution.
Proposition 3.5.14. The formula (3.5.4) corresponds to the characteristic function of a probability measure on Rd . Further, the parameters and V of the Gaussian random vector X are merely j = EXj and Vjk = Cov(Xj , Xk ), j, k = 1, . . . , d.
Proof. Any non-negative definite matrix V can be written as V = Ut D2 U
for some orthogonal matrix U (i.e., such that Ut U = I, the d d-dimensional
identity matrix), and some diagonal matrix D. Consequently,
(, V) = (A, A)
for A = DU and all Rd . We claim that (3.5.4) is the characteristic function
of the random vector X = At Y + , where Y = (Y1 , . . . , Yd ) has i.i.d. coordinates
Yk , each of which has the standard normal distribution. Indeed, by Exercise 3.5.11
Y () = exp( 12 (, )) is the product of the characteristic functions exp(k2 /2) of
the standard normal distribution (see Example 3.3.6), and by part (e) of Proposition
3.3.2, X () = exp(i(, ))Y (A), yielding the formula (3.5.4).
We have just shown that X has the N (, V) distribution if X = At Y + for a
Gaussian random vector Y (whose distribution is N (0, I)), such that EYj = 0 and
Cov(Yj , Yk ) = 1j=k for j, k = 1, . . . , d. It thus follows by linearity of the expectation and the bi-linearity of the covariance that EXj = j and Cov(Xj , Xk ) =

[EAt Y (At Y )t ]jk = (At IA)jk = Vjk , as claimed.
Definition 3.5.13 allows for V that is non-invertible, so for example the constant
random vector X = is considered a Gaussian random vector though it obviously
does not have a density. The reason we make this choice is to have the collection
of multivariate normal distributions closed with respect to L2 -convergence, as we
prove below to be the case.
148
Proposition 3.5.15. Suppose Gaussian random vectors X n converge in L2 to

a random vector X , that is, E[kX n X k2 ] 0 as n . Then, X is
a Gaussian random vector, whose parameters are the limits of the corresponding
parameters of X n .
Proof. Recall that the convergence in L2 of X n to X implies that n = EX n
converge to = EX and the element-wise convergence of the covariance matrices Vn to the corresponding covariance matrix V . Further, the L2 -convergence
implies the corresponding convergence in probability and hence, by bounded con1
vergence X n () X () for each Rd . Since X n () = e 2 (,Vn ) ei(,n ) ,
for any n < , it follows that the same applies for n = . It is a well known fact
of linear algebra that the element-wise limit V of non-negative definite matrices
Vn is necessarily also non-negative definite. In view of Definition 3.5.13, we see
that the limit X is a Gaussian random vector, whose parameters are the limits
of the corresponding parameters of X n .

One of the main reasons for the importance of the multivariate normal distribution
is the following clt (which is the multivariate extension of Proposition 3.1.2).
1
Theorem 3.5.16 (Multivariate clt). Let Sbn = n 2
n
X
k=1
(X k ), where {X k }
are i.i.d. random vectors with finite second moments and such that = EX 1 .
D
Then, Sbn G, with G having the N (0, V) distribution and where V is the d ddimensional covariance matrix of X 1 .
Proof. Consider the i.i.d. random vectors Y k = X k each having also the
covariance matrix V. Fixing an arbitrary vector Rd we proceed to show that
D
(, Sbn ) (, G), which in view of the Cramer-Wold device completes the proof
1 P
of the theorem. Indeed, note that (, Sbn ) = n 2 nk=1 Zk , where Zk = (, Y k ) are
i.i.d. R-valued random variables, having zero mean and variance
v = Var(Z1 ) = E[(, Y 1 )2 ] = (, E[Y 1 Y t1 ] ) = (, V) .
Observing that the clt of Proposition 3.1.2 thus applies to (, Sbn ), it remains only
to verify that the resulting limit distribution N (0, v ) is indeed the law of (, G).
To this end note that by Definitions 3.5.4 and 3.5.13, for any s R,
1
(,G) (s) = G (s) = e 2 s
(,V)
= ev s
/2
which is the characteristic function of the N (0, v ) distribution (see Example 3.3.6).
Since the characteristic function uniquely determines the law (see Corollary 3.3.15),
we are done.

Here is an explicit example for which the multivariate clt applies.
P
Example 3.5.17. The simple random walk on Zd is S n = nk=1 X k where X,
X k are i.i.d. random vectors such that
1
P(X = +ei ) = P(X = ei ) =
i = 1, . . . , d,
2d
and ei is the unit vector in the i-th direction, i = 1, . . . , d. In this case EX = 0
and if i 6= j then EXi Xj = 0, resulting with the covariance matrix V = (1/d)I for
the multivariate normal limit in distribution of n1/2 S n .
149
Building on Lindebergs clt for weighted sums of i.i.d. random variables, the
following multivariate normal limit is the basis for the convergence of random walks
to Brownian motion, to which Section 9.2 is devoted.
Exercise 3.5.18. Suppose {k } are i.i.d. with E1 = 0 and E12 = 1. Consider
P[t]
the random functions Sbn (t) = n1/2 S(nt) where S(t) = k=1 k + (t [t])[t]+1
and [t] denotes the integer part of t.
P
(a) Verify that Lindebergs clt applies for Sbn = nk=1 an,k k whenever the
non-random
Pn{an,k } are such that rn = max{|an,k | : k = 1, . . . , n} 0
and vn = k=1 a2n,k 1.
(b) Let c(s, t) = min(s, t) and fixing 0 = t0 t1 < < td , denote by C the
d d matrix of entries Cjk = c(tj , tk ). Show that for any Rd ,
d
X
r=1
r
X
j )2 = (, C) ,
(tr tr1 )(
j=1
D
(c) Using the Cramer-Wold device deduce that (Sbn (t1 ), . . . , Sbn (td )) G
with G having the N (0, C) distribution.
As we see in the next exercise, there is more to a Gaussian random vector than
each coordinate having a normal distribution.
Exercise 3.5.19. Suppose X1 has a standard normal distribution and S is independent of X1 and such that P(S = 1) = P(S = 1) = 1/2.
(a) Check that X2 = SX1 also has a standard normal distribution.
(b) Check that X1 and X2 are uncorrelated random variables, each having
the standard normal distribution, while X = (X1 , X2 ) is not a Gaussian
random vector and where X1 and X2 are not independent variables.
Motivated by the proof of Proposition 3.5.14 here is an important property of

Gaussian random vectors which may also be considered to be an alternative to
Definition 3.5.13.
Exercise 3.5.20. A random vector X has the multivariate normal distribution
Pd
if and only if ( i=1 aji Xi , j = 1, . . . , m) is a Gaussian random vector for any
non-random coefficients a11 , a12 , . . . , amd R.
The classical definition of the multivariate normal density applies for a strict subset
of the distributions we consider in Definition 3.5.13.
Definition 3.5.21. We say that X has a non-degenerate multivariate normal

distribution if the matrix V is invertible, or alternatively, when V is (strictly)
positive definite matrix, that is (, V) > 0 whenever 6= 0.
We next relate the density of a random vector with its characteristic function, and
provide the density for the non-degenerate multivariate normal distribution.
Exercise 3.5.22.
R
(a) Show that if Rd |X ()|d < , then X has the bounded continuous
probability density function
Z
1
(3.5.5)
fX (x) =
ei(,x) X ()d .
(2)d Rd
150
(b) Show that a random vector X with a non-degenerate multivariate normal

distribution N (, V) has the probability density function
1

fX (x) = (2)d/2 (detV)1/2 exp (x , V1 (x )) .
2
Here is an application to the uniform distribution over the sphere in Rn , as n .
2
Exercise 3.5.23. Suppose
variables with EY1 = 1 and
Pn {Yk }2 are i.i.d. random
1
EY1 = 0. Let Wn = n
k=1 Yk and Xn,k = Yk / Wn for k = 1, . . . , n.
D
a.s.
(a) Noting that Wn 1 deduce that Xn,1 Y1 .

Pn
D
(b) Show that n1/2 k=1 Xn,k G whose distribution is N (0, 1).
(c) Show that if {Yk } are standard normal random variables, then the random vector X n = (Xn,1 , . . . , Xn,n
) has the uniform distribution over the
surface of the sphere of radius n in Rn (i.e., the unique measure supported on this sphere and invariant under orthogonal transformations),
and interpret the preceding results for this special case.
Next you find an interesting property about the coordinate of maximal value in
certain Gaussian random vectors.
Exercise 3.5.24. Suppose random vector X = (X1 , X2 , , Xd ) has the multivariate normal distribution N (0, V), with Vii = 1 for all i and Vij < 1 for all
i 6= j.
(a) Show that for each 1 j d, the random variable Xj is independent of
nX V X o
i
ij j
.
Mj := max
1id,i6=j
1 Vij
(b) Check that with probability one, the index
j := arg max Xj ,
1jd
is uniquely attained and that j = j if and only if Xj Mj .

(c) Deduce that U (Xj , Mj ) is uniformly distributed on (0, 1), where U (x, m) :=
(1 FG (x))/(1 FG (m)) for x, m R and a standard normal variable
G.
We conclude the section with the following exercise, which is a multivariate, Lindebergs type clt.
Exercise 3.5.25. Let y t denotes the transpose of the vector y Rd and kyk
its Euclidean norm. The independent random vectors {Y k } on Rd are such that
D
Y k = Y k ,
n
X
lim
P(kY k k > n) = 0,
n
k=1
and for some symmetric, (strictly) positive definite matrix V and any fixed
(0, 1],
n
X
lim n1
E(Y k Y tk IkY k kn ) = V.
n
Pn
k=1
(a) Let T n = k=1 X n,k for X n,k = n1/2 Y k IkY k kn . Show that T n
G, with G having the N (0, V) multivariate normal distribution.
Pn
D
(b) Let Sbn = n1/2 k=1 Y k and show that Sbn G.
D
(c) Show that (Sbn )t V1 Sbn Z and identify the law of Z.
151
CHAPTER 4
Conditional expectations and probabilities

The most important concept in probability theory is the conditional expectation
to which this chapter is devoted. In contrast with the elementary definition often
used for a finite or countable sample space, the conditional expectation, as defined
in Section 4.1, is itself a random variable. Section 4.2 details the important properties of the conditional expectation. Section 4.3 provides a representation of the
conditional expectation as an orthogonal projection in Hilbert space. Finally, in
Section 4.4 we represent the conditional expectation also as the expectation with
respect to the random regular conditional probability distribution.
4.1. Conditional expectation: existence and uniqueness
In Subsection 4.1.1 we review the elementary definition of the conditional expectation E(X|Y ) in case of discrete valued R.V.-s X and Y . This motivates our
formal definition of the conditional expectation for any pair of R.V.s. such that
X is integrable. The existence and uniqueness of the conditional expectation is
shown there based on the Radon-Nikodym theorem, the proof of which we provide
in Subsection 4.1.2.
4.1.1. Conditional expectation: motivation and definition. Suppose
the R.V.s X and Z on a probability space (, F , P) are both simple functions.
More precisely, let X take the distinct values x1 , . . . , xm R and Z take the
distinct values z1 , . . . , zn R, where without loss of generality we assume that
P(Z = zi ) > 0 for i = 1, . . . , n. Then, from elementary probability theory, we know
that for any i = 1, . . . , n, j = 1, . . . , m,
P(X = xj |Z = zi ) =
P(X = xj , Z = zi )
,
P(Z = zi )
and we can compute the corresponding conditional expectation

E[X|Z = zi ] =
m
X
j=1
xj P(X = xj |Z = zi ) .
Noting that this conditional expectation is a function of (via the value of

Z()), we define the R.V. Y = E[X|Z] on the same probability space such that
Y () = E[X|Z = zi ] whenever is such that Z() = zi .
Example 4.1.1. Suppose that X = 1 and Z = 2 on the probability space F =
2 , = {1, 2}2 with
P(1, 1) = .5, P(1, 2) = .1, P(2, 1) = .1, P(2, 2) = .3.
153
154
4. CONDITIONAL EXPECTATIONS AND PROBABILITIES
Then,
P(X = 1|Z = 1) =
implying that P(X = 2|Z = 1) =
1
6
5
P(X = 1, Z = 1)
= ,
P(Z = 1)
6
and
5
1
7
+2 = .
6
6
6
Likewise, check that E[X|Z = 2] = 74 , hence E[X|Z] = 67 IZ=1 + 74 IZ=2 .
E[X|Z = 1] = 1
Partitioning into the discrete collection of Z-atoms, namely the sets Gi = { :

Z() = zi } for i = 1, . . . , n, observe that Y () is constant on each of these sets.
The -algebra G = F Z = (Z) = {Z 1 (B), B B} is in this setting merely
the collection of all 2n possible unions of various Z-atoms. Hence, G is finitely
generated and since Y () is constant on each generator Gi of G, we seeSthat Y ()
is measurable on (, G). Further, since any G G is of the form G = iI Gi for
the disjoint sets Gi and some I {1, . . . , n}, we find that
X
X
E[Y IG ] =
E[Y IGi ] =
E[X|Z = zi ]P(Z = zi )
iI
iI
m
XX
xj P(X = xj , Z = zi ) = E[XIG ] .
iI j=1
To summarize, in case X and Z are simple functions and G = (Z), we have

Y = E[X|Z] as a R.V. on (, G) such that E[Y IG ] = E[XIG ] for all G G.
Since both properties make sense for any -algebra G and any integrable R.V. X
this suggests the definition of the conditional expectation as given by the following
theorem.
Theorem 4.1.2. Given X L1 (, F , P) and G F a -algebra there exists a
R.V. Y called the conditional expectation (C.E.) of X given G, denoted by E[X|G],
such that Y L1 (, G, P) and for any G G,
(4.1.1)
E [(X Y ) IG ] = 0 .
Moreover, if (4.1.1) holds for any G G and R.V.s Y and Ye , both of which are in
L1 (, G, P), then P(Ye = Y ) = 1. In other words, the C.E. is uniquely defined for
P-almost every .
Remark. We call Y L1 (, G, P) that satisfies (4.1.1) for all G G a version

of the C.E. E[X|G]. In view of the preceding theorem, unless stated otherwise we
consider all versions of the C.E. as being the same R.V.
Given our motivation for Theorem 4.1.2, we let E[X|Z] stand for E[X|F Z ] and
likewise E[X|Z1 , Z2 , . . .] stand for E[X|F Z ], where F Z = (Z1 , Z2 , . . .).
To check whether a R.V. is a C.E. with respect to a given -algebra G, it suffices

to verify (4.1.1) for some -system that contains and generates G, as you show in
the following exercise. This useful general observation is often key to determining
an explicit formula for the C.E.
Exercise 4.1.3. Suppose that P is a -system of subsets of such that P
and G = (P) F . Show that if X L1 (, F , P) and Y L1 (, G, P) are such
that E[XIG ] = E[Y IG ] for every G P then Y = E[X|G].
4.1. CONDITIONAL EXPECTATION: EXISTENCE AND UNIQUENESS
155
To prove the existence of the C.E. we need the following definition of absolute
continuity of measures.
Definition 4.1.4. Let and be two measures on measurable space (S, F ). We
say that is absolutely continuous with respect to , denoted by , if
(A) = 0
(A) = 0
for any set A F .

Recall Proposition 1.3.56 that given a measure on (S, F ), any f mF+ induces a
new measure f on (S, F ). The next theorem, whose proof is deferred to Subsection
4.1.2, shows that all absolutely continuous -finite measures with respect to a finite measure are of this form.
Theorem 4.1.5 (Radon-Nikodym theorem). If and are two -finite measures on (S, F ) such that , then there exists f mF+ finite valued such that
= f . Further, if f = g then ({s : f (s) 6= g(s)}) = 0.
Remark. The assumption in Radon-Nikodym theorem that is a -finite measure
can be somewhat relaxed, but not completely dispensed off.
Definition 4.1.6. The function f such that = f is called the Radon-Nikodym
d
.
derivative (or density) of with respect to and denoted f = d
We note in passing that a real-valued R.V. has a probability density function f
if and only if its law is absolutely continuous with respect to the completion
of Lebesgue measure on (R, B), with f being the corresponding Radon-Nikodym
derivative (c.f. Example 1.3.60).
Proof of Theorem 4.1.2. Given two versions Y and Ye of E[X|G] we apply
(4.1.1) for the set G = { : Y () Ye () > } to see that (by linearity of the
expectation),
0 = E [XIG ] E [XIG ] = E [(Y Ye ) IG ] P(G ) .
Hence, P(G ) = 0. Since this applies for any > 0 and G G0 as 0, we deduce
that P(Y Ye > 0) = 0. The same argument applies with the roles of Y and Ye
reversed, so P(Y Ye = 0) = 1 as claimed.
We turn to the existence of the C.E. assuming first that X L1 (, F , P) is also
non-negative. Let denote the probability measure obtained by restricting P to
the measurable space (, G) and denote the measure obtained by restricting XP
of Proposition 1.3.56 to this measurable space, noting that is a finite measure
(since () = (XP)() = E[X] < ). If G G is such that (G) = P(G) = 0,
then by definition also (G) = (XP)(G) = 0. Therefore, is absolutely continuous
with respect to , and by the Radon-Nikodym theorem there exists Y mG+ such
that = Y . This implies that for any G G,
E[XIG ] = P(XIG ) = (G) = (Y )(G) = (Y IG ) = E[Y IG ]
(and in particular, that E[Y ] = () < ), proving the existence of the C.E. for
non-negative R.V.s.
Turning to deal with the case of a general integrable R.V. X we use the representation X = X+ X with X+ 0 and X 0 such that both E[X+ ] and E[X ] are
finite. Set Y = Y + Y where the integrable, non-negative R.V.s Y = E[X |G]
156
exist by the preceding argument. Then, Y mG is integrable, and by definition of

Y we have that for any G G
E[Y IG ] = E[Y + IG ] E[Y IG ] = E[X+ IG ] E[X IG ] = E[XIG ] .
This establishes (4.1.1) and completes the proof of the theorem.
Remark. Beware that for Y = E[X|G] often Y 6= E[X |G] (for example, take
the trivial G = {, } and P(X = 1) = P(X = 1) = 1/2 for which Y = 0 while
E[X |G] = 1).
Exercise 4.1.7. Suppose either E(Yk )+ is finite or E(Yk ) is finite for random
variables Yk , k = 1, 2 on (, F , P) such that E[Y1 IA ] E[Y2 IA ] for any A F .
Show that then P(Y1 Y2 ) = 1.
In the next exercise you show that the Radon-Nikodym density preserves the
product structure.
Exercise 4.1.8. Suppose that k k are pairs of -finite measures on (Sk , Fk )
for k = 1, . . . , n with the corresponding Radon-Nikodym derivatives fk = dk /dk .
(a) Show that the -finite product measure = 1 n on the product
space (S, F ) is absolutely continuous with respect toQthe -finite measure
n
= 1 n on (S, F ), with d/d(s) = k=1 fk (sk ) for s =
(s1 , . . . , sn ).
(b) Suppose and are probability measures on S = {(s1 , . . . , sn ) : sk
Sk , k = 1, . . . , n}. Show that fk (sk ), k = 1, . . . , n, are both mutually
-independent and mutually -independent.
4.1.2. Proof of the Radon-Nikodym theorem. This section is devoted
to proving the Radon-Nikodym theorem, which we have already used for establishing the existence of C.E. This is done by proving the more general Lebesgue
decomposition, based on the following definition.
Definition 4.1.9. Two measures 1 and 2 on the same measurable space (S, F )
are mutually singular if there is a set A F such that 1 (A) = 0 and 2 (Ac ) = 0.
This is denoted by 1 2 , and we sometimes state that 1 is singular with respect
to 2 , instead of 1 and 2 mutually singular.
Equipped with the concept of mutually singular measures, we next state the
Lebesgue decomposition and show that the Radon-Nikodym theorem is a direct
consequence of this decomposition.
Theorem 4.1.10 (Lebesgue decomposition). Suppose and are measures
on the same measurable space (S, F ) such that (S) and (S) are finite. Then,
= ac + s where the measure s is singular with respect to and ac = f for
some f mF+ . Further, such a decomposition of is unique (per given ).
Remark. To build your intuition, note that Lebesgue decomposition is quite
explicit for -finite measures on a countable space S (with F = 2S ). Indeed, then
ac and s are the restrictions of to the support S = {s S : ({s}) > 0}
of and its complement, respectively, with f (s) = ({s})/({s}) for s S the
Radon-Nikodym derivative of ac with respect to (see Exercise 1.2.48 for more
on the support of a measure).
4.1. CONDITIONAL EXPECTATION: EXISTENCE AND UNIQUENESS
157
Proof of the Radon-Nikodym theorem. Assume first that (S) and (S)
are finite. Let = ac + s be the unique Lebesgue decomposition induced by .
Then, by definition there exists a set A F such that s (Ac ) = (A) = 0. Further,
our assumption that implies that s (A) (A) = 0 as well, hence s (S) = 0,
i.e. = ac = f for some f mF+ .
Next, in case and are -finite measures the sample space S is a countable union
of disjoint sets An F such that both (An ) and (An ) are finite. Considering the
measures n = IAn and n = IAn such that n (S) = (An ) and n (S) = (An )
are finite, our assumption that implies that n n . Hence, by the
preceding
argument for each n there exists fn mF+ such that n = fn n . With
P
(by the composition relation of Proposition 1.3.56),
= n n and n = (fn IAn ) P
it follows that = f for f = n fn IAn mF+ finite valued.
As for the uniqueness of the Radon-Nikodym derivative f , suppose
that f = g
T
for some g mF+ and a -finite measure . Consider En = Dn {s : g(s) f (s)
1/n, g(s) n} and measurable Dn S such that (Dn ) < . Then, necessarily
both (f IEn ) and (gIEn ) are finite with
n1 (En ) ((g f )IEn ) = (g)(En ) (f )(En ) = 0 ,
implying that (En ) = 0. Considering the union over n = 1, 2, . . . we deduce

that ({s : g(s) > f (s)}) = 0, and upon reversing the roles of f and g, also
({s : g(s) < f (s)}) = 0.

Remark. Following the same argument as in the preceding proof of the RadonNikodym theorem, one easily concludes that Lebesgue decomposition applies also
for any two -finite measures and .
Our next lemma is the key to the proof of Lebesgue decomposition.
Lemma 4.1.11. If the finite measures and on (S, F ) are not mutually singular,
then there exists B F and > 0 such that (B) > 0 and (A) (A) for all
A FB .
The proof of this lemma is based on the Hahn-Jordan decomposition of a finite
signed measure to its positive and negative parts (for a definition of a finite signed
measure see the remark after Definition 1.1.2).
Theorem 4.1.12 (Hahn decomposition). For any finite signed measure :
F 7 R there exists D F such that + = ID and = IDc are measures on
(S, F ).
See [Bil95, Theorem 32.1] for a proof of the Hahn decomposition as stated here,
or [Dud89, Theorem 5.6.1] for the same conclusion in case of a general, that is
[, ]-valued signed measure, where uniqueness of the Hahn-Jordan decomposition of a signed measure as the difference between the mutually singular measures
is also shown (see also [Dur10, Theorems A.4.3 and A.4.4]).
Remark. If IB is a measure we call B F a positive set for the signed measure
and if IB is a measure we say that B F is a negative set for . So, the Hahn
decomposition provides a partition of S into a positive set (for ) and a negative
set (for ).
S
Proof of Lemma 4.1.11. Let A = n Dn where Dn , n = 1, 2 . . ., is a positive set for the Hahn decomposition of the finite signed measure n1 . Since Ac
158
is contained in the negative set Dnc for n1 , it follows that (Ac ) n1 (Ac ).
Taking n we deduce that (Ac ) = 0. If (Dn ) = 0 for all n then (A) = 0
and necessarily is singular with respect to , contradicting the assumptions of
the lemma. Therefore, (Dn ) > 0 for some finite n. Taking = n1 and B = Dn
results with the thesis of the lemma.

Proof of Lebesgue decomposition. Our goal is to construct f mF+
such that the measure s = f is singular with respect to . Since necessarily
s (A) 0 for any A F , such a function f must belong to
H = {h mF+ : (A) (h)(A),
for all A F }.
Indeed, we take f to be an element of H for which (f )(S) is maximal. To show

that such f exists note first that H is closed under non-decreasing passages to the
limit (by monotone convergence). Further, if h and e
h are both in H then also
max{h, e
h} H since with = {s : h(s) > e
h(s)} we have that for any A F ,
h}IA ) .
(A) = (A ) + (A c ) (hIA ) + (e
hIAc ) = (max{h, e
That is, H is also closed under the formation of finite maxima and in particular, the function limn max(h1 , . . . , hn ) is in H for any hn H. Now let =
sup{(h)(S) : h H} noting that (S) is finite. Choosing hn H such
that (hn )(S) n1 results with f = limn max(h1 , . . . , hn ) in H such that
(f )(S) limn (hn )(S) = . Since f is an element of H both ac = f and
s = f are finite measures.
If s fails to be singular with respect to then by Lemma 4.1.11 there exists
B F and > 0 such that (B) > 0 and s (A) (IB )(A) for all A F . Since
= s +f , this implies that f +IB H. However, ((f +IB ))(S) +(B) >
contradicting the fact that is the finite maximal value of (h)(S) over h H.
Consequently, this construction of f has = f + s with a finite measure s that
is singular with respect to .
Finally, to prove the uniqueness of the Lebesgue decomposition, suppose there
exist f1 , f2 mF+ , such that both f1 and f2 are singular with respect to
. That is, there exist A1 , A2 F such that (Ai ) = 0 and ( fi )(Aci ) = 0 for
i = 1, 2. Considering A = A1 A2 it follows that (A) = 0 and ( fi )(Ac ) = 0
for i = 1, 2. Consequently, for any E F we have that ( f1 )(E) = (E A) =
( f2 )(E), proving the uniqueness of s , and hence of the decomposition of as
ac + s .

We conclude with a simple application of Radon-Nikodym theorem in conjunction
with Lemma 1.3.8.
Exercise 4.1.13. Suppose and are two -finite measures on the same measurable space (S, F ) such that (A) (A) for all A F . Show that if (g) = (g)
is finite for some g mF such that ({s : g(s) 0}) = 0 then () = ().
4.2. Properties of the conditional expectation
In some generic settings the C.E. is rather explicit. One such example is when X
is measurable on the conditioning -algebra G.
Example 4.2.1. If X L1 (, G, P) then Y = X mG satisfies (4.1.1) so

E[X|G] = X. In particular, if X = c is a constant R.V. then E[X|G] = c for
any -algebra G.
4.2. PROPERTIES OF THE CONDITIONAL EXPECTATION
159
Here is an extension of this example.

Exercise 4.2.2. Suppose that (Y, Y)-valued random variable Y is measurable on
G and (X, X)-valued random variable Z is P-independent of G. Show that if is
measurable on the product space (X Y, X Y) and (Z, Y ) is integrable, then
E[(Z, Y )|G] = g(Y ) where g(y) = E[(Z, y)].
Since only constant random variables are measurable on F0 = {, }, by definition
of the C.E. clearly E[X|F0 ] = EX. We show next that E[X|H] = EX also whenever
the conditioning -algebra H is independent of (X) (and in particular, when H is
P-trivial).
Proposition 4.2.3. If X L1 (, F , P) and the -algebra H is independent of
((X), G), then
E[X|(H, G)] = E[X| G] .
For G = {, } this implies that
H independent of (X)
E[X|H] = EX .
Remark. Recall that a P-trivial -algebra H F is independent of (X) for

any X mF . Hence, by Proposition 4.2.3 in this case E[X|H] = EX for all
X L1 (, F , P ).
Proof. Let Y = E[X|G] mG. Because H is independent of (G, (X)), it
follows that for any G G and H H the random variable IH is independent of
both XIG and Y IG . Consequently,
E[XIGH ] = E[XIG IH ] = E[XIG ]EIH
E[Y IGH ] = E[Y IG IH ] = E[Y IG ]EIH
Further, E[XIG ] = E[Y IG ] by the definition of Y , hence E[XIA ] = E[Y IA ] for
any A A = {G H : G G, H H}. Applying Exercise 4.1.3 with Y
L1 (, G, P) L1 (, (H, G), P) and A a -system of subsets containing and
generating (G, H), we thus conclude that
as claimed.
E[X|(G, H)] = Y = E[X|G]
We turn to derive various properties of the C.E. operation, starting with its positivity and linearity (per fixed conditioning -algebra).
Proposition 4.2.4. Let X L1 (, F ,P) and set Y = E[X|G] for some -algebra
G F . Then,
(a) EX = EY
(b) ( Positivity) X 0 = Y 0 a.s. and X > 0 = Y > 0
a.s.
Proof. Considering G = G in the definition of the C.E. we find that
EX = E[XIG ] = E[Y IG ] = EY .
Turning to the positivity of the C.E. note that if X 0 a.s. then 0 E[XIG ] =
E[Y IG ] 0 for G = { : Y () 0} G. Hence, in this case E[Y IY 0 ] = 0.
That is, almost surely Y 0. Further, P(X > , Y 0) E[XIX> IY 0 ]
E[XIY 0 ] = 0 for any > 0, so P(X > 0, Y = 0) = 0 as well.

We next show that the C.E. operator is linear.
160
Proposition 4.2.5. ( Linearity) Let X, Y L1 (, F ,P) and G F a -algebra.

Then, for any , R,
E[X + Y |G] = E[X|G] + E[Y |G] .
Proof. Let Z = E[X|G] and V = E[Y |G]. Since Z, V L1 (, F , P) the

same applies for Z + V . Further, for any G G, by linearity of the expectation
operator and the definition of the C.E.
E[(Z + V )IG ] = E[ZIG ] + E[V IG ] = E[XIG ] + E[Y IG ] = E[(X + Y )IG ],
as claimed.
From its positivity and linearity we immediately get the monotonicity of the C.E.
Corollary 4.2.6 (Monotonicity). If X, Y L1 (, F , P) are such that X Y ,
then E[X|G] E[Y |G] for any -algebra G F .
In the following exercise you are to combine the linearity and positivity of the
C.E. with Fubinis theorem.
Exercise 4.2.7. Show that if X, Y L1 (, F , P) are such that E[X|Y ] = Y and
E[Y |X] = X then almost surely X = Y .
Hint: First show that E[(X Y )I{X>cY } ] = 0 for any non-random c.
We next deal with the relationship between the C.E.s of the same R.V. for nested
conditioning -algebras.
Proposition 4.2.8 (Tower property). Suppose X L1 (, F ,P) and the algebras H and G are such that H G F . Then, E[X|H] = E[E(X|G)|H].
Proof. Recall that Y = E[X|G] is integrable, hence Z = E[Y |H] is integrable.

Fixing A H we have that E[Y IA ] = E[ZIA ] by the definition of the C.E. Z. Since
H G, also A G hence E[XIA ] = E[Y IA ] by the definition of the C.E. Y . We
deduce that E[XIA ] = E[ZIA ] for all A H. It then follows from the definition of
the C.E. that Z is a version of E[X|H].
Remark. The tower property is also called the law of iterated expectations.
Any -algebra G contains the trivial -algebra F0 = {, }. Applying the tower
property with H = F0 and using the fact that E[Y |F0 ] = EY for any integrable
random variable Y , it follows that for any -algebra G
(4.2.1)
EX = E[X|F0 ] = E[E(X|G)|F0 ] = E[E(X|G)] .
Here is an application of the tower property, leading to stronger conclusion than

what one has from Proposition 4.2.3.
Lemma 4.2.9. If integrable R.V. X and -algebra G are such that E[X|G] is independent of X, then E[X|G] = E[X].
Proof. Let Z = E[X|G]. Applying the tower property for H = (Z) G we
have that E[X|H] = E[Z|H]. Clearly, E[Z|H] = Z (see Example 4.2.1), whereas
our assumption that X is independent of Z implies that E[X|H] = E[X] (see
Proposition 4.2.3). Consequently, Z = E[X], as claimed.

As shown next, we can take out what is known when computing the C.E.
Proposition 4.2.10. Suppose Y mG and X L1 (, F ,P) are such that XY
L1 (, F ,P). Then, E[XY |G] = Y E[X|G].
161
Proof. Let Z = E[X|G] which is well defined due to our assumption that
E|X| < . With Y Z mG and E|XY | < , it suffices to check that
(4.2.2)
E[XY IA ] = E[ZY IA ]
for all A G. Indeed, if Y = IB for B G then Y IA = IG for G = B A G so

(4.2.2) follows from the definition of the C.E. Z. By linearity of the expectation,
this extends to Y which is a simple function on (, G). Recall that for X 0 by
positivity of the C.E. also Z 0, so by monotone convergence (4.2.2) then applies
for all Y mG+ . In general, let X = X+ X and Y = Y+ Y for Y mG+
and the integrable X 0. Since |XY | = (X+ + X )(Y+ + Y ) is integrable, so
are the products X Y and (4.2.2) holds for each of the four possible choices of the
pair (X , Y ), with Z = E[X |G] instead of Z. Upon noting that Z = Z + Z
(by linearity of the C.E.), and XY = X+ Y+ X+ Y X Y+ + X Y , it readily
follows that (4.2.2) applies also for X and Y .

Adopting hereafter the notation P(A|G) for E[IA |G], the following exercises illustrate some of the many applications of Propositions 4.2.8 and 4.2.10.
Exercise 4.2.11. For any -algebras Gi F , i = 1, 2, 3, let Gij = (Gi , Gj ) and
prove that the following conditions are equivalent:
(a) P[A3 |G12 ] = P[A3 |G2 ] for all A3 G3 .
(b) P[A1 A3 |G2 ] = P[A1 |G2 ]P[A3 |G2 ] for all A1 G1 and A3 G3 .
(c) P[A1 |G23 ] = P[A1 |G2 ] for all A1 G1 .
Remark. Taking G1 = (Xk , k < n), G2 = (Xn ) and G3 = (Xk , k > n),
condition (a) of the preceding exercise states that the sequence of random variables
{Xk } has the Markov property. That is, the conditional probability of a future
event A3 given the past and present information G12 is the same as its conditional
probability given the present G2 alone. Condition (c) makes the same statement,
but with time reversed, while condition (b) says that past and future events A1 and
A3 are conditionally independent given the present information, that is, G2 .
Exercise 4.2.12. Let Z = (X, Y ) be a uniformly chosen point in (0, 1)2 . That
is, X and Y are independent random variables, each having the U (0, 1) measure
of Example 1.1.26. Set T = 2IA (Z) + 10IB (Z) + 4IC (Z) where A = {(x, y) :
0 < x < 1/4, 3/4 < y < 1}, B = {(x, y) : 1/4 < x < 3/4, 0 < y < 1/2} and
C = {(x, y) : 3/4 < x < 1, 1/4 < y < 1}.
(a) Find an explicit formula for the conditional expectation W = E(T |X)
and use it to determine the conditional expectation U = E(T X|X).
(b) Find the value of E[(T W ) sin(eX )].
Exercise 4.2.13. Fixing a positive integer k, compute E(X|Y ) in case Y = kX
[kX] for X having the U (0, 1) measure of Example 1.1.26 (and where [x] denotes
the integer part of x).
Exercise 4.2.14. Fixing t R and X integrable random variable, let Y =
max(X, t) and Z = min(X, t). Setting at = E[X|X t] and bt = E[X|X t],
show that E[X|Y ] = Y IY >t + at IY =t and E[X|Z] = ZIZ<t + bt IZ=t .
Exercise 4.2.15. Let X, Y be i.i.d. random variables. Suppose is independent
of (X, Y ), with P( = 1) = p, P( = 0) = 1 p. Let Z = (Z1 , Z2 ) where
Z1 = X + (1 )Y and Z2 = Y + (1 )X.
162
(a) Prove that Z and are independent.

(b) Obtain an explicit expression for E[g(X, Y )|Z], in terms of Z1 and Z2 ,
where g : R2 7 R is a bounded Borel function.
Exercise 4.2.16. Suppose EX 2 < and define Var(X|G) = E[(X E(X|G))2 |G].
(a) Show that, E[Var(X|G2 )] E[Var(X|G1 )] for any two -algebras G1 G2
(that is, the dispersion of X about its conditional mean decreases as the
algebra grows).
(b) Show that for any -algebra G,
Var[X] = E[Var(X|G)] + Var[E(X|G)] .
Exercise 4.2.17. Suppose N is a non-negative, integer valued R.V. which is independentPof the independent, integrable R.V.-s i on the same probability space,
and that i P(N i)E|i | is finite.
(a) Check that
N ()
X
i () ,
X() =
i=1
P
is integrable and deduce that EX = i P(N i)Ei .
(b) Suppose in addition that i are identically distributed, in which case this
is merely Walds identity EX = EN E1 . Show that if both 1 and N are
square-integrable, then so is X and
Var(X) = Var(1 )EN + Var(N )(E1 )2 .

Suppose XY , X and Y are integrable. Combining Proposition 4.2.10 and (4.2.1)
convince yourself that if E[X|Y ] = EX then E[XY ] = EXEY . Recall that if X and
Y are independent and integrable then E[X|Y ] = EX (c.f. Proposition 4.2.3). As
you show next, the converse implications are false and further, one cannot dispense
of the nesting relationship between the two -algebras in the tower property.
Exercise 4.2.18. Provide examples of X, Y {1, 0, 1} such that
(a) E[XY ] = EXEY but E[X|Y ] 6= EX.
(b) E[X|Y ] = EX but X is not independent of Y .
(c) For = {1, 2, 3} and Fi = ({i}), i = 1, 2, 3,
E[E(X|F1 )|F2 ] 6= E[E(X|F2 )|F1 ].
As shown in the sequel, per fixed conditioning -algebra we can interpret the
C.E. as an expectation in a different (conditional) probability space. Indeed, every
property of the expectation has a corresponding extension to the C.E. For example,
the extension of Jensens inequality is
Proposition 4.2.19 (Jensens inequality). Suppose g() is a convex function
on an open interval G of R, that is,
g(x) + (1 )g(y) g(x + (1 )y)
x, y G,
0 1.
If X is an integrable R.V. with P(X G) = 1 and g(X) is also integrable, then

almost surely E[g(X)|H] g(E[X|H]) for any -algebra H.
Proof. Recall our derivation of (1.3.3) showing that
g(x) g(c) + (D g)(c)(x c)
c, x G
163
Further, with (D g)() a finite, non-decreasing function on G where g() is continuous, it follows that
g(x) = sup {g(c) + (D g)(c)(x c)} = sup{an x + bn }
n
cGQ
for some sequences {an } and {bn } in R and all x G.

Since P(X G) = 1, almost surely g(X) an X + bn and by monotonicity of
the C.E. also E[g(X)|H] an Y + bn for Y = E[X|H]. Further, P(Y G) = 1
due to the linearity and positivity of the C.E., so almost surely E[g(X)|H]
supn {an Y + bn } = g(Y ), as claimed.

Example 4.2.20. Fixing q 1 and applying (the conditional) Jensens inequality
for the convex function g(x) = |x|q , we have that E[|X|q |H] |E[X|H]|q for any
X Lq (, F , P). So, by the tower property and the monotonicity of the expectation,
kXkqq = E|X|q = E[E(|X|q |H)]
E[|E(X|H)|q ] = kE(X|H)kqq .
In conclusion, kXkq kE(X|H)kq for all q 1.
Exercise 4.2.21. Let Z = E[X|G] for an integrable random variable X and a

-algebra G.
(a) Show that if EZ 2 = EX 2 < then Z = X a.s.
(b) Suppose that Z = E[X|G] has the same law as X. Show that then Z = X
a.s. even if EX 2 = .
Hint: Show that E[(|X| X)IA ] = 0 for A = {Z 0} G, so X 0 for
almost every A. Applying this for X c with c non-random deduce that
P(X < c Z) = 0 and conclude that X Z a.s.
In the following exercises you are to derive the conditional versions of Markovs
and H
olders inequalities.
Exercise 4.2.22. Suppose p > 0 is non-random and X is a random variable in
(, F , P) with E|X|p finite.
(a) Prove that for every -algebra G F , with probability one
Z
p
E[|X| |G] =
pxp1 P(|X| > x |G)dx .
0
(b) Deduce the conditional version of Markovs inequality, that for any a > 0
P(|X| a |G) ap E[|X|p |G]
(compare with Lemma 1.4.31 and Example 1.3.14).

Exercise 4.2.23. Suppose E|X|p < and E|Y |q < for some p, q > 1 such
olders inequality
that p1 + 1q = 1. Prove the conditional H
E[|XY | |G] (E[|X|p |G])1/p (E[|Y |q |G])1/q
(compare with Proposition 1.3.17).
Here are the corresponding extensions of some of the convergence theorems of

Section 1.3.3.
Theorem 4.2.24 (Monotone convergence for C.E.). If 0 Xm X a.s.
and EX < , then E[Xm |G] E[X |G].
164
Proof. Let Ym = E[Xm |G] mG+ . By monotonicity of the C.E. we have that
the sequence Ym is a.s. non-decreasing, hence it has a limit Y mG+ (possibly
infinite). We complete the proof by showing that Y = E[X |G]. Indeed, for any
G G,
E[Y IG ] = lim E[Ym IG ] = lim E[Xm IG ] = E[X IG ],
m
where since Ym Y and Xm X the first and third equalities follow by the
monotone convergence theorem (the unconditional version), and the second equality
from the definition of the C.E. Ym . Considering G = we see that Y is integrable.
In conclusion, E[Xm |G] = Ym Y = E[X |G], as claimed.

Lemma 4.2.25 (Fatous lemma for C.E.). If the non-negative, integrable Xn
on same measurable space (, F ) are such that lim inf n Xn is integrable, then
for any -algebra G F ,

E lim inf Xn G lim inf E[Xn |G] a.s.
n
Proof. Applying the monotone convergence theorem for the C.E. of the nondecreasing sequence of non-negative R.V.s Zn = inf{Xk : k n} (whose limit is
the integrable lim inf n Xn ), results with

(4.2.3)
E lim inf Xn |G = E( lim Zn |G) = lim E[Zn |G] a.s.
n
Since Zn Xn it follows that E[Zn |G] E[Xn |G] for all n and
(4.2.4)
lim E[Zn |G] = lim inf E[Zn |G] lim inf E[Xn |G]
a.s.
Upon combining (4.2.3) and (4.2.4) we obtain the thesis of the lemma.
Fatous lemma leads to the C.E. version of the dominated convergence theorem.
Theorem 4.2.26 (Dominated convergence for C.E.). If supm |Xm | is intea.s
a.s.
grable and Xm X , then E[Xm |G] E[X |G].
Proof. Let Y = supm |Xm | and Zm = Y Xm 0. Applying Fatous lemma
for the C.E. of the non-negative, integrable R.V.s Zm 2Y , we see that

E lim inf Zm G lim inf E[Zm |G] a.s.
m
Since Xm converges, by the linearity of the C.E. and integrability of Y this is

equivalent to

E lim Xm G lim sup E[Xm |G] a.s.
m
Applying the same argument for the non-negative, integrable R.V.s Wm = Y + Xm

results with

E lim Xm G lim inf E[Xm |G] a.s. .
m
We thus conclude that a.s. the lim inf and lim sup of the sequence E[Xm |G] coincide
and are equal to E[X |G], as stated.

Exercise 4.2.27. Let X1 , X2 be random variables defined on same probability
space (, F , P) and G F a -algebra. Prove that (a), (b) and (c) below are
equivalent.
(a) For any Borel sets B1 and B2 ,
P(X1 B1 , X2 B2 |G) = P(X1 B1 |G)P(X2 B2 |G) .
165
(b) For any bounded Borel functions h1 and h2 ,

E[h1 (X1 )h2 (X2 )|G] = E[h1 (X1 )|G]E[h2 (X2 )|G] .
(c) For any bounded Borel function h,
E[h(X1 )|(G, (X2 ))] = E[h(X1 )|G] .
Definition 4.2.28. If one of the equivalent conditions of Exercise 4.2.27 holds we
say that X1 and X2 are conditionally independent given G.
Exercise 4.2.29. Suppose that X and Y are conditionally independent given (Z)
and that X and Z are conditionally independent given F , where F (Z). Prove
that then X and Y are conditionally independent given F .
Our next result shows that the C.E. operation is continuous with respect to Lq
convergence.
Lq
Theorem 4.2.30. Suppose Xn X . That is, Xn , X Lq (, F , P) are such

Lq
that E(|Xn X |q ) 0. Then, E[Xn |G] E[X |G] for any -algebra G F .
Proof. We saw already in Example 4.2.20 that E[Xn |G] are in Lq (, G, P)

for n . Further, by the linearity of C.E., Jensens Inequality for the convex
function |x|q as in this example, and the tower property of (4.2.1),
E[|E(Xn |G) E(X |G)|q ] = E[|E(Xn X |G)|q ]
E[E(|Xn X |q |G)] = E[|Xn X |q ] 0,
by our hypothesis, yielding the thesis of the theorem.
As you will show, the C.E. operation is also continuous with respect to the following topology of weak Lq convergence.
Definition 4.2.31. Let L (, F , P) denote the collection of all random variables
on (, F ) which are P-a.s. bounded, with kY k denoting the smallest non-random
K such that P(|Y | K) = 1. Setting p(q) : [1, ] [1, ] via p(q) = q/(q 1), we
wLq
say that Xn converges weakly in Lq to X , denoted Xn X , if Xn , X Lq

and E[(Xn X )Y ] 0 for each fixed Y such that kY kp(q) is finite (compare with
Definition 1.3.26).
Exercise 4.2.32. Show that E[Y E(X|G)] = E[XE(Y |G)] for any -algebra G
F , provided that for some q 1 and p = q/(q 1) both kXkq and kY kp are finite.
wLq
wLq
Deduce that if Xn X then E[Xn |G] E[X |G] for any -algebra G F .
In view of Example 4.2.20 we already know that for each integrable random variable X the collection {E[X|G] : G F is a -algebra} is a bounded in L1 (, F , P).
As we show next, this collection is even uniformly integrable (U.I.), a key fact in
our study of uniformly integrable martingales (see Subsection 5.3.1).
Proposition 4.2.33. For any X L1 (, F , P), the collection {E[X|H] : H F
is a -algebra} is U.I.
Proof. Fixing > 0, let = (X, ) > 0 be as in part (b) of Exercise 1.3.43
and set the finite constant M = 1 E|X|. By Markovs inequality and Example
4.2.20 we get that M P(A) E|Y | E|X| for A = {|Y | M } H and Y =
E[X|H]. Hence, P(A) by our choice of M , whereby our choice of results
166
with E[|X|IA ] (c.f. part (b) of Exercise 1.3.43). Further, by (the conditional)
Jensens inequality |Y | E[|X| |H] (see Example 4.2.20). Therefore, by definition
of the C.E. E[|X| |H],
E[|Y |I|Y |>M ] E[|Y |IA ] E[E[|X||H]IA ] = E[|X|IA ] .
Since this applies for any -algebra H F and the value of M = M (X, ) does not
depend on Y , we conclude that the collection of such Y = E[X|H] is U.I.

To check your understanding of the preceding derivation, prove the following natural extension of Proposition 4.2.33.
Exercise 4.2.34. Let C be a uniformly integrable collection of random variables
a.s.
on (, F , P). Show that the collection D of all R.V. Y such that Y = E[X|H] for
some X C and -algebra H F , is U.I.
Here is a somewhat counter intuitive fact about the conditional expectation.
a.s
Exercise 4.2.35. Suppose Yn Y in (, F , P) when n and {Yn } are

uniformly integrable.
L1
(a) Show that E[Yn |G] E[Y |G] for any -algebra G F .
(b) Provide an example of such sequence {Yn } and a -algebra G F such
that E[Yn |G] does not converge almost surely to E[Y |G].
4.3. The conditional expectation as an orthogonal projection
It readily follows from our next proposition that for X L2 (, F , P) and algebras G F the C.E. Y = E[X|G] is the unique Y L2 (, G, P) such that
(4.3.1)
kX Y k2 = inf{kX W k2 : W L2 (, G, P)}.
Proposition 4.3.1. For any X L2 (, F , P) and -algebras G F , a R.V.

Y L2 (, G, P) is optimal in the sense of (4.3.1) if and only if it satisfies the
orthogonality relations
(4.3.2)
E[(X Y )Z] = 0
for all
Z L2 (, G, P) .
Further, any such R.V. Y is a version of E[X|G].

Proof. If Y L2 (, G, P) satisfies (4.3.1) then considering W = Y + Z it
follows that for any Z L2 (, G, P) and R,
0 kX Y Zk22 kX Y k22 = 2 EZ 2 2E[(X Y )Z] .
By elementary calculus, this inequality holds for all R if and only if E[(X
Y )Z] = 0. Conversely, suppose Y L2 (, G, P) satisfies (4.3.2) and fix W
L2 (, G, P). Then, considering (4.3.2) for Z = W Y we see that
kX W k22 = kX Y k22 2E[(X Y )(W Y )] + kW Y k22 kX Y k22 ,
so necessarily Y satisfies (4.3.1). Finally, since IG L2 (, G, P) for any G G, if
Y satisfies (4.3.2) then it also satisfies the identity (4.1.1) which characterizes the
C.E. E[X|G].
4.3. THE CONDITIONAL EXPECTATION AS AN ORTHOGONAL PROJECTION
167
Example 4.3.2. If G = (A1 , . . . , An ) for finite n and disjoint sets Ai such that
P(Ai )P> 0 for i = 1, . . . , n, then L2 (, G, P) consists of all variables of the form
W = ni=1 vi IAi , vi R. A R.V. Y of this form satisfies (4.3.1) if and only if the
corresponding {vi } minimizes
E[(X
n
X
i=1
vi IAi )2 ] EX 2 =
n
nX
i=1
P(Ai )vi2 2
n
X
i=1
o
vi E[XIAi ] ,
Pn
which amounts to vi = E[XIAi ]/P(Ai ). In particular, if Z =
i=1 zi IAi for
distinct zi -s, then (Z) = G and we thus recover our first definition of the C.E.
E[X|Z] =
n
X
E[XIZ=zi ]
IZ=zi .
P(Z = zi )
i=1
As shown in the sequel, using (4.3.1) as an alternative characterization of the

C.E. of X L2 (, F , P) we can prove the existence of the C.E. without invoking
the Radon-Nikodym theorem. We start by defining the relevant concepts from the
theory of Hilbert spaces on which this approach is based.
Definition 4.3.3. A linear vector space is a set H that is closed under operations
of addition and multiplication by (real-valued) scalars. That is, if h1 , h2 H then
h1 + h2 H and h H for all R, where (h1 + h2 ) = h1 + h2 , ( + )h =
h + h, (h) = ()h and 1h = h. A normed vector space is a linear vector
space H equipped with a norm k k. That is, a non-negative function on H such
that khk = ||khk for all R and d(h1 , h2 ) = kh1 h2 k is a metric on H.
Definition 4.3.4. A sequence {hn } in a normed vector space is called a Cauchy
sequence if supk,mn khk hm k 0 as n and we say that {hn } converges to
h H if khn hk 0 as n . A Banach space is a normed vector space in
which every Cauchy sequence converges.
Building on the preceding, we define the concept of inner product and the corresponding Hilbert spaces and sub-spaces.
Definition 4.3.5. A Hilbert space is a Banach space H whose norm is of the
form (h, h)1/2 for a bi-linear, symmetric function (h1 , h2 ) : H H 7 R such that
(h, h) 0 and we call such (h1 , h2 ) an inner product for H. A subset K of a Hilbert
space which is closed under addition and under multiplication by a scalar is called
a Hilbert sub-space if every Cauchy sequence {hn } K has a limit in K.
Here are two elementary properties of inner products we use in the sequel.
1
Exercise 4.3.6. Let khk = (h, h) 2 with (h1 , h2 ) an inner product for a linear
vector space H. Show that Schwarz inequality
(u, v)2 kuk2 kvk2 ,
and the parallelogram law ku+vk2 +kuvk2 = 2kuk2 +2kvk2 hold for any u, v H.
Our next proposition shows that for each finite q 1 the space Lq (, F , P) is a
Banach space for the norm k kq , the usual addition of R.V.s and the multiplication
of a R.V. X() by a non-random (scalar) constant. Further, L2 (, G, P) is a Hilbert
sub-space of L2 (, F , P) for any -algebras G F .
168
Proposition 4.3.7. Upon identifying R-valued R.V. which are equal with probability one as being in the same equivalence class, for each q 1 and a -algebra F ,
the space Lq (, F , P) is a Banach space for the norm k kq . Further, L2 (, G, P) is
then a Hilbert sub-space of L2 (, F , P) for the inner product (X, Y ) = EXY and
any -algebras G F .
Proof. Fixing q 1, we identify X and Y such that P(X 6= Y ) = 0 as
being the same element of Lq (, F , P). The resulting set of equivalence classes is a
normed vector space. Indeed, both kkq , the addition of R.V. and the multiplication
by a non-random scalar are compatible with this equivalence relation. Further, if
X, Y Lq (, F , P) then kXkq = ||kXkq < for all R and by Minkowskis
inequality kX + Y kq kXkq + kY kq < . Consequently, Lq (, F , P) is closed
under the operations of addition and multiplication by a non-random scalar, with
k kq a norm on this collection of equivalence classes.
Suppose next that {Xn } Lq is a Cauchy sequence for k kq . Then, by definition,
there exist kn such that kXr Xs kqq < 2n(q+1) for all r, s kn . Observe that
by Markovs inequality
P(|Xkn+1 Xkn | 2n ) 2nq kXkn+1 Xkn kqq < 2n ,
and consequently the sequence

P(|Xkn+1 Xkn | 2n ) is summable. By BorelP
Cantelli I it follows that n |Xkn+1 () Xkn ()| is finite with probability one, in
which case clearly
n1
X
(Xki+1 Xki )
Xkn = Xk1 +
i=1
converges to a finite limit X(). Next let, X = lim supn Xkn (which per Theorem 1.2.22 is an R-valued R.V.). Then, fixing n and r kn , for any t n,
i
h
E |Xr Xkt |q = kXr Xkt kqq 2nq ,
so that by the a.s. convergence of Xkt to X and Fatous lemma
i
h
E|Xr X|q = E lim |Xr Xkt |q lim inf E|Xr Xkt |q 2nq .
t
This inequality implies that Xr X Lq and hence also X Lq . As r so

Lq
does n and we can further
deduce from the preceding inequality that Xr X.
Recall that |EXY | EX 2 EY 2 by the Cauchy-Schwarz inequality. Thus, the bilinear, symmetric function (X, Y ) = EXY on L2 L2 is real-valued and compatible
with our equivalence relation. As kXk22 = (X, X), the Banach space L2 (, F , P) is
a Hilbert space with respect to this inner product.
Finally, observe that for any -algebra G F the subset L2 (, G, P) of the Hilbert
space L2 (, F , P) is closed under addition of R.V.s and multiplication by a nonrandom constant. Further, as shown before, the L2 limit of a Cauchy sequence
{Xn } L2 (, G, P) is lim supn Xkn which also belongs to L2 (, G, P). Hence, the
latter is a Hilbert subspace of L2 (, F , P).

Remark. With minor notational modifications, this proof shows that for any
measure on (S, F ) and q 1 finite, the set Lq (S, F , ) of -a.e. equivalence classes
of R-valued, measurable functions f such that (|f |q ) < , is a Banach space. This
is merely a special case of a general extension of this property, corresponding to
Y = R in your next exercise.
4.3. THE CONDITIONAL EXPECTATION AS AN ORTHOGONAL PROJECTION
169
Exercise 4.3.8. For q 1 finite and a given Banach space (Y, k k), consider
the space Lq (S, F , ; Y) of all -a.e. equivalence classes of functions f : S 7 Y,
measurable with respect to the Borel -algebra induced on Y by k k and such that
(kf ()kq ) < .
(a) Show that kf kq = (kf ()kq )1/q makes Lq (S, F , ; Y) into a Banach
space.
(b) For future applications of the preceding, verify that the space Y = Cb (T)
of bounded, continuous real-valued functions on a topological space T is
a Banach space for the supremum norm kf k = sup{|f (t)| : t T}.
Your next exercise extends Proposition 4.3.7 to the collection L (, F , P) of all

R-valued R.V. which are in equivalence classes of bounded random variables.
Exercise 4.3.9. Fixing a probability space (, F , P) prove the following facts:
(a) L (, F , P) is a Banach space for kXk = inf{M : P(|X| M ) = 1}.
(b) kXkq kXk as q , for any X L (, F , P).
(c) If E[|X|q ] < for some q > 0 then E|X|q P(|X| > 0) as q 0.
(d) The collection SF of simple functions is dense in Lq (, F , P) for any
1 q .
(e) The collection Cb (R) of bounded, continuous real-valued functions, is
dense in Lq (R, B, ) for any q 1 finite.
Hint: The (bounded) monotone class theorem might be handy.
In view of Proposition 4.3.7, the existence of the C.E. of X L2 which satisfies

(4.3.1), or the equivalent condition (4.3.2), is a special instance of the following
fundamental geometric property of Hilbert spaces.
Theorem 4.3.10 (Orthogonal projection). Given h H and a Hilbert subspace G of H, let d = inf{kh gk : g G}. Then, there exists a unique b
h G,
b
called the orthogonal projection of h on G, such that d = kh hk. This is also the
unique b
h G such that (h b
h, f ) = 0 for all f G.
Proof. We start with the existence of b
h G such that d = kh b
hk. To this
end, let gn G be such that kh gn k d. Applying the parallelogram law for
u = h 21 (gm + gk ) and v = 12 (gm gk ) we find that
1
1
1
kh gk k2 + kh gmk2 = 2kh (gm + gk )k2 + 2k (gm gk )k2 2d2 + kgm gk k2
2
2
2
since 21 (gm + gk ) G. Taking k, m , both kh gk k2 and kh gm k2 approach
d2 and hence by the preceding inequality kgm gk k 0. In conclusion, {gn } is a
Cauchy sequence in the Hilbert sub-space G, which thus converges to some b
h G.
Recall that kh b
hk d by the definition of d. Since for n both kh gnk d
and kgn b
hk 0, the converse inequality is a consequence of the triangle inequality
kh b
hk kh gn k + kgn b
hk.
Next, suppose there exist g1 , g2 G such that (h gi , f ) = 0 for i = 1, 2 and
all f G. Then, by linearity of the inner product (g1 g2 , f ) = 0 for all f G.
Considering f = g1 g2 G we see that (g1 g2 , g1 g2 ) = kg1 g2 k2 = 0 so
necessarily g1 = g2 .
We complete the proof by showing that b
h G is such that kh b
hk2 kh gk2
b
for all g G if and only if (h h, f ) = 0 for all f G. This is done exactly as in
170
the proof of Proposition 4.3.1. That is, by symmetry and bi-linearity of the inner
product, for all f G and R,
kh b
h f k2 kh b
hk2 = 2 kf k2 2(h b
h, f )
We arrive at the stated conclusion upon noting that fixing f , this function is nonnegative for all if and only if (h b
h, f ) = 0.
Applying Theorem 4.3.10 for the Hilbert subspace G = L2 (, G, P) of L2 (, F , P)

(see Proposition 4.3.7), you have the existence of a unique Y G satisfying (4.3.2)
for each non-negative X L2 .
Exercise 4.3.11. Show that for any non-negative integrable X, not necessarily in
L2 , the sequence Yn G corresponding to Xn = min(X, n) is non-decreasing and
that its limit Y satisfies (4.1.1). Verify that this allows you to prove Theorem 4.1.2
without ever invoking the Radon-Nikodym theorem.
Exercise 4.3.12. Suppose G F is a -algebra.
(a) Show that for any X L1 (, F , P) there exists some G G such that
E[XIG ] = sup E[XIA ] .
AG
Any G with this property is called G-optimal for X.

(b) Show that Y = E[X|G] almost surely, if and only if for any r R, the
event { : Y () > r} is G-optimal for the random variable (X r).
Here is an alternative proof of the existence of E[X|G] for non-negative X L2

which avoids the orthogonal projection, as well as the Radon-Nikodym theorem
(the general case then follows as in Exercise 4.3.11).
Exercise 4.3.13. Suppose X L2 (, F , P) is non-negative. Assume first that
the -algebra G F is countably generated. That is G = (B1 , B2 , . . .) for some
Bk F .
(a) Let Yn = E[X|Gn ] for the finitely generated Gn = (Bk , k n) (for its
existence, see Example 4.3.2). Show that Yn is a Cauchy sequence in
L2 (, G, P), hence it has a limit Y L2 (, G, P).
(b) Show that Y = E[X|G].
Hint: Theorem 2.2.10 might be of some help.
Assume now that G is not countably generated.
(c) Let H1 H2 be finite -algebras. Show that
E[E(X|H1 )2 ] E[E(X|H2 )2 ] .
(d) Let = sup E[E(X|H)2 ], where the supremum is over all finite -algebras
H G. Show that is finite, and that there exists an increasing sequence
of finite -algebras Hn such that E[E(X|Hn )2 ] as n .
(e) Let H = (n Hn ) and Yn = E[X|Hn ] for the Hn in part (d). Explain
why your proof of part (b) implies the L2 convergence of Yn to a R.V. Y
such that E[Y IA ] = E[XIA ] for any A H .
( f ) Fixing A G such that A
/ H , let Hn,A = (A, Hn ) and Zn =
E[X|Hn,A ]. Explain why some sub-sequence of {Zn } has an a.s. and
L2 limit, denoted Z. Show that EZ 2 = EY 2 = and deduce that
E[(Y Z)2 ] = 0, hence Z = Y a.s.
(g) Show that Y is a version of the C.E. E[X|G].
4.4. REGULAR CONDITIONAL PROBABILITY DISTRIBUTIONS
171
4.4. Regular conditional probability distributions

We first show that if the random vector (X, Z) R2 has a probability density function fX,Z (x, z) (per Definition 3.5.5), then the C.E. E[X|Z] can be computed out of the corresponding conditional probability densityR (as done in a typical elementary
probability course). To this end, let fZ (z) = R fX,Z (x, z)dx and
R
fX (x) = R fX,Z (x, z)dz denote the probability density functions of Z and X. That
is, fZ (z) = (fX,Z (, z)) and fX (x) = (fX,Z (x, )) for Lebesgue measure and the
Borel function fX,Z on R2 . Recall that fZ () and fX () are non-negative Borel
functions (for example, consider our proof of Fubinis theorem in case of Lebesgue
measure on (R2 , BR2 ) and the non-negative integrable Borel function h = fX,Z ).
So, defining the conditional probability density function of X given Z as
( f (x,z)
X,Z
if fZ (z) > 0 ,
fZ (z)
fX|Z (x|z) =
fX (x)
otherwise ,
R
guarantees that fX|Z : R2 7 R+ is Borel measurable and R fX|Z (x|z)dx = 1 for
all z R.
Proposition 4.4.1. Suppose the random vector (X, Z) has a probability density
function fX,Z (x, z) and g() is a Borel function on R such that E|g(X)| < . Then,
gb(Z) is a version of E[g(X)|Z] for the Borel function
Z
(4.4.1)
g(z) =
b
g(x)fX|Z (x|z)dx ,
in case
|g(x)|fX,Z (x, z)dx is finite (taking otherwise gb(z) = 0).
Proof. Since the Borel function h(x, z) = g(x)fX,Z (x, z) is integrable with
respect to Lebesgue measure on (R2 , BR2 ), it follows that gb() is also a Borel function
(c.f. our proof of Fubinis theorem). Further, by Fubinis
theorem the integrability
R
of g(X) implies that (R \ A) = 0 for A = {z : |g(x)|fX,Z (x, z)dx < }, and
with PZ = fZ this implies that P(Z A) = 1. By Jensens inequality,
Z
|b
g(z)| |g(x)|fX|Z (x|z)dx ,
z A .
Thus, by Fubinis theorem and the definition of fX|Z we have that
Z
Z
hZ
i
> E|g(X)| = |g(x)|fX (x)dx |g(x)|
fX|Z (x|z)fZ (z)dz dx
A
Z
Z hZ
i
|b
g(z)|fZ (z)dz = E|b
g (Z)| .
|g(x)|fX|Z (x|z)dx fZ (z)dz
=
A
So, b
g(Z) is integrable. With (4.4.1) holding for all z A and P(Z A) = 1, by
Fubinis theorem and the definition of fX|Z we have that for any Borel set B,
Z hZ
Z
i
gb(z)fZ (z)dz =
g(x)fX|Z (x|z)dx IBA (z)fZ (z)dz
E[b
g (Z)IB (Z)] =
ZBA
g(x)IBA (z)fX,Z (x, z)dxdz = E[g(X)IB (Z)] .
=
R2
This amounts to E[b

g(Z)IG ] = E[g(X)IG ] for any G (Z) = {Z 1 (B) : B B}
so indeed gb(Z) is a version of E[g(X)|Z].
172
To each conditional probability density fX|Z (|) corresponds the collection of conR
b X|Z (B, ) =
f
(x|Z())dx. The remainder of
ditional probability measures P
B X|Z
this section deals with the following generalization of the latter object.
Definition 4.4.2. Let Y : 7 S be an (S, S)-valued R.V. in the probability space
b Y |G (, ) :
(, F , P), per Definition 1.2.1, and G F a -algebra. The collection P
S 7 [0, 1] is called the regular conditional probability distribution (R.C.P.D.)
of Y given G if:
b Y |G (A, ) is a version of the C.E. E[IY A |G] for each fixed A S.
(a) P
b Y |G (, ) is a probability measure
(b) For any fixed , the set function P
on (S, S).
In case S = , S = F and Y () = , we call this collection the regular conditional
b
probability (R.C.P.) on F given G, denoted also by P(A|G)().
If the R.C.P. exists, then we can define all conditional expectations through the
R.C.P. Unfortunately, the R.C.P. might not exist (see [Bil95, Exercise 33.11] for
an example in which there exists no R.C.P. on F given G).
Recall that each C.E. is uniquely determined only a.e. Hence, for any countable
collection of disjoint sets An F there is possibly a set of of probability zero
for which a given collection of C.E. is such that
[
X
P( An |G)() 6=
P(An |G)() .
n
In case we need to examine an uncountable number of such collections in order to

see whether P(|G) is a measure on (, F ), the corresponding exceptional sets of
can pile up to a non-negligible set, hence the reason why a R.C.P. might not exist.
Nevertheless, as our next proposition shows, the R.C.P.D. exists for any conditioning -algebra G and any real-valued random variable X. In this setting, the
R.C.P.D. is the analog of the law of X as in Definition 1.2.34, but now given the
information contained in G.
Proposition 4.4.3. For any real-valued random variable X and any -algebra
b X|G (, ).
G F , there exists a R.C.P.D. P
Proof. Consider the random variables H(q, ) = E[I{Xq} |G](), indexed by

q Q. By monotonicity of the C.E. we know that if q r then H(q, ) H(r, )
for all
/ Ar,q where Ar,q G is such that P(Ar,q ) = 0. Further, by linearity
and dominated convergence of C.E.s H(q + n1 , ) H(q, ) as n for all
/ Bq , where Bq G is such that P(Bq ) = 0. For the same reason, H(q, ) 0

as q and H(q, ) 1 as q for all
/ S
C, whereSC G is such
that P(C) = 0. Since Q is countable, the set D = C r,q Ar,q q Bq is also in
G with P(D) = 0. Next, for a fixed non-random distribution function G(), let
F (x, ) = inf{G(r, ) : r Q, r > x}, where G(r, ) = H(r, ) if
/ D and
G(r, ) = G(r) otherwise. Clearly, for all the non-decreasing function
x 7 F (x, ) converges to zero when x and to one when x , as C D.
Furthermore, x 7 F (x, ) is right continuous, hence a distribution function, since
lim F (xn , ) = inf{G(r, ) : r Q, r > xn for some n}
xn x
= inf{G(r, ) : r Q, r > x} = F (x, ) .
173
b ) on (R, B)
Thus, to each corresponds a unique probability measure P(,
b
such that P((,
x], ) = F (x, ) for all x R (recall Theorem 1.2.37 for its
existence and Proposition 1.2.45 for its uniqueness).
Note that G(q, ) mG for all q Q, hence so is F (x, ) for each x R (see
b
Theorem 1.2.22). It follows that {B B : P(B,
) mG} is a -system (see
Corollary 1.2.19 and Theorem 1.2.22), containing the -system P = {R, (, q] :
b
q Q}, hence by Dynkins theorem P(B,
) mG for all B B. Further, for
/ D and q Q,
H(q, ) = G(q, ) F (q, ) G(q + n1 , ) = H(q + n1 , ) H(q, )
as n (specifically, the left-most inequality holds for

/ r Ar,q and the rightb
most limit holds for
/ Bq ). Hence, P(B,
) = E[I{XB} |G]() for any B P
and
/ D. Since P(D) = 0 it follows from the definition of the C.E. that for any
G G and B P,
Z
b
P(B,
)dP() = E[I{XB} IG ] .
G
Fixing G G, by monotone convergence and linearity of the expectation, the set L

of B B for which this equation holds is a -system. Consequently, L = (P) = B.
b
Since this applies for all G G, we conclude that P(B,
) is a version of E[IXB |G]
b
for each B B. That is, P(B, ) is per Definition 4.4.2 the R.C.P.D. of X given
G.
Remark. The reason behind Proposition 4.4.3 is that (X) inherits the structure
of the Borel -algebra B which in turn is not too big due to the fact the rational
numbers are dense in R. Indeed, as you are to deduce in the next exercise, there
exists a R.C.P.D. for any (S, S)-valued R.V. X with a B-isomorphic (S, S).
Exercise 4.4.4. Suppose (S, S) is B-isomorphic, that is, there exists a Borel set
T (equipped with the induced Borel -algebra T = {B T : B B}) and a one to
one and onto mapping g : S 7 T such that both g and g 1 are measurable. For
b Y |G (, ) denote the R.C.P.D. of the
any -algebra G and (S, S)-valued R.V. X let P
real-valued random variable Y = g(X).
b Y |G (T, ) = 1 for all .
(a) Explain why without loss of generality P
(b) Verify that for any A S both { : X() A} = { : Y () g(A)} and
g(A) B.
b
b Y |G (g(A), ) is the R.C.P.D. of X given G.
(c) Deduce that Q(A,
) = P
Our next exercise provides a generalization of Proposition 4.4.3 which is key to

the canonical construction of Markov chains in Section 6.1. We note in passing
that to conform with the notation for Markov chains, we reverse the order of the
b X|Y (y, A) with respect to that of the
arguments in the transition probabilities P
b X|(Y ) (A, ).
R.C.P.D. P
Exercise 4.4.5. Suppose (S, S) is B-isomorphic and X and Y are (S, S)-valued
R.V. in the same probability space (, F , P). Prove that there exists (regular) tranb X|Y (, ) : S S 7 [0, 1] such that
sition probability P
b X|Y (y, A) is a measurable function and
(a) For each A S fixed, y 7 P
b
PX|Y (Y (), A) is a version of the C.E. E[IXA |(Y )]().
174
b X|Y (Y (), ) is a probability

(b) For any fixed , the set function P
measure on (S, S).
Hint: With g : S 7 T as before, show that (Y ) = (g(Y )) and deduce from
b X|(g(Y )) (A, ) = f (A, g(Y ()) for each A S, where z 7
Theorem 1.2.26 that P
f (A, z) is a Borel function.
Here is the extension of the change of variables formula (1.3.14) to the setting of
conditional distributions.
Exercise 4.4.6. Suppose X mF and Y mG for some -algebras G F are
real-valued. Prove that, for any Borel function h : R2 7 R such that E|h(X, Y )| <
, almost surely,
Z
b X|G (x, ) .
E[h(X, Y )|G] =
h(x, Y ())dP
R
For an integrable R.V. X (and a non-random constant Y = c), this exercise

provides the representation
Z
b X|G (x, ) ,
E[X|G] =
xdP
R
of the C.E. in terms of the corresponding R.C.P.D. (with the right side denoting the
b X|G (, )).
Lebesgues integral of Definition 1.3.1 for the probability space (R, B, P
Solving the next exercise should improve your understanding of the relation between the R.C.P.D. and the conditional probability density function.
Exercise 4.4.7. Suppose that the random vector (X, Y, Z) has a probability density
function fX,Y,Z per Definition 3.5.5.
b Y |(X,Z) in terms of fX,Y,Z .
(a) Express the R.C.P.D. P
(b) Using this expression show that if X is independent of (Y, Z), then
E[Y |X, Z] = E[Y |Z] .
(c) Provide an example of random variables X, Y, Z, such that X is independent of Y and

E[Y |X, Z] 6= E[Y |Z] .
Pn
Exercise 4.4.8. Let Sn = k=1 k for i.i.d. integrable random variables k .
(a) Show that E[1 |Sn ] = n1 Sn .
Hint: Consider E[(1) ISn B ] for B B and a uniformly chosen random permutation of {1, . . . , n} which is independent of {k }.
(b) Find P(1 b|S2 ) in case the i.i.d. k are Exponential of parameter .
Hint: See the representation of Exercise 3.4.11.
Exercise 4.4.9. Let E[X|X < Y ] = E[XIX<Y ]/P(X < Y ) for integrable X and
Y such that P(X < Y ) > 0. For each of the following statements, either show that
it implies E[X|X < Y ] EX or provide a counter example.
(a) X and Y are independent.
(b) The random vector (X, Y ) has the same joint law as the random vector
(Y, X) and P(X = Y ) = 0.
(c) EX 2 < , EY 2 < and E[XY ] EXEY .
175
Exercise 4.4.10. Suppose (X, Y ) are distributed according to a multivariate normal distribution, with EX = EY = 0 and EY 2 > 0. Show that E[X|Y ] = Y with
= E[XY ]/EY 2 .
CHAPTER 5
Discrete time martingales and stopping times

In this chapter we study a collection of stochastic processes called martingales.
To simplify our presentation we focus on discrete time martingales and filtrations,
also called discrete parameter martingales and filtrations, with definitions and examples provided in Section 5.1 (indeed, a discrete time stochastic process is merely
a sequence of random variables defined on the same probability space). As we shall
see in Section 5.4, martingales play a key role in computations involving stopping
times. Martingales share many other useful properties, chiefly among which are
tail bounds and convergence theorems. Section 5.2 deals with martingale representations and tail inequalities, some of which are applied in Section 5.3 to prove
various convergence theorems. Section 5.5 further demonstrates the usefulness of
martingales in the study of branching processes, likelihood ratios, and exchangeable
processes.
5.1. Definitions and closure properties
Subsection 5.1.1 introduces the concepts of filtration, martingale and stopping
time and provides a few illustrating examples and interpretations. Subsection 5.1.2
introduces the related super-martingales and sub-martingales, as well as the powerful martingale transform and other closure properties of this collection of stochastic
processes.
5.1.1. Martingales, filtrations and stopping times: definitions and
examples. Intuitively, a filtration represents any procedure of collecting more and
more information as times goes on. Our starting point is the following rigorous
mathematical definition of a (discrete time) filtration.
Definition 5.1.1. A filtration is a non-decreasing family of sub--algebras {Fn }
of our measurable space (, F ). That is, F0 F1 F2 Fn F and
Fn is a -algebra for each n.SWe denote by Fn F a filtration {Fn } and the
associated -algebra F = ( k Fk ) such that the relation Fk F applies for all
0 k .
Given a filtration, we are interested in stochastic processes (S.Ps) such that for
each n the information gathered by that time suffices for evaluating the value of
the n-th element of the process. That is,
Definition 5.1.2. A S.P. {Xn , n = 0, 1, . . .} is adapted to a filtration {Fn }, also
denoted Fn -adapted, if (Xn ) Fn for each n (that is, Xn mFn for each n).
At this point you should convince yourself that {Xn } is adapted to the filtration
{Fn } if and only if (X0 , X1 , . . . , Xn ) Fn for all n. That is,
177
178
5. DISCRETE TIME MARTINGALES AND STOPPING TIMES
Definition 5.1.3. The filtration {FnX } with FnX = (X0 , X1 , . . . , Xn ) is the minimal filtration with respect to which {Xn } is adapted. We therefore call it the
canonical filtration for the S.P. {Xn }.
Whenever clear from the context what it means, we shall use the notation Xn
both for the whole S.P. {Xn } and for the n-th R.V. of this process, and likewise we
may sometimes use Fn to denote the whole filtration {Fn }.
A martingale consists of a filtration and an adapted S.P. which can represent the
outcome of a fair gamble. That is, the expected future reward given current
information is exactly the current value of the process, or as a rigorous definition:
Definition 5.1.4. A martingale (denoted MG) is a pair (Xn , Fn ), where {Fn } is
a filtration and {Xn } is an integrable S.P., that is, E|Xn | < for all n, adapted
to this filtration, such that
(5.1.1)
E[Xn+1 |Fn ] = Xn
n,
a.s.
Remark. The slower a filtration n 7 Fn grows, the easier it is for an adapted

S.P. to be a martingale. That is, if Hn Fn for all n and S.P. {Xn } adapted to
filtration {Hn } is such that (Xn , Fn ) is a martingale, then by the tower property
(Xn , Hn ) is also a martingale. In particular, if (Xn , Fn ) is a martingale then {Xn }
is also a martingale with respect to its canonical filtration. For this reason, hereafter
the statement {Xn } is a MG (without explicitly specifying the filtration), means
that {Xn } is a MG with respect to its canonical filtration FnX = (Xk , k n).
We next provide an alternative characterization of the martingale property.
Pn
Proposition 5.1.5. If Xn = k=0 Dk then the canonical filtration for {Xn } is
the same as the canonical filtration for {Dn }. Further, (Xn , Fn ) is a martingale if
and only if {Dn } is an integrable S.P., adapted to {Fn }, such that E[Dn+1 |Fn ] = 0
a.s. for all n.
Remark. The martingale differences associated with {Xn } are Dn = Xn Xn1 ,
n 1 and D0 = X0 .
Proof. With both the transformation from (X0 , . . . , Xn ) to (D0 , . . . , Dn ) and
its inverse being continuous (hence Borel), it follows that FnX = FnD for each n
(c.f. Exercise 1.2.33). Therefore, {Xn } is adapted to a given filtration {Fn } if and
only if {Dn } is adapted to this filtration (see Definition 5.1.3). It is easy to show
by induction on n that E|Xk | < for k = 0, . . . , n if and only if E|Dk | < for
k = 0, . . . , n. Hence, {Xn } is an integrable S.P. if and only if {Dn } is. Finally, with
Xn mFn it follows from the linearity of the C.E. that
E[Xn+1 |Fn ] Xn = E[Xn+1 Xn |Fn ] = E[Dn+1 |Fn ] ,
and the alternative expression for the martingale property follows from (5.1.1).
Our first example of a martingale, is the random walk, perhaps the most fundamental stochastic process.
Pn
Definition 5.1.6. The random walk is the stochastic process Sn = S0 + k=1 k
with real-valued, independent, identically distributed {k } which are also independent of S0 . Unless explicitly stated otherwise, we always set S0 to be zero. We say
that the random walk is symmetric if the law of k is the same as that of k . We
5.1. DEFINITIONS AND CLOSURE PROPERTIES
179
call it a simple random walk (on Z), in short srw, if k {1, 1}. The srw is
completely characterized by the parameter p = P(k = 1) which is always assumed
to be in (0, 1) (or alternatively, by q = 1 p = P(k = 1)). Thus, the symmetric
srw corresponds to p = 1/2 = q (and the asymmetric srw corresponds to p 6= 1/2).
The random walk is a MG (with respect to its canonical filtration), whenever
E|1 | < and E1 = 0.
Remark. More generally, such partial sums {Sn } form a MG even when the
independent and integrable R.V. k of zero mean have non-identical distributions,
and the canonical filtration of {Sn } is merely {Fn }, where Fn = (1 , . . . , n ).
Indeed, this is an application of Proposition 5.1.5 for independent, integrable Dk =
Sk Sk1 = k , k 1 (with D0 = 0), where E[Dn+1 |D0 , D1 , . . . , Dn ] = EDn+1 = 0
for all n 0 by our assumption that Ek = 0 for all k.
Definition 5.1.7. We say that a stochastic process {Xn } is square-integrable if
EXn2 < for all n. Similarly, we call a martingale (Xn , Fn ) such that EXn2 <
for all n, an L2 -MG (or a square-integrable MG).
Square-integrable martingales have zero-mean, uncorrelated differences and admit
an elegant decomposition of conditional second moments.
Exercise 5.1.8. Suppose (Xn , Fn ) and (Yn , Fn ) are square-integrable martingales.
(a) Show that the corresponding martingale differences Dn are uncorrelated

and that each Dn , n 1, has zero mean.
(b) Show that for any n 0,
E[X Y |Fn ] Xn Yn = E[(X Xn )(Y Yn )|Fn ]
=
k=n+1
E[(Xk Xk1 )(Yk Yk1 )|Fn ] .
(c) Deduce that if supk |Xk | C non-random then for any 1,

E
h X
k=1
Dk2
2 i
6C 4 .
Remark. A square-integrable stochastic process with zero-mean mutually independent differences is necessarily a martingale (consider Proposition 5.1.5). So, in
view of part (a) of Exercise 5.1.8, the MG property is between the more restrictive
requirement of having zero-mean, independent differences, and the not as useful
property of just having zero-mean, uncorrelated differences. While in general these
three conditions are not the same, as you show next they do coincide in case of
Gaussian stochastic processes.
Exercise 5.1.9. A stochastic process {Xn } is Gaussian if for each n the random vector (X1 , . . . , Xn ) has the multivariate normal distribution (c.f. Definition
3.5.13). Show that having independent or uncorrelated differences are equivalent
properties for such processes, which together with each of these differences having
a zero mean is then also equivalent to the MG property.
Products of R.V. is another classical source for martingales.
180
Qn
Example 5.1.10. Consider the stochastic process Mn = k=1 Yk for independent,
integrable random variables Yk 0. Its canonical filtration coincides with FnY (see
Exercise 1.2.33), and taking out what is known we get by independence that
E[Mn+1 |FnY ] = E[Yn+1 Mn |FnY ] = Mn E[Yn+1 |FnY ] = Mn E[Yn+1 ] ,
so {Mn } is a MG, which we then call the product martingale, if and only if
EYk = 1 for all k 1 (for general sequence {Yn } we need instead that a.s.
E[Yn+1 |Y1 , . . . , Yn ] = 1 for all n).
Remark. In investment applications, the MG condition EYk = 1 corresponds to
a neutral return rate, and is not the same as the condition E[log Yk ] = 0 under
which the associated partial sums Sn = log Mn form a MG.
We proceed to define the important concept of stopping time (in the simpler
context of a discrete parameter filtration).
Definition 5.1.11. A random variable taking values in {0, 1, . . . , n, . . . , } is a
stopping time for the filtration {Fn } (also denoted Fn -stopping time), if the event
{ : () n} is in Fn for each finite n 0.
Remark. Intuitively, a stopping time corresponds to a situation where the decision whether to stop or not at any given (non-random) time step is based on the
information available by that time step. As we shall amply see in the sequel, one of
the advantages of MGs is in providing a handle on explicit computations associated
with various stopping times.
The next two exercises provide examples of stopping times. Practice your understanding of this concept by solving them.
Exercise 5.1.12. Suppose that and are stopping times for the same filtration
{Fn }. Show that then , and + are also stopping times for this filtration.
Exercise 5.1.13. Show that the first hitting time () = min{k 0 : Xk () B}
of a Borel set B R by a sequence {Xk }, is a stopping time for the canonical
filtration {FnX }. Provide an example where the last hitting time = sup{k 0 :
Xk B} of a set B by the sequence, is not a stopping time (not surprising, since
we need to know the whole sequence {Xk } in order to verify that there are no visits
to B after a given time n).
Here is an elementary application of first hitting times.
Exercise 5.1.14 (Reflection principle). Suppose {Sn } is a symmetric random
walk starting at S0 = 0 (see Definition 5.1.6).
(a) Show that P(Sn Sk 0) 1/2 for k = 1, 2, . . . , n.
(b) Fixing x > 0, let = inf{k 0 : Sk > x} and show that
P(Sn > x)
n
X
k=1
P( = k, Sn Sk 0)
1X
P( = k) .
2
(c) Deduce that for any n and x > 0,

n
P(max Sk > x) 2P(Sn > x) .

k=1
k=1
181
(d) Considering now the symmetric srw, show that for any positive integers
n, x,
n
P(max Sk x) = 2P(Sn x) P(Sn = x)
k=1
D
Z2n+1 =
(|S2n+1 |1)/2, where Zn denotes the number of (strict)

and that
sign changes within {S0 = 0, S1 , . . . , Sn }.
Hint: Show that P(Z2n+1 r|S1 = 1) = P(max2n+1
k=1 Sk 2r 1|S1 =
1) by reflecting (the signs of ) the increments occurring between the odd
and the even strict sign changes of the srw.
We conclude this subsection with a useful sufficient condition for the integrability
of a stopping time.
Exercise 5.1.15. Suppose the Fn -stopping time is such that a.s.
P[ n + r|Fn ]
for some positive integer r, some > 0 and all n.

(a) Show that P( > kr) (1 )k for any positive integer k.
Hint: Use induction on k.
(b) Deduce that in this case E < .
5.1.2. Sub-martingales, super-martingales and stopped martingales.
Often when operating on a MG, we naturally end up with a sub-martingale or a
super-martingale, as defined next. Moreover, these processes share many of the
properties of martingales, so it is useful to develop a unified theory for them.
Definition 5.1.16. A sub-martingale (denoted sub-MG) is an integrable S.P.
{Xn }, adapted to the filtration {Fn }, such that
E[Xn+1 |Fn ] Xn
n,
a.s.
E[Xn+1 |Fn ] Xn
n,
a.s.
A super-martingale (denoted sup-MG) is an integrable S.P. {Xn }, adapted to the

filtration {Fn } such that
(A typical S.P. {Xn } is neither a sub-MG nor a sup-MG, as the sign of the R.V.
E[Xn+1 |Fn ] Xn may well be random, or possibly dependent upon n).
Remark 5.1.17. Note that {Xn } is a sub-MG if and only if {Xn } is a sup-MG.
By this identity, all results about sub-MGs have dual statements for sup-MGs and
vice verse. We often state only one out of each such pair of statements. Further,
{Xn } is a MG if and only if {Xn } is both a sub-MG and a sup-MG. As a result,
every statement holding for either sub-MGs or sup-MGs, also hold for MGs.
Example 5.1.18. Expanding on Example 5.1.10, if the non-negative, integrable
random
variables Yk are such that E[Yn |Y1 , . . . , Yn1 ] 1 a.s. for all n then Mn =
Qn
k=1 Yk is a sub-MG, and if E[Yn |Y1 , . . . , Yn1 ] 1 a.s. for all n then {Mn } is a
sup-MG. Such martingales appear for example in mathematical finance, where Yk
denotes the random proportional change in the value of a risky asset at the k-th
trading round. So, positive conditional mean return rate yields a sub-MG while
negative conditional mean return rate gives a sup-MG.
The sub-martingale (and super-martingale) property is closed with respect to the
addition of S.P.
182
Exercise 5.1.19. Show that if {Xn } and {Yn } are sub-MGs with respect to a
filtration {Fn }, then so is {Xn + Yn }. In contrast, show that for any sub-MG {Yn }
there exists integrable {Xn } adapted to {FnY } such that {Xn + Yn } is not a sub-MG
with respect to any filtration.
Here are some of the properties of sub-MGs (and of sup-MGs).
Proposition 5.1.20. If (Xn , Fn ) is a sub-MG, then a.s. E[X |Fm ] Xm for
any > m. Consequently, for s sub-MG necessarily n 7 EXn is non-decreasing.
Similarly, for a sup-MG a.s. E[X |Fm ] Xm (with n 7 EXn non-increasing),
and for a martingale a.s. E[X |Fm ] = Xm for all > m (with E[Xn ] independent
of n).
Proof. Suppose {Xn } is a sub-MG and = m + k for k 1. Then,
E[Xm+k |Fm ] = E[E(Xm+k |Fm+k1 )|Fm ] E[Xm+k1 |Fm ]
with the equality due to the tower property and the inequality by the definition
of a sub-MG and monotonicity of the C.E. Iterating this inequality for decreasing
values of k we deduce that E[Xm+k |Fm ] E[Xm |Fm ] = Xm for all non-negative
integers k, m, as claimed. Next taking the expectation of this inequality, we have
by monotonicity of the expectation and (4.2.1) that E[Xm+k ] E[Xm ] for all
k, m 0, or equivalently, that n 7 EXn is non-decreasing.
To get the corresponding results for a super-martingale {Xn } note that then
{Xn } is a sub-martingale, see Remark 5.1.17. As already mentioned there, if
{Xn } is a MG then it is both a super-martingale and a sub-martingale, hence
both E[X |Fm ] Xm and E[X |Fm ] Xm , resulting with E[X |Fm ] = Xm , as
stated.

Exercise 5.1.21. Show that a sub-martingale (Xn , Fn ) is a martingale if and only
if EXn = EX0 for all n.
We next detail a few examples in which sub-MGs or sup-MGs naturally appear,
starting with an immediate consequence of Jensens inequality
Proposition 5.1.22. Suppose : R 7 R is convex and E[|(Xn )|] < for all
n.
(a) If (Xn , Fn ) is a martingale then ((Xn ), Fn ) is a sub-martingale.
(b) If x 7 (x) is also non-decreasing, ((Xn ), Fn ) is a sub-martingale even
when (Xn , Fn ) is only a sub-martingale.
Proof. With (Xn ) integrable and adapted, it suffices to check that a.s.
E[(Xn+1 )|Fn ] (Xn ) for all n. To this end, since () is convex and Xn is
integrable, by the conditional Jensens inequality,
E[(Xn+1 )|Fn ] (E[Xn+1 |Fn ]) ,
so it remains only to verify that (E[Xn+1 |Fn ]) (Xn ). This clearly applies
when (Xn , Fn ) is a MG, and even for a sub-MG (Xn , Fn ), provided that () is
monotone non-decreasing.

Example 5.1.23. Typical convex functions for which the preceding proposition is
often applied are (x) = |x|p , p 1, (x) = (xc)+ , (x) = max(x, c) (for c R),
(x) = ex and (x) = x log x (the latter only for non-negative S.P.). Considering
instead () concave leads to a sup-MG, as for example when (x) = min(x, c) or
183
(x) = xp for some p (0, 1) or (x) = log x (latter two cases restricted to nonnegative S.P.). For example, if {Xn } is a sub-martingale then (Xn c)+ is also
a sub-martingale (since (x c)+ is a convex, non-decreasing function). Similarly,
if {Xn } is a super-martingale, then min(Xn , c) is also a super-martingale (since
Xn is a sub-martingale and the function min(x, c) = max(x, c) is convex
and non-decreasing).
Here is a concrete application of Proposition 5.1.22.
Exercise 5.1.24. Suppose {i } are mutually independent, Ei = 0 and Ei2 = i2 .
Pn
Pn
(a) Let Sn = i=1 i and s2n = i=1 i2 . Show that {Sn2 } is a sub-martingale
and {Sn2 s2n } is a martingale. Q
n
(b) Show that if in addition mn = i=1 Eei are finite, then {eSn } is a
sub-martingale and Mn = eSn /mn is a martingale.
Remark. A special case of Exercise 5.1.24 is the random walk Sn of Definition
5.1.6, with Sn2 nE12 being a MG when 1 is square-integrable and of zero mean.
Likewise, eSn is a sub-MG whenever E1 = 0 and Ee1 is finite. Though eSn is in
general not a MG, the normalized Mn = eSn /[Ee1 ]n is merely the product MG of
Example 5.1.10 for the i.i.d. variables Yi = ei /E(e1 ).
Here is another family of super-martingales, this time related to super-harmonic
functions.
Definition 5.1.25. A lower semi-continuous function f : Rd 7 R is superharmonic if for any x and r > 0,
Z
1
f (y)dy
f (x)
|B(0, r)| B(x,r)
where B(x, r) = {y : |x y| r} is the ball of radius r centered at x and |B(x, r)|

denotes its volume.
Pn
Exercise 5.1.26. Suppose Sn = x+ k=1 k for i.i.d. k that are chosen uniformly
on the ball B(0, 1) in Rd (i.e. using Lebesgues measure on this ball, scaled by
its volume). Show that if f () is super-harmonic on Rd then f (Sn ) is a supermartingale.
Hint: When checking the integrability of f (Sn ) recall that a lower semi-continuous
function is bounded below on any compact set.
We next define the important concept of a martingale transform, and show that
it is a powerful and flexible method for generating martingales.
Definition 5.1.27. We call a sequence {Vn } predictable (or pre-visible) for the
filtration {Fn }, also denoted Fn -predictable, if Vn is measurable on Fn1 for all
n 1. The sequence of random variables
n
X
Yn =
Vk (Xk Xk1 ) , n 1, Y0 = 0
k=1
is called the martingale transform of the Fn -predictable {Vn } with respect to a sub
or super martingale (Xn , Fn ).
Theorem 5.1.28. Suppose {Yn } is the martingale transform of Fn -predictable
{Vn } with respect to a sub or super martingale (Xn , Fn ).
184
(a) If Yn is integrable and (Xn , Fn ) is a martingale, then (Yn , Fn ) is also a

martingale.
(b) If Yn is integrable, Vn 0 and (Xn , Fn ) is a sub-martingale (or supermartingale) then (Yn , Fn ) is also a sub-martingale (super-martingale, respectively).
(c) For the integrability of Yn it suffices in both cases to have |Vn | cn for
some non-random finite constants cn , or alternatively to have Vn Lq
and Xn Lp for all n and some p, q > 1 such that q1 + p1 = 1.
Proof. With {Vn } and {Xn } adapted to the filtration Fn , it follows that
Vk Xl mFk mFn for all l k n. By inspection Yn mFn as well (see
Corollary 1.2.19), i.e. {Yn } is adapted to {Fn }.
Turning to prove part (c) of the theorem, note that for each n the variable Yn is
a finite sum of terms of the form Vk Xl . If Vk Lq and Xl Lp for some p, q > 1
olders inequality Vk Xl is integrable. Alternatively,
such that 1q + p1 = 1, then by H
since a super-martingale Xl is in particular integrable, Vk Xl is integrable as soon
as |Vk | is bounded by a non-random finite constant. In conclusion, if either of these
conditions applies for all k, l then obviously {Yn } is an integrable S.P.
Recall that Yn+1 Yn = Vn+1 (Xn+1 Xn ) and Vn+1 mFn (since {Vn } is Fn predictable). Therefore, taking out Vn+1 which is measurable on Fn we find that
E[Yn+1 Yn |Fn ] = E[Vn+1 (Xn+1 Xn )|Fn ] = Vn+1 E[Xn+1 Xn |Fn ] .
This expression is zero when (Xn , Fn ) is a MG and non-negative when Vn+1 0
and (Xn , Fn ) is a sub-MG. Since the preceding applies for all n, we consequently
have that (Yn , Fn ) is a MG in the former case and a sub-MG in the latter. Finally,
to complete the proof also in case of a sup-MG (Xn , Fn ), note that then Yn is the
MG transform of {Vn } with respect to the sub-MG (Xn , Fn ).

Here are two concrete examples of a martingale transform.
Pn
Example 5.1.29. The S.P. Yn =
k=1 Xk1 (Xk Xk1 ) is a MG whenever
Xn L2 (, F , P) is a MG (indeed, Vn = Xn1 is predictable for the canonical
filtration of {Xn } and consider p = q = 2 in part (c) of Theorem 5.1.28).
Example 5.1.30. Given an integrable process {Vn } suppose that for each k 1 the
bounded k has
P zero mean and is independent of Fk1 = (1 , . . . , k1 , V1 , . . . , Vk ).
Then, Yn = nk=1 Vk k is a martingale
Pn for the filtration {Fn }. Indeed, by assumption, the differences n of Xn =
k=1 k are such that E[k |Fk1 ] = 0 for all
k 1. Hence, (Xn , Fn ) is a martingale (c.f. Proposition 5.1.5), and {Yn } is
the martingale transform of the Fn -predictable {Vn } with respect to the martingale
(Xn , Fn ) (where the integrability of Yn is a consequence of the boundedness of each
k and integrability of each Vk ). In discrete mathematics applications one often
uses a special case of this construction, with an auxiliary sequence of random i.i.d.
signs k {1, 1} such that P(1 = 1) = 21 and {n } is independent of the given
integrable S.P. {Vn }.
We next define the important concept of a stopped stochastic process and then
use the martingale transform to show that stopped sub and super martingales are
also sub-MGs (sup-MGs, respectively).
185
Definition 5.1.31. Given a stochastic process {Xn } and a a random variable

taking values in {0, 1, . . . , n, . . . , }, the stopped at stochastic process, denoted
{Xn }, is given by
(
Xn (),
n ()
Xn () =
X () (), n > ()
Theorem 5.1.32. If (Xn , Fn ) is a sub-MG (or a sup-MG or a MG) and
are stopping times for {Fn }, then (Xn Xn , Fn ) is also a sub-MG (or sup-MG
or MG, respectively). In particular, taking = 0 we have that (Xn , Fn ) is then
a sub-MG (or sup-MG or MG, respectively).
Proof. We may and shall assume that (Xn , Fn ) is a sub-MG (just consider
Xn in case Xn is a sup-MG and both when Xn is a MG). Let Vk () = I{()<k ()} .
Since are two Fn -stopping times, it follows that Vk () = I{()(k1)}
I{ ()(k1)} is measurable on Fk1 for all k 1. Thus, {Vn } is a bounded,
non-negative Fn -predictable sequence. Further, since
Xn () Xn () =
n
X
k=1
I{()<k ()} (Xk () Xk1 ())
is the martingale transform of {Vn } with respect to sub-MG (Xn , Fn ), we know from
Theorem 5.1.28 that (Xn Xn , Fn ) is also a sub-MG. Finally, considering the
latter sub-MG for = 0 and adding to it the sub-MG (X0 , Fn ), we conclude that
(Xn , Fn ) is a sub-MG (c.f. Exercise 5.1.19 and note that Xn0 = X0 ).

Theorem 5.1.32 thus implies the following key ingredient in the proof of Doobs
optional stopping theorem (to which we return in Section 5.4).
Corollary 5.1.33. If (Xn , Fn ) is a sub-MG and are Fn -stopping times,
then EXn EXn for all n. The reverse inequality holds in case (Xn , Fn ) is a
sup-MG, with EXn = EXn for all n in case (Xn , Fn ) is a MG.
Proof. Suffices to consider Xn which is a sub-MG for the filtration Fn . In
this case we have from Theorem 5.1.32 that Yn = Xn Xn is also a sub-MG
for this filtration. Noting that Y0 = 0 we thus get from Proposition 5.1.20 that
EYn 0. Theorem 5.1.32 also implies the integrability of Xn so by linearity of
the expectation we conclude that EXn EXn .

An important concept associated with each stopping time is the stopped -algebra
defined next.
Definition 5.1.34. The stopped -algebra F associated with the stopping time
for a filtration {Fn } is the collection of events A F such that A { : ()
n} Fn for all n.
With Fn representing the information known at time n, think of F as quantifying
the information known upon stopping at . Some of the properties of these stopped
-algebras are detailed in the next exercise.
Exercise 5.1.35. Let and be Fn -stopping times.
(a) Verify that F is a -algebra and if () = n is non-random then F =
Fn .
186
(b) Suppose Xn mFn for all n (including n = unless is finite for all ).
Show that then X mF . Deduce that ( ) F and Xk I{ =k} mF
for any k non-random.
(c) Show that for any integrable {Yn } and non-random k,
E[Y I{ =k} |F ] = E[Yk |Fk ]I{ =k} .
(d) Show that if then F F .
Our next exercise shows that the martingale property is equivalent to the strong
martingale property whereby conditioning at stopped -algebras F replaces the
one at Fn for non-random n.
Exercise 5.1.36. Given an integrable stochastic process {Xn } adapted to a filtration {Fn }, show that (Xn , Fn ) is a martingale if and only if E[Xn |F ] = X for
any non-random, finite n and all Fn -stopping times n.
For non-integrable stochastic processes we generalize the concept of a martingale
into that of a local martingale.
Exercise 5.1.37. The pair (Xn , Fn ) is called a local martingale if {Xn } is adapted
to the filtration {Fn } and there exist Fn -stopping times k such that k with
probability one and (Xnk , Fn ) is a martingale for each k. Show that any martingale is a local martingale and any integrable, local martingale is a martingale.
We conclude with the renewal property of stopping times with respect to the
canonical filtration of an i.i.d. sequence.
Exercise 5.1.38. Suppose is an a.s. finite stopping time with respect to the
canonical filtration {FnZ } of a sequence {Zk } of i.i.d. R.V-s.
(a) Show that TZ = (Z +k , k 1) is independent of the stopped -algebra
FZ .
(b) Provide an example of a finite FnZ -stopping time and independent {Zk }
for which TZ is not independent of FZ .
5.2. Martingale representations and inequalities
In Subsection 5.2.1 we show that martingales are at the core of all adapted processes. We further explore there the structure of certain sub-martingales, introducing the increasing process associated with square-integrable martingales. This
is augmented in Subsection 5.2.2 by the study of maximal inequalities for submartingales (and martingales). Such inequalities are an important technical tool
in many applications of probability theory. In particular, they are the key to the
convergence results of Section 5.3.
5.2.1. Martingale decompositions. To demonstrate the relevance of martingales to the study of general stochastic processes, we start with a representation
of any adapted, integrable, discrete-time S.P. as the sum of a martingale and a
predictable process.
Theorem 5.2.1 (Doobs decomposition). Given an integrable stochastic process
{Xn }, adapted to a discrete parameter filtration {Fn }, n 0, there exists a decomposition Xn = Yn + An such that (Yn , Fn ) is a MG and {An } is an Fn -predictable
sequence. This decomposition is unique up to the value of Y0 mF0 .
5.2. MARTINGALE REPRESENTATIONS AND INEQUALITIES
187
Proof. Let A0 = 0 and for n 1 set

An = An1 + E[Xn Xn1 |Fn1 ].
By definition of the conditional expectation we see that Ak Ak1 is measurable
on
PnFk1 for any k 1. Since Fk1 Fn1 for all k n and An = A0 +
k=1 (Ak Ak1 ), it follows that {An } is Fn -predictable. We next check that
Yn = Xn An is a MG. To this end, recall that since {Xn } is integrable so is
{Xn Xn1 }, whereas the C.E. only reduces the L1 norm (see Example 4.2.20).
Therefore, E|An An1 | E|Xn Xn1 | < . Hence, An is integrable, as
is Xn , implying by Minkowskis inequality that Yn is integrable as well. With
{Xn } adapted and {An } predictable, hence adapted, to {Fn }, we see that {Yn } is
also Fn -adapted. It remains to check the martingale condition, that almost surely
E[Yn Yn1 |Fn1 ] = 0 for all n 1. Indeed, by linearity of the C.E. and the
construction of the Fn -predictable sequence {An }, for any n 1,
E[Yn Yn1 |Fn1 ] = E[Xn Xn1 (An An1 )|Fn1 ]
= E[Xn Xn1 |Fn1 ] (An An1 ) = 0 .
We finish the proof by checking that such a decomposition is unique up to the

en are two such
choice of Y0 . To this end, suppose that Xn = Yn + An = Yen + A
e
en . Since
decompositions of a given stochastic process {Xn }. Then, Yn Yn = An A
en } are both Fn -predictable sequences while (Yn , Fn ) and (Y
fn , Fn ) are
{An } and {A
martingales, we find that
en = E[An A
en |Fn1 ] = E[Yen Yn |Fn1 ]
An A
en1 .
= Yen1 Yn1 = An1 A
en is independent of n and if in addition Y0 = Ye0 then An A

en =
Thus, An A
e0 = Ye0 Y0 = 0 for all n. In conclusion, both sequences {An } and {Yn } are
A0 A
uniquely determined as soon as we determine Y0 , a R.V. measurable on F0 .

Doobs decomposition has more structure when (Xn , Fn ) is a sub-MG.
Exercise 5.2.2. Check that the predictable part of Doobs decomposition of a submartingale (Xn , Fn ) is a non-decreasing sequence, that is, An An+1 for all n.
Remark. As shown in Subsection 5.3.2, Doobs decomposition is particularly
useful in connection with square-integrable martingales {Xn }, where one can relate
the limit of Xn as n with that of the non-decreasing sequence {An } in the
decomposition of {Xn2 }.
We next evaluate Doobs decomposition for two classical sub-MGs.
Pn
Example 5.2.3. Consider the sub-MG {Sn2 } for the random walk Sn = k=1 k ,
where k are i.i.d. random variables with E1 = 0 and E12 = 1. Since Yn = Sn2 n
is a martingale (see Exercise 5.1.24), and Doobs decomposition Sn2 = Yn + An is
unique, it follows that the non-decreasing predictable part in the decomposition of
Sn2 is An = n.
In contrast with the preceding example, the non-decreasing predictable part in
Doobs decomposition is for most sub-MGs a non-degenerate random sequence, as
is the case in our next example.
188
Qn
Example 5.2.4. Consider the sub-MG (Mn , FnZ ) where Mn = i=1 Zi for i.i.d.
integrable Zi 0 such that EZ1 > 1 (see Example 5.1.10). The non-decreasing
predictable part of its Doobs decomposition is such that for n 1
An+1 An = E[Mn+1 Mn |FnZ ] = E[Zn+1 Mn Mn |FnZ ]
= Mn E[Zn+1 1|FnZ ] = Mn (EZ1 1)
Pn1
(since Zn+1 is independent of FnZ ). In this case An = (EZ1 1) k=1 Mk + A1 ,
where we are free to choose for A1 any non-random constant. We see that {An } is
a non-degenerate random sequence (assuming the R.V. Zi are not a.s. constant).
We conclude with the representation of any L1 -bounded martingale as the difference of two non-negative martingales (resembling the representation X = X+ X
for an integrable R.V. X and non-negative X ).
Exercise 5.2.5. Let (Xn , Fn ) be a martingale with supn E|Xn | < . Show that
there is a representation Xn = Yn Zn with (Yn , Fn ) and (Zn , Fn ) non-negative
martingales such that supn E|Yn | < and supn E|Zn | < .
5.2.2. Maximal and up-crossing inequalities. Martingales are rather tame
stochastic processes. In particular, as we see next, the tail of maxkn Xk is bounded
by moments of Xn . This is a major improvement over Markovs inequality, relating the typically much smaller tail of the R.V. Xn to its moments (see part (b) of
Example 1.3.14).
Theorem 5.2.6 (Doobs inequality). For any sub-martingale {Xn } and x > 0
let x = min{k 0 : Xk x}. Then, for any finite n 0,
n
P(max Xk x) x1 E[Xn I{x n} ] x1 E[(Xn )+ ] .
(5.2.1)
k=0
Proof. Since Xx x whenever x is finite, setting

n
An = { : x () n} = { : max Xk () x} ,
k=0
it follows that
E[Xnx ] = E[Xx Ix n ] + E[Xn Ix >n ] xP(An ) + E[Xn IAcn ].
With {Xn } a sub-MG and x a pair of FnX -stopping times, it follows from
Corollary 5.1.33 that E[Xnx ] E[Xn ]. Therefore, E[Xn ] E[Xn IAcn ] xP(An )
which is exactly the left inequality in (5.2.1). The right inequality there holds by
monotonicity of the expectation and the trivial fact XIA (X)+ for any R.V. X
and any measurable set A.

Remark. Doobs inequality generalizes Kolmogorovs maximal inequality. Indeed, consider Xk = Zk2 for the L2 -martingale Zk = Y1 + + Yk , where {Yl } are
mutually independent with EYl = 0 and EYl2 < . By Proposition 5.1.22 {Xk } is
a sub-MG, so by Doobs inequality we obtain that for any z > 0,
P( max |Zk | z) = P( max Xk z 2 ) z 2 E[(Xn )+ ] = z 2 Var(Zn )
1kn
1kn
which is exactly Kolmogorovs maximal inequality of Proposition 2.3.16.

Combining Doobs inequality with Doobs decomposition of non-negative submartingales, we arrive at the following bounds, due to Lenglart.
189
Lemma 5.2.7. Let Vn = maxnk=0 Zk and An denote the Fn -predictable sequence in

Doobs decomposition of a non-negative submartingale (Zn , Fn ) with Z0 = 0. Then,
for any Fn -stopping time and all x, y > 0,
P(V x, A y) x1 E(A y) .
(5.2.2)
Further, in this case E[Vp ] cp E[Ap ] for cp = 1 + 1/(1 p) and any p (0, 1).
Proof. Since Mn = Zn An is a MG with respect to the filtration {Fn }
(starting at M0 = 0), by Theorem 5.1.32 the same applies for the stopped stochastic
process Mn , with any Fn -stopping time. By the same reasoning Zn = Mn +
An is a sub-MG with respect to {Fn }. Applying Doobs inequality (5.2.1) for
this non-negative sub-MG we deduce that for any n and x > 0,
n
P(Vn x) = P(max Zk x) x1 E[ Zn ] = x1 E[ An ] .
k=0
Both Vn and An are non-negative and non-decreasing in n (see Exercise 5.2.2),

so by monotone convergence we have that P(V x) x1 EA . In particular,
fixing y > 0, since {An } is Fn -predictable, = min{n 0 : An+1 > y} is an
Fn -stopping time. Further, with An non-decreasing, < if and only if A > y
in which case A y (by the definition of ). Consequently, A A y and as
{V x, A y} {V x} we arrive at the inequality (5.2.2).
Next, considering (5.2.2) for x = y we see that for Y = A and any y > 0,
P(V y) P(Y y) + E[min(Y /y, 1)] .
Multiplying both sides of this inequality by py p1 and integrating over y (0, ),

upon taking r = 1 > p in part (a) of Lemma 1.4.31 we conclude that
EVp EY p + (1 p)1 EY p ,
as claimed.
To practice your understanding, adapt the proof of Doobs inequality en-route to

the following dual inequality (which is often called Doobs second sub-MG inequality).
Exercise 5.2.8. Show that for any sub-MG {Xn }, finite n 0 and x > 0,
(5.2.3)
P(min Xk x) x1 (E[(Xn )+ ] E[X0 ]) .

k=0
Here is a typical example of an application of Doobs inequality.

Exercise 5.2.9. Fixing s > 0, the independent variables Zn are such that P(Zn =
1) = P(Zn = 1) = ns /2 and P(Zn = 0) = 1 ns . Starting at Y0 = 0, for
n 1 let
Yn = ns Yn1 |Zn | + Zn I{Yn1 =0} .
(a) Show that {Yn } is a martingale and that for any x > 0 and n 1,
n1
P(max Yk x)
k=1
X
1
[1 +
(k + 1)s (1 k s )] .
2x
k=1
a.s.
(b) Show that Yn 0 as n and further Yn 0 if and only if s > 1,

L
but there is no value of s for which Yn 0.
190
Martingales also provide bounds on the probability that the sum of bounded independent variables is too close to its mean (in lieu of the clt).
Pn
Exercise 5.2.10. Let Sn =P k=1 k where {k } are independent and Ek = 0,
n
|k | K for all k. Let s2n = k=1 Ek2 . Using Corollary 5.1.33 for the martingale
Sn2 s2n and a suitable stopping time show that
n
P(max |Sk | x) (x + K)2 /s2n .

k=1
If the positive part of the sub-MG has finite p-th moment you can improve the
rate of decay in x in Doobs inequality by an application of Proposition 5.1.22 for
the convex non-decreasing (y) = max(y, 0)p , denoted hereafter by (y)p+ . Further,
in case of a MG the same argument yields comparable bounds on tail probabilities
for the maximum of |Yk |.
Exercise 5.2.11.
(a) Show that for any sub-MG {Yn }, p 1, finite n 0 and y > 0,
i
h
n
P(max Yk y) y p E max(Yn , 0)p .
k=0
(b) Show that in case {Yn } is a martingale, also

i
h
n
P(max |Yk | y) y p E |Yn |p .
k=1
(c) Suppose the martingale {Yn } is such that Y0 = 0. Using the fact that
(Yn + c)2 is a sub-martingale and optimizing over c, show that for y > 0,
n
P(max Yk y)
k=0
EYn2
EYn2 + y 2
Here is the version of Doobs inequality for non-negative sup-MGs and its application for the random walk.
Exercise 5.2.12.
(a) Show that if is a stopping time for the canonical filtration of a nonnegative super-martingale {Xn } then EX0 EXn E[X I n ] for
any finite n.
(b) Deduce that if {Xn } is a non-negative super-martingale then for any x >
0
P(sup Xk x) x1 EX0 .
k
(c) Suppose Sn is a random walk with E1 = < 0 and Var(1 ) = 2 > 0.

Let = /( 2 + 2 ) and f (x) = 1/(1 + (z x)+ ). Show that f (Sn ) is
a super-martingale and use this to conclude that for any z > 0,
1
.
P(sup Sk z)
1 + z
k
Hint: Taking v(x) = f (x)2 1x<z show that gx (y) = f (x) + v(x)[(y
x) + (y x)2 ] f (y) for all x and y. Then show that f (Sn ) =
E[gSn (Sn+1 )|Sk , k n].
Integrating Doobs inequality we next get bounds on the moments of the maximum
of a sub-MG.
191
Corollary 5.2.13 (Lp maximal inequalities). If {Xn } is a sub-MG then for

any n and p > 1,

(5.2.4)
E (max Xk )p+ q p E (Xn )p+ ,
kn
where q = p/(p 1) is a finite universal constant. Consequently, if {Yn } is a MG

then for any n and p > 1,

p

(5.2.5)
E max |Yk |
q p E |Yn |p .
kn
Proof. The bound (5.2.4) is obtained by applying part (b) of Lemma 1.4.31
for the non-negative variables X = (Xn )+ and Y = (maxkn Xk )+ . Indeed, the
hypothesis P(Y y) y 1 E[XIY y ] of this lemma is provided by the left inequality in (5.2.1) and its conclusion that EY p q p EX p is precisely (5.2.4). In
case {Yn } is a martingale, we get (5.2.5) by applying (5.2.4) for the non-negative
sub-MG Xn = |Yn |.

Remark. A bound such as (5.2.5) can not hold for all sub-MGs. For example,
the non-random sequence Yk = (k n) 0 is a sub-MG with |Y0 | = n but Yn = 0.
The following two exercises show that while Lp maximal inequalities as in Corollary
5.2.13 can not hold for p = 1, such an inequality does hold provided we replace
E(Xn )+ in the bound by E[(Xn )+ log min(Xn , 1)].
Qn
Exercise 5.2.14. Consider the martingale Mn = k=1 Yk for i.i.d. non-negative
random variables {Yk } with EY1 = 1 and P(Y1 = 1) < 1.
(a) Explain why E(log Y1 )+ is finite and why the strong law of large numbers
a.s.
implies that n1 log Mn < 0 when n .
a.s.
(b) Deduce that Mn 0 as n and that consequently {Mn } is not
uniformly integrable.
(c) Show that if (5.2.4) applies for p = 1 and some q < , then any nonnegative martingale would have been uniformly integrable.
Exercise 5.2.15. Show that if {Xn } is a non-negative sub-MG then

E max Xk (1 e1 )1 {1 + E[Xn (log Xn )+ ]} .
kn
Hint: Apply part (c) of Lemma 1.4.31 and recall that x(log y)+ e1 y + x(log x)+
for any x, y 0.
We just saw that in general L1 -bounded martingales might not be U.I. Nevertheless, as you show next, for sums of independent zero-mean random variables these
two properties are equivalent.
Pn
Exercise 5.2.16. Suppose Sn = k=1 k with k independent.
(a) Prove Ottavianis inequality. Namely, show that for any n and t, s 0,
k=1
k=1
k=1
P(max |Sk | t + s) P(|Sn | t) + P(max |Sk | t + s) max P(|Sn Sk | > s) .

(b) Suppose further that {k } is integrable and supn E|Sn | < . Show that
then E[supk |Sk |] is finite.
192
In the spirit of Doobs inequality bounding the tail probability of the maximum
of a sub-MG {Xk , k = 0, 1, . . . , n} in terms of the value of Xn , we will bound the
oscillations of {Xk , k = 0, 1, . . . , n} over an interval [a, b] in terms of X0 and Xn .
To this end, we require the following definition of up-crossings.
Definition 5.2.17. The number of up-crossings of the interval [a, b] by {Xk (), k =
0, 1, . . . , n}, denoted Un [a, b](), is the largest Z+ such that Xsi () < a and
Xti () > b for 1 i and some 0 s1 < t1 < < s < t n.
For example, Fig. 1 depicts two up-crossings of [a, b].
Figure 1. Illustration of up-crossings of [a, b] by Xk ()

Our next result, Doobs up-crossing inequality, is the key to the a.s. convergence
of sup-MGs (and sub-MGs) on which Section 5.3 is based.
Lemma 5.2.18 (Doobs up-crossing inequality). If {Xn } is a sup-MG then
(5.2.6)
(b a)E(Un [a, b]) E[(Xn a) ] E[(X0 a) ]
a < b .
Proof. Fixing a < b, let V1 = I{X0 <a} and for n = 2, 3, . . ., define recursively
Vn = I{Vn1 =1,Xn1 b} + I{Vn1 =0,Xn1 <a} . Informally, the sequence Vk is zero
while waiting for the process {Xn } to enter (, a) after which time it reverts
to one and stays so while waiting for this process to enter (b, ). See Figure 1
for an illustration in which black circles depict indices k such that Vk = 1 and
open circles indicate those values of k with Vk = 0. Clearly, the sequence {Vn } is
predictable for the canonical filtration of {Xn }. Let {Yn } denote the martingale
5.3. THE CONVERGENCE OF MARTINGALES
193
transform of {Vn } with respect to {Xn } (per Definition 5.1.27). By the choice of
V every up-crossing of the interval [a, b] by {Xk , k = 0, 1, . . . , n} contributes to Yn
the difference between the value of X at the end of the up-crossing (i.e. the last in
the corresponding run of black circles), which is at least b and its value at the start
of the up-crossing (i.e. the last in the preceding run of open circles), which is at
most a. Thus, each up-crossing increases Yn by at least (b a) and if X0 < a then
the first up-crossing must have contributed at least (b X0 ) = (b a) + (X0 a)
to Yn . The only other contribution to Yn is by the up-crossing of the interval [a, b]
that is in progress at time n (if there is such), and since it started at value at most
a, its contribution to Yn is at least (Xn a) . We thus conclude that
Yn (b a)Un [a, b] + (X0 a) (Xn a)
for all . With {Vn } predictable, bounded and non-negative it follows that {Yn }
is a super-martingale (see parts (b) and (c) of Theorem 5.1.28). Thus, considering
the expectation of the preceding inequality yields the up-crossing inequality (5.2.6)
since 0 = EY0 EYn for the sup-MG {Yn }.

Doobs up-crossing inequality implies that the total number of up-crossings of [a, b]
by a non-negative sup-MG has a finite expectation. In this context, Dubins upcrossing inequality, which you are to derive next, provides universal (i.e. depending
only on a/b), exponential bounds on tail probabilities of this random variable.
Exercise 5.2.19. Suppose (Xn1 , Fn ) and (Xn2 , Fn ) are both sup-MGs and is an
Fn -stopping time such that X1 X2 .
(a) Show that Wn = Xn1 I >n + Xn2 I n is a sup-MG with respect to Fn and
deduce that so is Yn = Xn1 I n + Xn2 I <n (this is sometimes called the
switching principle).
(b) For a sup-MG Xn 0 and constants b > a > 0 define the FnX -stopping
times 0 = 1, = inf{k > : Xk a} and +1 = inf{k > : Xk
b}, = 0, 1, . . .. That is, the -th up-crossing of (a, b) by {Xn } starts at
1 and ends at . For = 0, 1, . . . let Zn = a b when n [ , ) and
Zn = a1 b Xn for n [ , +1 ). Show that (Zn , FnX ) is a sup-MG.
(c) For b > a > 0 let U [a, b] denote the total number of up-crossings of the
interval [a, b] by a non-negative super-martingale {Xn }. Deduce from the
preceding that for any positive integer ,
a
E[min(X0 /a, 1)]
P(U [a, b] )
b
(this is Dubins up-crossing inequality).
5.3. The convergence of Martingales
As we shall see in this section, a sub-MG (or a sup-MG), has an integrable

limit under relatively mild integrability assumptions. For example, in this context L1 -boundedness (i.e. the finiteness of supn E|Xn |), yields a.s. convergence
(see Doobs convergence theorem), while the L1 -convergence of {Xn } is equivalent
to the stronger hypothesis of uniform integrability of this process (see Theorem
5.3.12). Finally, the even stronger Lp -convergence applies for the smaller sub-class
of Lp -bounded martingales (see Doobs Lp martingale convergence).
194
Indeed, these convergence results are closely related to the fact that the maximum
and up-crossings counts of a sub-MG do not grow too rapidly (and same applies
for sup-MGs and martingales). To further explore this direction, we next link the
finiteness of the total number of up-crossings U [a, b] of intervals [a, b], b > a, by a
process {Xn } to its a.s. convergence.
a.s
Lemma 5.3.1. If for each b > a almost surely U [a, b] < , then Xn X
where X is an R-valued random variable.
Proof. Note that the event that Xn has an almost sure (R-valued) limit as
n is the complement of
[
a,b ,
=
a,bQ
a a,

a,b = { : lim inf Xn () < a b
and lim inf n Xn () < a are both limit points of the sequence {Xn ()}, hence the
total number of up-crossings of the interval [a, b] by this sequence is infinite. That
is, a,b { : U [a, b]() = }. So, from our hypothesis that U [a, b] is finite
almost surely it follows that P(a,b ) = 0 for each a < b, resulting with the stated
conclusion.

Combining Doobs up-crossing inequality of Lemma 5.2.18 with Lemma 5.3.1 we
now prove Doobs a.s. convergence theorem for sup-MGs (and sub-MGs).
Theorem 5.3.2 (Doobs convergence theorem). Suppose sup-MG (Xn , Fn )
a.s.
is such that supn {E[(Xn ) ]} < . Then, Xn X and E|X | lim inf n E|Xn |
is finite.
Proof. Fixing b > a, recall that 0 Un [a, b] U [a, b] as n , where
U [a, b] denotes the total number of up-crossings of [a, b] by the sequence {Xn }.
Hence, by monotone convergence E(U [a, b]) = supn E(Un [a, b]). Further, with
(xa) |a|+x , we get from Doobs up-crossing inequality and the monotonicity
of the expectation that

1
1
|a| + sup E[(Xn ) ] .
E(Xn a)
E(Un [a, b])
(b a)
(b a)
n
Thus, our hypothesis that supn E[(Xn ) ] < implies that E(U [a, b]) is finite,
hence in particular U [a, b] is finite almost surely.
a.s
Since this applies for any b > a, we have from Lemma 5.3.1 that Xn X .
Further, with Xn a sup-MG, we have that E|Xn | = EXn + 2E(Xn ) EX0 +
2E(Xn ) for all n. Using this observation in conjunction with Fatous lemma for
a.s.
0 |Xn | |X | and our hypothesis, we find that
E|X | lim inf E|Xn | EX0 + 2 sup{E[(Xn ) ]} < ,
n
as stated.
195
Remark. In particular, Doobs convergence theorem implies that if (Xn , Fn ) is

a.s.
a non-negative sup-MG then Xn X for some integrable X (and in this
case EX EX0 ). The same convergence applies for a non-positive sub-MG and
more generally, for any sub-MG with supn {E(Xn )+ } < . Further, the following
exercise provides alternative equivalent conditions for the applicability of Doobs
convergence theorem.
Exercise 5.3.3. Show that the following five conditions are equivalent for any
sub-MG {Xn } (and if {Xn } is a sup-MG, just replace (Xn )+ by (Xn ) ).
(a) limn E|Xn | exists and is finite.
(b) supn E|Xn | < .
(c) lim inf n E|Xn | < .
(d) limn E(Xn )+ exists and is finite.
(e) supn E(Xn )+ < .
Our first application of Doobs convergence theorem extends Doobs inequality
(5.2.1) to the following bound on the maximal value of a U.I. sub-MG.
Corollary 5.3.4. For any U.I. sub-MG {Xn } and x > 0,
(5.3.1)
P(Xk x for some k < ) x1 E[X Ix < ] x1 E[(X )+ ] ,
where x = min{k 0 : Xk x}.
Proof. Let An = {x n} = {maxkn Xk x} and A = {x < } =

{Xk x for some k < }. Then, An A and as the U.I. sub-MG {Xn } is
a.s.
L1 -bounded, we have from Doobs convergence theorem that Xn X . Consequently, Xn IAn and (Xn )+ converge almost surely to X IA and (X )+ , respectively. Since these two sequences are U.I. we further have that E[Xn IAn ]
E[X IA ] and E[(Xn )+ ] E[(X )+ ]. Recall Doobs inequality (5.2.1) that
(5.3.2)
P(An ) x1 E[Xn IAn ] x1 E[(Xn )+ ]
for any n finite. Taking n we conclude that
P(A ) x1 E[X IA ] x1 E[(X )+ ]
which is precisely our stated inequality (5.3.1).
Applying Doobs convergence theorem we also find that martingales of bounded

differences either converge to a finite limit or oscillate between and +.
Proposition 5.3.5. Suppose {Xn } is a martingale of uniformly bounded differences. That is, almost surely supn |Xn Xn1 | c for some finite non-random
constant c. Then, P(A B) = 1 for the events
A = { : lim Xn () exists and is finite},
n
B = { : lim sup Xn () = and lim inf Xn () = }.

n
Proof. We may and shall assume without loss of generality that X0 = 0

(otherwise, apply the proposition for the MG Yn = Xn X0 ). Fixing a positive integer k, consider the stopping time k () = inf{n 0 : Xn () k} for
the canonical filtration of {Xn } and the associated stopped sup-MG Yn = Xnk
(per Theorem 5.1.32). By definition of k and our hypothesis of Xn having uniformly bounded differences, it follows that Yn () k c for all n. Consequently,
196
supn E(Yn ) k + c and by Doobs convergence theorem Yn () Y () R

for all
/ k and some measurable k such that P(k ) = 0. In particular, if
k () = and
/ k then Xn () = Yn () has a finite limit, so A. This
shows that Ac {k < } k for all k, and hence Ac B k k where
B = k {k < } = { : lim inf n Xn () = }. With P(k ) = 0 for all k,
we thus deduce that P(A B ) = 1. Applying the preceding argument for the
sup-MG {Xn } we find that P(A B+ ) = 1 for B+ = { : lim supn Xn () = }.
Combining these two results we conclude that P(A (B B+ )) = 1 as stated.
Pn
Remark. Consider a random walk Sn = k=1 k with zero-mean, bounded increments {k } (i.e. |k | c with c a finite non-random constant), such that the
finite v = Ek2 is non-zero. Then, the event A where Sn () S () as n for
some S () finite, implies that Sbn () = (nv)1/2 Sn () 0. Combining the clt
D
Sbn G with Fatous lemma and part (d) of the Portmanteau theorem we find
that for any > 0,
P(A) E[lim inf I|Sbn | ] lim inf P(|Sbn | ) = P(|G| ) .
n
Taking 0 we deduce that P(A) = 0. Hence, by Proposition 5.3.5 such random

walk is an example of a non-converging MG for which a.s.
lim sup Sn = = lim inf Sn .
n
Here is another application of the preceding proposition.

Exercise 5.3.6. Consider the Fn -adapted Wn 0, such that supn |Wn+1 Wn |
K for some finite non-random constant K and W0 = 0. Suppose there exist nonrandom, positive constants a and b such that for all n 0,
Pn
E[Wn+1 Wn + a|Fn ]I{Wn b} 0 .
With Nn = k=1 I{Wk <b} , show that P(N is finite) = 0.

Hint: Check that Xn = Wn + an (K + a)Nn1 is a sup-MG of uniformly bounded
differences.
As we show next, Doobs convergence theorem leads to the integrability of X for
any L1 bounded sub-MG Xn and any stopping time .
Lemma 5.3.7. If (Xn , Fn ) is a sub-MG and supn E[(Xn )+ ] < then E|X | <
for any Fn -stopping time .
Proof. Since ((Xn )+ , Fn ) is a sub-MG (see Proposition 5.1.22), it follows
that E[(Xn )+ ] E[(Xn )+ ] for all n (consider Theorem 5.1.32 for the sub-MG
(Xn )+ and = ). Thus, our hypothesis that supn E[(Xn )+ ] is finite results with
supn E[(Yn )+ ] finite, where Yn = Xn . Applying Doobs convergence theorem for
a.s

the sub-MG (Yn , Fn ) we have that Yn Y with Y = X integrable.
We further get the following relation, which is key to establishing Doobs optional
stopping for certain sup-MGs (and sub-MGs).
Proposition 5.3.8. Suppose (Xn , Fn ) is a non-negative sup-MG and are
stopping times for the filtration {Fn }. Then, EX EX are finite valued.
197
Proof. From Theorem 5.1.32 we know that Zn = Xn Xn is a sup-MG

(as are Xn and Xn ), with Z0 = 0. Thus, E[Xn ] E[Xn ] are finite and
since , subtracting from both sides the finite E[Xn In ] we find that
E[X I<n ] E[X I <n ] + E[Xn I n I<n ] .
a.s.
The sup-MG {Xn } is non-negative, so by Doobs convergence theorem Xn X

and in view of Fatous lemma
lim inf E[Xn I n I<n ] E[X I = I< ] .
n
Further, by monotone convergence E[X I <n ] E[X I < ] and E[X I<n ]
E[X I< ]. Hence, taking n results with
E[X I< ] E[X I < ] + E[X I = I< ] .
Adding the identity E[X I= ] = E[X I= ], which holds for , yields the
stated inequality E[X ] E[X ]. Considering 0 we further see that E[X0 ]
E[X ] E[X ] 0 are finite, as claimed.

Solving the next exercise should improve your intuition about the domain of validity of Proposition 5.1.22 and of Doobs convergence theorem.
Exercise 5.3.9.
(a) Provide an example of a sub-martingale {Xn } for which {Xn2 } is a supermartingale and explain why it does not contradict Proposition 5.1.22.
(b) Provide an example of a martingale which converges a.s. to and
explain why it does
P not contradict Theorem 5.3.2.
Hint: Try Sn = ni=1 i , with zero-mean, independent but not identically
distributed i .
We conclude this sub-section with few additional applications of Doobs convergence theorem.
Exercise 5.3.10. Suppose {Xn } and {Y
Pn } are non-negative, integrable processes
adapted to the filtration Fn such that
n1 Yn < a.s. and E[Xn+1 |Fn ]
(1 + Yn)Xn + Yn for all n. Show that Xn converges a.s. to a finite limit as n .
Hint: Find a non-negative super-martingale (Wn , Fn ) whose convergence implies
that of Xn .
Exercise 5.3.11. Let {Xk } be mutually independent but not necessarily integrable
random variables, such that
P
(c)
(a) Fixing c < non-random, let Yn = nk=1 |Sk1 |I|Sk1 |c Xk I|Xk |c .
(c)
Show that Yn is a martingale with respect to the filtration {FnX } and

(c)
that supn kYn k2 < .
(c)
Hint: Kolmogorovs three series theorem may help in proving that {Yn }
2
is L -bounded. P
n
(b) Show that Yn = k=1 |Sk1 |Xk converges a.s.
5.3.1. Uniformly integrable martingales. The main result of this subsection is the following L1 convergence theorem for uniformly integrable (U.I.) subMGs (and sup-MGs).
198
Theorem 5.3.12. If (Xn , Fn ) is a sub-MG, then {Xn } is U.I. (c.f. Definition

L1
a.s.
1.3.47), if and only if Xn X , in which case also Xn X and Xn

E[X |Fn ] for all n.
Remark. If {Xn } is uniformly integrable then supn E|Xn | is finite (see Lemma
1.3.48). Thus, the assumption of Theorem 5.3.12 is stronger than that of Theorem
5.3.2, as is its conclusion.
Proof. If {Xn } is U.I. then supn E|Xn | < . For {Xn } sub-MG it thus
a.s.
follows by Doobs convergence theorem that Xn X with X integrable. Obp
viously, this implies that Xn X . Similarly, if we start instead by assuming
L1
that Xn X then also Xn X . Either way, Vitalis convergence theorem

(i.e. Theorem 1.3.49), tells us that uniform integrability is equivalent to L1 conp
vergence when Xn X . We thus deduce that for sub-MGs the U.I. property is
equivalent to L1 convergence and either one of these yields also the corresponding
a.s. convergence.
Turning to show that Xn E[X |Fn ] for all n, recall that Xm E[X |Fm ] for
L1
all > m and any sub-MG (see Proposition 5.1.20). Further, since X X it
L1
follows that E[X |Fm ] E[X |Fm ] as , per fixed m (see Theorem 4.2.30).
The latter implies the convergence a.s. of these conditional expectations along some
sub-sequence k (c.f. Theorem 2.2.10). Hence, we conclude that for any m, a.s.
Xm lim inf E[X |Fm ] E[X |Fm ] ,
i.e., Xn E[X |Fn ] for all n.
The preceding theorem identifies the collection of U.I. martingales as merely the
set of all Doobs martingales, a concept we now define.
Definition 5.3.13. The sequence Xn = E[X|Fn ] with X an integrable R.V. and
{Fn } a filtration, is called Doobs martingale of X with respect to {Fn }.
Corollary 5.3.14. A martingale (Xn , Fn ) is U.I. if and only if Xn = E[X |Fn ]
L1
is a Doobs martingale with respect to {Fn }, or equivalently if and only if Xn X .

Proof. Theorem 5.3.12 states that a sub-MG (hence also a MG) is U.I. if and
only if it converges in L1 and in this case Xn E[X |Fn ]. Applying this theorem
also for Xn we deduce that a U.I. martingale is necessarily a Doobs martingale
of the form Xn = E[X |Fn ]. Conversely, the sequence Xn = E[X|Fn ] for some
integrable X and a filtration {Fn } is U.I. (see Proposition 4.2.33).

We next generalize Theorem 4.2.26 about dominated convergence of C.E.
Theorem 5.3.15 (L
evys upward theorem). Suppose supm |Xm | is integrable,
a.s.
Xn X and Fn F . Then E[Xn |Fn ] E[X |F ] both a.s. and in L1 .
Remark. Levys upward theorem is trivial if {Xn } is adapted to {Fn } (which
is obviously not part of its assumptions). On the other hand, recall that in view
a.s.
of part (b) of Exercise 4.2.35, having {Xn } U.I. and Xn X is in general not
enough even for the a.s. convergence of E[Xn |G] to E[X |G].
199
Proof. Consider first the special case where Xn = X does not depend on
n. Then, Yn = E[X|Fn ] is a U.I. martingale. Therefore, E[Y |Fn ] = E[X|Fn ]
for all n, where Y denotes the a.s. and L1 limit of Yn (see Corollary 5.3.14).
As Yn mFn mF clearly Y = limn Yn mF . Further,
S by definition of
the C.E. E[XIA ] = E[Y IA ] for all A in the -system P = n Fn hence with
F = (P) it follows that Y = E[X|F ] (see Exercise 4.1.3).
a.s.
Turning to the general case, with Z = supm |Xm | integrable and Xm X , we
deduce that X and Wk = sup{|Xn X | : n k} 2Z are both integrable. So,
the conditional Jensens inequality and the monotonicity of the C.E. imply that for
all n k,
|E[Xn |Fn ] E[X |Fn ]| E[|Xn X | |Fn ] E[Wk |Fn ] .
Consequently, considering n we find by the special case of the theorem where

Xn is replaced by Wk independent of n (which we already proved), that
lim sup |E[Xn |Fn ] E[X |Fn ]| lim E[Wk |Fn ] = E[Wk |F ] .
n
a.s
Similarly, we know that E[X |Fn ] E[X |F ]. Further, by definition Wk 0

a.s. when k , so also E[Wk |F ] 0 by the usual dominated convergence of
C.E. (see Theorem 4.2.26). Combining these two a.s. convergence results and the
a.s
preceding inequality, we deduce that E[Xn |Fn ] E[X |F ] as stated. Finally,
since |E[Xn |Fn ]| E[Z|Fn ] for all n, it follows that {E[Xn |Fn ]} is U.I. and hence
the a.s. convergence of this sequence to E[X |F ] yields its convergence in L1 as
well (c.f. Theorem 1.3.49).

Considering Levys upward theorem for Xn = X = IA and A F yields the
following corollary.
a.s.
Corollary 5.3.16 (L
evys 0-1 law). If Fn F , A F , then E[IA |Fn ]
IA .
As shown in the sequel, Kolmogorovs 0-1 law about P-triviality of the tail algebra T X = n TnX of independent random variables is a special case of Levys
0-1 law.
S
Proof of Corollary 1.4.10. Let F X = ( n FnX ). Recall Definition 1.4.9
a.s.
that T X TnX F X for all n. Thus, by Levys 0-1 law E[IA |FnX ] IA for
any A T X . By assumption {Xk } are P-mutually independent, hence for any
A T X the R.V. IA mTnX is independent of the -algebra FnX . Consequently,
a.s.
a.s.
E[IA |FnX ] = P(A) for all n. We deduce that P(A) = IA , implying that P(A)
X
{0, 1} for all A T , as stated.

The generalization of Theorem 4.2.30 which you derive next also relaxes the assumptions of Levys upward theorem in case only L1 convergence is of interest.
L1
L1
Exercise 5.3.17. Show that if Xn X and Fn F then E[Xn |Fn ]

E[X |F ].
Here is an example of the importance of uniform integrability when dealing with
convergence.
a.s.
Exercise 5.3.18. Suppose Xn 0 are [0, 1]-valued random variables and {Mn }
is a non-negative MG.
200
(a) Provide an example where E[Xn Mn ] = 1 for all n finite.

(b) Show that if {Mn } is U.I. then E[Xn Mn ] 0.
Definition 5.3.19. A continuous function x : [0, 1) 7 R is absolutely continuous
if for every > 0 there exists > 0 such that for all k < , s1 < t1 s2 < t2
sk < tk [0, 1)
k
X
=1
|t s |
k
X
=1
|x(t ) x(s )| .
The next exercise uses convergence properties of MGs to prove a classical result
in real analysis, namely, that an absolutely continuous function is differentiable for
Lebesgue a.e. t [0, 1).
Exercise 5.3.20. On the probability space ([0, 1), B, U ) consider the events
Ai,n = [(i 1)2n , i2n )
for
i = 1, . . . , 2n ,
n = 0, 1, . . . ,
and the associated -algebras Fn = (Ai,n , i = 1, . . . , 2 ).

(a) Write an explicit formula for E[h|Fn ] and h L1 ([0, 1), B, U ).
P2n
(b) For hi,n = 2n (x(i2n )x((i1)2n )), show that Xn (t) = i=1 hi,n IAi,n (t)
is a martingale with respect to {Fn }.
(c) Show that for absolutely continuous x() the martingale {Xn } is U.I.
Hint: Show that P(|Xn | > ) c/ for some constant c < and all
n, > 0.
(d) Show that then there exists h L1 ([0, 1), B, U ) such that
Z t
x(t) x(s) =
h(u)du
for all
1 > t s 0.
s
R s+
a.s
(e) Recall Lebesgues theorem, that 1 s
|h(s) h(u)|du 0 as 0,
dx
for a.e. s [0, 1). Using it, conclude that dt = h for almost every
t [0, 1).
Here is another consequence of MG convergence properties (this time, of relevance

for certain economics theories).
Exercise 5.3.21. Given integrable random variables X, Y0 and Z0 on the same
probability space (, F , P), and two -algebras A F , B F , for k = 1, 2, . . ., let
Yk := E[X|(A, Z0 , . . . , Zk1 )] ,
Zk := E[X|(B, Y0 , . . . , Yk1 )] .
Show that Yn Y and Zn Z a.s. and in L1 , for some integrable random

variables Y and Z . Deduce that a.s. Y = Z , hence Yn Zn 0 a.s. and in
L1 .
Recall that uniformly bounded p-th moment for some p > 1 implies U.I. (see
Exercise 1.3.54). Strengthening the L1 convergence of Theorem 5.3.12, the next
proposition shows that an Lp -bounded martingale converges to its a.s. limit also
in Lp (provided p > 1). In contrast to the preceding convergence results, this
one does not hold for sub-MGs (or sup-MGs) which are not MGs (for example, let
= inf{k 1 : k = 0} for independent {k } such that P(k 6= 0) = k 2 /(k + 1)2 ,
a.s.
so P( n) = n2 and verify that Xn = nI{n } 0 but EXn2 = 1, so this
L2 -bounded sup-MG does not converge to zero in L2 ).
201
Proposition 5.3.22 (Doobs Lp martingale convergence). If the MG {Xn }

is such that supn E|Xn |p < for some p > 1, then there exists a R.V. X such
that Xn X almost surely and in Lp (so kXn kp kX kp ).
Proof. Being Lp bounded, the MG {Xn } is L1 bounded and Doobs martina.s.

gale convergence theorem applies here, so Xn X for some integrable R.V. X .
Further, applying Fatous lemma for |Xn |p 0 we have that,
lim inf E(|Xn |p ) E(lim inf |Xn |p ) = E|X |p
n
as claimed, with X Lp . It thus suffices to verify that E|Xn X |p 0 as

n (as in Exercise 1.3.28, this implies that kXn kp kX kp ). To this end, with
c = supn E|Xn |p finite we have by the Lp maximal inequality of (5.2.5) that EZn
q p c for Zn = maxkn |Xk |p and any finite n. Since 0 Zn Z = supk< |Xk |p , we
have by monotone convergence that EZ q p c is finite. As X is the a.s. limit of Xn
it follows that |X |p Z as well. Hence, Yn = |Xn X |p (|Xn |+|X |)p 2p Z.
a.s.
With Yn 0 and Yn 2p Z for integrable Z, we deduce by dominated convergence
that EYn 0 as n , thus completing the proof of the proposition.

Remark. Proposition 5.3.22 does not have an L1 analog. Indeed, as we have
seen already in Exercise 5.2.14, there exists a non-negative MG {Mn } such that
EMn = 1 for all n and Mn M = 0 almost surely, so obviously, Mn does not
converge to M in L1 .
Pn
Example 5.3.23. Consider the martingale Sn = k=1
Pk for2 independent, square2
integrable,
zero-mean
random
variables
such
that
k
k Ek < . Since ESn =
Pn
2
k=1 Ek , it follows from Proposition 5.3.22 that the random series Sn ()
S () almost surely and in L2 (see also Theorem 2.3.17 for a direct proof of this
result, based on Kolmogorovs maximal inequality).
Pn
Exercise 5.3.24. Suppose Zn = 1n k=1 k for i.i.d. k L2 (, F , P) of zeromean and unit variance. Let Fn = (k , k n) and F = (k , k < ).
(a) Prove that EW Zn 0 for any fixed W L2 (, F , P).
(b) Deduce that the same applies for any W L2 (, F , P) and conclude that
Zn does not converge in L2 .
D
(c) Show that though Zn G, a standard normal variable, there exists no
p
Z mF such that Zn Z .
We conclude this sub-section with the application of martingales to the study of
P
olyas urn scheme.
lyas urn). Consider an urn that initially contains r red and
Example 5.3.25 (Po
b blue marbles. At the k-th step a marble is drawn at random from the urn, with all
possible choices being equally likely, and it and
Pnck more marbles of the same color are
then returned to the urn. With Nn = r+b+ k=1 ck counting the number of marbles
in the urn after n iterations of this procedure, let Rn denote the number of red
marbles at that time and Mn = Rn /Nn the corresponding fraction of red marbles.
Since Rn+1 {Rn , Rn +cn } with P(Rn+1 = Rn +cn |FnM ) = Rn /Nn = Mn it follows
that E[Rn+1 |FnM ] = Rn + cn Mn = Nn+1 Mn . Consequently, E[Mn+1 |FnM ] = Mn
for all n with {Mn } a uniformly bounded martingale.
For the study of P
olyas urn scheme we need the following definition.
202
Definition 5.3.26. The beta density with parameters > 0 and > 0 is
( + ) 1
u
(1 u)1 1u[0,1] ,
f (u) =
()()
R
where () = 0 s1 es ds is finite and positive (compare with Definition 1.4.45).
In particular, = = 1 corresponds to the density fU (u) of the uniform measure
on (0, 1], as in Example 1.2.41.
Exercise 5.3.27. Let {Mn } be the martingale of Example 5.3.25.
(a) Show that Mn M a.s. and in Lp for any p > 1.
(b) Assuming further that ck = c for all k 1, show that for = 0, . . . , n,
Qn1
Q1
(b + jc)
n
i=0 (r + ic)
j=0
P(Rn = r + c) =
,
Qn1
k=0 (r + b + kc)
and deduce that M has the beta density with parameters = b/c and
= r/c (in particular, M has the law of U (0, 1] when r = b = ck > 0).
(c) For r = b = ck > 0 show that P(supk1 Mk > 3/4) 2/3.
Exercise 5.3.28 (Bernard Friedmans urn). Consider the following variant

of P
olyas urn scheme, where after the k-th step one returns to the urn in addition
to the marble drawn and ck marbles of its color, also dk 1 marbles of the opposite
a.s.
color. Show that if ck , dk are uniformly bounded and r + b > 0, then Mn 1/2.
M
Hint: With Xn = (Mn 1/2)2 check that E[Xn |Fn1
] = (1 an )Xn1P+ un , where
P
the non-negative constants an and un are such that k uk < and k ak = .
Exercise 5.3.29. Fixing bn [, 1] for some > 0, suppose {Xn } are [0, 1]-valued,
Fn -adapted such that Xn+1 = (1 bn )Xn + bn Bn , n 0, and P(Bn = 1|Fn ) =
a.s.
1 P(Bn = 0|Fn ) = Xn . Show that Xn X {0, 1} and P(X = 1|F0 ) = X0 .
5.3.2. Square-integrable martingales. If (Xn , Fn ) is a square-integrable

martingale then (Xn2 , Fn ) is a sub-MG, so by Doobs decomposition Xn2 = Mn + An
for a non-decreasing Fn -predictable sequence {An } and a MG (Mn , Fn ) with M0 =
0. In the course of proving Doobs decomposition we saw that An An1 =
2
E[Xn2 Xn1
|Fn1 ] and part (b) of Exercise 5.1.8 provides an alternative expression
An An1 = E[(Xn Xn1 )2 |Fn1 ], motivating the following definition.
Pn
Definition 5.3.30. The sequence An = X02 + k=1 E[(Xk Xk1 )2 |Fk1 ] is
called the predictable compensator of an L2 -martingale (Xn , Fn ) and denoted by
angle-brackets, that is, An = hXin .
With EMn = EM0 = 0 it follows that EXn2 = EhXin , so {Xn } is L2 -bounded if
and only if supn EhXin is finite. Further, hXin is non-decreasing, so it converges
to a limit, denoted hereafter hXi . With hXin hXi0 = X02 integrable it further
follows by monotone convergence that EhXin EhXi , so {Xn } is L2 bounded
if and only if hXi is integrable, in which case Xn converges a.s. and in L2 (see
Doobs L2 convergence theorem). As we show in the sequel much more can be said
about the relation between convergence of Xn () and the random variable hXi .
To simplify the notations assume hereafter that X0 = 0 = hXi0 so hXin 0 (the
transformation of our results to the general case is trivial).
We start with the following explicit bounds on E[supn |Xn |p ] for p 2, from which
1/2
we deduce that {Xn } is U.I. (hence converges a.s. and in L1 ), whenever hXi is
integrable.
203
Proposition 5.3.31. There exist finite constants cq , q (0, 1], such that if
(Xn , Fn ) is an L2 -MG with X0 = 0, then
E[sup |Xk |2q ] cq E[ hXiq ] .
k
Remark. Our proof gives cq = (2 q)/(1 q) for q < 1 and c1 = 4.

Proof. Let Vn = maxnk=0 |Xk |2 , noting that Vn V = supk |Xk |2 when
n . As already explained EXn2 EhXi for n . Thus, applying the
bound (5.2.5) of Corollary 5.2.13 for p = 2 we find that
E[Vn ] 4EXn2 4EhXi ,
and considering n we get our thesis for q = 1 (by monotone convergence).
Turning to the case of 0 < q < 1, note that (V )q = supk |Xk |2q . Further, the
Fn -predictable part in Doobs decomposition of the non-negative sub-martingale
Zn = Xn2 is An = hXin . Hence, applying Lemma 5.2.7 with p = q and =
yields the stated bound.

Here is an application of Proposition 5.3.31 to the study of a certain class of
random walks.
Pn
Exercise 5.3.32. Let Sn = k=1 k for i.i.d. {k } of zero mean and finite second
moment. Suppose is an Fn -stopping time such that E[ ] is finite.

(a) Compute the predictable compensator of the L2 -martingale (Sn , Fn ).
(b) Deduce that {Sn } is U.I. and that ES = 0.
We deduce from Proposition 5.3.31 that Xn () converges a.s. to a finite limit

1/2
when hXi is integrable. A considerable refinement of this conclusion is offered
by our next result, relating such convergence to hXi being finite at !
Theorem 5.3.33. Suppose (Xn , Fn ) is an L2 martingale with X0 = 0.
(a) Xn () converges to a finite limit for a.e. for which hXi () is finite.
(b) Xn ()/hXin () 0 for a.e. for which hXi () is infinite.
(c) If the martingale differences Xn Xn1 are uniformly bounded then the
converse to part (a) holds. That is, hXi () is finite for a.e. for
which Xn () converges to a finite limit.
Proof. (a) Recall that for any n and Fn -stopping time we have the identity
2
Xn
= Mn + hXin with EMn = 0, yielding by monotone convergence
2
that supn E[Xn
] = EhXi . While proving Lemma 5.2.7 we noted that v =
min{n 0 : hXin+1 > v} are Fn -stopping times such that hXiv v. Thus,
setting Yn = Xnk for a positive integer k, the martingale (Yn , Fn ) is L2 -bounded
and as such it almost surely has a finite limit. Further, if hXi () is finite, then
by definition k () = for some random positive integer k = k(), in which case
Xnk = Xn for all n. As we consider only countably many values of k, this yields
the thesis of part (a) of the theorem.
(b). Since Vn = (1 + hXin )1 isP
an Fn -predictable sequence of bounded variables,
n
its martingale transform Yn = k=1 Vk (Xk Xk1 ) with respect to the squareintegrable martingale {Xn } is also a square-integrable martingale for the filtration
204
{Fn } (c.f. Theorem 5.1.28). Further, since Vk mFk1 it follows that for all k 1,
hY ik hY ik1 = E[(Yk Yk1 )2 |Fk1 ] = Vk2 E[(Xk Xk1 )2 |Fk1 ]
=
1
1
hXik hXik1
(1 + hXik )2
1 + hXik1
1 + hXik
(as (x y)/(1 + x)2 (1 + y)1 (1 + x)1 for all x y 0 and hXik 0 is

non-decreasing in k). With hY i0 = hXi0 = 0, adding the preceding inequalities
over k = 1, . . . , n, we deduce that hY in 1 1/(1 + hXin ) 1 for all n. Thus,
by part (a) of the theorem,
for almost every , Yn () has a finite limit. That is,
P
for a.e. the series n xn /bn converges,
Pnwhere bn = 1 + hXin () is a positive,
non-decreasing sequence and Xn () =
k=1 xk for all n. If in addition to the
convergence of this series also hXi () = then bn and by Kroneckers
lemma Xn ()/bn 0. In this case bn /(bn 1) 1 so we conclude that then also
Xn /(bn 1) 0, which is exactly the thesis of part (b) of the theorem.

(c). Suppose that P hXi = , supn |Xn | < > 0. Then, there exists some r
such that P hXi = , r = > 0 for the Fn -stopping time r = inf{m 0 :
|Xm | > r}. Since supm |Xm Xm1 | c for some non-random finite constant c, we
2
(r+c)2
have that |Xnr | r+c, from which we deduce that EhXinr = EXn
r
for all n. With 0 hXinr hXir , by monotone convergence also

E hXi Ir = E hXir (r + c)2 .

This contradicts our assumption that P hXi = , r = > 0. In conclusion,
necessarily, P hXi = , supn |Xn | < = 0. Consequently, with supn |Xn |
finite on the set of values for which Xn () converges to a finite limit, it follows
that hXi () is finite for a.e. such .

We next prove Levys extension of both Borel-Cantelli lemmas (which is a neat
application of the preceding theorem).
Proposition 5.3.34 (Borel-Cantelli
III). Consider events An Fn for some
P
filtration {Fn }. Let SnP= nk=1 IAk count the number of events occurring among
the first n, with SP
=
k IAk the corresponding total number of occurrences. Simn
ilarly, let Zn =
k=1 k denote
P the sum of the first n conditional probabilities
k = P(Ak |Fk1 ) and Z = k k . Then, for almost every ,
(a) If Z () is finite, then so is S ().
(b) If Z () is infinite, then Sn ()/Zn () 1.
Remark. Given any sequence of events, by the tower property Ek = P(Ak ) for
all k and settingP
Fn = (Ak , k n) guarantees that Ak Fk for all k. Hence,
(a) If EZ P
=
k P(Ak ) is finite, then from part (a) of Proposition 5.3.34 we
deduce that k IAk is finite a.s., thus recovering the first Borel-Cantelli lemma.
(b) For Fn = (Ak , k n) and mutually independent events {Ak } we have that
k = P(Ak ) and Zn = ESn for all n. Thus, in this case, part (b) of Proposition
P
a.s.
5.3.34 is merely the statement that Sn /ESn 1 when k P(Ak ) = , which is
your extension of the second Borel-Cantelli via Exercise 2.2.26.
Proof. Clearly, Mn = Sn Zn is square-integrable and Fn -adapted. Further,
follows that
as Mn Mn1 = IAn E[IAn |Fn1 ] and Var(IAn |Fn1 ) = n (1n ), itP
n
the predictable compensator of the L2 martingale (Mn , Fn ) is hM in = k=1 k (1
k ). Hence, hM in Zn for all n, and if Z () is finite, then so is hM i (). By
205
part (a) of Theorem 5.3.33, for a.e. such the finite limit M () of Mn () exists,
implying that S = M + Z is finite as well.
With Sn = Mn + Zn , it suffices for part (b) of the proposition to show that
Mn /Zn 0 for a.e. for which Z () = . To this end, note first that by
the preceding argument, the finite limit M () exists also for a.e. for which
Z () = while hM i () is finite. For such we have that Mn /Zn 0 (since
Mn () is a bounded sequence while Zn () is unbounded). Finally, from part (b)
of Theorem 5.3.33 we know that Mn /hM in Mn /Zn converges to zero for a.e.
for which hM i () is infinite.

Here is a direct application of Theorem 5.3.33.
Exercise 5.3.35. Given a martingale (MP
n , Fn ) and positive, non-random bn ,
2
2
show that b1
M
0
for
a.e.
such
that
n
n
k1 bk E[(Mk Mk1 ) |Fk1 ] is finite.
Pn
1
Hint: Consider Xn = k=1 bk (Mk Mk1 ) and recall Kroneckers lemma.
The following extension of Kolmogorovs three series theorem uses both Theorem
5.3.33 and Levys extension of the Borel-Cantelli lemmas.
Exercise 5.3.36. Suppose {Xn } is adapted to filtration {Fn } and for any n, the
R.C.P.D. of Xn given Fn1 equals the R.C.P.D. of Xn given Fn1 . For non(c)
random c > 0 let Xn = Xn I|Xn |c be the corresponding truncated variables.
Pn
(c)
(a) Verify that (Zn , Fn ) is a MG, where Zn = k=1 Xk .
(b) Considering the series
X
X
(5.3.3)
P(|Xn | > c |Fn1 ),
and
Var(Xn(c) |Fn1 ),
n
P
show that for a.e. the series n Xn () has a finite limit if and only if
both series in (5.3.3) converge.
(c) Provide an example where the convergence in part (b) occurs with probability 0 < p < 1.
We now consider sufficient conditions for convergence almost surely of the martingale transform.
P
Exercise 5.3.37. Suppose Yn = nk=1 Vk (Zk Zk1 ) is the martingale transform
of the Fn -predictable {Vn } with respect to the martingale (Zn , Fn ), per Definition
5.1.27.
(a) Show that if {Zn } is L2 -bounded and {Vn } is uniformly bounded then
a.s.
Yn Y finite.
(b) Deduce that for L2 -bounded MG {Zn } the sequence Yn () converges to a
finite limit for a.e. for which supk1 |Vk ()| is finite.
(c) Suppose now that {Vk } is predictable for the canonical filtration {Fn } of
D
the i.i.d. {k }. Show that
P if k = k and u 7 uP(|1 | u) is bounded
above,
then the series
n Vn n has a finite limit for a.e. for which
P
|V
()|
is
finite.
k
k1
Hint: Consider Exercise 5.3.36 for the adapted sequence Xk = Vk k .
Here is another application of Levys extension of the Borel-Cantelli lemmas.

P
Exercise 5.3.38. Suppose Xn = 1 + nk=1 Dk , n 0, where the {1, 1}-valued
Dk is Fk -adapted and such that E[Dk |Fk1 ] for some non-random 1 > > 0
and all k 1.
206
(a) Show that (Xn , Fn ) is a sub-martingale and provide its Doob decomposition.
(b) Using this decomposition and Levys extension of the Borel-Cantelli lemmas, show that Xn almost surely.
(c) Let Zn = Xn for = (1 )/(1 + ). Show that (Zn , Fn ) is a supermartingale and deduce that P(inf n Xn 0) .
As we show next, the predictable compensator controls the exponential tails for
martingales of bounded differences.
Exercise 5.3.39. Fix > 0 non-random and an L2 martingale (Mn , Fn ) with
M0 = 0 and bounded differences supk |Mk Mk1 | 1.
(a) Show that Nn = exp(Mn (e 1)hM in ) is a sup-MG for {Fn }.
Hint: Recall part (a) of Exercise 1.4.40.
(b) Show that for any a.s. finite Fn -stopping time and constants u, r > 0,
P(M u, hM i r) exp(u + r(e 1)) .
(c) Applying (a) show that if the martingale {Sn } of Example 5.3.23 has
uniformly
P bounded differences |k | 1, then E exp(S ) is finite for
S = k k and any R.
Applying part (c) of the preceding exercise, you are next to derive the following tail
estimate, due to Dvoretsky, in the context of Levys extension of the Borel-Cantelli
lemmas.
Pn
ExerciseP5.3.40. Suppose Ak Fk for some filtration {Fk }. Let Sn = k=1 IAk
n
and Zn = k=1 P(Ak |Fk1 ). Show that P(Sn r + u, Zn r) eu (r/(r + u))r+u
for all n and u, r > 0, then deduce that for any 0 < r < 1,
n
n
X

[

P
Ak er + P
P(Ak |Fk1 ) > r .
k=1
k=1
Hint: Recall the proof of Borel-Cantelli III that the L2 -martingale Mn = Sn Zn

has differences bounded by one and hM in Zn .
We conclude this section with a refinement of the well known Azuma-Hoeffding

concentration inequality for martingales of bounded differences, from which we
deduce the strong law of large numbers for martingales of bounded differences.
Exercise 5.3.41. Suppose (Mn , Fn ) is a martingale with M0 = 0 and differences
Dk = Mk Mk1 , k 1 such that for some finite k and all u [0, 1],
E[Dk2 euDk |Fk1 ] k2 E[euDk |Fk1 ] < .
2
(a) Show that Nn =
n rn /2) is a sup-MG for Fn provided
Pexp(M
n
2
[0, 1] and rn = k=1 k .
Hint: Recall part (b) of Exercise 1.4.40.
(b) Deduce that for I(x) = (x 1)(2x x 1) and any u 0,
P(Mn u) exp(rn I(u/rn )/2) .

a.s.
(c) Conclude that b1

n Mn 0 for any martingale {M
n } of uniformly bounded
differences and non-random {bn } such that bn / n log n .
5.4. THE OPTIONAL STOPPING THEOREM
207
5.4. The optional stopping theorem

This section is about the use of martingales in computations involving stopping
times. The key tool for doing so is the following theorem.
Theorem 5.4.1 (Doobs optional stopping). Suppose are Fn -stopping
times and Xn = Yn +Vn for sub-MGs (Vn , Fn ), (Yn , Fn ) such that Vn is non-positive
and {Yn } is uniformly integrable. Then, the R.V. X and X are integrable and
EX EX EX0 (where X () and X () are set as lim supn Xn () in case
the corresponding stopping time is infinite).
Remark 5.4.2. Doobs optional stopping theorem holds for any sub-MG (Xn , Fn )
such that {Xn } is uniformly integrable (just set Vn = 0). Alternatively, it holds
also whenever E[X |Fn ] Xn for some integrable X and all n (for then the
martingale Yn = E[X |Fn ] is U.I. by Corollary 5.3.14, hence {Yn } also U.I. by
Proposition 5.4.4, and the sub-MG Vn = Xn Yn is by assumption non-positive).
By far the most common application has (Xn , Fn ) a martingale, in which case
it yields that EX0 = EX for any Fn -stopping time such that {Xn } is U.I.
(for example, whenever is bounded, or under the more general conditions of
Proposition 5.4.4).
Proof. By linearity of the expectation, it suffices to prove the claim separately for Yn = 0 and for Vn . Dealing first with Yn = 0, i.e. with a non-positive
sub-MG (Vn , Fn ), note that (Vn , Fn ) is then a non-negative sup-MG. Thus, the
inequality E[V ] E[V ] E[V0 ] and the integrability of V and V are immediate
consequences of Proposition 5.3.8.
Considering hereafter the sub-MG (Yn , Fn ) such that {Yn } is U.I., since
are Fn -stopping times it follows by Theorem 5.1.32 that Un = Yn , Zn = Yn and
Un Zn are all sub-MGs with respect to Fn . In particular, EUn EZn EZ0 for
all n. Our assumption that the sub-MG (Un , Fn ) is U.I. results with Un U a.s.
and in L1 (see Theorem 5.3.12). Further, as we show in part (c) of Proposition 5.4.4,
in this case Un = Zn is U.I. so by the same reasoning, Zn Z a.s. and in L1 .
We thus deduce that EU EZ EZ0 . By definition, U = limn Yn = Y
and Z = limn Yn = Y . Consequently, EY EY EY0 , as claimed.

We complement Theorem 5.4.1 by first strengthening its conclusion and then providing explicit sufficient conditions for the uniform integrability of {Yn }.
Lemma 5.4.3. Suppose {Xn } is adapted to filtration {Fn } and the Fn -stopping
time is such that for any Fn -stopping time the R.V. X is integrable and
E[X ] E[X ]. Then, also E[X |F ] X a.s.
Proof. Fixing A F set = IA + IAc . Note that is also an

Fn -stopping time since for any n,
[
{ n} = (A { n}) (Ac { n})
[
= (A { n}) ((Ac { n}) { n}) Fn
because both A and Ac are in F and { n} Fn (c.f. Definition 5.1.34 of

the -algebra F ). By assumption, X , X , X are integrable and EX EX .
Since X = X IA + X IAc subtracting the finite E[X IAc ] from both sides of this
inequality results with E[X IA ] E[X IA ]. This holds for all A F and with
E[X IA ] = E[ZIA ] for Z = E[X |F ] (by definition of the conditional expectation),
208
we see that E[(Z X )IA ] 0 for all A F . Since both Z and X are measurable
on F (see part (b) of Exercise 5.1.35), it thus follows that a.s. Z X , as
claimed.

Proposition 5.4.4. Suppose {Yn } is integrable and is a stopping time for a
filtration {Fn }. Then, {Yn } is uniformly integrable if any one of the following
conditions hold.
(a) E < and a.s. E[|Yn Yn1 ||Fn1 ] c for some finite, non-random
c.
(b) {Yn I >n } is uniformly integrable and Y I < is integrable.
(c) (Yn , Fn ) is a uniformly integrable sub-MG (or sup-MG).
Proof. (a) Clearly, |Yn | Zn , where
Zn = |Y0 | +
n
X
k=1
|Yk Yk1 | = |Y0 | +
n
X
k=1
|Yk Yk1 |I k ,
is non-decreasing in n. Hence, supn |Yn | Z , implying that {Yn } is U.I.

whenever EZ is finite (c.f. Lemma 1.3.48). Proceeding to show that this is the
case under condition (a), recall that I k mFk1 for all k (since is an Fn stopping time). Thus, taking out what is known, by the tower property we find
that under condition (a),
E[|Yk Yk1 |I k ] = E[E(|Yk Yk1 | |Fk1 )I k ] c P( k)
for all k 1. Summing this bound over k = 1, 2, . . . results with
X
EZ E|Y0 | + c
P( k) = E|Y0 | + c E ,
k=1
with the integrability of Z being a consequence of the hypothesis in condition (a)

that is integrable.
(b) Next note that |Xn | |X |I < + |Xn |I >n for every n, any sequence of
random variables {Xn } and any {0, 1, 2, . . . , }. Condition (b) states that the
sequence {|Yn |I >n } is U.I. and that the variable |Y |I < is integrable. Thus,
taking the expectation of the preceding inequality in case Xn = Yn I|Yn |>M , we find
that when condition (b) holds,
sup E[|Yn |I|Yn |>M ] E[|Y |I|Y |>M I < ] + sup E[|Yn |I|Yn |>M I >n ] ,
n
converges to zero as M . That is, {|Yn |} is then a U.I. sequence.
(c) The hypothesis of (c) that {Yn } is U.I. implies that {Yn I >n } is also U.I. and
that supn E[(Yn )+ ] is finite. With an Fn -stopping time and (Yn , Fn ) a sub-MG,
it further follows by Lemma 5.3.7 that Y I < is integrable. Having arrived at the
hypothesis of part (b), we are done.
Since {Yn } is U.I. whenever is bounded, we have the following immediate

consequences of Doobs optional stopping theorem, Remark 5.4.2 and Lemma 5.4.3.
Corollary 5.4.5. For any sub-MG (Xn , Fn ) and any non-decreasing sequence
{k } of Fn -stopping times, (Xk , Fk , k 0) is a sub-MG when either supk k
a non-random finite integer, or a.s. Xn E[X |Fn ] for an integrable X and all
n 0.
209
Check that by part (b) of Exercise 5.2.16 and part (c) of Proposition 5.4.4 it follows
from Doobs optional stopping theorem that P
ES = 0 for any stopping time with
respect to the canonical filtration of Sn = nk=1 k provided the independent k
are integrable with Ek = 0 and supn E|Sn | < .
Sometimes Doobs optional stopping theorem is applied en-route to a useful contradiction. For example,
Exercise 5.4.6. Show that if {Xn } is a sub-martingale such that EX0 0 and
inf n Xn < 0 a.s. then necessarily E[supn Xn ] = .
Hint: Assuming first that supn |Xn | is integrable, apply Doobs optional stopping
theorem to arrive at a contradiction. Then consider the same argument for the
sub-MG Zn = max{Xn , 1}.
Exercise 5.4.7. Fixing b > 0, let b = min{n 0 : Sn b} for the random walk
{Sn } of Definition 5.1.6 and suppose n = Sn Sn1 are uniformly bounded, of
zero mean and positive variance.
(a) Show that b is almost surely finite.
Hint: See Proposition 5.3.5.
(b) Show that E[min{Sn : n b }] = .
Martingales often provide much information about specific stopping times. We
detail below one such example, pertaining to the srw of Definition 5.1.6.
Corollary 5.4.8 (Gamblers Ruin). Fixing positive integers a and b the probability that a srw {Sn }, starting at S0 = 0, hits a before first hitting +b is
r = (eb 1)/(eb ea ) for = log[(1 p)/p] 6= 0. For the symmetric srw, i.e.
when p = 1/2, this probability is r = b/(a + b).
Remark. The probability r is often called the gamblers ruin, or ruin probability
for a gambler with initial capital of +a, betting on the outcome of independent
rounds of the same game, a unit amount per round, gaining or losing an amount
equal to his bet in each round and stopping when either all his capital is lost (the
ruin event), or his accumulated gains reach the amount +b.
Proof. Consider the stopping time a,b = inf{n 0 : Sn b, or Sn a} for
the canonical filtration of the srw. That is, a,b is the first time that the srw exits
the interval (a, b). Since (Sk + k)/2 has the Binomial(k, p) distribution it is not
hard to check that sup P(Sk = ) 0 hence P(a,b > k) P(a < Sk < b) 0
as k . Consequently, a,b is finite a.s. Further, starting at S0 (a, b) and
using only increments k {1, 1}, necessarily Sa,b {a, b} with probability
one. Our goal is thus to compute the ruin probability r = P(Sa,b = a). To
k
this end, note that Ee

+ (1 p)e = 1 for = log[(1 p)/p]. Thus,
Qn =pe
k
is, for such , a non-negative MG with M0 = 1
Mn = exp(Sn ) = k=1 e
(c.f. Example 5.1.10). Clearly, Mna,b = exp(Sna,b ) exp(|| max(a, b)) is
uniformly bounded (in n), hence uniformly integrable. So, applying Doobs optional
stopping theorem for this MG and stopping time, we have that
1 = EM0 = E[Ma,b ] = E[eSa,b ] = rea + (1 r)eb ,
which easily yields the stated explicit formula for r in case 6= 0 (i.e. p 6= 1/2).
Finally, recall that {Sn } is a martingale for the symmetric srw, with Sna,b uniformly bounded, hence uniformly integrable. So, applying Doobs optional stopping
210
theorem for this MG, we find that in the symmetric case

0 = ES0 = E[Sa,b ] = ar + b(1 r) ,
that is, r = b/(a + b) when p = 1/2.
Here is an interesting consequence of the Gamblers ruin formula.

Example 5.4.9. Initially, at step k = 0 zero is the only occupied site in Z. Then,
at each step a new particle starts at zero and follows a symmetric srw, independently of the previous particles, till it lands on an unoccupied site, whereby it stops
and thereafter occupies this site. The set of occupied sites after k steps is thus
an interval of length k + 1 and we let Rk {1, . . . , k + 1} count the number of
non-negative integers occupied after k steps (starting at R0 = 1).
Clearly, Rk+1 {Rk , Rk + 1} and P(Rk+1 = Rk |FkM ) = Rk /(k + 2) by the
preceding Gamblers ruin formula. Thus, {Rk } follows the evolution of Bernard
Friedmans urn with parameters dk = r = b = 1 and ck = 0. Consequently, by
a.s.
Exercise 5.3.28 we have that (n + 1)1 Rn 1/2.
You are now to derive Walds identities about stopping times for the random
walk, and use them to gain further information about the stopping times a,b of the
preceding corollary.
Exercise 5.4.10. Let be an integrable stopping time for the canonical filtration
of the random walk {Sn }.
(a) Show that if 1 is integrable, then Walds
P identity ES = E1 E holds.
Hint: Use the representation S = k=1 k Ik and independence.
(b) Show that if in addition 1 is square-integrable, then Walds second identity E[(S E1 )2 ] = Var(1 )E holds as well.
Hint: Explain why you may assume that E1 = 0, prove the identity with
n instead of and use Doobs L2 convergence theorem.
(c) Show that if 1 0 then Walds identity applies also when E =
(under the convention that 0 = 0).
Exercise 5.4.11. For the srw Sn and positive integers a, b consider the stopping
time a,b = min{n 0 : Sn
/ (a, b)} as in proof of Corollary 5.4.8.
(a) Check that E[a,b ] < .
Hint: See Exercise 5.1.15.
(b) Combining Corollary 5.4.8 with Walds identities, compute the value of
E[a,b ].
(c) Show that a,b b = min{n 0 : Sn = b} for a (where the
minimum over the empty set is ), and deduce that Eb = b/(2p 1)
when p 1/2.
(d) Show that b is almost surely finite when p 1/2.
(e) Find constants c1 and c2 such that Yn = Sn4 6nSn2 + c1 n2 + c2 n is a
martingale for the symmetric srw, and use it to evaluate E[(b,b )2 ] in
this case.
We next provide a few applications of Doobs optional stopping theorem, starting
with information on the law of b for srw (and certain other random walks).
Exercise 5.4.12. Consider the stopping time b = inf{n 0 : Sn = b} and the
martingale Mn = exp(Sn )M ()n for a srw {Sn }, with b a positive integer and
M () = E[e1 ].
211
(a) Show that if p = 1 q [1/2, 1) then eb E[M ()b ] = 1 for every > 0.
(b) Deduce that for p [1/2, 1) and every 0 < s < 1,
i
p
1 h
E[s1 ] =
1 1 4pqs2 ,
2qs
and E[sb ] = (E[s1 ])b .

(c) Show that if 0 0.
(d) Deduce that for p (0, 1/2) the variable Z = 1 + maxk0 Sk has a Geometric distribution of success probability 1 e .
Exercise 5.4.13. Consider b = min{n 0 : Sn b} for b > 0, in case the i.i.d.
increments n = Sn Sn1 of the random walk {Sn } are such that P(1 > 0) > 0
and {1 |1 > 0} has the Exponential law of parameter .
(a) Show that for any n finite, conditional on {b = n} the law of Sb b is
also Exponential of parameter .
Hint: Recall the memory-less property of the exponential distribution.
(b) With M () = E[e1 ] and 0 denoting the maximal solution of
M () = 1, verify the existence of a monotone decreasing, continuous
function u : (0, 1] 7 [ , ) such that M (u(s)) = 1/s.
(c) Evaluate E[sb Ib < ], 0 < s < 1, and P(b < ) in terms of u(s) and
.
Exercise 5.4.14. A monkey types a random sequence of capital letters {k } that
are chosen independently of each other, with each k chosen uniformly from amongst
the 26 possible values {A, B, . . . , Z}.
(a) Suppose that just before each time step n = 1, 2, , a new gambler
arrives on the scene and bets $1 that n = P . If he loses, he leaves,
whereas if he wins, he receives $26, all of which he bets on the event
n+1 = R. If he now loses, he leaves, whereas if he wins, he bets his
current fortune of $262 on the event that n+2 = O, and so on, through
the word P ROBABILIT Y . Show that the amount of money Mn that the
gamblers have collectively earned by time n is a martingale with respect
to {Fn }.
(b) Let Ln denote the number of occurrences of the word P ROBABILIT Y in
the first n letters typed by the monkey and b = inf{n 11 : Ln = 1} the
first time by which it produced this word. Using Doobs optional stopping
theorem show that Eb
= a for a = 2611 . Does the same apply for the
first time by which the monkey produces the word ABRACADABRA
and if not, what is E ?
a.s.
D
(c) Show that n1 Ln a1 and further that (Ln n/a)/ vn G for some
finite, positive constant v.
Hint: Show that the renewal theory clt of Exercise 3.2.9 applies here.
Exercise 5.4.15. Consider a fair game consisting of successive turns whose outcome are the i.i.d. signs k {1, 1} such that P(1 = 1) = 12 , and where
upon betting
the wagers {Vk } in each turn, your gain (or loss) after n turns is
Pn
Yn =
k=1 k Vk . Here is a betting system {Vk }, predictable with respect to the

canonical filtration {Fn }, as in Example 5.1.30, that surely makes a profit in this
fair game!
212
Choose a finite sequence x1 , x2 , . . . , x of non-random positive numbers. For each

k 1, wager an amount Vk that equals the sum of the first and last terms in your
sequence prior to your k-s turn. Then, to update your sequence, if you just won
your bet delete those two numbers while if you lost it, append their sum as an extra
term x+1 = x1 + x at the right-hand end of the sequence. You play iteratively
according to this rule till your sequence is empty (and if your sequence ever consists
of one term only, you wager that amount, so upon wining you delete this term, while
upon losing you append it to the sequence to obtain two terms).
P
(a) Let v = i=1 xi . Show that the sum of terms in your sequence after n
turns is a martingale Sn = v Yn with respect to {Fn }. Deduce that
with probability one you terminate playing with a profit v at the finite
Fn -stopping time = inf{n 0 : Sn = 0}.
(b) Show that E is finite.
Hint: Consider the number of terms Nn in your sequence after n turns.
(c) Show that the expected value of your aggregate maximal loss till termination, namely EL for L = mink Yk , is infinite (which is why you are
not to attempt this gambling scheme).
In the next exercise you derive a time-reversed version of the L2 maximal inequality
(5.2.4) by an application of Corollary 5.4.5.
Exercise 5.4.16. Associate to any given martingale (Yn , Hn ) the record times
k+1 = min{j 0 : Yj > Yk }, k = 0, 1, . . . starting at 0 = 0.
(a) Fixing m finite, set k = k m and explain why (Yk , Hk ) is a MG.
(b) Deduce that if EYm2 is finite then
m
X
k=1
E[(Yk Yk1 )2 ] = EYm2 EY02 .
Hint: Apply Exercise 5.1.8.

(c) Conclude that for any martingale {Yn } and all m
E[(max Y Ym )2 ] EYm2 .
m
5.5. Reversed MGs, likelihood ratios and branching processes

With martingales applied throughout probability theory, we present here just a
few selected applications. Our first example, Sub-section 5.5.1, deals with the
analysis of extinction probabilities for branching processes. We then study in Subsection 5.5.2 the likelihood ratios for independent experiments with the help of
Kakutanis theorem about product martingales. Finally, in Sub-section 5.5.3 we
develop the theory of reversed martingales and applying it, provide zero-one law
and representation results for exchangeable processes.
5.5.1. Branching processes: extinction probabilities. We use martingales to study the extinction probabilities of branching processes, the object we
define next.
Definition 5.5.1 (Branching process). The branching process is a discrete
time stochastic process {Zn } taking non-negative integer values, such that Z0 = 1
5.5. REVERSED MGS, LIKELIHOOD RATIOS AND BRANCHING PROCESSES
and for any n 1,
213
Zn1
Zn =
(n)
Nj ,
j=1
(n)
Nj
for j = 1, 2, . . . are i.i.d. non-negative integer valued R.V.s with

where N and
finite mean mN = EN < , and where we use the convention that if Zn1 = 0
then also Zn = 0. We call a branching process sub-critical when mN < 1, critical
when mN = 1 and super-critical when mN > 1.
Remark. The S.P. {Zn } is interpreted as counting the size of an evolving popula(n)
tion, with Nj being the number of offspring of j th individual of generation (n 1)
and Zn being the size of the n-th generation. Associated with the branching process
(n)
is the family tree with the root denoting the 0-th generation and having Nj edges
from vertex j at distance n from the root to vertices of distance (n+1) from the root.
Random trees generated in such a fashion are called Galton-Watson trees and are
the subject of much research. We focus here on the simpler S.P. {Zn } and shall use
(k)
throughout the filtration Fn = ({Nj , k n, j = 1, 2, . . .}). We note in passing
that in general FnZ is a strict subset of Fn (since in general one can not recover the
number of offspring of each individual knowing only the total population sizes at
the different generations). Though not dealt with here, more sophisticated related
models have also been successfully studied by probabilists. For example, branching
process with immigration, where one adds to Zn an external random variable In
that count the number of individuals immigrating into the population at the nth
generation; Age-dependent branching process where individuals have random lifetimes during which they produce offspring according to age-dependent probability
generating function; Multi-type branching process where each individual is assigned
a label (type), possibly depending on the type of its parent and with a different law
for the number of offspring in each type, and branching process in random environment where the law of the number of offspring per individual is itself a random
variable (part of the a-apriori given random environment).
Our goal here is to find the probability pex of population extinction, formally
defined as follows.
Definition 5.5.2. The extinction probability of a branching process is
pex := P({ : Zn () = 0 for all n large enough}) .
Obviously, pex = 0 whenever P(N = 0) = 0 and pex = 1 whenever P(N = 0) = 1.
Hereafter we exclude these degenerate cases by assuming that 1 > P(N = 0) > 0.
To this end, we first deduce that with probability one, conditional upon nonextinction the branching process grows unboundedly.
Lemma 5.5.3. If P(N = 0) > 0 then with probability one either Zn or
Zn = 0 for all n large enough.
Proof. We start by proving that for any filtration Fn F and any S.P.
Zn 0 if for A F , some non-random k > 0 and all large positive integers k, n
(5.5.1)
P(A|Fn )I[0,k] (Zn ) k I[0,k] (Zn ) ,
then P(AB) = 1 for B = {limn Zn = }. Indeed, Ck = {Zn k, i.o. in n} are by

(5.5.1) such that Ck {P(A|Fn ) k , i.o. in n}. By Levys 0-1 law P(A|Fn ) IA
214
except on a set D such that P(D) = 0, hence also Ck D {IA k } = D A for

all k. With Ck B c it follows that B c DA yielding our claim that P(AB) = 1.
Turning now to the branching process Zn , let A = { : Zn () = 0 for all n large
(n+1)
= 0, j = 1, . . . , k, then
enough} which is in F , noting that if Zn k and Nj
(n+1)
Zn+1 = 0 hence A. Consequently, by the independence of {Nj

and Fn it follows that
, j = 1, . . .}
E[IA |Fn ]I{Zn k} E[I{Zn+1 =0} |Fn ]I{Zn k} P(N = 0)k I{Zn k}
for all n and k. That is, (5.5.1) holds in this case for k = P(N = 0)k > 0. As
shown already, this implies that with probability one either Zn or Zn = 0 for
all n large enough.

The generating function
L(s) = E[sN ] = P(N = 0) +
(5.5.2)
P(N = k)sk
k=1
plays a key role in analyzing the branching process. In this task, we employ the
following martingales associated with branching process.
Lemma 5.5.4. Suppose 1 > P(N = 0) > 0. Then, (Xn , Fn ) is a martingale where
Xn = mn
N Zn . In the super-critical case we also have the martingale (Mn , Fn ) for
Mn = Zn and (0, 1) the unique solution of s = L(s). The same applies in the
sub-critical case if there exists a solution (1, ) of s = L(s).
(k)
Proof. Since the value of Zn is a non-random function of {Nj , k n, j =

1, 2, . . .}, it follows that both Xn and Mn are Fn -adapted. We proceed to show by
induction on n that the non-negative processes Zn and sZn for each s > 0 such that
L(s) max(s, 1) are integrable with
(5.5.3)
E[sZn+1 |Fn ] = L(s)Zn .
E[Zn+1 |Fn ] = mN Zn ,
(n+1)
of finite mean mN are indeIndeed, recall that the i.i.d. random variables Nj
pendent of Fn on which Zn is measurable. Hence, by linearity of the expectation
it follows that for any A Fn ,
E[Zn+1 IA ] =
X
j=1
(n+1)
E[Nj
I{Zn j} IA ] =
(n+1)
E[Nj
j=1
]E[I{Zn j} IA ] = mN E[Zn IA ] .
This verifies the integrability of Zn 0 as well as the identity E[Zn+1 |Fn ] = mN Zn

of (5.5.3), which amounts to the martingale condition E[Xn+1 |Fn ] = Xn for Xn =
mn
N Zn . Similarly, fixing s > 0,
sZn+1 =
X
=0
I{Zn =}
j=1
(n+1)
sN j

(n+1)
Hence, by linearity of the expectation and independence of sNj

E[sZn+1 IA ] =
X
=0
X
=0
E[I{Zn =} IA
E[I{Zn =} IA ]
j=1
and Fn ,
j=1
(n+1)
E[sNj
(n+1)
sN j
215
]=
X
=0
E[I{Zn =} IA ]L(s) = E[L(s)Zn IA ] .
Since Zn 0 and L(s) max(s, 1) this implies that EsZn+1 1 + EsZn and the
integrability of sZn follows by induction on n. Given that sZn is integrable and the
preceding identity holds for all A Fn , we have thus verified the right identity in
(5.5.3), which in case s = L(s) is precisely the martingale condition for Mn = sZn .
Finally, to prove that s = L(s) has a unique solution in (0, 1) when mN = EN > 1,
note that the function s 7 L(s) of (5.5.2) is continuous and bounded on [0, 1].
Further, since L(1) = 1 and L (1) = EN > 1, it follows that L(s) < s for some
0 < s < 1. With L(0) = P(N = 0) > 0 we have by continuity that s = L(s)
for some s (0, 1). To show the uniqueness of such solution note that EN > 1
P
k(k 1)P(N = k)sk2
implies that P(N = k) > 0 for some k > 1, so L (s) =
k=2
is positive and finite on (0, 1). Consequently, L() is strictly convex there. Hence,
if (0, 1) is such that = L(), then L(s) < s for s (, 1), so such a solution
(0, 1) is unique.
Remark. Since Xn = mn
N Zn is a martingale with X0 = 1, it follows that EZn =
mnN for all n 0. Thus, a sub-critical branching process, i.e. when mN < 1, has
mean total population size
X
X
1
< ,
E[
Zn ] =
mnN =
1 mN
n=0
n=0
which is finite.
We now determine the extinction probabilities for branching processes.

Proposition 5.5.5. Suppose 1 > P(N = 0) > 0. If mN 1 then pex = 1. In
a.s.
a.s.
contrast, if mN > 1 then pex = , with mn
N Zn X and Zn Z {0, }.
Remark. In words, we find that for sub-critical and non-degenerate critical branching processes the population eventually dies off, whereas non-degenerate supercritical branching processes survive forever with positive probability and conditional
upon such survival their population size grows unboundedly in time.
Proof. Applying Doobs martingale convergence theorem to the non-negative
a.s.
MG Xn of Lemma 5.5.4 we have that Xn X with X almost surely finite.
In case mN 1 this implies that Zn = mnN Xn is almost surely bounded (in n),
hence by Lemma 5.5.3 necessarily Zn = 0 for all large n, i.e. pex = 1. In case
a.s.
mN > 1 we have by Doobs martingale convergence theorem that Mn M for
the non-negative MG Mn = Zn of Lemma 5.5.4. Since (0, 1) and Zn 0, it
follows that this MG is bounded by one, hence U.I. and with Z0 = 1 it follows that
a.s.
EM = EM0 = (see Theorem 5.3.12). Recall Lemma 5.5.3 that Zn Z
Z
{0, 1} with
{0, }, so M =
pex = P(Z = 0) = P(M = 1) = EM =
216
as stated.
Remark. For a non-degenerate critical branching process (i.e. when mN = 1

and P(N = 0) > 0), we have seen that the martingale {Zn } converges to 0 with
probability one, while EZn = EZ0 = 1. Consequently, this MG is L1 -bounded but
not U.I. (for another example, see Exercise 5.2.14). Further, as either Zn = 0 or
Zn 1, it follows that in this case 1 = E(Zn |Zn 1)(1 qn ) for qn = P(Zn = 0).
Further, here qn pex = 1 so we deduce that conditional upon non-extinction, the
mean population size E(Zn |Zn 1) = 1/(1 qn ) grows to infinity as n .
As you show next, if super-critical branching process has a square-integrable offspring distribution then mn
N Zn converges in law to a non-degenerate random variable. The Kesten-Stigum L log L-theorem, (which we do not prove here), states
that the latter property holds if and only if E[N log N ] is finite.
Exercise 5.5.6. Consider a super-critical branching process {Zn } where the number of offspring is of mean mN = E[N ] > 1 and variance vN = Var(N ) < .
(a) Compute E[Xn2 ] for Xn = mn
N Zn .
(b) Show that P(X > 0) > 0 for the a.s. limit X of the martingale Xn .
(c) Show that P(X = 0) = and deduce that for a.e. , if the branching
process survives forever, that is Zn () > 0 for all n, then X () > 0.
The generating function L(s) = E[sN ] yields information about the laws of Zn
and that of X of Proposition 5.5.5.
Proposition 5.5.7. Consider the generating functions Ln (s) = E[sZn ] for s
[0, 1] and a branching process {Zn } starting with Z0 = 1. Then, L0 (s) = s and
Ln (s) = L[Ln1 (s)] for n 1 and L() of (5.5.2). Consequently, the generating
b (s) = L[L
b (s1/mN )] which
b (s) = E[sX ] of X is a solution of L
function L
converges to one as s 1.
Remark. In particular, the probability qn = P(Zn = 0) = Ln (0) that the branching process is extinct after n generations is given by the recursion qn = L(qn1 )
for n 1, starting at q0 = 0. Since the continuous function L(s) is above s on
the interval from zero to the smallest positive solution of s = L(s) it follows that
qn is a monotone non-decreasing sequence that converges to this solution, which is
thus the value of pex . This alternative evaluation of pex does not use martingales.
Though implicit here, it instead relies on the Markov property of the branching
process (c.f. Example 6.1.10).
(1)
Proof. Recall that Z1 = N1 and if Z1 = k then the branching process Zn

for n 2 has the same law as the sum of k i.i.d. variables, each having the same
law as Zn1 (with the j th such variable counting the number of individuals in the
nth generation who are descendants of the j th individual of the first generation).
Consequently, E[sZn |Z1 = k] = E[sZn1 ]k for all n 2 and k 0. Summing over
the disjoint events {Z1 = k} we have by the tower property that for n 2,
X
P(N = k)Ln1 (s)k = L[Ln1 (s)]
Ln (s) = E[E(sZn |Z1 )] =
k=0
for L() of (5.5.2), as claimed. Obviously, L0 (s) = s and L1 (s) = E[sN ] = L(s).
b n (s) = L[L
b n1 (s1/mN )] for L
b n (s) = E[sXn ]
From this identity we conclude that L
217
a.s.
and Xn = mn
N Zn . With Xn X we have by bounded convergence that
b n (s) L
b (s) = E[sX ], which by the continuity of r 7 L(r) on [0, 1] is thus a
L
b (s) = L[L
b (s1/mN )]. Further, by monotone convergence
solution of the identity L
b
b
L (s) L (1) = 1 as s 1.

Remark. Of course, qn = P(T n) provides the distribution function of the time
of extinction T = min{k 0 : Zk = 0}. For example, if N has the Bernoulli(p)
distribution for some 0 < p < 1 then T is merely a Geometric(1 p) random
variable, but in general the law of T is more involved.
b () determines the law of X 0 (see Exercise 3.2.40).

The generating function L
For example, as you show next, in the special case where N has the Geometric
distribution, conditioned on non-extinction X is an exponential random variable.
Exercise 5.5.8. Suppose Zn is a branching process with Z0 = 1 and N + 1 having
a Geometric(p) distribution for some 0 1/2, critical if p = 1/2 and super-critical if p < 1/2.
(a) Check that L(s) = p/(1 (1 p)s) and = 1/m. Then verify that
Ln (s) = (pmn (1 s) + (1 p)s p)/((1 p)(1 s)mn + (1 p)s p)
except in the critical case for which Ln (s) = (n (n 1)s)/((n + 1) ns).
b (e ) = + (1 )2 /( + (1 ))
(b) Show that in the super-critical case L
for all 0 and deduce that conditioned on non-extinction X has the
exponential distribution of parameter (1 ).
(c) Show that in the sub-critical case E[sZn |Zn 6= 0] (1 m)s/[1 ms] and
deduce that then the law of Zn conditioned upon non-extinction converges
weakly to a Geometric(1 m) distribution.
(d) Show that in the critical case E[eZn /n |Zn 6= 0] 1/(1+) for all 0
and deduce that then the law of n1 Zn conditioned upon non-extinction
converges weakly to an exponential distribution (of parameter one).
The following exercise demonstrates that martingales are also useful in the study
of Galton-Watson trees.
Exercise 5.5.9. Consider a super-critical branching process Zn such that 1
N for some non-random finite . A vertex of the corresponding Galton-Watson
tree T is called a branch point if it has more than one offspring. For each vertex
v T let C(v) count the number of branch points one encounters when traversing
along a path from the root of the tree to v (possibly counting the root, but not
counting v among these branch points).
(a) Let Tn denote the set of vertices in T of distance n from the root.
Show that for each > 0,
X
eC(v)
Xn := M ()n
vTn
is a martingale when M () = mN e + P(N = 1)(1 e ).

(b) Let Bn = min{C(v) : v Tn }. Show that a.s. lim inf n n1 Bn
where > 0 is non-random (and possibly depends on the offspring
distribution).
218
5.5.2. Product martingales and Radon-Nikodym derivatives. We start

with an explicit characterization of uniform integrability for the product martingale
of Example 5.1.10.
Theorem 5.5.10 (Kakutanis
Theorem). Let M denote the a.s. limit of the
Qn
product martingale Mn = k=1 Yk , with M0 = 1 and independent, integrable
Yk 0
such that EYk = 1 for all k 1. By Jensens inequality, ak = E[ Yk ] is in (0, 1]

for all k 1. The following five statements are then equivalent:
L1
(a) {Mn } is U.I. , (b) Mn M ; (c) EM = 1;

Y
X
(d)
ak > 0 ; (e)
(1 ak ) < ,
k
and if any (every) one of them fails, then M = 0 a.s.

Proof. Statement (a) implies statement (b) because any U.I. martingale converges in L1 (see Theorem 5.3.12). Further, the L1 convergence per statement (b)
implies that EMn EM and since EMn = EM0 = 1 for all n, this results with
EM = 1 as well, which is statement (c).
Qn
Considering the non-negative martingale Nn = k=1 ( Yk /ak ) we next show that
(c) implies (d) by proving the contra-positive. Indeed, by Doobs convergence
a.s.
theorem Nn N with N finite a.s. Hence, if statement (d) fails to hold (that
Qn
Q
a.s.
is, k=1 ak 0), then Mn = Nn2 ( nk=1 ak )2 0. So in this case M = 0 a.s.
and statement (c) also fails to hold.
In contrast, if statement (d) holds then {Nn } is L2 -bounded since for all n,
ENn2 = (
n
Y
k=1
ak )2 EMn
Y
k
ak
2
= c < .
Thus, with Mk Nk2 it follows by the L2 -maximal inequality that for all n,
n

n

E max Mk E max Nk2 4E[Nn2 ] 4c .
k=0
k=0
Hence, Mk 0 are such that supk Mk is integrable and in particular, {Mn } is U.I.
(that is, (a) holds).
Finally, to see why the statements (d) and (e) are equivalent note that upon
applying the Borel Cantelli P
lemmas for independent events An with P(An ) = 1an
the divergence of the series k (1ak ) isQequivalent to P(Acn eventually) = 0, which

for strictly positive ak is equivalent to k ak = 0.
We next consider another martingale that is key to the study of likelihood ratios
in sequential statistics. To this end, let P and Q be two probability
measures on
the same measurable space (, F ) with Pn = PFn and Qn = QFn denoting the
restrictions of P and Q to a filtration Fn F .
Theorem 5.5.11. Suppose Qn Pn for all n, with Mn = dQn /dPn denoting
the corresponding Radon-Nikodym derivatives on (, Fn ). Then,
a.s.
(a) (Mn , Fn ) is a martingale on the probability space (, F , P) where Mn

M as n and M is P-a.s. finite.
(b) If {Mn } is uniformly P-integrable then Q P and dQ/dP = M .
219
(c) More generally, the Lebesgue decomposition of Q to its absolutely continuous and singular parts with respect to P is
(5.5.4)
Q = Qac + Qs = M P + I{M =} Q .
Remark. From the decomposition of (5.5.4) it follows that if Q P then both

Q(M < ) = 1 and P(M ) = 1 while if QP then both Q(M = ) = 1 and
P(M = 0) = 1.
Example 5.5.12. Suppose Fn = (n ) and the countable partitions n = {Ai,n }
F of are nested (that is, for each n the partition n+1 is a refinement of n ). It
is not hard to check directly that
X
Q(Ai,n )
Mn =
IA ,
P(Ai,n ) i,n
{i:P(Ai,n )>0}
is an Fn -sup-MG for (, F , P) and is further an Fn -martingale if Q(Ai,n ) = 0

whenever P(Ai,n ) = 0 (which is precisely the assumption made in Theorem 5.5.11).
We have seen this construction in Exercise 5.3.20, where n are the dyadic partitions of = [0, 1), P is taken to be Lebesgues measure on [0, 1) and Q([s, t)) =
x(t) x(s) is the signed measure associated with the function x().
Proof. (a). By the Radon-Nikodym theorem, Mn mFn is non-negative
and P-integrable (since Pn (Mn ) = Qn () = 1). Further, Q(A) = Qn (A) =
Mn Pn (A) = Mn P(A) for all A Fn . In particular, if k n and A Fk then
(since Fk Fn ),
P(Mn IA ) = Q(A) = P(Mk IA ) ,
so in (, F , P) we have Mk = E[Mn |Fk ] by definition of the conditional expectation. Finally, by Doobs convergence theorem the non-negative MG Mn converges
P-a.s. to M which is P-a.s. finite.
(b). We have seen already that if A Fk then Q(A) = P(Mn IA ) for all n k.
Hence, if {Mn } is further uniformly P-integrable then also P(Mn IA ) P(M IA ),
so taking n we deduce that in this case Q(A) = P(M IA ) for any A k Fk
(and in particular for A = ). Since the probability measures Q and M P then
coincide on the -system k Fk they agree also on the -algebra F generated by
this -system (recall Proposition 1.1.39).
(c). To deal with the general case, where Mn is not necessarily uniformly Pintegrable, consider the probability measure S = (P+Q)/2 and its restrictions Sn =
(Pn + Qn )/2 to Fn . As Pn Sn and Qn Sn there exist Vn = dPn /dSn 0 and
Wn = dQn /dSn 0 such that Vn + Wn = 2. Per part (a) the bounded (Vn , Fn )
and (Wn , Fn ) are martingales on (, F , S), having the S-a.s. finite limits V and
W , respectively. Further, as shown in part (b), V = dP/dS and W = dQ/dS.
Recall that Wn Sn = Qn = Mn Pn = Mn Vn Sn , so S-a.s. Mn Vn = Wn = 2 Vn
for all n. Consequently, S-a.s. Vn > 0 (since Qn Pn ), and Mn = (2 Vn )/Vn .
Considering n we deduce that S-a.s. Mn M = (2 V )/V = W /V ,
possibly infinite, and I{M <} = I{V >0} . Thus,
Q = W S = I{V >0} M V S + I{V =0} W S
= I{M <} M P + I{M =} Q ,
and since M is finite P-a.s. this is precisely the stated Lebesgue decomposition
of Q with respect to P.
220
Combining Theorem 5.5.11 and Kakutanis theorem we next deduce that if the
marginals of one infinite product measure are absolutely continuous with respect to
those of another, then either the former product measure is absolutely continuous
with respect to the latter, or these two measures are mutually singular. This
dichotomy is a key result in the treatment by theoretical statistics of the problem
of hypothesis testing (with independent observables under both the null hypothesis
and the alternative hypothesis).
Proposition 5.5.13. Suppose that P and Q are product measures on (RN , Bc )
which make the coordinates Xn () = n independent with the respective laws Q
1
Xk1 P Xk1 for each k N. Let Yk () = d(Q Xk1 )/d(P
Q Xk )(Xk ()) then
denote the likelihood ratios ofthe marginals. Then, M = k Yk exists a.s. under
Q
both P and Q. If = k P( Yk ) is positive then Q is absolutely continuous with
respect to P with dQ/dP = M , whereas if = 0 then Q is singular with respect
to P such that Q-a.s. M = while P-a.s. M = 0.
Remark 5.5.14. Note that the preceding Yk are identically distributed when both
P and Q
are products of i.i.d. random variables. Hence in this
case > 0 if and
only if P( Y1 ) = 1, which with P(Y1 ) = 1 is equivalent to P[( Y1 1)2 ] = 0, i.e. to
having P-a.s. Y1 = 1. The latter condition implies that P-a.s. M = 1, so Q = P.
We thus deduce that any Q 6= P that are both products of i.i.d. random variables,
are mutually singular, and for n large enough the likelihood test of comparing Mn
to a fixed threshold decides correctly between the two hypothesis regarding the law
of {Xk }, since P-a.s. Mn 0 while Q-a.s. Mn .
Proof. We are in the setting of Theorem 5.5.11 for = RN and the filtration
FnX = (Xk : 1 k n) F X = (Xk , k < ) = Bc
(c.f. Exercise 1.2.14 and the definitionQof Bc preceding Kolmogorovs extension

theorem). Here Mn = dQn /dPn = nk=1 Yk and the mutual independence of
{Xk } imply that Yk m(Xk ) are both mutually P-independent and mutually
Q-independent (c.f. part (b) of Exercise 4.1.8), with P(Yk ) = Q Xk1 (R) = 1
(see Theorem 1.3.61). In the course of proving part (c) of Theorem 5.5.11 we have
shown that Mn M both P-a.s. and Q-a.s. Further, recall part (a) of Theorem
5.5.11 that Mn is a martingale on (, F X , P). From Kakutanis theorem we know
that the product martingale {Mn } is uniformly P-integrable when > 0 (see (d)
implying (a) there), whereas if = 0 then P-a.s. M = 0. By part (b) of Theorem
5.5.11 the uniform P-integrability of Mn results with Q = M P P. In contrast,
when P-a.s. M = 0 we get from the decomposition of part (c) of Theorem 5.5.11
that Qac = 0 and Q = I{M =} Q so in this case Q-a.s. M = and QP.
Here is a concrete application of the preceding proposition.
Exercise 5.5.15. Suppose P and Q are two product probability measures on the
set = {0, 1}N of infinite binary sequences equipped with the product -algebra
generated by its cylinder sets, with pk = P({ : k = 1}) strictly between zero and
one and qk = Q({ : k = 1}) [0, 1].
(a) Deduce from Proposition 5.5.13 that Q is p
absolutely continuous with reP
(1
spect to P if and
only
if
p
q
(1 pk )(1 qk )) is finite.
k
k
k
P
(b) Show that if k |pk qk | is finite then Q is absolutely continuous with
respect to P.
221
(c) Show that ifPpk , qk [, 1 ] for some > 0 and all k, then Q P if
2
and only if P
k (pk qk ) < .P
(d) Show that P
if k qk < and k pk = then QP so in general the
condition k (pk qk )2 < is not sufficient for absolute continuity of
Q with respect to P.
In the spirit of Theorem 5.5.11, as you show next, a positive martingale (Zn , F
n)
induces a collection of probability measures Qn that are equivalent to Pn = PF
n
(i.e. both Qn Pn and Pn Qn ), and satisfy a certain martingale Bayes rule.
In particular, the following discrete time analog of Girsanovs theorem, shows that
such construction can significantly simplify certain computations upon moving from
Pn to Qn .
Exercise 5.5.16. Suppose (Zn , Fn ) is a (strictly) positive MG on (, F , P), normalized so that EZ0 = 1. Let Pn = PFn and consider the equivalent probability
measure Qn on (, Fn ) of Radon-Nikodym derivative dQn /dPn = Zn .

(a) Show that Qk = Qn F for any 0 k n.
k
(b) Fixing 0 k m n and Y L1 (, Fm , P) show that Qn -a.s. (hence
also P-a.s.), EQn [Y |Fk ] = E[Y Zm |Fk ]/Zk .
(c) For Fn = Fn , the canonical filtration of i.i.d. standard normal variables
{k } and any bounded, Fn -predictable Vn , consider the P
measures Qn inn
1
2
duced by
the
exponential
martingale
Z
=
exp(Y
n
n
k=1 Vk ), where
2 P
Pn
m
Yn = k=1 k Vk . Show that X of coordinates Xm = k=1 (k Vk ),
1 m n, is under
Pm Qn a Gaussian random vector whose law is the
same as that of { k=1 k : 1 m n} under P.
Hint: Use characteristic functions.
5.5.3. Reversed martingales and 0-1 laws. Reversed martingales which

we next define, though less common than martingales, are key tools in the proof of
many asymptotics (e.g. 0-1 laws).
Definition 5.5.17. A reversed martingale (in short RMG), is a martingale indexed by non-positive integers. That is, integrable Xn , n 0, adapted to a filtration
Fn , n 0, such that E[Xn+1 |Fn ] = Xn for all n 1. We
T denote by Fn F
a filtration {Fn }n0 and the associated -algebra F = n0 Fn such that the
relation Fk F applies for any k 0.
Remark. One similarly defines reversed subMG-s (and supMG-s), by replacing
E[Xn+1 |Fn ] = Xn for all n 1 with the condition E[Xn+1 |Fn ] Xn , for all
n 1 (or the condition E[Xn+1 |Fn ] Xn , for all n 1, respectively). Since
(Xn+k , Fn+k ), k = 0, . . . , n, is then a MG (or sub-MG, or sup-MG), any result
about subMG-s, sup-MG-s and MG-s that does not involve the limit as n
(such as, Doobs decomposition, maximal and up-crossing inequalities), shall apply
also for reversed subMG-s, reversed supMG-s and RMG-s.
As we see next, RMG-s are the dual of Doobs martingales (with time moving
backwards), hence U.I. and as such each RMG converges both a.s. and in L1 as
n .
Theorem 5.5.18 (L
evys downward theorem). With X0 integrable, (Xn , Fn ),
n 0 is a RMG if and only if Xn = E[X0 |Fn ] for all n 0. Further, E[X0 |Fn ]
E[X0 |F ] almost surely and in L1 when n .
222
Remark. Actually, (Xn , Fn ) is a RMG for Xn = E[Y |Fn ], n 0 and any integrable Y (possibly Y 6 mF0 ). Further, E[Y |Fn ] E[Y |F ] almost surely
and in L1 . This is merely a restatement of Levys downward theorem, since for
X0 = E[Y |F0 ] we have by the tower property that E[Y |Fn ] = E[X0 |Fn ] for any
n 0.
Proof. Suppose (Xn , Fn ) is a RMG. Then, fixing n < 0 and applying Proposition 5.1.20 for the MG (Yk , Gk ) with Yk := Xn+k and Gk := Fn+k , k = 0, . . . , n
(taking there = n > m = 0), we deduce that E[X0 |Fn ] = Xn . Conversely, suppose Xn = E[X0 |Fn ] for X0 integrable and all n 0. Then, Xn L1 (, Fn , P) by
the definition of C.E. and further, with Fn Fn+1 , we have by the tower property
that
Xn = E[X0 |Fn ] = E[E(X0 |Fn+1 )|Fn ] = E[Xn+1 |Fn ] ,
so any such (Xn , Fn ) is a RMG.

Setting hereafter Xn = E[X0 |Fn ], note that for each n 0 and a < b , by
Doobs up-crossing inequality for the MG (Yk , Gk ), k = 0, . . . , n, we have that
E(Un [a, b]) (b a)1 E[(X0 a) ] (where Un [a, b] denotes the number of upcrossings of the interval [a, b] by {Xk (), k = n, . . . , 0}). By monotone convergence
this implies that E(U [a, b]) (b a)1 E[(X0 a) ] is finite (for any a < b).
Repeating the proof of Lemma 5.3.1, now for n , we thus deduce that
a.s.
Xn X as n . Recall Proposition 4.2.33 that {E[X0 |Fn ]} is U.I. hence
L1
by Vitalis convergence theorem also Xn X when n (and in particular

the random variable X is integrable).
We now complete the proof by showing that X = E[X0 |F ]. Indeed, fixing
k 0, since Xn mFk for all n k it follows that X T
= lim supn Xn
is also in mFk . This applies for all k 0, hence X m[ k0 Fk ] = mF .
Further, E[Xn IA ] E[X IA ] for any A F (by the L1 convergence of
Xn to X ), and as A F Fn also E[X0 IA ] = E[Xn IA ] for all n 0.
Thus, E[X IA ] = E[X0 IA ] for all A F , so by the definition of conditional
expectation, X = E[X0 |F ].

Similarly to Levys upward theorem, as you show next, Levys downward theorem
can be extended to accommodate a dominated sequences of random variables and
Lp
if X0 Lp for some p > 1, then Xn X as n (which is the analog of
Doobs Lp martingale convergence).
a.s.
Exercise 5.5.19. Suppose Fn F and Yn Y as n . Show that if

a.s.
supn |Yn | is integrable, then E[Yn |Fn ] E[Y |F ] when n .
Exercise 5.5.20. Suppose (Xn , Fn ) is a RMG. Show that if E|X0 |p is finite and
Lp
p > 1, then Xn X when n
Not all reversed sub-MGs are U.I. but here is an explicit characterization of those
that are.
Exercise 5.5.21. Show that a reversed sub-MG {Xn } is U.I. if and only if inf n EXn
is finite.
Our first application of RMG-s is to provide an alternative proof of the strong law
of large numbers of Theorem 2.3.3, with the added bonus of L1 convergence.
223
Pn
Theorem 5.5.22 (Strong law of large numbers). Suppose Sn = k=1 k
for i.i.d. integrable {k }. Then, n1 Sn E1 a.s. and in L1 when n .
Proof. Let Xm = (m + 1)1 Sm+1 for m 0, and define the corresponding
filtration Fm = (Xk , k m). Recall part (a) of Exercise 4.4.8, that Xn =
E[1 |Xn ] for each n 0. Further, clearly Fn = (Gn , Tn ) for Gn = (Xn ) and
X
T = (r , r > ). With Tn
independent of ((1 ), Gn ), we thus have that
Xn = E[1 |Fn ] for each n 0 (see Proposition 4.2.3). Consequently, (Xn , Fn ) is
a RMG which by Levys downward theorem converges for n both a.s. and
in L1 to the finite valued random variable X = E[1 |F ]. Combining this
and the tower property leads to EX = E1 so it only remains to show that
P(X 6= c) = 0 for some non-random constant c. To this end, note that for any
finite,
m
m
1 X
1 X
X = lim sup
k = lim sup
k .
m m
m m
k=1
k=+1
X
Clearly,
T X mT for any so X is also measurable on the tail -algebra
T = T of the sequence {k }. We complete the proof upon noting that the
-algebra T is P-trivial (by Kolmogorovs 0-1 law and the independence of k ), so
in particular, a.s. X equals a non-random constant (see Proposition 1.2.47).
In this context, you find next that while any RMG Xm is U.I., it is not necessarily
dominated by an integrable variable, and its a.s. convergence may not translate to
conditional expectations E[Xm |H].
Exercise 5.5.23. Consider integrable i.i.d. copies of 1 , having distribution function F1 (x) = 1 x1 (log x)2 for x e and P(1 = e/(e 1)) = 1 e1 , so
E1 = 0. Let H = (An , n 3) for An = {n en/(log n)} and recall Theorem
5.5.22 that for m the U.I. RMG Xm = (m + 1)1 Sm+1 converges a.s. to
zero.
(a) Verify that m1 E[m |H] IAm for all m 3 and deduce that a.s.
lim supm m1 E[m |H] 1.
(b) Conclude that E[Xm |H] does not converge to zero a.s. and supm |Xm |
is not integrable.
In preparation for the Hewitt-Savage 0-1 law and de-Finettis theorem we now
define the exchangeable -algebra and random variables.
Definition 5.5.24 (exchangeable -algebra and random variables). Consider the product measurable space (RN , Bc ) as in Kolmogorovs extension theorem.
Let Em Bc denote the -algebra of events that are invariant under permutations of
the first m coordinates; that is, A Em if ((1) , . . . , (m) , m+1 , . . .) A for any
permutation
of {1, . . . , m} and all (1 , 2 , . . .) A. The exchangeable -algebra
T
E = m Em consists of all events that are invariant under all finite permutations of
coordinates. Similarly, we call an infinite sequence of R.V.s {k }k1 on the same
D
measurable space exchangeable if (1 , . . . , m ) = ((1) , . . . , (m) ) for any m and

any permutation of {1, . . . , m}; that is, the joint law of an exchangeable sequence
is invariant under finite permutations of coordinates.
Our next lemma summarizes the use of RMG-s in this context.
224
Lemma 5.5.25. Suppose k () = k is an exchangeable sequence of random variables on (RN , Bc ). For any bounded Borel function : R 7 R and m let
P
1
Sbm () = (m)
i (i1 , . . . , i ), where i = (i1 , . . . , i ) is an -tuple of distinct in
m!
is the number of such -tuples. Then,
tegers from {1, . . . , m} and (m) = (m)!
(5.5.5)
Sbm () E[(1 , . . . , )|E]
a.s. and in L1 when m .
Proof. Fixing m since the value of Sbm () is invariant under any permutation of the first m coordinates of we have that Sbm () is measurable on Em .
Further, this bounded R.V. is obviously integrable, so
1 X
(5.5.6)
Sbm () = E[Sbm ()|Em ] =
E[(i1 , . . . , i )|Em ] .
(m) i
Fixing any -tuple of distinct integers i1 , . . . , i from {1, . . . , m}, by our exchangeability assumption, the probability measure on (RN , Bc ) is invariant under any permutation of the first m coordinates of such that (ik ) = k for k = 1, . . . , . Consequently, E[(i1 , . . . , i )IA ] = E[(1 , . . . , )IA ] for any A Em , implying that
E[(i1 , . . . , i )|Em ] = E[(1 , . . . , )|Em ]. Since this applies for any -tuple of distinct integers from {1, . . . , m} it follows by (5.5.6) that Sbm () = E[(1 , . . . , )|Em ]
for all m . In conclusion, considering the filtration Fn = En , n 0 for which
F = E, we have in view of the remark following Levys downward theorem that
(Sbn (), En ), n 0 is a RMG and the convergence in (5.5.5) holds a.s. and in
L1 .
Remark. Noting that any sequence of i.i.d. random variables is also exchangeable,
our first application of Lemma 5.5.25 is the following zero-one law.
Theorem 5.5.26 (Hewitt-Savage 0-1 law). The exchangeable -algebra E of a
sequence of i.i.d. random variables k () = k is P-trivial (that is, P(A) {0, 1}
for any A E).
Remark. Given the Hewitt-Savage 0-1 law, we can simplify the proof of Theorem
5.5.22 upon noting that for each m the -algebra Fm is contained in Em+1 , hence
F E must also be P-trivial.
Proof. As the i.i.d. k () = k are exchangeable, from Lemma 5.5.25 we
have that for any bounded Borel : R R, almost surely Sbm () Sb () =
E[(1 , . . . , )|E].
We proceed to show that Sb () = E[(1 , . . . , )]. To this end, fixing a finite
integer r m let
X
1
(i1 , . . . , i )
Sbm,r () =
(m)
{i: i1 >r,...,i >r}
denote the contribution of the -tuples i that do not intersect {1, . . . , r}. Since
there are exactly (m r) such -tuples and is bounded, it follows that
(m r)
c
|Sbm () Sbm,r ()| [1
]k()k
(m)
m
225
for some c = c(r, , ) finite and all m. Consequently, for any r,

Sb () = lim Sbm () = lim Sbm,r () .
m
Further, by the mutual independence of {k } we have that Sbm,r () are independent

of Fr , hence the same applies for their limit Sb (). Applying Lemma 4.2.9 for
X = (1 , . . . , ) we deduce that E[(1 , . . . , )|E] = E[(1 , . . . , )]. Recall
that IG is, for each G F , a bounded Borel function of (1 , . . . , ). Hence,
E[IG |E] = E[IG ]. Thus, by the tower property and taking out what is known,
P(A G) = E[IA E[IG |E]] = E[IA ]E[IG ] for any A E. That is, E and F are
independent for each finite , so by Lemma 1.4.8 we conclude that E is a P-trivial
-algebra, as claimed.

The proof of de Finettis theorem requires the following algebraic identity which
we leave as an exercise for the reader.
Exercise 5.5.27. Fixing bounded Borel functions f : R1 7 R and g : R 7 R, let
hj (x1 , . . . , x ) = f (x1 , . . . , x1 )g(xj ) for j = 1, . . . , . Show that for any sequence
{k } and any m ,
1
Sbm (h ) =
X
m
1
Sbm (hj ) .
Sbm (f )Sbm (g)
m+1
m + 1 j=1
Theorem 5.5.28 (de Finettis theorem). If k () = k is an exchangeable

sequence then conditional on E the random variables k , k 1 are mutually independent and identically distributed.
Remark. For example, if exchangeable {k } are {0, 1}-valued, then by de Finettis
theorem these are i.i.d. Bernoulli variables of parameter p, conditional on E. The
joint (unconditional) law of {k } is thus that of a mixture of i.i.d. Bernoulli(p)
sequences with p a [0, 1]-valued random variable, measurable on E.
Proof. In view of Exercise 5.5.27, upon applying (5.5.5) of Lemma 5.5.25
for the exchangeable sequence {k } and bounded Borel functions f , g and h , we
deduce that
E[f (1 , . . . , 1 )g( )|E] = E[f (1 , . . . , 1 )|E]E[g( )|E] .
By induction on this leads to the identity
E
Y
k=1
Y
gk (k )|E =
E[gk (k )|E]
k=1
for all and bounded Borel gk : R 7 R. Taking gk = IBk for Bk B we have

P[(1 , . . . , ) B1 B |E] =
k=1
P(k Bk |E)
which implies that conditional on E the R.V.-s {k } are mutually independent (see
Proposition 1.4.21). Further, E[g(1 )IA ] = E[g(r )IA ] for any A E, bounded
Borel g(), positive integer r and exchangeable variables k () = k , from which it
follows that conditional on E these R.V.-s are also identically distributed.
226
We conclude this section with exercises detailing further applications of RMG-s

for the study of a certain U -statistics, for solving the ballots problem and in the
context of mixing conditions.
Exercise 5.5.29. Suppose {k } are i.i.d. random variables and h : R2 7 R a
Borel function such that E[|h(1 , 2 )|] < . For each m 2 let
X
1
h(i , j ) .
W2m =
m(m 1)
1i6=jm
Pm
1 Pm
1
2
For example, note that W2m = m1
k=1 (k m
i=1 i ) is of this form,
2
corresponding to h(x, y) = (x y) /2.
(a) Show that Wn = E[h(1 , 2 )|FnW ] for n 0 hence (Wn , FnW ) is a RMG
and determine its almost sure limit as n .
(b) Assuming in addition that v = E[h(1 , 2 )2 ] is finite, find the limit of
E[Wn2 ] as n .
Pk
Exercise 5.5.30 (The ballot problem). Let Sk = i=1 i for i.i.d. integer
valued j 0 and for n 2 consider the event n = {Sj < j for 1 j n}.
(a) Show that Xk = k 1 Sk is a RMG for the filtration Fk = (Sj , j k)
and that = inf{ n : X 1} 1 is a stopping time for it.
(b) Show that In = 1X whenever Sn n, hence P(n |Sn ) = (1Sn /n)+ .
The name ballot problem is attached to Exercise 5.5.30 since for j {0, 2} we
interpret 0s and 2s as n votes for two candidates A and B in a ballot, with n = {A
leads B throughout the counting} and P(n |B gets r votes) = (1 2r/n)+ .
As you find next, the ballot problem yields explicit formulas for the probability
distributions of the stopping times b = inf{n 0 : Sn = b} associated with the
srw {Sn }.
Exercise 5.5.31. Let R = inf{ 1 : S = 0} denote the first visit to zero by the
srw {Sn }. Using a path reversal counting argument followed by the ballot problem,
show that for any positive integers n, b,
b
P(b = n|Sn = b) = P(R > n|Sn = b) =
n
and deduce that for any k 0,
(b + 2k 1)! b+k k
P(b = b + 2k) = b
p q .
k!(k + b)!
Exercise 5.5.32. Show that for any A F and -algebra G F
sup |P(A B) P(A)P(B)| E[|P(A|G) P(A)|] .
BG
Next, deduce that if Gn G as n and G is P-trivial, then

lim
sup
m BGm
|P(A B) P(A)P(B)| = 0 .
CHAPTER 6
Markov chains
The rich theory of Markov processes is the subject of many text books and one can
easily teach a full course on this subject alone. Thus, we limit ourselves here to the
discrete time Markov chains and to their most fundamental properties. Specifically,
in Section 6.1 we provide definitions and examples, and prove the strong Markov
property of such chains. Section 6.2 explores the key concepts of recurrence, transience, invariant and reversible measures, as well as the asymptotic (long time)
behavior for time homogeneous Markov chains of countable state space. These concepts and results are then generalized in Section 6.3 to the class of Harris Markov
chains.
6.1. Canonical construction and the strong Markov property
We start with the definition of a Markov chain.
Definition 6.1.1. Given a filtration {Fn }, an Fn -adapted stochastic process {Xn }
taking values in a measurable space (S, S) is called an Fn -Markov chain with state
space (S, S) if for any A S,
(6.1.1)
P[Xn+1 A|Fn ] = P[Xn+1 A|Xn ]
n,
a.s.
Remark. We call {Xn } a Markov chain in case Fn = (Xk , k n), noting that
if {Xn } is an Fn -Markov chain then it is also a Markov chain. Indeed, FnX =
(Xk , k n) Fn since {Xn } is adapted to {Fn }, so by the tower property we
have that for any Fn -Markov chain, any A S and all n, almost surely,
P[Xn+1 A|FnX ] = E[E[IXn+1 A |Fn ]|FnX ] = E[E[IXn+1 A |Xn ]|FnX ]
= E[IXn+1 A |Xn ] = P[Xn+1 A|Xn ] .
The key object in characterizing an Fn -Markov chain are its transition probabilities, as defined next.
Definition 6.1.2. A set function p : S S 7 [0, 1] is a transition probability if
(a) For each x S, A 7 p(x, A) is a probability measure on (S, S).
(b) For each A S, x 7 p(x, A) is a measurable function on (S, S).
We say that an Fn -Markov chain {Xn } has transition probabilities pn (x, A), if
almost surely P[Xn+1 A|Fn ] = pn (Xn , A) for every n 0 and every A S and
call it a homogeneous Fn -Markov chain if pn (x, A) = p(x, A) for all n, x S and
A S.
With bS mS denoting the collection of all bounded (R, BR )-valued measurable
mappings on (S, S), we next express E[h(Xk+1 )|Fk ] for h bS in terms of the
transition probabilities of the Fn -Markov chain {Xn }.
227
228
6. MARKOV CHAINS
Lemma 6.1.3. If {Xn } is an Fn -Markov chain with state space (S, S) and transition probabilities pn ( , ), then for any h bS and all k 0
(6.1.2)
E[h(Xk+1 )|Fk ] = (pk h)(Xk ) ,

R
where h 7 (pk h) : bS 7 bS and (pk h)(x) = pk (x, dy)h(y) denotes the Lebesgue
integral of h() under the probability measure pk (x, ) per fixed x S.
Proof. Let H bS denote the collection of bounded, measurable R-valued

functions h() for which (pk h)(x) bS and (6.1.2) holds for all k 0. Since pk (, )
are transition probabilities of the chain, IA H for all A S (c.f. Definition
6.1.2). Thus, we complete the proof of the lemma upon checking that H satisfies the
conditions of the (bounded version of the) monotone class theorem (i.e. Theorem
1.2.7). To this end, for a constant h we have that pk h is also constant and evidently
(6.1.2) then holds. Further, with bS a vector space over R, due to the linearity of
both the conditional expectation on the left side of (6.1.2) and the expectation on
its right side, so is H. Next, suppose hm H, hm 0 and hm h bS. Then,
by monotone convergence (pk hm )(x) (pk h)(x) for each x S and all k 0. In
particular, with pk hm bS and pk h bounded by the bound on h, it follows that
pk h bS. Further, by the monotone convergence of conditional expectations and
the boundedness of h(Xk+1 ) also E[hm (Xk+1 )|Fk ] E[h(Xk+1 )|Fk ]. It thus follows
that h H and with all conditions of the monotone class theorem holding for H
and the -system S, we have that bS H, as stated.

Our construction of product measures extends to products of transition probabilities. Indeed, you should check at this point that the proof of Theorem 1.4.19 easily
adapts to yield the following proposition.
Proposition 6.1.4. Given a -finite measure 1 on (X, X) and 2 : X S 7 [0, 1]
such that B 7 2 (x, B) is a probability measure on (S, S) for each fixed x X
and x 7 2 (x, B) is measurable on (X, X) for each fixed B S, there exists a
unique -finite measure on the product space (X S, X S), denoted hereafter by
= 1 2 , such that
Z
(A B) = 1 (dx)2 (x, B), A X, B S .
A
We turn to show how relevant the preceding proposition is for Markov chains.
Proposition 6.1.5. To any -finite measure on (S, S) and any sequence of
transition probabilities pn (, ) there correspond unique -finite measures k =
p0 pk1 on (Sk+1 , S k+1 ), k = 1, 2, . . . such that
Z
Z
Z
k (A0 Ak ) = (dx0 ) p0 (x0 , dx1 ) pk1 (xk1 , dxk )
A0
A1
Ak
for any Ai S, i = 0, . . . , k. If is a probability measure, then k is a consistent

sequence of probability measures (that is, k+1 (A S) = k (A) for any k finite and
A Sk+1 ).
Further, if {Xn } is a Markov chain with state space (S, S), transition probabilities
pn ( , ) and initial distribution (A) = P(X0 A) on (S, S), then for any h bS
6.1. CANONICAL CONSTRUCTION AND THE STRONG MARKOV PROPERTY
and all k 0,
(6.1.3)
E[
k
Y
h (X )] =
=0
(dx0 )h0 (x0 )
229
pk1 (xk1 , dxk )hk (xk ) ,
so in particular, {Xn } has the finite dimensional distributions (f.d.d.)

(6.1.4)
P(X0 A0 , . . . , Xn An ) = p0 pn1 (A0 . . . An ) .
Proof. Starting at a -finite measure 1 = on (S, S) and applying Proposition 6.1.4 for 2 (x, B) = p0 (x, B) on S S yields the -finite measure 1 =
p0 on (S2 , S 2 ). Applying this proposition once more, now with 1 = 1 and
2 ((x0 , x1 ), B) = p1 (x1 , B) for x = (x0 , x1 ) S S yields the -finite measure
2 = p0 p1 on (S3 , S 3 ) and upon repeating this procedure k times we arrive
at the -finite measure k = p0 pk1 on (Sk+1 , S k+1 ). Since pn (x, S) = 1
for all n and x S, it follows that if is a probability measure, so are k which by
construction are also consistent.
Suppose next that the Markov chain {Xn } has transition probabilities pn (, ) and
initial distribution . Fixing k and h bS we have by the tower property and
(6.1.2) that
E[
k
Y
=0
h (X )] = E[
k1
Y
=0
X
h (X )E(hk (Xk )|Fk1
)] = E[
k1
Y
h (X )(pk1 hk )(Xk1 )] .
=0
Further, with pk1 hk bS (see Lemma 6.1.3), alsoRhk1 (pk1 hk ) bS and we

get (6.1.3) by induction on k starting at Eh0 (X0 ) = (dx0 )h0 (x0 ). The formula
(6.1.4) for the f.d.d. is merely the special case of (6.1.3) corresponding to indicator
functions h = IA .

Remark 6.1.6. Using (6.1.1) we deduce from Exercise 4.4.5 that any Fn -Markov
chain with a B-isomorphic state space has transition probabilities. We proceed to
define the law of such a Markov chain and building on Proposition 6.1.5 show that
it is uniquely determined by the initial distribution and transition probabilities of
the chain.
Definition 6.1.7. The law of a Markov chain {Xn } with a B-isomorphic state
space (S, S) and initial distribution is the unique probability measure P on
(S , Sc ) with S = SZ+ , per Corollary 1.4.25, with the specified f.d.d.
P ({s : si Ai , i = 0, . . . , n}) = P(X0 A0 , . . . , Xn An ) ,
for Ai S. We denote by Px the law P in case (A) = IxA (i.e. when X0 = x
is non-random).
Remark. Definition 6.1.7 provides the (joint) law for any stochastic process {Xn }
with a B-isomorphic state space (that is, it applies for any sequence of (S, S)-valued
R.V. on the same probability space).
Here is our canonical construction of Markov chains out of their transition probabilities and initial distributions.
Theorem 6.1.8. If (S, S) is B-isomorphic, then to any collection of transition
probabilities pn : S S 7 [0, 1] and any probability measure on (S, S) there
230
6. MARKOV CHAINS
corresponds a Markov chain Yn (s) = sn on the measurable space (S , Sc ) with

state space (S, S), transition probabilities pn (, ), initial distribution and f.d.d.
(6.1.5)
P ({s : (s0 , . . . , sk ) A}) = p0 pk1 (A) A S k+1 , k < .
Remark. In particular, this construction implies that for any probability measure
on (S, S) and all A Sc
Z
(6.1.6)
P (A) = (dx)Px (A) .
We shall use the latter identity as an alternative definition for P , that is applicable
even for a non-finite initial measure (namely, when (S) = ), noting that if is
-finite then P is also the unique -finite measure on (S , Sc ) for which (6.1.5)
holds (see the remark following Corollary 1.4.25).
Proof. The given transition probabilities pn (, ) and probability measure

on (S, S) determine the consistent probability measures k = p0 pk1
per Proposition 6.1.5 and thereby via Corollary 1.4.25 yield the stochastic process
Yn (s) = sn on (S , Sc ), of law P , state space (S, S) and f.d.d. k . Taking k = 0
in (6.1.5) confirms that its initial distribution is indeed . Further, fixing k 0
finite, let Y = (Y0 , . . . , Yk ) and note that for any A S k+1 and B S
Z
E[I{YA} I{Yk+1 B} ] = k+1 (A B) =
k (dy)pk (yk , B) = E[I{YA} pk (Yk , B)]
A
(where the first and last equalities are due to (6.1.5)). Consequently, for any B S
and k 0 finite, pk (Yk , B) is a version of the C.E. E[I{Yk+1 B} |FkY ] for FkY =
(Y0 , . . . , Yk ), thus showing that {Yn } is a Markov chain of transition probabilities
pn (, ).
Remark. Conversely, given a Markov chain {Xn } of state space (S, S), applying this construction for its transition probabilities and initial distribution yields a
Markov chain {Yn } that has the same law as {Xn }. To see this, recall (6.1.4) that
the f.d.d. of a Markov chain are uniquely determined by its transition probabilities and initial distribution, and further for a B-isomorphic state space, the f.d.d.
uniquely determine the law P of the corresponding stochastic process. For this
reason we consider (S , Sc , P ) to be the canonical probability space for Markov
chains, with Xn () = n given by the coordinate maps.
The evaluation of the f.d.d. of a Markov chain is considerably more explicit when
the state space S is a countable set (in which case S = 2S ), as then
X
pn (x, A) =
pn (x, y) ,
yA
for any
P A S, so the transition probabilities are determined by pn (x, y) 0 such
that yS pn (x, y) = 1 for all n and x S (and all Lebesgue integrals are in this case
merely sums). In particular, if S is a finite set and the chain is homogeneous, then
identifying S with {1, . . . , m} for some m < , we view p(x, y) as the (x, y)-th entry
of an m m dimensional transition probability matrix, and express probabilities of
interest in terms of powers of the latter matrix.
For homogeneous Markov chains whose state space is S = Rd (or a product of
closed intervals thereof), equipped with the corresponding Borel -algebra, computations are relatively explicit when for each x S the transition probability p(x, )
231
is absolutely continuous with respect to (the completion of) Lebesgue measure on

S. Its non-negative Radon-Nikodym derivative p(x, y)R is then called the transition
probability kernel of the chain. In this case (ph)(x) = h(y)p(x, y)dy and the right
side of (6.1.4) amounts to iterated integrations of the kernel p(x, y) with respect to
Lebesgue measure on S.
Here are few homogeneous Markov chains of considerable interest in probability
theory and its applications.
n
P
k , where {k }
Example 6.1.9 (Random walk). The random walk Sn = S0 +
k=1
are i.i.d. Rd -valued random variables that are also independent of S0 is an example
of a homogeneous Markov chain. Indeed, Sn+1 = Sn + n+1 with n+1 independent
of FnS = (S0 , . . . , Sn ). Hence, P[Sn+1 A|FnS ] = P[Sn + n+1 A|Sn ]. With
n+1 having the same law as 1 , we thus get that P[Sn + n+1 A|Sn ] = p(Sn , A)
for the transition probabilities p(x, A) = P(1 {y x : y A}) (c.f. Exercise
4.2.2) and the state space S = Rd (with its Borel -algebra).
Example 6.1.10 (Branching process). Another homogeneous Markov chain

is the branching process {Zn } of Definition 5.5.1 having the countable state space
S
S = {0, 1, 2, . . .} (and
Px the -algebra S = 2 ). The transition probabilities are in this
case p(x, A) = P( j=1 Nj A), for integer x 1 and p(0, A) = 10A .
P
Example 6.1.11 (Renewal Markov chain). Suppose qk 0 and k=1 qk = 1.
Taking S = {0, 1, 2, . . .} (and S = 2S ), a homogeneous Markov chain with transition
probabilities p(0, j) = qj+1 for j 0 and p(i, i 1) = 1 for i 1 is called a renewal
chain.
As you are now to show, in a renewal (Markov) chain {Xn } the value of Xn is
the amount of time from n to the first of the (integer valued) renewal times {Tk }
in [n, ), where m = Tm Tm1 are i.i.d. and P(1 = j) = qj (compare with
Example 2.3.7).
Exercise 6.1.12. Suppose {k } are i.i.d.
P positive integer valued random variables
with P(1 = j) = qj . Let Tm = T0 + m
k=1 k for non-negative integer random
variable T0 which is independent of {k }.
(a) Show that N = inf{k 0 : Tk }, = 0, 1, . . ., are finite stopping
times for the filtration Gn = (T0 , k , k n).
(b) Show that for each fixed non-random , the random variable N +1 is
independent of the stopped -algebra GN and has the same law as 1 .
(c) Let Xn = min{(Tk n)+ : Tk n}. Show that Xn+1 = Xn +Nn +1 IXn =0
1 is a homogeneous Markov chain whose transition probabilities are given
in Example 6.1.11.
Example 6.1.13 (Birth and death chain). A homogeneous Markov chain {Xn }
whose state space is S = {0, 1, 2, . . .} and for which Xn+1 Xn {1, 0, 1} is called
a birth and death chain.
Exercise 6.1.14 (Bayesian estimator). Let and {Uk } be independent
Pn random
variables, each of which is uniformly distributed on (0, 1). Let Sn = k=1 Xk for
Xk = sgn( Uk ). That is, first pick according to the uniform distribution and
then generate a srw Sn with each of its increments being +1 with probability and
1 otherwise.
232
6. MARKOV CHAINS
(a) Compute P(Xn+1 = 1|X1 , . . . , Xn ).

(b) Show that {Sn } is a Markov chain. Is it a homogeneous chain?
Exercise 6.1.15 (First order auto-regressive process). The first order
auto-regressive process {Xk } is defined via Xn = Xn1 + n for n 1, where is
a non-random scalar constant and {k } are i.i.d. Rd -valued random variables that
are independent of X0 .
(a) With Fn = (X0 , k , k n) verify that {Xn } is a homogeneous Fn Markov chain of state space S = Rd (equipped with its Borel -algebra),
and provide its transition probabilities.
(b) Suppose || < 1 and X0 = 0 for non-random scalar , with each k
having the multivariate normal distribution N (0, V) of zero mean and
covariance matrix V. Find the values of for which the law of Xn is
independent of n.
As we see in the sequel, our next result, the strong Markov property, is extremely
useful. It applies to any homogeneous Markov chain with a B-isomorphic state
space and allows us to handle expectations of random variables shifted by any
stopping time with respect to the canonical filtration of the chain.
Proposition 6.1.16 (Strong Markov property). Consider a canonical probability space (S , Sc , P ), a homogeneous Markov chain Xn () = n constructed
on it via Theorem 6.1.8, its canonical filtration FnX = (Xk , k n) and the shift
operator : S 7 S such that ()k = k+1 for all k 0 (with the corresponding
iterates (n )k = k+n for k, n 0). Then, for any {hn } bSc with supn, |hn ()|
finite, and any FnX -stopping time
(6.1.7)
E [h ( ) | FX ]I{ <} = EX [h ] I{ <} .
Remark. Here FX is the stopped -algebra associated with the stopping time
(c.f. Definition 5.1.34) and E (or Ex ) indicates expectation taken with respect
to P (Px , respectively). Both sides of (6.1.7) are set to zero when () =
and otherwise its right hand side is g(n, x) = Ex [hn ] evaluated at n = () and
x = X () ().
The strong Markov property is a significant extension of the Markov property:
(6.1.8)
E [ h(n ) | FnX ] = EXn [h] ,
holding almost surely for any non-negative integer n and fixed h bSc (that is,
the identity (6.1.7) with = n non-random). This in turn generalizes Lemma 6.1.3
where (6.1.8) is proved in the special case of h(1 ) and h bS.
Proof. We first prove (6.1.8) for h() =
k
Q
=0
g ( ) with g bS, = 0, . . . , k.
To this end, fix B S n+1 and recall that m = p p are the f.d.d. for P .
233
Consequently, by (6.1.3) and the definition of n ,

E [h(n )IB (0 , . . . , n )] = n+k [IB (x0 , . . . , xn )
k
Y
g (x+n )]
=0
Z
Z
h
i
= n IB (x0 , . . . , xn )g0 (xn ) p(xn , dy1 )g1 (y1 ) p(yk1 , dyk )gk (yk )
= E [IB (X0 , . . . , Xn )EXn (h) ] .
This holds for all B S n+1 , which by definition of the conditional expectation
amounts to (6.1.8).
The collection H bSc of bounded, measurable h : S R for which (6.1.8)
holds, clearly contains the constant functions and is a vector space over R (by
linearity of the expectation and the conditional expectation). Moreover, by the
monotone convergence theorem for conditional expectations, if hm H are nonnegative and hm h which is bounded, then also h H. Taking in the preceding
g = IB we see that IA H for any A in the -system P of cylinder sets (i.e.
whenever A = { : 0 B0 , . . . , k Bk } for some k finite and B S). We thus
deduce by the (bounded version of the) monotone class theorem that H = bSc , the
collection of all bounded functions on S that are measurable with respect to the
-algebra Sc generated by P.
Having established the Markov property (6.1.8), fixing {hn } bSc and a FnX stopping time , we proceed to prove (6.1.7) by decomposing both sides of the
latter identity according to the value of . Specifically, the bounded random variables Yn = hn (n ) are integrable and applying (6.1.8) for h = hn we have that
E [Yn |FnX ] = g(n, Xn ). Hence, by part (c) of Exercise 5.1.35, for any finite integer
k 0,
E [h ( )I{ =k} |FX ] = g(k, Xk )I{ =k} = g(, X )I{ =k}
The identity (6.1.7) is then established by taking out the FX -measurable indicator
on { = k} and summing over k = 0, 1, . . . (where the finiteness of supn, |hn ()|
provides the required integrability).

Exercise 6.1.17. Modify the
Plast step of the proof of Proposition 6.1.16 to show
that (6.1.7) holds as soon as k EXk [ |hk | ]I{ =k} is P -integrable.
Here are few applications of the Markov and strong Markov properties.
Exercise 6.1.18. Consider a homogeneous Markov

S chain {Xn } with B-isomorphic
state space (S, S). Fixing {Bl } S, let n = l>n {Xl Bl } and = {Xl Bl
i.o.}.
(a) Using the Markov property and Levys upward theorem (Theorem 5.3.15),
a.s.
show that P(n |Xn ) I .
(b) Show that P({Xn An i.o.} \ ) = 0 for any {An } S such that for
some > 0 and all n, with probability one,
P(n |Xn ) I{Xn An } .
(c) Suppose A, B S are such that Px (Xl B for some l 1) for some
> 0 and all x A. Deduce that
P({Xn A finitely often} {Xn B i.o.}) = 1 .
234
6. MARKOV CHAINS
Exercise 6.1.19 (Reflection principle). Consider a symmetric random walk

Pn
D
Sn = k=1 k , that is, {k } are i.i.d. real-valued and such that 1 = 1 . With
n = Sn , use the strong Markov property for the stopping time = inf{k n :
k > b} and hk () = I{nk >b} to show that for any b > 0,
P(max Sk > b) 2P(Sn > b) .
kn
Derive also the following, more precise result for the symmetric srw, where for any
integer b > 0,
P(max Sk b) = 2P(Sn > b) + P(Sn = b) .
kn
The concept of invariant measure for a homogeneous Markov chain, which we now
introduce, plays an important role in our study of such chains throughout Sections
6.2 and 6.3.
Definition 6.1.20. A measure on (S, S) such that (S) > 0 is called a positive
or non-zero measure. An event A Sc is called shift invariant if A = 1 A (i.e.
A = { : () A}), and a positive measure on (S , Sc ) is called shift invariant
if 1 () = () (i.e. (A) = ({ : () A}) for all A Sc ). We say
that a stochastic process {Xn } with a B-isomorphic state space (S, S) is (strictly)
stationary if its joint law is shift invariant. A positive -finite measure on
a B-isomorphic space (S, S) is called an invariant measure for a transition probability p(, ) if it defines via (6.1.6) a shift invariant measure P (). In particular,
starting at X0 chosen according to an invariant probability measure results with
a stationary Markov chain {Xn }.
Lemma 6.1.21. Suppose a -finite measure and transition probability p0 (, ) on
(S, S) are such that p0 (S A) = (A) for any A S. Then, for all k 1 and
A S k+1 ,
p0 pk (S A) = p1 pk (A) .
Proof. Our assumption that ((p0 f )) = (f ) for f = IA and any A S
extends by the monotone class theorem to all f bS. Fixing Ai S and k 1
let fk (x) = IA0 (x)p1 pk (x, A1 Ak ) (where p1 pk (x, ) are the
probability measures of Proposition 6.1.5 in case = x is the probability measure
supported on the singleton {x} and p0 (y, {y}) = 1 for all y S). Since (pj h) bS
for any h bS and j 1 (see Lemma 6.1.3), it follows that fk bS as well.
Further, (fk ) = p1 pk (A) for A = A0 A1 Ak . By the same
reasoning also
Z
Z
p0 (y, dx)p1 pk (x, A1 Ak ) = p0 pk (SA) .
((p0 fk )) = (dy)
S
A0
Thus, the stated identity holds for the -system of product sets A = A0 Ak
which generates S k+1 and since p1 pk (Bn Sk ) = (Bn ) < for some
Bn S, this identity extends to all of S k+1 (see the remark following Proposition
1.1.39).

Remark 6.1.22. Let k = k p denote the -finite measures of Proposition 6.1.5
in case pn (, ) = p(, ) for all n (with 0 = 0 p = ). Specializing Lemma 6.1.21 to
this setting we see that if 1 (SA) = 0 (A) for any A S then k+1 (SA) = k (A)
for all k 0 and A S k+1 .
6.2. MARKOV CHAINS WITH COUNTABLE STATE SPACE
235
Building on the preceding remark we next characterize the invariant measures for
a given transition probability.
Proposition 6.1.23. A positive -finite measure () on B-isomorphic (S, S) is an
invariant measure for transition probability p(, ) if and only if p(S A) = (A)
for all A S.
Proof. With a positive -finite measure, so are the measures P and P
1 on (S , Sc ) which for a B-isomorphic space (S, S) are uniquely determined by
their finite dimensional distributions (see the remark following Corollary 1.4.25).
By (6.1.5) the f.d.d. of P are the -finite measures k (A) = k p(A) for A S k+1
and k = 0, 1, . . . (where 0 = ). By definition of the corresponding f.d.d. of
P 1 are k+1 (S A). Therefore, a positive -finite measure is an invariant
measure for p(, ) if and only if k+1 (S A) = k (A) for any non-negative integer
k and A S k+1 , which by Remark 6.1.22 is equivalent to p(S A) = (A) for
all A S.

6.2. Markov chains with countable state space
Throughout this section we restrict our attention to homogeneous Markov chains
{Xn } on a countable (finite or infinite), state space S, setting as usual S = 2S and
p(x, y) = Px (X1 = y) for the corresponding transition probabilities. Noting that
such chains admit the canonical construction of Theorem 6.1.8 since their state
space is B-isomorphic (c.f. Proposition 1.4.27 for M = S equipped with the metric
d(x, y) = 1x6=y ), we start with a few useful consequences of the Markov and strong
Markov properties that apply for any homogeneous Markov chain on a countable
state space.
Proposition 6.2.1 (Chapman-Kolmogorov). For any x, y S and non-negative
integers k n,
X
(6.2.1)
Px (Xn = y) =
Px (Xk = z)Pz (Xnk = y)
zS
Proof. Using the canonical construction of the chain whereby Xn () = n ,

we combine the tower property with the Markov property for h() = I{nk =y}
followed by a decomposition according to the value z of Xk to get that
n
o
Px (Xn = y) = Ex [h(k )] = Ex Ex h(k ) | FkX
X
= Ex [EXk (h)] =
Px (Xk = z)Ez (h) .
zS
This concludes the proof as Ez (h) = Pz (Xnk = y).
Remark. The Chapman-Kolmogorov equations of (6.2.1) are a concrete special

case of the more general Chapman-Kolmogorov semi-group representation pn =
pk pnk of the n-step transition probabilities pn (x, y) = Px (Xn = y). See [Dyn65]
for more on this representation, which is at the core of the analytic treatment of
general Markov chains and processes (and beyond our scope).
We proceed to derive some results about first hitting times of subsets of the state
space by the Markov chain, where by convention we use A = inf{n 0 : Xn A}
in case the initial state matters and the strictly positive TA = inf{n 1 : Xn A}
when it does not, with y = {y} and Ty = T{y} . To this end, we start with the first
236
6. MARKOV CHAINS
entrance decomposition of {Xn = y} according to the value of Ty (which serves as

an alternative to the Chapman-Kolmogorov decomposition of the same event via
the value in S of Xk ).
Exercise 6.2.2 (First entrance decomposition).
For a homogeneous Markov chain {Xn } on (S, S), let Ty,r = inf{n r : Xn = y}
(so Ty = Ty,1 and y = Ty,0 ).
(a) Show that for any x, y S, B S and positive integers r n,
Px (Xn B, Ty,r n) =
nr
X
k=0
Px (Ty,r = n k)Py (Xk B) .
(b) Deduce that in particular,

n
X
Px (Xn = y) =
Px (Ty,r = k)Py (Xnk = y).
k=r
(c) Conclude that for any y S and non-negative integers r, ,
X
j=0
Py (Xj = y)
+r
X
Py (Xn = y).
n=r
In contrast, here is an application of the last entrance decomposition.

Exercise 6.2.3 (Last entrance decomposition). Show that for a homogeneous Markov chain {Xn } on state space (S, S), all x, y S, B S and n 1,
Px (Xn B, Ty n) =
n1
X
k=0
Px (Xnk = y)Py (Xk B, Ty > k) .
Hint: With Ln = max{1 n : X = y} denoting the last visit of y by the

chain during {1, . . . , n}, observe that {Ty n} is the union of the disjoint events
{Ln = n k}, k = 0, . . . , n 1.
Next, we express certain hitting probabilities for Markov chains in terms of harmonic functions for these chains.
Definition 6.2.4. Extending Definition 5.1.25 we say that f : S 7 R which is
either bounded below or bounded above
P is super-harmonic for the transition probability p(x, y) at x S when f (x) yS p(x, y)f (y). Likewise, f () is sub-harmonic
at x when this inequality is reversed and harmonic at x in case an equality holds.
Such a function is called super-harmonic (or sub-harmonic, harmonic, respectively)
for p(, ) (or for the corresponding chain {Xn }), if it is super-harmonic (or, subharmonic, harmonic, respectively), at all x S. Equivalently, f () which is either
bounded below or bounded above is harmonic provided {f (Xn )} is a martingale
whenever the initial distribution of the chain is such that f (X0 ) is integrable. Similarly, f () bounded below is super-harmonic if {f (Xn )} is a super-martingale whenever f (X0 ) is integrable.
Exercise 6.2.5. Suppose S \ C is finite, inf xC
/ Px (C < ) > 0 and A C,
B = C \ A are both non-empty.
(a) Show that there exist N < and > 0 such that Py (C > kN ) (1)k
for all k 1 and y S.
(b) Show that g(x) = Px (A < B ) is harmonic at every x
/ C.
237
(c) Show that if a bounded function g() is harmonic at every x

/ C then
g(XnC ) is a martingale.
(d) Deduce that g(x) = Px (A < B ) is the only bounded function harmonic
at every x
/ C for which g(x) = 1 when x A and
P g(x) = 0 when x B.
(e) Show that if f : S 7 R+ satisfies f (x) = 1 + yS p(x, y)f (y) at every
x
/ C then Mn := n C + f (XnC ) is a martingale, provided P(X0
C) = 0. Deduce that if in addition f (x) = 0 for x C then f (x) = Ex C
for all x S.
The next exercise demonstrates few of the many interesting explicit formulas one
may find for finite state Markov chains.
Exercise 6.2.6. Throughout, {Xn } is a Markov chain on S = {0, 1, . . . , N } of
transition probability p(x, y).
(a) Use induction to show that in case N = 1, p(0, 1) = and p(1, 0) =
such that + > 0,

+ (1 )n (0)
.
P (Xn = 0) =
+
+
(b) Fixing (0) and 1 6= 0 non-random, suppose = and conditional
on {Xn } the variables Bk are independent Bernoulli(
P Xk ). Evaluate the
mean and variance of the additive functional Sn = nk=1 Bk .
(c) Verify that Ex [(Xn N/2)] = (12/N )n (xN/2) for the Ehrenfest chain
whose transition probabilities are p(x, x 1) = x/N = 1 p(x, x + 1).
6.2.1. Classification of states, recurrence and transience. We start
with the partition of a countable state space of a homogeneous Markov chains
to its intercommunicating (equivalence) classes, as defined next.
Definition 6.2.7. Let xy = Px (Ty < ) denote the probability that starting
at x the chain eventually visits the state y. State y is said to be accessible from
state x 6= y if xy > 0 (or alternatively, we then say that x leads to y). Two
states x 6= y, each accessible to the other, are said to intercommunicate, denoted
by x y. A non-empty collection of states C S is called irreducible if each two
states in C intercommunicate, and closed if there is no y
/ C and x C such that
y is accessible from x.
Remark. Evidently an irreducible set C may be a non-closed set and vice verse.
For example, if p(x, y) > 0 for any x, y S then S\{z} is irreducible and non-closed
(for any z S). More generally, adopting hereafter the convention that x x,
any non-empty proper subset of an irreducible set is irreducible and non-closed.
Conversely, when there exists y S such that p(x, y) = 0 for all x S \ {y}, then S
is closed and reducible. More generally, a closed set that has a closed proper subset
is reducible. Note however the following elementary properties.
Exercise 6.2.8.
(a) Show that if xy > 0 and yz > 0 then also xz > 0.
(b) Deduce that intercommunication is an equivalence relation (that is, x
x, if x y then also y x and if both x y and y z then also
x z).
(c) Explain why its equivalence classes partition S into maximal irreducible
sets such that the directed graph indicating which one leads to each other
238
6. MARKOV CHAINS
is both transitive (i.e. if C1 leads to C2 and C2 leads to C3 then also C1

leads to C3 ), and acyclic (i.e. if C1 leads to C2 then C2 does not lead to
C1 ).
For our study of the qualitative behavior of such chains we further classify each
state as either a transient state, visited only finitely many times by the chain or as
a recurrent state to which the chain returns with certainty (infinitely many times)
once it has been reached by the chain. To this end, we make use of the following
formal definition and key proposition.
Definition 6.2.9. A state y S is called recurrent (or persistent) if yy = 1 and
transient if yy < 1.
Proposition 6.2.10. With Ty0 = 0, let Tyk = inf{n > Tyk1 : Xn = y} for k 1
denote the time of the k-th return to state y S (so Ty1 = Ty > 0 regardless of X0 ).
Then, for any x, y S and k 1,
k1
Px (Tyk < ) = xy yy
.
(6.2.2)
Further, let N (y) denote the number of visits to state y by the Markov chain at
xy
is positive if and only if xy > 0, in which
positive times. Then, Ex N (y) = 1
yy
case it is finite when y is transient and infinite when y is recurrent.
Proof. The identity (6.2.2) is merely the observation that starting at x, in
order to have k visits to y, one has to first reach y andS
then to have k1 consecutive
returns to y. More formally, the event {Ty < } = n {Ty n} is in Sc so fixing
k 2 the strong Markov property applies for the stopping time = Tyk1 and the
indicator function h = I{Ty <} . Further, < implies that h( ) = I{Tyk <} ()
and X = y so EX h = Py (Ty < ) = yy . Combining the tower property with
the strong Markov property we thus find that
Px (Tyk < ) = Ex [h( )I < ] = Ex [Ex [h( ) | FX ]I < ]
= Ex [yy I < ] = yy Px (Tyk1 < ) ,
and (6.2.2) follows by induction on k, starting with the trivial case k = 1.

Next note that if the chain makes at least k visits to state y, then the k-th return
to y occurs at finite time, and vice verse. That is, {Tyk < } = {N (y) k}, and
from the identity (6.2.2), we get that
Ex N (y) =
(6.2.3)
k=1
X
k=1
as claimed.
Px (N (y) k) =
k1
xy yy
=
Px (Tyk < )
k=1
xy
1yy , xy
0,
>0
xy = 0
In the same spirit as the preceding proof you next show that successive returns to
the same state by a Markov chain are renewal times.
Exercise 6.2.11. Fix a recurrent state y S of a Markov chain {Xn }. Let
Rk = Tyk and rk = Rk Rk1 the number of steps between consecutive returns to
y.
239
(a) Deduce from the strong Markov property that under Py the random vectors Y k = (rk , XRk1 , . . . , XRk 1 ) for k = 1, 2, . . . are independent and
identically distributed.
(b) Show that for any probability measure , under P and conditional on the
event {Ty < }, the random vectors Y k are independent of each other
D
and further Y k = Y 2 for all k 2, with Y 2 having then the law of Y 1

under Py .
Here is a direct consequence of Proposition 6.2.10.

Corollary 6.2.12. Each of the following characterizes a recurrent state y:
(a) yy = 1;
(b) Py (Tyk < ) = 1 for all k;
(c) Py (Xn = y, i.o.) = 1;
(d) Py (N (y) = ) = 1;
(e) Ey N (y) = .
Proof. Considering (6.2.2) for x = y we have that (a) implies (b). Given
(b) we have w.p.1. that Xnk = y for infinitely many nk = Tyk , k = 1, 2, . . .,
which is (c). Clearly, the events in (c) and (d) are identical, and evidently (d)
implies (e). To complete the proof simply note that if yy < 1 then by (6.2.3)
Ey N (y) = yy /(1 yy ) is finite.

We are ready for the main result of this section, a decomposition of the recurrent
states to disjoint irreducible closed sets.
Theorem 6.2.13 (Decomposition theorem). A countable state space S of a
homogeneous Markov chain can be partitioned uniquely as
S = T R1 R2 . . .
where T is the set of transient states and the Ri are disjoint, irreducible closed sets
of recurrent states with xy = 1 whenever x, y Ri .
Remark. An alternative statement of the decomposition theorem is that for any
pair of recurrent states xy = yx {0, 1} while xy = 0 if x is recurrent and y
is transient (so x 7 {y S : xy > 0} induces a unique partition of the recurrent
states to irreducible closed sets).
Proof. Suppose x y. Then, xy > 0 implies that Px (XK = y) > 0 for
some finite K and yx > 0 implies that Py (XL = x) > 0 for some finite L. By the
Chapman-Kolmogorov equations we have for any integer n 0,
X
Px (XK+n+L = x) =
Px (XK = z)Pz (Xn = v)Pv (XL = x)
z,vS
(6.2.4)
Px (XK = y)Py (Xn = y)Py (XL = x) .
As Ey N (y) = n=1 Py (Xn = y), summing the preceding inequality over n 1

we find that Ex N (x) cEy N (y) with c = Px (XK = y)Py (XL = x) positive.
If x is a transient state then Ex N (x) is finite (see Corollary 6.2.12), hence the
same applies for y. Reversing the roles of x and y we conclude that any two
intercommunicating states x and y are either both transient or both recurrent.
More generally, an irreducible set of states C is either transient (i.e. every x C
is transient) or recurrent (i.e. every x C is recurrent).
240
6. MARKOV CHAINS
We thus consider the unique partition of S to (disjoint) maximal irreducible equivalence classes of (see Exercise 6.2.8), with Ri denoting those equivalence classes
that are recurrent and proceed to show that if x is recurrent and xy > 0 for y 6= x,
then yx = 1. The latter implies that any y accessible from x R must intercommunicate with x, so with R a maximal irreducible set, necessarily such y is also
in R . We thus conclude that each R is closed, with xy = 1 whenever x, y R ,
as claimed.
To complete the proof fix a state y 6= x that is accessible by the chain from
a recurrent state x, noting that then L = inf{n 1 : Px (Xn = y) > 0} is
finite. Further, because L is the minimal such value there exist y0 = x, yL = y
QL
and yi 6= x for 1 i L such that k=1 p(yk1 , yk ) > 0. Consequently, if
Py (Tx = ) = 1 yx > 0, then
Px (Tx = )
L
Y
k=1
p(yk1 , yk )(1 yx ) > 0 ,
in contradiction of the assumption that x is recurrent.
The decomposition theorem motivates the following definition, as an irreducible

chain is either a recurrent chain or a transient chain.
Definition 6.2.14. A homogeneous Markov chain is called an irreducible Markov
chain (or in short, irreducible), if S is irreducible, a recurrent Markov chain (or in
short, recurrent), if every x S is recurrent and a transient Markov chain (or in
short, transient), if every x S is transient.
By definition once the chain enters a closed set, it remains forever in this set.
Hence, if X0 R we may as well take R to be the whole state space. The case
of X0 T is more involved, for then the chain either remains forever in the set of
transient states, or it lies eventually in the first irreducible set of recurrent states
it entered. As we next show, the first of these possibilities does not occur when T
(or S) is finite (and any irreducible chain of finite state space is recurrent).
Proposition 6.2.15. If F is a finite set of transient states then for any initial
distribution P (Xn F i.o.) = 0. Hence, any finite closed set C contains at least
one recurrent state, and if C is also irreducible then C is recurrent.
P
Proof. Let N (F ) = yF N (y) denote the totality of positive time the
chain spends at a set F . If F is a finite set of transient states then by Proposition
6.2.10 and linearity of the expectation Ex N (F ) is finite, hence Px (N (F ) =
) = 0. With S countable and x arbitrary, it follows that P (N (F ) = ) = 0
for any initial distribution . This is precisely our first claim (as N (F ) is infinite
if and only if Xn F for infinitely many values of n). If C is a closed set then
starting at x C the chain stays in C forever. Thus, Px (N (C) = ) = 1 and to
not contradict our first claim, if such C is finite, then it must contain at least one
recurrent state, which is our second claim. Finally, while proving the decomposition
theorem we showed that if an irreducible set contains a recurrent state then all its
states are recurrent, thus yielding our third and last claim.

We proceed to study the recurrence versus transience of states for some homogeneous Markov chains we have encountered in Section 6.1. To this end, starting
with the branching process we make use of the following definition.
241
Definition 6.2.16. If a singleton {x} is a closed set of a homogeneous Markov

chain, then we call x an absorbing state for the chain. Indeed, once the chain visits
an absorbing state it remains there (so an absorbing state is recurrent).
Example 6.2.17 (Branching Processes). By our definition of the branching
process {Zn } we have that 0 is an absorbing state (as p(0, 0) = 1, hence 0k = 0
for all k 1). If P(N = 0) > 0 then clearly k0 p(k, 0) = P(N = 0)k > 0 and
kk 1 k0 < 1 for all k 1, so all states other than 0 are transient.
Exercise 6.2.18. Suppose a homogeneous Markov chain {Xn } with state space
S = {0, 1, . . . , N } is a martingale for any initial distribution.
(a) Show that 0 and N are absorbing states, that is, p(0, 0) = p(N, N ) = 1.
(b) Show that if also Px ({0,N } < ) > 0 for all x then all other states are
transient and xN = Px (N < 0 ) = x/N .
(c) Check that this applies for the symmetric srw on S (with absorption at
0 and N ), in which case also Ex {0,N } = x(N x).
Example 6.2.19 (Renewal Markov chain). The renewal Markov chain of Example 6.1.11 has p(i, i 1) = 1 for i 1 so evidently i0 = 1 for all i 1 and
hence also 00 = 1, namely 0 is a recurrent state. Recall that p(0, j) = qj+1 , so if
{k : qk > 0} is unbounded, then 0j > 0 for all j so the only closed set containing
0 is S = Z+ . Consequently, in this case the renewal chain is recurrent. If on the
other hand K = sup{k : qk > 0} < then R = {0, 1, . . . , K 1} is an irreducible
closed set of recurrent states and all other states are transient. Indeed, starting at
any positive integer j this chain enters its recurrent class of states after at most j
steps and stays there forever.
Your next exercise pursues another approach to the classification of states, expressing the return probabilities xx in terms of limiting values of certain generating
functions. Applying this approach to the asymmetric srw on the integers provides
us with an example of a transient (irreducible) chain.
Exercise 6.2.20. Given a homogeneous Markov chain of countable state space S
and x S, consider for 1 < s < 1 the generating functions f (s) = Ex [sTx ] and
X
X
k
u(s) =
Ex [sTx ] =
Px (Xn = x)sn .
k0
n0
(a) Show that u(s) = u(s)f (s) + 1.

(b) Show that u(s) 1 + Ex [N (x)] as s 1, while f (s) xx and deduce
that Ex [N (x)] = xx /(1 xx ).
(c) Consider the srw on Z with p(i, i + 1) = p and p(i, i 1) = q = 1 p.
Show that in this case u(s) = (1 4pqs2)1/2 is independent of the initial
state x.
2m m
P
Hint: Recall that (1 t)1/2 = m=0 2m
t for any 0 t < 1.
m 2
(d) Deduce that the srw on Z has xx = 2 min(p, q) for all x so for 0 < p < 1,
p 6= 1/2 this irreducible chain is transient, whereas for p = 1/2 it is
recurrent.
Our next proposition explores a powerful method for proving recurrence of an
irreducible chain by the construction of super-harmonic functions (per Definition
6.2.4).
242
6. MARKOV CHAINS
Proposition 6.2.21. Suppose S is irreducible for a chain {Xn } and there exists
h : S 7 [0, ) of finite level sets Gr = {x : h(x) < r} that is super-harmonic at
S \ Gr for this chain and some finite r. Then, the chain {Xn } is recurrent.
Proof. If S is finite then the chain is recurrent by Proposition 6.2.15. Assuming hereafter that S is infinite, fix r0 large enough so the finite set F = Gr0 is
non-empty and h() is super-harmonic at x
/ F . By Proposition 6.2.15 and part
(c) of Exercise 6.1.18 (for B = F = S \ A), if Px (F < ) = 1 for all x S then F
contains at least one recurrent state, so by irreducibility of S the chain is recurrent,
as claimed. Proceeding to show that Px (F < ) = 1 for all x S, fix r > r0 and
C = Cr = F (S \ Gr ). Note that h() super-harmonic at x
/ C, hence h(XnC )
is a non-negative sup-MG under Px for any x S. Further, S \ C is a subset of Gr
hence a finite set, so it follows by irreducibility of S that Px (C < ) = 1 for all
x S (see part (a) of Exercise 6.2.5). Consequently, from Proposition 5.3.8 we get
that
h(x) Ex h(XC ) r Px (C < F )
(since h(XC ) r when C < F ). Thus,
Px (F < ) Px (F C ) 1 h(x)/r
and taking r we deduce that Px (F < ) = 1 for all x S, as claimed.
Here is a concrete application of Proposition 6.2.21.

Exercise 6.2.22. Suppose {Sn } is an irreducible random walk on Z with zeromean increments {k } such that |k | r for some finite integer r. Show that {Sn }
is a recurrent chain.
The following exercises complement Proposition 6.2.21.
Exercise 6.2.23. Suppose that S is irreducible for some homogeneous Markov
chain. Show that this chain is recurrent if and only if the only non-negative superharmonic functions for it are the constant functions.
Exercise 6.2.24. Suppose {Xn } is an irreducible birth and death chain with
pi = p(i, i + 1), qi = p(i, i 1) and ri = 1 pi qi = p(i, i) 0, where pi and qi
are positive for i > 0, q0 = 0 and p0 > 0. Let
h(m) =
m1
k
XY
k=0
qj
,
p
j=1 j
for m 1 and h(0) = 0.

(a) Check that h() is harmonic for the chain at all positive integers.
(b) Fixing a < x < b in S = Z+ verify that Px (C < ) = 1 for C = {a, b}
and that h(XnC ) is a bounded martingale under Px . Deduce that
Px (Ta < Tb ) =
h(b) h(x)
h(b) h(a)
(c) Considering a = 0 and b show that the chain is transient if and

only if h() is bounded above.
(d) Suppose i(pi /qi 1) c as i . Show that the chain is recurrent if
c < 1 and transient if c > 1, so in particular, when pi = p = 1 qi for
all i > 0 the chain is recurrent if and only if p 21 .
243
6.2.2. Invariant, excessive and reversible measures. Recall Proposition

6.1.23 that an invariant measure for the transition probability p(x, y) is uniquely
determined by a non-zero : S 7 [0, ) such that
X
(6.2.5)
(y) =
(x)p(x, y) ,
y S .
xS
To simplify our notations we thus regard such a function as the corresponding

invariant measure. Similarly,
P we say that : S 7 [0, ) is a finite, positive, or
probability measure, when x (x) is finite (positive, or equals one, respectively),
and call {x : (x) > 0} the support of the measure .
Definition 6.2.25. Relaxing the notion of invariance we say that a non-zero
: S 7 [0, ] is an excessive measure if
X
(y)
(x)p(x, y) ,
y S .
xS
Example 6.2.26. Some chains do not have any invariant measure. For example,
in a birth and death chain with pi = 1, i 0 the identity (6.2.5) is merely (0) = 0
and (i) = (i 1) for i 1, whose only solution is the zero function. However,
the totally asymmetric srw on Z with p(x, x + 1) = 1 at every integer x has an
invariant measure (x) = 1, although just as in the preceding birth and death chain
all its states are transient with the only closed set being the whole state space.
Nevertheless, as we show next, to every recurrent state corresponds an invariant
measure.
Proposition 6.2.27. Let Tz denote the possibly infinite return time to a state z
by a homogeneous Markov chain {Xn }. Then,
z (y) = Ez
z 1
h TX
n=0
i
I{Xn =y} ,
is an excessive measure for {Xn }, the support of which is the closed set of all states
accessible from z. If z is recurrent then z () is an invariant measure, whose support
is the closed and recurrent equivalence class of z.
Remark. We have by the second claim of Proposition 6.2.15 (for the closed set
S), that any chain with a finite state space has at least one recurrent state. Further,
recall that any invariant measure is -finite, which for a finite state space amounts
to being a finite measure. Hence, by Proposition 6.2.27 any chain with a finite state
space has at least one invariant probability measure.
Example 6.2.28. For a transient state z the excessive measure z (y) may be
infinite at some y S. For example, the transition probability p(x, 0) = 1 for all
x S = {0, 1} has 0 as an absorbing (recurrent) state and 1 as a transient state,
with T1 = and 1 (1) = 1 while 1 (0) = .
Proof. Using the canonical construction of the chain, we set
Tz ()1
hk (, y) =
n=0
I{n+k =y} ,
244
6. MARKOV CHAINS
so that z (y) = Ez h0 (, y). By the tower property and the Markov property of the
chain,
Ez h1 (, y) = Ez
=
hX
n=0
XX
xS n=0
XX
xS n=0
I{Tz >n} I{Xn+1 =y}
X
xS
I{Xn =x}
h
i
Ez I{Tz >n} I{Xn =x} Pz (Xn+1 = y|FnX )
h
i
X
z (x)p(x, y) .
Ez I{Tz >n} I{Xn =x} p(x, y) =
xS
The key to the proof is the observation that if 0 = z then h0 (, y) h1 (, y) for

any y S, with equality when y 6= z or Tz () < (in which case Tz () = 0 ).
Consequently, for any state y,
X
z (y) = Ez h0 (, y) Ez h1 (, y) =
z (x)p(x, y) ,
xS
with equality when y 6= z or z is recurrent (in which case Pz (Tz < ) = 1).
By definition z (z) = 1, so z () is an excessive measure.
Iterating the preceding
P
inequality k times we further deduce that z (y) x z (x)Px (Xk = y) for any
k 1 and y S, with equality when z is recurrent. If zy = 0 then clearly
z (y) = 0, while if zy > 0 then Pz (Xk = y) > 0 for some k finite, hence z (y)
z (z)Pz (Xk = y) > 0. The support of z is thus the closed set of states accessible
from z, which for z recurrent is its equivalence class. Finally, note that if x z
then Px (Xk = z) > 0 for some k finite, so 1 = z (z) z (x)Px (Xk = z) implying
that z (x) < . That is, if z is recurrent then z is a -finite, positive invariant
measure, as claimed.

What about uniqueness of the invariant measure for a given transition probability?
By definition the set of invariant measures for p(, ) is a convex cone (that is, if 1
and 2 are invariant measures, possibly the same, then for any positive c1 and c2
the measure c1 1 + c2 2 is also invariant). Thus, hereafter we say that the invariant
measure is unique whenever it is unique up to multiplication by a positive constant.
The first negative result in this direction comes from Proposition 6.2.27. Indeed,
the invariant measures z and x are clearly mutually singular (and in particular,
not constant multiple of each other), whenever the two recurrent states x and z do
not intercommunicate. In contrast, your next exercise yields a positive result, that
the invariant measure supported within each recurrent equivalence class of states
is unique (and given by Proposition 6.2.27).
Exercise 6.2.29. Suppose : S 7 (0, ) is a strictly positive invariant measure
for the transition probability p(, ) of a Markov chain {Xn } on the countable set S.
(a) Verify that q(x, y) = (y)p(y, x)/(x) is a transition probability on S.
(b) Verify that if : S 7 [0, ) is an excessive measure for p(, ) then
h(x) = (x)/(x) is super-harmonic for q(, ).
(c) Show that if p(, ) is irreducible and recurrent, then so is q(, ). Deduce
from Exercise 6.2.23 that then h(x) is a constant function, hence (x) =
c(x) for some c > 0 and all x S.
245
Proposition 6.2.30. If R is a recurrent equivalence class of states then the

invariant measure whose support is contained in R is unique (and has R as its
support). In particular, the invariant measure of an irreducible, recurrent chain is
unique (up to multiplication by a constant) and strictly positive.
Proof. Recall the decomposition theorem that R is closed, hence the restriction of p(, ) to R is also a transition probability and when considering invariant
measures supported within R we may as well take S = R. That is, hereafter we
assume that the chain is recurrent. In this case we have by Proposition 6.2.27 a
strictly positive invariant measure = z on S = R. To complete the proof recall the conclusion of Exercise 6.2.29 that any -finite excessive measure (and in
particular any invariant measure), is then a constant multiple of .

Propositions 6.2.27 and 6.2.30 provide a complete picture of the invariant measures
supported outside the set T of transient states, as the convex cone generated by
the mutually singular, unique invariant measures z () supported on each closed
recurrent equivalence class. Complementing it, your next exercise shows that
an invariant measure must be zero at all transient states that lead to at least one
recurrent state and if it is positive at some v T then it is also positive at any
y T accessible from v.
Exercise 6.2.31. Let () be an invariant measure for a Markov chain {Xk } on
S.
P
(a) Iterating (6.2.5) verify that (y) = x (x)Px (Xk = y) for all k 1
and y S.
(b) Deduce that if (v) > 0 for some v S then (y) > 0 for any y accessible
from v.
(c) Show that if R is a recurrent equivalence class then (x)p(x, y) = 0
for all x
/ R and y R.
Hint: Exercise 6.2.29 may be handy here.
(d) Deduce that if such R is accessible from v 6 R then (v) = 0.
We complete our discussion of (non)-uniqueness of the invariant measure with an
example of a transient chain having two strictly positive invariant measures that
are not constant multiple of each other.
Example 6.2.32 (srw on Z). Consider the srw, a homogeneous Markov chain
with state space Z and transition probability p(x, x + 1) = 1 p(x, x 1) = p for
e
some 0 < p < 1. You can easily verify that both the counting measure (x)
1
and 0 (x) = (p/(1 p))x are invariant measures for this chain, with 0 a constant
e only in the symmetric case p = 1/2. Recall Exercise 6.2.20 that this
multiple of
chain is transient for p 6= 1/2 and recurrent for p = 1/2 and observe that neither
e nor 0 is a finite measure. Indeed, as we show in the sequel, a finite invariant
measure of a Markov chain must be zero at all transient states.
Remark. Evidently, having a uniform (or counting) invariant measure (i.e. (x)
c > 0 for all x S), as in the preceding example,
is equivalent to the transition
P
probability being doubly stochastic, that is, xS p(x, y) = 1 for all y S.
Example 6.2.32 motivates our next subject, which are the conditions under which
a Markov chain is reversible, starting with the relevant definitions.
246
6. MARKOV CHAINS
Definition 6.2.33. A non-zero : S 7 [0, ) is called a reversible measure

for the transition probability p(, ) if the detailed balance relation (x)p(x, y) =
(y)p(y, x) holds for all x, y S. We say that a transition probability p(, ) (or the
corresponding Markov chain) is reversible if it has a reversible measure.
Remark. Every reversible measure is an invariant measure, for summing the
detailed balance relation over x S yields the identity (6.2.5), but there are nonreversible invariant measures. For example, the uniform invariant measure of a
doubly stochastic transition probability p(, ) is non-reversible as soon as p(x, y) 6=
p(y, x) for some x, y S. Indeed, for the asymmetric srw of Example 6.2.32 (i.e.,
e is non-reversible while 0 is a
when p 6= 1/2), the (constant) counting measure
reversible measure (as you can easily check on your own).
As their name suggest, reversible measures have to do with the time reversed chain
(and the corresponding adjoint transition probability), which we now define.
Definition 6.2.34. If () is an invariant measure for transition probability p(x, y),
then q(x, y) = (y)p(y, x)/(x) is a transition probability on the support of (),
which we call the adjoint (or dual) of p(, ) with respect to . The corresponding
chain of law Q is called the time reversed chain (with respect to ).
It is not hard, and left to the reader, to check that for any invariant probability
measure the stationary Markov chains {Yn } of law Q and {Xn } of law P are
D
such that (Yk , . . . , Y ) = (X , . . . , Xk ) for any k finite. Indeed, this is why {Yn }
is called the time reversed chain.
Also note that () is a reversible measure if and only if p(, ) is self-adjoint with
respect to () (that is, q(x, y) = p(x, y) on the support of ()). Alternatively put,
() is a reversible measure if and only if P = Q , that is, the shift invariant law
of the chain induced by is the same as that of its time reversed chain.
By Definition 6.2.33 the set of reversible measures for p(, ) is a convex cone.
The following exercise affirms that reversible measures are zero outside the closed
equivalence classes of the chain and uniquely determined by it within each such
class. It thus reduces the problem of characterizing reversible chains (and measures)
to doing so for irreducible chains.
Exercise 6.2.35. Suppose (x) is a reversible measure for the transition probability p(x, y) of a Markov chain {Xn } with a countable state space S.
(a) Show that (x)Px (Xk = y) = (y)Py (Xk = x) for any x, y S and all
k 1.
(b) Deduce that if (x) > 0 then any y accessible from x must intercommunicate with x.
(c) Conclude that the support of () is a disjoint union of closed equivalence classes, within each of which the measure is uniquely determined
by p(, ) up to a non-negative constant multiple.
We proceed to characterize reversible irreducible Markov chains as random walks
on networks.
Definition 6.2.36. A network (or weighted graph) consists of a countable (finite
or infinite) set of vertices V with a symmetric weight function w : V PV 7 [0, )
(i.e. wxy = wyx for all x, y V). Further requiring that (x) =
yV wxy is
247
finite and positive for each x V, a random walk on the network is a homogeneous
Markov chain of state space V and transition probability p(x, y) = wxy /(x). That
is, when at state x the probability of the chain moving to state y is proportional to
the weight wxy of the pair {x, y}.
Remark. For example, an undirected graph is merely a network the weights wxy
of which are either one (indicating an edge in the graph whose ends are x and y)
or zero (no such edge). Assuming such graph has positive and finite degrees, the
random walker moves at each time step to a vertex chosen uniformly at random
from those adjacent in the graph to its current position.
Exercise 6.2.37. Check P
that a random walk on a network has a strictly positive
reversible measure (x) = y wxy and that a Markov chain is reversible if and only
if there exists an irreducible closed set V on which it is a random walk (with weights
wxy = (x)p(x, y)).
Example 6.2.38 (Birth and death chain). We leave for the reader to check
that the irreducible birth and death chain of Exercise 6.2.24 is a random walk on
the network Z+ with weights wx,x+1 = px (x) = qx+1 (x + 1), wxx
rx (x) and
Q=
x
wxy = 0 for |x y| > 1, and the unique reversible measure (x) = i=1 pi1
qi (with
(0) = 1).
Remark. Though irreducibility does not imply uniqueness of the invariant measure (c.f. Example 6.2.32), if is an invariant measure of the preceding birth and
death chain then (x + 1) is determined by (6.2.5) from (x) and (x 1), so
starting at (0) = 1 we conclude that the reversible measure of Example 6.2.38 is
also the unique invariant measure for this chain.
We conclude our discussion of reversible measures with an explicit condition for
reversibility of an irreducible chain, whose proof is left for the reader (for example,
see [Dur10, Theorem 6.5.1]).
Exercise 6.2.39 (Kolmogorovs cycle condition). Show that an irreducible
chain of transition probability p(x, y) is reversible if and only if p(x, y) > 0 whenever
p(y, x) > 0 and
k
k
Y
Y
p(xi , xi1 ) ,
p(xi1 , xi ) =
i=1
i=1
for any k 3 and any cycle x0 , x1 , . . . , xk = x0 .
Remark. The renewal Markov chain of Example 6.1.11 is one of the many recurrent chains that fail to satisfy Kolmogorovs condition (and thus are not reversible).
Turning to investigate the existence and support of finite invariant measures (or
equivalently, that of invariant probability measures), we further partition the recurrent states of the chain according to the integrability (or lack thereof) of the
corresponding return times.
Definition 6.2.40. With Tz denoting the first return time to state z, a recurrent
state z is called positive recurrent if Ez (Tz ) < and null recurrent otherwise.
Indeed, invariant probability measures require the existence of positive recurrent
states, on which they are supported.
248
6. MARKOV CHAINS
Proposition 6.2.41. If () is an invariant probability measure then all states z

with (z) > 0 are positive recurrent. Further, if the support of () is an irreducible
set R of positive recurrent states then (z) = 1/Ez (Tz ) for all z R.
Proof. Recall PropositionP6.2.10 that for any initial probability measure ()
the number of visits N (z) = n1 IXn =z to a state z by the chain is such that
n=1
P (Xn = z) = E N (z) =
X
xS
(x)Ex N (z) =
X
xS
(x)
1
xz
1 zz
1 zz
(since xz 1 for all x). Starting at X0 chosen according to an invariant probability measure () results with a stationary Markov chain {Xn } and in particular
P (Xn = z) = (z) for all n. The left side of the preceding inequality is thus
infinite for positive (z) and invariant probability measure (). Consequently, in
this case zz = 1, or equivalently z must be a recurrent state of the chain. Since
this applies for any z S we conclude that () is supported outside the set T of
transient states.
Next, recall that for any z S,
z (S) =
X
yS
z (y) = Ez
z 1
h X TX
yS n=0
i
I{Xn =y} = Ez Tz ,
so z is a finite measure if and only if z is a positive recurrent state of the chain.

If the support of () is an irreducible equivalence class R then we deduce from
Propositions 6.2.27 and 6.2.30 that z is a finite measure and (z) = z (z)/z (S) =
1/Ez Tz for any z R. Consequently, R must be a positive recurrent equivalence
class, that is, all states of R are positive recurrent.
To complete the proof, note that by the decomposition theorem any invariant
probability measure () is a mixture of such invariant probability measures, each
supported on a different closed recurrent class Ri , which by the preceding argument
must all be positive recurrent.

In the course of proving Proposition 6.2.41 we have shown that positive and null
recurrence are equivalence class properties. That is, an irreducible set of states
C is either positive recurrent (i.e. every z C is positive recurrent), null recurrent
(i.e. every z C is null recurrent), or transient. Further, recall the discussion
after Proposition 6.2.27, that any chain with a finite state space has an invariant
probability measure, from which we get the following corollary.
Corollary 6.2.42. For an irreducible Markov chain the existence of an invariant
probability measure is equivalent to the existence of a positive recurrent state, in
which case every state is positive recurrent. We call such a chain positive recurrent
and note that any irreducible chain with a finite state space is positive recurrent.
For the remainder of this section we consider the existence and non-existence of
invariant probability measures for some Markov chains of interest.
Example 6.2.43. Since the invariant measure of a recurrent chain is unique up to
a constant multiple (see Proposition 6.2.30) and a transient chain has no invariant
probability measure (seeP
Corollary 6.2.42), if an irreducible chain has an invariant
measure () for which x (x) = then it has no invariant probability measure.
249
e is an invariant measure for the (irreFor example, since the counting measure
ducible) srw of Example 6.2.32, this chain does not have an invariant probability
measure, regardless of the value of p. For the same reason, the symmetric srw on
Z (i.e. where p = 1/2), is a null recurrent chain.
Similarly, the irreducible birth and death chain of Exercise 6.2.24
Qxhas an invariant
probability measure if and only if its reversible measure (x) = i=1 pi1
qi is finite
(c.f. Example 6.2.38). In particular, if pj = 1 qj = p for all j 1 then this chain
is positive recurrent with an invariant probability measure when p < 1/2 but null
recurrent for p = 1/2 (and transient when 1 > p > 1/2).
Finally, a random walk on a graph is irreducible if and only if the graph is connected. With (v) 1 for all v V (see Definition 6.2.36), it is positive recurrent
only for finite graphs.
P
Exercise 6.2.44. Check that (j) =
k>j qk is an invariant measure for the
recurrent renewal Markov chain of Example 6.1.11 in case {k : qk > 0} is unbounded
(see
P Example 6.2.19). Conclude that this chain is positive recurrent if and only if
k kqk is finite.
In the next exercise you find how the invariant probability measure is modified by
the introduction of holding times.
Exercise 6.2.45. Let () be the unique invariant probability measure of an irreducible, positive recurrent Markov chain {Xn } with transition probability p(x, y)
such that p(x, x) = 0 for all x S. Fixing r(x) (0, 1), consider the Markov chain
{Yn } whose transition probability is q(x, x) = 1 r(x) and q(x, y) = r(x)p(x, y) for
all y 6= x. Show that {Yn } is an irreducible, recurrent chain of invariant measure
(x) = (x)/r(x) and deduce that {Yn } is further positive recurrent if and only if
P
x (x)/r(x) < .
Though we have established the next result in a more general setting, the proof
we outline here is elegant, self-contained and instructive.
Exercise 6.2.46. Suppose g() is a strictly concave bounded function on [0, )
and () is a strictly positive invariant probability measure
Pfor irreducible transition
probability p(x, y). For any : S 7 [0, ) let (p)(y) = xS (x)p(x, y) and
X (y)
(y) .
E() =
g
(y)
yS
(a) Show that E(p) E().

(b) Assuming p(x, y) > 0 for all x, y S deduce from part (a) that any
invariant measure () for p(x, y) is a constant multiple of ().
(c) Extend this conclusion to any irreducible p(x, y) by checking that
pb(x, y) =
n=1
2n Px (Xn = y) > 0 ,
x, y S ,
and that invariant measures for p(x, y) are also invariant for pb(x, y).
Here is an introduction to the powerful method of Lyapunov (or energy) functions.

Exercise 6.2.47. Let z = inf{n 0 : Zn = z} and FnZ = (Zk , k n), for
Markov chain {Zn } of transition probabilities p(x, y) on a countable state space S.
250
6. MARKOV CHAINS
(a) Show that Vn = Znz is a FnZ -Markov chain and compute its transition
probabilities q(x, y).
(b) Suppose
h : S 7 [0, ) is such that h(z) = 0, the function (ph)(x) =
P
p(x,
y)h(y)
is finite everywhere and h(x) (ph)(x) + for some
y
> 0 and all x 6= z. Show that (Wn , FnZ ) is a sup-MG under Px for
Wn = h(Vn ) + (n z ) and any x S.
(c) Deduce that Ex z h(x)/ for any x S and conclude that z is positive
recurrent in the stronger sense that Ex Tz is finite for all x S.
(d) Fixing > 0 consider i.i.d. random vectors vk = (k , k ) such that
P(v1 = (1, 0)) = P(v1 = (0, 1)) = 0.25 and P(v1 = (1, 0)) =
P(v1 = (0, 1)) = 0.25 + . The chain Zn = (Xn , Yn ) on Z2 is such
that Xn+1 = Xn + sgn(Xn )n+1 and Yn+1 = Yn + sgn(Yn )n+1 , where
sgn(0) = 0. Prove that (0, 0) is positive recurrent in the sense of part (c).
Exercise 6.2.48. Consider the Markov chain Zn = n + (Zn1 1)+ , n 1,
on S = {0, 1, 2, . . .}, where n are i.i.d. S-valued such that P(1 > 1) > 0 and
E1 = 1 for some > 0.
(a) Show that {Zn } is positive recurrent.
(b) Find its invariant probability measure () in case P(1 = k) = p(1 p)k ,
k S, for some p (1/2, 1).
(c) Is this Markov chain reversible?
6.2.3. Aperiodicity and limit theorems. Building on our classification of
states and study of the invariant measures of homogeneous Markov chains with
countable state space S, we focus here on the large n asymptotics of the state
Xn () of the chain and its law.
We start with the asymptotic behavior of the occupation time
n
X
Nn (y) =
IX =y ,
=1
of state y by the Markov chain during its first n steps.
Proposition 6.2.49. For any probability measure on S and all y S,

1
I{T <}
P -a.s.
(6.2.6)
lim n1 Nn (y) =
n
Ey (Ty ) y
Remark. This special case of the strong law of large numbers for Markov additive
functionals (see Exercise 6.2.62 for its generalization), tells us that if a Markov chain
visits a positive recurrent state then it asymptotically occupies it for a positive
fraction of time, while the fraction of time it occupies each null recurrent or transient
state is zero (hence the reason for the name null recurrent ).
Proof. First note that if y is transient then Ex N (y) is finite by (6.2.3) for
any x S. Hence, P -a.s. N (y) is finite and consequently n1 Nn (y) 0 as
n . Furthermore, since Py (Ty = ) = 1 yy > 0, in this case Ey (Ty ) =
and (6.2.6) follows.
Turning to consider recurrent y S, note that if Ty () = then Nn (y)() = 0
for all n and (6.2.6) trivially holds. Thus, assuming hereafter that Ty () < ,
we have by recurrence of y that a.s. Tyk () < for all k (see Corollary 6.2.12).
Recall part (b) of Exercise 6.2.11, that under P and conditional on {Ty < },
the positive, finite random variables k = Tyk Tyk1 are independent of each other,
251
with {k , k 2} further identically distributed and of mean value Ey (Ty ). Since

Nn (y) = sup{k 0 : Tyk n}, as you have showed in part (b) of Exercise 2.3.8,
a.s.
it follows from the strong law of large numbers that n1 Nn (y) 1/Ey (Ty ) for
n . This completes the proof, as by assumption I{Ty <} = 1 in the present
case.

Here is a direct application of Proposition 6.2.49.
Exercise 6.2.50. Consider the positions {Xn } of a particle starting at X0 = x S
and moving in S = {0, . . . , r} according to the following rules. From any position
1 y r 1 the particle moves to y 1 or y + 1, and each such move is made
with probability 1/2 independently of all other moves, whereas from positions 0 and
r the particle moves in one step to position k S.
(a) Fixing y S and k {1, . . . , r 1} find the almost sure limit (k, y) of
n1 Nn (y) as n .
(b) Find the almost sure limit (y) of n1 Nn (y) in case upon reaching either
0 or r the particle next moves to an independently and uniformly chosen
position K {1, . . . , r 1}.
Your next task is to prove the following ratio limit theorem for the occupation
times Nn (y) within each irreducible, closed recurrent set of states. In particular,
it refines the limited information provided by Proposition 6.2.49 in case y is a null
recurrent state.
Exercise 6.2.51. Suppose y S is a recurrent state for the chain {Xn }. Let y ()
denote the invariant measure of the chain per Proposition 6.2.27, whose support is
the closed and recurrent equivalence class Ry of y. Decomposing the path {X }
at the successive return times Tyk show that for any x, w Ry ,
lim
Nn (w)
= y (w),
Nn (y)
Px -a.s.
Hint: Use Exercise 6.2.11 and the monotonicity of n 7 Nn (w).

Proceeding to study the asymptotics of Px (Xn = y) we start with the following
consequence of Proposition 6.2.49.
Corollary 6.2.52. For all x, y S,
n
1X
xy
(6.2.7)
lim
.
Px (X = y) =
n n
Ey (Ty )
=1
Further, for any transient state y T

(6.2.8)
lim Px (Xn = y) =
xy
.
Ey (Ty )
Proof. Since supn n Nn (y) 1, the convergence in (6.2.7) follows from

Proposition 6.2.49 by bounded convergence (i.e. Corollary 1.3.46).
For a transient state y the sequence Px (Xn = y) is summable (to the finite value
Ex N (y), c.f. Proposition 6.2.10), hence Px (Xn = y) 0 as n . Further,
this amounts to (6.2.8) as in this case Ey (Ty ) = .

Corollary 6.2.52 tells us that for every Markov chain the Ces`aro averages of
Px (Xn = y) converge. In contrast, our next example shows that even for an
252
6. MARKOV CHAINS
irreducible chain of finite state space the sequence n 7 Px (Xn = y) may fail to
converge pointwise.
Example 6.2.53. Consider the Markov chain {Xn } on state space S = {0, 1}
with transition probabilities p(x, y) = 1x6=y . Then, Px (Xn = y) = 1{n even} when
x = y and Px (Xn = y) = 1{n odd} when x 6= y, so the sequence n 7 Px (Xn = y)
alternates between zero and one, having no limit for any fixed (x, y) S2 .
Nevertheless, as we prove in the sequel (more precisely, in Theorem 6.2.59), periodicity of the state y is the only reason for such non-convergence of Px (Xn = y).
Definition 6.2.54. The period dx of a state x S of a Markov chain {Xn } is
the greatest common divisor (g.c.d.) of the set Ix = {n 1 : Px (Xn = x) > 0},
with dx = 0 in case Ix is empty. Similarly, we say that the chain is of period d if
dx = d for all x S. A state x is called aperiodic if dx 1 and a Markov chain is
called aperiodic if every x S is aperiodic.
As the first step in this program, we show that the period is constant on each
irreducible set.
Lemma 6.2.55. The set Ix contains all large enough integer multiples of dx and
if x y then dx = dy .
Proof. Considering (6.2.4) for x = y and L = 0 we find that Ix is closed
under addition. Hence, this set contains all large enough integer multiples of dx
because every non-empty set I of positive integers which is closed under addition
must contain all large enough integer multiples of its g.c.d. d. Indeed, it suffices
to prove this fact when d = 1 since the general case then follows upon considering
the non-empty set I = {n 1 : nd I} whose g.c.d. is one (and which is
also closed under addition). Further, note that any integer n 2 is of the form
n = 2 + k + r = r( + 1) + ( r + k) for some k 0 and 0 r < . Hence, if
two consecutive integers and + 1 are in I then so are all integers n 2 . We
thus complete the proof by showing that K = inf{m : m, I, m > > 0} > 1
is in contradiction with I having g.c.d. d = 1. Indeed, both m0 and m0 + K are
in I for some positive integer m0 and if d = 1 then I must contain also a positive
integer of the form m1 = sK + r for some 0 < r < K and s 0. With I closed
under addition, (s + 1)(m0 + K) > (s + 1)m0 + m1 must then both be in I but
their difference is (s + 1)K m1 = K r < K, in contradiction with the definition
of K.
If x y then in view of the inequality (6.2.4) there exist finite K and L such that
K + n + L Ix whenever n Iy . Moreover, K + L Ix so every n Iy must
also be an integer multiple of dx . Consequently, dx is a common divisor of Iy and
therefore dy , being the greatest common divisor of Iy , is an integer multiple of dx .
Reversing the roles of x and y we likewise have that dx is an integer multiple of dy
from which we conclude that in this case dx = dy .

The key for determining the asymptotics of Px (Xn = y) is to handle this question
for aperiodic irreducible chains, to which end the next lemma is most useful.
Lemma 6.2.56. Consider two independent copies {Xn } and {Yn } of an aperiodic,
irreducible chain on a countable state space S with transition probabilities p(, ). The
Markov chain Zn = (Xn , Yn ) on S2 of transition probabilities p2 ((x , y ), (x, y)) =
253
p(x , x)p(y , y) is then also aperiodic and irreducible. If {Xn } has invariant probability measure () then {Zn } is further positive recurrent and has the invariant
probability measure 2 (x, y) = (x)(y).
Remark. Example 6.2.53 shows that for periodic p(, ) the chain of transition
probabilities p2 (, ) may not be irreducible.
Proof. Fix states z = (x , y ) S2 and z = (x, y) S2 . Since p(, ) are the

transition probabilities of an irreducible chain, there exist K and L finite such that
Px (XK = x) > 0 and Py (YL = y) > 0. Further, by the aperiodicity of this chain
we have from Lemma 6.2.55 that both Px (Xn+L = x) > 0 and Py (YK+n = y ) > 0
for all n large enough, in which case from (6.2.4) we deduce that Pz (ZK+n+L =
z) > 0 as well. As this applies for any z , z S2 , the chain {Zn } is irreducible.
Further, considering z = z we see that Iz contains all large enough integers, hence
{Zn } is also aperiodic. Finally, it is easy to verify that if () is an invariant
probability measure for p(, ) then 2 (x, y) = (x)(y) is an invariant probability
measure for p2 (, ), whose existence implies positive recurrence of the chain {Zn }
(see Corollary 6.2.42).

The following Markovian coupling complements Lemma 6.2.56.
Theorem 6.2.57. Let {Xn } and {Yn } be two independent copies of an aperiodic,
irreducible Markov chain. Suppose further that the irreducible chain Zn = (Xn , Yn )
is recurrent. Then, regardless of the initial distribution of (X0 , Y0 ), the first meeting
time = min{ 0 : X = Y } of the two processes is a.s. finite and for any n,
(6.2.9)
kPXn PYn ktv 2P( > n) ,
where k ktv denotes the total variation norm of Definition 3.2.22.
Proof. Recall Lemma 6.2.56 that the Markov chain Zn = (Xn , Yn ) on S2 is

irreducible. We have further assumed that {Zn } is recurrent, hence z = min{
0 : Z = z} is a.s. finite (for any z S S), regardless of the initial measure of
Z0 = (X0 , Y0 ). Consequently,
= inf{z : z = (x, x) for some x S}
is also a.s. finite, as claimed.

Turning to prove the inequality (6.2.9), fixing g bS bounded by one, recall
that the chains {Xn } and {Yn } have the same transition probabilities and further
X = Y . Thus, for any k n,
I{ =k} EXk [g(Xnk )] = I{ =k} EYk [g(Ynk )] .
By the Markov property and taking out the known I{ =k} it thus follows that
E[I{ =k} g(Xn )] = E(I{ =k} EXk [g(Xnk )])
= E(I{ =k} EYk [g(Ynk )]) = E[I{ =k} g(Yn )] .
Summing over 0 k n we deduce that E[I{ n} g(Xn )] = E[I{ n} g(Yn )] and

hence
Eg(Xn ) Eg(Yn ) = E[I{ >n} g(Xn )] E[I{ >n} g(Yn )]
= E[I{ >n} (g(Xn ) g(Yn ))] .
Since |g(Xn ) g(Yn )| 2, we conclude that |Eg(Xn ) Eg(Yn )| 2P( > n) for
any g bS bounded by one, which is precisely what is claimed in (6.2.9).
254
6. MARKOV CHAINS
Remark. Another Markovian coupling corresponds to replacing the transition

probabilities p2 ((x , y ), (x, y)) with p(x , x)1y=x whenever x = y . Doing so extends the identity Y = X to Yn = Xn for all n , thus yielding the bound
P(Xn 6= Yn ) P( > n) while each coordinate of the coupled chain evolves as
before according to the original transition probabilities p(, ).
The tail behavior of the first meeting time controls the rate of convergence of
n 7 Px (Xn = y). As you are to show next, this convergence is exponentially fast
when the state space is finite.
Exercise 6.2.58. Show that if the aperiodic, irreducible Markov chain {Xn } has
finite state space, then P( > n) exp(n) for the first meeting time of Theorem 6.2.57, some > 0 and any n large enough.
Hint: First assume that p(x, y) > 0 for all x, y S. Then show that Px (Xr = y) > 0
for some finite r and all x, y and consider the chain {Znr }.
The following consequence of Theorem 6.2.57 is a major step in our analysis of
the asymptotics of Px (Xn = y).
Theorem 6.2.59. The convergence (6.2.8) holds whenever y is an aperiodic state
of the Markov chain {Xn }. In particular, if this Markov chain is irreducible, positive
recurrent and aperiodic then for any x S,
lim kPx (Xn ) ()ktv = 0 .
Proof. If xy = 0 then Px (Xn = y) = 0 for all n and (6.2.8) trivially holds.

Otherwise,
X
xy =
Px (Ty = k) ,
k=1
is finite. Hence, in view of the first entrance decomposition

Px (Xn = y) =
n
X
Px (Ty = k)Py (Xnk = y)
k=1
(see part (b) of Exercise 6.2.2), the asymptotics (6.2.8) follows by bounded convergence (with respect to the law of Ty conditional on {Ty < }), from
(6.2.10)
lim Py (Xn = y) =
1
.
Ey (Ty )
Turning to prove (6.2.10), in view of Corollary 6.2.52 we may and shall assume
hereafter that y is an aperiodic recurrent state. Further, recall that by Theorem
6.2.13 it then suffices to consider the aperiodic, irreducible, recurrent chain {Xn }
obtained upon restricting the original Markov chain to the closed equivalence
class of y, which with some abuse of notation we denote hereafter also by S.
Suppose first that {Xn } is positive recurrent and so it has the invariant probability
measure (w) = 1/Ew (Tw ) (see Proposition 6.2.41). The irreducible chain Zn =
(Xn , Yn ) of Lemma 6.2.56 is then recurrent, so we apply Theorem 6.2.57 for X0 = y
and Y0 chosen according to the invariant probability measure . Since Yn is a
D
stationary Markov chain (see Definition 6.1.20), in particular Yn = Y0 has the law
for all n. Moreover, the corresponding first meeting time is a.s. finite. Hence,
P( > n) 0 as n and by (6.2.9) the law of Xn converges in total variation
255
to . This convergence in total variation further implies that Py (Xn = y) (y)

when n (c.f. Example 3.2.25), which is precisely the statement of (6.2.10).
Next, consider a null recurrent aperiodic, irreducible chain {Xn }, in which case our
thesis is that Py (Xn = y) 0 when n . This clearly holds if the irreducible
chain {Zn } of Lemma 6.2.56 is transient, for setting z = (y, y) we then have upon
applying Corollary 6.2.52 for the chain {Zn }, that as n
Pz (Zn = z) = Py (Xn = y)2 0 .
Proceeding to prove our thesis when the chain {Zn } is recurrent, suppose to the
contrary that the sequence n 7 Py (Xn = y) has a limit point (y) > 0. Then,
mapping S in a one to one manner into Z we deduce from Hellys theorem that
along a further sub-sequence n the distributions of Xn under Py converge vaguely,
hence pointwise (see Exercise 3.2.3), to some finite, positive measure on S. We
complete the proof of the theorem by showing that is an excessive measure for
the irreducible, recurrent chain {Xn }. Indeed, By part (c) of Exercise 6.2.29 this
would imply the existence of a finite invariant measure for {Xn }, in contradiction
with our assumption that this chain is null recurrent (see Corollary 6.2.42).
To prove that is an excessive measure, note first that considering Theorem
6.2.57 for Z0 = (x, y) we get from (6.2.9) that |Px (Xn = w) Py (Xn = w)| 0
as n , for any x, w S. Consequently, Px (Xn = w) (w) as , for
every x, w S. Moreover, from the Chapman-Kolmogorov equations we have that
for any w S, any finite set F S and all 1,
X
X
p(x, z)Pz (Xn = w) = Px (Xn +1 = w)
Px (Xn = z)p(z, w) .
zF
zS
In the limit this yields by bounded convergence (with respect to the probability measure p(x, ) on S), that for all w S
X
X
(w) =
p(x, z)(w)
(z)p(z, w) .
zS
zF
Taking F S we conclude by monotone convergence that () is an excessive measure on S, as we have claimed before.

Turning to the behavior of Px (Xn = y) for periodic state y, we start with the
following consequence of Theorem 6.2.59.
Corollary 6.2.60. The convergence (6.2.8) holds whenever y is a null recurrent
state of the Markov chain {Xn } and if y is a positive recurrent state of {Xn } having
period d = dy , then
(6.2.11)
lim Py (Xnd = y) =
d
.
Ey (Ty )
Proof. If y S has period d 1 for the chain {Xn } then Py (Xn = y) = 0

whenever n is not an integer multiple of d. Hence, the expected return time to such
state y by the Markov chain Yn = Xnd is precisely 1/d of the expected return time
Ey (Ty ) for {Xn }. Therefore, (6.2.11) is merely a reformulation of the limit (6.2.10)
for the chain {Yn } at its aperiodic state y S.
If y is a null recurrent state of {Xn } then Ey (Ty ) = so we have just established
that Py (Xn = y) 0 as n . It thus follows by the first entrance decomposition
at Ty that in this case Px (Xn = y) 0 for any x S (as in the opening of the
proof of Theorem 6.2.59).
256
6. MARKOV CHAINS
In the next exercise, you extend (6.2.11) to the asymptotic behavior of Px (Xn = y)
for any two states x, y in a recurrent chain (which is not necessarily aperiodic).
Exercise 6.2.61. Suppose {Xn } is an irreducible, recurrent chain of period d. For
each x, y S let Ix,y = {n 1 : Px (Xn = y) > 0}.
(a) Fixing z S show that there exist integers 0 ry < d such that if n Iz,y
then d divides n ry .
(b) Show that if n Ix,y then n = (ry rx ) mod d and deduce that Si =
{y S : ry = i}, i = 0, . . . , d 1 are the irreducible equivalence classes
of the aperiodic chain {Xnd } (Si are called the cyclic classes of {Xn }).
(c) Show that for all x, y, S,
lim Px (Xnd+ry rx = y) =
d
.
Ey (Ty )
Remark. It is not always true that if a recurrent state y has period d then
Px (Xnd+r = y) dxy /Ey (Ty ) for some r = r(x, y) {0, . . . , d 1}. Indeed, let
p(x, y) be the transition probabilities of the renewal chain with q1 = 0 and qk > 0
for k 2 (see Example 6.1.11), except for setting p(1, 2) = 1 (instead of p(1, 0) = 1
in the renewal chain). The corresponding Markov chain has precisely two recurrent
states, y = 1 and y = 2, both of period d = 2 and mean return times E1 (T1 ) =
E2 (T2 ) = 2.PFurther, 02 = 1 but P0 (Xnd = 2) and P0 (Xnd+1 = 2) 1 ,
where = k q2k is strictly between zero and one.
We next
Pnconsider the large n asymptotic behavior of the Markov additive functional
Afn = =1 f (X ), where {X } is an irreducible, positive recurrent Markov chain.
In the following two exercises you establish first the strong law of large numbers
(thereby generalizing Proposition 6.2.49), and then the central limit theorem for
such Markov additive functionals.
Exercise 6.2.62. Suppose {Xn } is an irreducible, positive recurrent chain of initial probability measure and invariant probability measure (). Let f : S 7 R be
such that (|f |) < .
(a) Fixing y S let Rk = Tyk . Show that the random variables
Zkf =
RX
k 1
f (X ) ,
=Rk1
k 1,
are mutually independent and moreover Zkf , k 2 are identically dis|f |

tributed with EZ2 finite.
Hint: Consider
6.2.11.
PNnExercise
(y) f
(b) With Snf = k=1
Zk+1 show that
lim n1 Snf =
EZ2f
= (f )
Ey (Ty )
P -a.s.
|f |
(c) Show that P -a.s. max{n1 Zk : k n} 0 when n and deduce

that n1 Afn (f ) with P probability one.
Exercise 6.2.63. For {Xn } as in Exercise 6.2.62 suppose that f : S 7 R is such
|f |
that (f ) = 0 and v|f | = Ey [(Z1 )2 ] is finite.
6.3. GENERAL STATE SPACE: DOEBLIN AND HARRIS CHAINS
257
D
(a) Show that n1/2 Snf uG as n , for u = vf /Ey (Ty ) finite and
G a standard normal variable.
Hint: See part (a) of Exercise 3.2.9.
|f |
(b)
Show that max{n1/2 Zk
uG.
: k n} 0 and deduce that n1/2 Afn
Building upon their strong law of large number, you are next to show that irreducible, positive recurrent chains have P-trivial tail -algebra and the laws of any
two such chains are mutually singular (for the analogous results for i.i.d. variables,
see Corollary 1.4.10 and Remark 5.5.14, respectively).
Exercise 6.2.64. Suppose {Xn } is an irreducible, positive recurrent chain of law
Px on (S , Sc ) (as in Definition 6.1.7).
(a) Show that Px (A) is independent of x S whenever A is in the tail algebra T X (of Definition 1.4.9).
(b) Deduce that T X is P-trivial.
Exercise 6.2.65. Suppose {Xn } is an irreducible, positive recurrent chain of transition probability p(x, y), initial and invariant probability measures () and (),
respectively.
(a) Show that {Xn , Xn+1 } is an irreducible, positive recurrent chain on S2+ =
{(x, y) : x, y S, p(x, y) > 0}, of initial and invariant measures (x)p(x, y)
and (x)p(x, y), respectively.
(b) Let P and P denote the laws of two irreducible, positive recurrent
chains on the same countable state space S, whose transition probabilities p(x, y) and p (x, y) are not identical. Show that P and P are
mutually singular measures (per Definition 4.1.9).
Hint: Consider the conclusion of Exercise 6.2.62 (for f () = 1x (), or, if
the invariant measures and are identical, then for f () = 1(x,y) ()
and the induced pair-chains of part (a)).
Exercise 6.2.66. Fixing 1 > > > 0 let P,
denote the law of (X0 , . . . , Xn )
n
for the Markov chain {Xk } of state space S = {1, 1} starting from X0 = 1
and evolving according to transition probability p(1, 1) = = 1 p(1, 1) and
p(1, 1) = = 1 p(1, 1). Fixing an
Pninteger b > 0 consider the stopping time
b = inf{n 0 : An = b} where An = k=1 Xk .
(a) Setting = log(/), h(1) = 1 and h(1) = (1 )/((1 )), show
,
that the Radon-Nikodym derivative Mn = dP,
is of the form
n /dPn
Mn = exp( An )h(Xn ).
(b) Deduce that P, (b < ) = exp( b)/h(1).
Exercise 6.2.67. Suppose {Xn } is a Markov chain of transition probability p(x, y)
and g() = (ph)() h() for some bounded function h() on S. Show that h(Xn )
Pn1
=0 g(X ) is then a martingale.
6.3. General state space: Doeblin and Harris chains
The refined analysis of homogeneous Markov chains with countable state space
is possible because such chains hit states with positive probability. This does not
happen in many important applications where the state space is uncountable. However, most proofs require only having one point of the state space that the chain
258
6. MARKOV CHAINS
hits with probability one. As we shall see, subject to the rather mild irreducibility and recurrence properties of Section 6.3.1, it is possible to create such a point
(called a recurrent atom), even in an uncountable state space, by splitting the chain
transitions. Guided by successive visits of the recurrent atom for the split chain, we
establish in Section 6.3.2 the existence and attractiveness of invariant (probability)
measures for the split chain (which then yield such results about the original chain).
6.3.1. Minorization, splitting, irreducibility and recurrence. Considering hereafter homogeneous Markov chains, we start by imposing a minorization
property of the transition probability p(, ) which yields the splitting of these transitions.
Definition 6.3.1. Consider a B-isomorphic state space (S, S). Suppose there
exists a non-zero measurable function v : S 7 [0, 1] and a probability measure q()
on (S, S) such that the transition probability of the chain {Xn } is of the form
(6.3.1)
p(x, ) = (1 v(x))b
p(x, ) + v(x)q() ,
for some transition probability pb(x, ) and v(x)q() pb(x, ). Amending the state
space to S = S {} with the corresponding -algebra S = {A, A {} : A S},
we then consider the split chain {X n } on (S, S) with transition probability
p(x, A) = (1 v(x))b
p(x, A)
p(x, {}) = v(x)

Z
p(, B) = q(dy)p(y, B)
x S, A S
xS
B S.
The transitions of {Xn } on S have been split by moving to the pseudo-atom

with probability v(x). The random times in which the split chain is at state are
regeneration times for {Xn }. That is, stopping times where future transitions are
decoupled from the past. Indeed, the event X n = corresponds to Xn moving to
a second copy of S where it is distributed according to the so called regeneration
measure q(), independently of Xn1 .
As the transitions of the split chain outside occur according to the excess probability (1 v(x))b
p(x, ), we can further merge the split chain to get back the original.
That is,
Definition 6.3.2. The merge transition probability m(, ) on (S, S) is such that
m(x, {x}) = 1 for all x S and m(, ) = q(). Associated
with it is the split
R
mapping f 7 f : bS 7 bS such that f () = (mf )() = m(, dy)f (y).
We note in passing that f (x) = f (x) for all x S and f () = q(f ), and further use in the sequel the following elementary fact about the closure of transition
probabilities under composition.
Corollary 6.3.3. Given any transition probabilities i : XR X 7 [0, 1], i = 1, 2,
the set function 1 2 : X X 7 [0, 1] such that 1 2 (x, A) = 1 (x, dy)2 (y, A) for
all x X and A X is a transition probability.
Proof. From Proposition 6.1.4 we see that
1 2 (x, A) = (1 (x, ) 2 )(X A) = (1 2 (, A))(x) .
259
Now, by the first equality, A 7 1 2 (x, A) is a probability measure on (S, S) for

each x S, and by the second equality, x 7 1 2 (x, A) is a measurable function on
(S, S) for each A S, as required in Definition 6.1.2.

Equipped with these notations we have the following coupling of {Xn } and {X n }.
Proposition 6.3.4. Consider the setup of Definitions 6.3.1 and 6.3.2.
(a). mp = p and the restriction of pm to (S, S) equals to p.
(b). Suppose {Zn } is an inhomogeneous Markov chain on (S, S) with transition
probability p2k = m and p2k+1 = p. Then, X n = Z2n is a Markov chain of transition probability p and Xn = Z2n+1 S is a Markov chain of transition probability p.
Setting an initial measure for Z0 = X 0 corresponds to having the initial measure
(A) = (A) + ({})q(A) for X0 S.
(c). E [f (Xn )] = E [f (X n )] for any f bS, any initial distribution on (S, S)
and all n 0.
Proof. (a). Since m(x, {x}) = 1 it follows that mp(x,
R B) = p(x, B) for all
x S and B S. Further, m(, ) = q() so mp(, B) = q(dy)p(y, B) which by
definition of p equals p(, B) (see Definition 6.3.1). Similarly, if either B = A S
or B = A{}, then by definition of the merge m and split p transition probabilities
we have as claimed that for any x S,
pm(x, B) = p(x, A) + p(x, {})q(A) = p(x, A) .
(b). As m(x, {}) = 0 for all x S, this follows directly from part (a). Indeed,
Z0 = X 0 of measure is mapped by transition m to X0 = Z1 S of measure
= m, then by transition p to X 1 = Z2 , followed by transition m to X1 = Z3 S
and so on. Therefore, the transition probability between X n1 and X n is mp = p
and the one between Xn1 and Xn is pm restricted to (S, S), namely p.
(c). Constructing Xn and X n as in part (b), if the initial distribution of X 0
assigns zero mass to then = with X0 = X 0 . Further, by construction
E [f (Xn )] = E [(mf )(X n )] which by definition of the split mapping is precisely
E [f (X n )], as claimed.

We plan to study existence and attractiveness of invariant (probability) measures
for the split chain {X n }, then apply Proposition 6.3.4 to transfer such results to
the original chain {Xn }. This however requires the recurrence of the atom . To
this end, we must restrict the so called small function v(x) of (6.3.1), motivating
the next definition.
Definition 6.3.5. A homogeneous Markov chain {Xn } on (S, S) is called a strong
Doeblin chain if the minorization condition (6.3.1) holds with a constant small
function. That is, when inf x p(x, A) q(A) for some probability measure q on
(S, S), a positive constant > 0 and all A S. We call {Xn } a Doeblin chain in
case Yn = Xrn is a strong Doeblin chain for some finite r, namely when Px (Xr
A) q(A) for all x S and A S.
The Doeblin condition allows us to construct a split chain {Y n } that visits its
atom at each time step with probability (0, ). Considering part (c) of
Exercise 6.1.18 (with A = S), it follows that P (Y n = i.o.) = 1 for any initial
distribution . So, in any Doeblin chain the atom is a recurrent state of the split
chain. Further, since T = inf{n 1 : Y n = } is such that Px (T = 1) =
260
6. MARKOV CHAINS
for all x S, by the Markov property of Y n (and Exercise 5.1.15), we deduce that
E [T ] 1/ is finite and uniformly bounded (in terms of the initial distribution
). Consequently, the atom is a positive recurrent, aperiodic state of the split
chain, which is accessible with probability one from each of its states.
As we see in Section 6.3.2, this is more than enough to assure that starting at
any initial state, PYn converges in total variation norm to the unique invariant
probability measure for {Yn }.
You are next going to examine which Markov chains of countable state space are
Doeblin chains.
Exercise 6.3.6. Suppose S = 2S with S a countable set.
(a) Show that a Markov chain of state space (S, S) is a Doeblin chain if and
only if there exists a S and r finite such that inf x Px (Xr = a) > 0.
(b) Deduce that for any Doeblin chain S = T R, where R = {y S : ay >
0} is a non-empty irreducible, closed set of positive recurrent, aperiodic
states and T = {y S : ay = 0} consists of transient states, all of which
lead to R.
(c) Verify that a Markov chain on a finite state space is a Doeblin chain if
and only if it has an aperiodic state a S that is accessible from any
other state.
(d) Check that branching processes with 0 < P(N = 0) < 1, renewal Markov
chains and birth and death chains are never Doeblin chains.
The preceding exercise shows that the Doeblin (recurrence) condition is too strong
for many chains of interest. We thus replace it by the weaker H-irreducibility
condition whereby the small function v(x) is only assumed bounded below on a
small, accessible set C. To this end, we start with the definitions of an accessible
set and weakly irreducible Markov chain.
Definition 6.3.7. We say that A S is accessible by the Markov chain {Xn } if
Px (TA < ) > 0 for all x S.
Given a non-zero -finite measure on (S, S), the chain is -irreducible if any set
A S with (A) > 0 is accessible by it. Finally, a homogeneous Markov chain on
(S, S) is called weakly irreducible if it is -irreducible for some non-zero -finite
measure (in particular, any Doeblin chain is weakly irreducible).
Remark. Modern texts on Markov chains typically refer to the preceding as the
standard definition of irreducibility but we use here the term weak irreducibility to
clearly distinguish it from the elementary definition for a countable S. Indeed, in
e denote the corresponding counting measure of S.
case S is a countable set, let
e
A Markov chain of state space S is then -irreducible
if and only if xy > 0 for
all x, y S, matching our Definition 6.2.14 of irreducibility, whereas a chain on S
countable is weakly irreducible if and only if xa > 0 for some a S and all x S.
In particular, a weakly irreducible chain of a countable state space S has exactly one
non-empty equivalence class of intercommunicating states (i.e. {y S : ay > 0}),
which is further accessible by the chain.
As we show next, a weakly irreducible chain has a maximal irreducibility measure
such that (A) > 0 if and only if A S is accessible by the chain.
261
Proposition 6.3.8. Suppose {Xn } is a weakly irreducible Markov chain on (S, S).
Then, there exists a probability measure on (S, S) such that for any A S,
(6.3.2)
(A) > 0
Px (TA < ) > 0
x S .
We call such a maximal irreducibility measure for the chain.

Remark. Clearly, if a chain is -irreducible, then any non-zero -finite measure
absolutely continuous with respect to (per Definition 4.1.4), is also an irreducibility measure for this chain. The converse holds in case of a maximal irreducibility
measure. That is, unraveling Definition 6.3.7 it follows from (6.3.2) that {Xn } is
-irreducible if and only if the non-zero -finite measure is absolutely continuous
with respect to .
Proof. Let be a non-zero -finite irreducibility measure of the given weakly
irreducible chain {Xn }. Taking D S such that (D) (0, ) we see that {Xn } is
also q-irreducible for the probability measure q() R= ( D)/(D). We claim that
(6.3.2) holds for the probability measure (A) = S q(dx)k(x, A) on (S, S), where
(6.3.3)
k(x, A) =
n=1
2n Px (Xn A) .
Indeed, with {TA < } = n1 {Xn A}, clearly Px (TA < ) > 0 if and only
if k(x, A) > 0. Consequently, if Px (TA < ) is positive for all x S then so is
k(x, A) and hence (A) > 0. Conversely, if (A) > 0 then necessarily q(C) > 0
for C = {x S : k(x, A) } and some > 0 small enough. In particular,
fixing x S, as {Xn } is q-irreducible, also Px (TC < ) > 0. That is, there exists
positive integer m = m(x) such that
P Px (Xm C) > 0. It now follows by the
Markov property at m (for h() = 1 2 I A ), that
k(x, A) 2m
X
=1
2 Px (Xm+ A)
= 2m Ex [k(Xm , A)] 2m Px (Xm C) > 0 .

Since this is equivalent to Px (TA < ) > 0 and applies for all x S, we have
established the identity (6.3.2).

We next define the notions of a small set and an H-irreducible chain.
Definition 6.3.9. An accessible set C S of a Markov chain {Xn } on (S, S) is
called r-small set if the transition probability (x, A) 7 Px (Xr A) satisfies the
minorization condition (6.3.1) with a small function that is constant and positive
on C. That is, when Px (Xr ) IC (x)q() for some positive constant > 0 and
probability measure q on (S, S).
We further use small set for 1-small set, and call the chain H-irreducible if it has
an r-small set for some finite r 1 and strong H-irreducible in case r = 1.
Clearly, a chain is Doeblin if and only if S is an r-small set for some r 1, and is
further strong Doeblin in case r = 1. In particular, a Doeblin chain is H-irreducible
and a strong Doeblin chain is also strong H-irreducible.
Exercise 6.3.10. Prove the following properties of H-irreducible chains.
(a) Show that an H-irreducible chain is q-irreducible, hence weakly irreducible.
262
6. MARKOV CHAINS
(b) Show that if {Xn } is strong H-irreducible then the atom of the split
chain {X n } is accessible by {X n } from all states in S.
(c) Show that in a countable state space every weakly irreducible chain is
strong H-irreducible.
Hint: Try C = {a} and q() = p(a, ) for some a S.
Actually, the converse to part (a) of Exercise 6.3.10 holds as well. That is, weak
irreducibility is equivalent to H-irreducibility (for the proof, see [Num84, Proposition 2.6]), and weakly irreducible chains can be analyzed via the study of an
appropriate split chain. For simplicity we focus hereafter on the somewhat more
restricted setting of strong H-irreducible chains. The following example shows that
it still applies for many Markov chains of interest.
Example 6.3.11 (Continuous transition densities). Let S = Rd with S =
BS . Suppose that for each x Rd the transition probability has a density p(x, y) with
respect to Lebesgue measure d () on Rd such that (x, y) 7 p(x, y) is continuous
jointly in x and y. Picking u and v such that p(u, v) > 0, there exists a neighborhood
C of u and a bounded neighborhood K of v, such that inf{p(x, y) : x C, y
K} > 0. Hence, setting q() to be the uniform measure on K (i.e. q(A) = d (A
K)/d (K) for any A S), such a chain is strong H-irreducible provided C is an
accessible set. For example, this occurs whenever p(x, u) > 0 for all x Rd .
Remark 6.3.12. Though our study of Markov chains has been mostly concerned
with measure theoretic properties of (S, S) (e.g. being B-isomorphic), quite often
S is actually a topological state space with S its Borel -algebra. As seen in the
preceding example, continuity properties of the transition probability are then of
much relevance in the study of Markov chains on S. In this context, we say that
p : S BS 7
R [0, 1] is a strong Feller transition probability, when the linear operator
(ph)() = p(, dy)h(y) of Lemma 6.1.3 maps every bounded BS -measurable function h to ph Cb (S), a continuous bounded function on S. In case of continuous
transition densities, as in Example 6.3.11, the transition probability is strong Feller
whenever the collection of probability measures {p(x, ), x S} is uniformly tight
(per Definition 3.2.31).
In case S = BS we further have the following topological notions of reachability
and irreducibility.
Definition 6.3.13. Suppose {Xn } is a Markov chain on a topological space S

equipped with its Borel -algebra S = BS . We call x S a reachable state of {Xn }
if any neighborhood of x is accessible by this chain and call the chain O-irreducible
(or open set irreducible), if every x S is reachable, that is, every open set is
accessible by {Xn }.
Remark. Equipping a countable state space S with its discrete topology yields
the Borel -algebra S = 2S , in which case O-irreducibility is equivalent to our
earlier Definition 6.2.14 of irreducibility.
For more general topological state spaces (such as S = Rd ), by their definitions,
a weakly irreducible chain is O-irreducible if and only if its maximal irreducibility
measure is such that (O) > 0 for any open subset O of S. Conversely,
Exercise 6.3.14. Show that if a strong Feller transition probability p(, ) has a
reachable state x S, then it is weakly irreducible.
Hint: Try the irreducibility measure () = p(x, ).
263
Remark. The minorization (6.3.1) may cause the maximal irreducibility measure
for the split chain to be supported on a smaller subset of the state space than the
one for the original chain. For example, consider the trivial Doeblin chain of i.i.d.
{Xn }, that is, p(x, ) = q(). In this case, taking v(x) = 1 results with the split
chain X n = for all n 1, so the maximal irreducibility measures = and
= q of {Xn } and {X n } are then mutually singular.
This is of course precluded by our additional requirement that v(x)q() pb(x, ).
For a strong H-irreducible chain {Xn } it is easily accommodated by, for example,
setting v(x) = IC (x) with = /2 > 0, and then the restriction of to S is a
maximal irreducibility measure for {Xn }.
Strong H-irreducible chains with a recurrent atom are called H-recurrent chains.
That is,
Definition 6.3.15. A strong H-irreducible chain {Xn } is called H-recurrent if
P (T < ) = 1. By the strong Markov property of X n at the consecutive visit
times Tk of , H-recurrence further implies that P (Tk finite for all k) = 1, or
equivalently P (X n = i.o.) = 1.
Here are a few examples and exercises to clarify the concept of H-recurrence.
Example 6.3.16. Many strong H-irreducible chains are not H-recurrent. For example, combining part (c) of Exercise 6.3.10 with the remark following Definition
6.3.7 we see that such are all irreducible transient chains on a countable state space.
By the same reasoning, a Markov chain of countable state space S is H-recurrent if
and only if S = T R with R a non-empty irreducible, closed set of recurrent states
and T a collection of transient states that lead to R (c.f. part (b) of Exercise 6.3.6
for such a decomposition in case of Doeblin chains). In particular, such chains are
not necessarily recurrent in the sense of Definition 6.2.14. For example, the chain
on S = {1, 2, . . .} with transitions p(k, 1) = 1 p(k, k + 1) = k s for some constant
s > 0, is H-recurrent but has only one recurrent state, i.e. R = {1}. Further,
k1 < 1 for all k 6= 1 when s > 1, while k1 = 1 for all k when s 1.
Remark. Advanced texts on Markov chains refer to what we call H-recurrence
as the standard definition of recurrence and call such chains Harris recurrent when
in addition Px (T < ) = 1 for all x S. As seen in the preceding example,
both notions are weaker than the elementary notion of recurrence for countable
S, per Definition 6.2.14. For this reason, we adopt here the convention of calling H-recurrence (with H after Harris), what is not the usual definition of Harris
recurrence.
Exercise 6.3.17. Verify that any strong Doeblin chain is also H-recurrent. Conversely show that for any H-recurrent chain {Xn } there exists C S and a probability distribution q on (S, S) such that Pq (TCk finite for all k) = 1 and the Markov
chain Zk = XT k+1 for k 0 is then a strong Doeblin chain.
C
The next proposition shows that similarly to the elementary notion of recurrence,
H-recurrence is transferred from the atom to all sets that are accessible from
it. Building on this proposition, you show in Exercise 6.3.19 that the same applies
when starting at any irreducibility probability measure of the split chain and that
every set in S is either almost surely visited or almost surely never reached from
by the split chain.
264
6. MARKOV CHAINS
Proposition 6.3.18. For an H-recurrent chain {Xn } consider the probability measure
X
(B) =
2n P (X n B) .
(6.3.4)
n=1
Then, P (X n B i.o.) = 1 whenever (B) > 0.

Proof. Clearly, (B) > 0 if and only if P (TB < ) > 0. Further, if
= P (TB < ) > 0, then considering the split chain starting at X 0 = , we
have from part (c) of Exercise 6.1.18 that
P ({X n = finitely often } {X n B i.o.}) = 1 .
As P (X n = i.o.) = 1 by the assumed H-recurrence, our thesis that P (X n B
i.o.) = 1 follows.

Exercise 6.3.19. Suppose is the probability measure of (6.3.4) for an H-recurrent
chain.
(a) Argue that {} is accessible by the split chain {X n } and show that is
a maximal irreducibility measure for it.
(b) Show that P (D) = P (D) for any shift invariant D S c (i.e. where
D = 1 D).
(c) In case B S is such that (B) > 0 explain why Px (X n B i.o.) = 1
for -a.e. x S and P (X n B i.o.) = 1 for any probability measure
.
(d) Show that P (TB < ) {0, 1} for all B S.
Given a strong H-irreducible chain {Xn } there is no unique way to select the small
set C, regeneration measure q() and > 0 such that p(x, ) IC (x)q(). Consequently, there are many different split chains for each chain {Xn }. Nevertheless, as
you show next, H-recurrence is determined by the original chain {Xn }.
Exercise 6.3.20. Suppose {X n } and X n } are two different split chains for the
same strong H-irreducible chain {Xn } with the corresponding atoms and .
Relying on Proposition 6.3.18 prove that P (T < ) = 1 if and only if P (T <
) = 1.
The concept of H-recurrence builds on measure theoretic properties of the chain,
namely the minorization associated with strong H-irreducibility. In contrast, for
topological state space we have the following topological concept of O-recurrence,
built on reachability of states.
Definition 6.3.21. A state x of a Markov chain {Xn } on (topological) state space
(S, BS ) is called O-recurrent (or open set recurrent), if Px (Xn O i.o.) = 1 for
any neighborhood O of x in S. All states x S which are not O-recurrent are called
O-transient. Such a chain is then called O-recurrent if every x S is O-recurrent
and O-transient if every x S is O-transient.
Remark. As was the case with O-irreducibility versus irreducibility, for a countable state space S equipped with its discrete topology, being O-recurrent (or Otransient), is equivalent to being recurrent (or transient, respectively), per Definitions 6.2.9 and 6.2.14.
265
The concept of O-recurrence is in particular suitable for the study of random

walks. Indeed,
Pn
Exercise 6.3.22. Suppose Sn = S0 + k=1 k is a random walk on Rd .
(a) Show that if {Sn } has one reachable state, then it is O-irreducible.
(b) Show that either {Sn } is an O-recurrent chain or it is an O-transient
chain.
(c) Show that if {Sn } is O-recurrent, then
S = {x Rd : Px (kXn k < r i.o. ) > 0, for all r > 0} ,
is a closed sub-group of Rd (i.e. 0 S and if x, y S then also xy S),

with respect to which {Sn } is O-irreducible (i.e. Py (TB(x,r) < ) > 0
for all r > 0 and x, y S).
In case of one-dimensional random walks, you are to recover next the Chung-Fuchs
theorem, stating that if n1 Sn converges to zero in probability, then this Markov
chain is O-recurrent.
Exercise 6.3.23 (Chung-Fuchs theorem). Suppose {Sn } is a random walk on
S R.
(a) Show that such random walk is O-recurrent if and only if for each r > 0,
X
P0 (|Sn | < r) = .
n=0
(b) Show that for any r > 0 and k Z,
X
X
P0 (Sn [kr, (k + 1)r))
P0 (|Sm | < r) ,
n=0
m=0
and deduce that suffices to check divergence of the series in part (a) for
large r.
p
(c) Conclude that if n1 Sn 0 as n , then {Sn } is O-recurrent.
6.3.2. Invariant measures, aperiodicity and asymptotic behavior. We
consider hereafter an H-recurrent Markov chain {Xn } of transition probability p(, )
on the B-isomorphic state space (S, S) with its recurrent pseudo-atom and the
corresponding split and merge chains p(, ), m(, ) on (S, S) per Definitions 6.3.1
and 6.3.2.
The following lemma characterizes the invariant measures of the split chain p(, )
and their relation to the invariant measures for p(, ). To this end, we use hereafter
1 2 also for the measure 1 2 (A) = 1 (2 (, A)) on (X, X ) in case 1 is a measure
on (X, X ) and 2 is a transition probability on this space and let pn (x, B) denote
the transition probability Px (X n B) on (S, S).
Lemma 6.3.24. A measure
on (S, S) is invariant for the split chain p(, ) of a
strong H-irreducible chain if and only if
=
p and 0 <
({}) < . Further,
m is then an invariant measure for the original chain p(, ). Conversely, if is

an invariant measure for p(, ) then the measure p is invariant for the split chain.
Proof. Recall Proposition 6.1.23 that a measure
is invariant for the split
chain if and only if
is positive, -finite and
p(B)
(B) =
p(S B) =
B S .
266
6. MARKOV CHAINS
Likewise, a measure is invariant for p if and only if is positive, -finite and

(A) = p(A) for all A S.
m is invariant for p. Indeed,
We first show that if
is invariant for p then =
note that from Definition 6.3.2 it follows that
(6.3.5)
(A) =
(A) +
({})q(A)
A S
and in particular, such is a positive, -finite measure on (S, S) for any -finite
on (S, S), and any probability measure q() on (S, S). Further, starting the
inhomogeneous Markov chain {Zn } of Proposition 6.3.4 with initial measure
for
Z0 = X 0 yields the measure for Z1 = X0 . By construction, the measure of
Z2 = X 1 is then p and that of Z3 = X1 is (p)m = (pm). Next, the invariance
of
for p implies that the measure of X 1 equals that of X 0 . Consequently, the
measure of X1 must equal that of X0 , namely = (pm). With m(, {}) 0
necessarily ({}) = 0 and the identity = (pm) holds also for the restrictions
to (S, S) of both and pm. Since the latter equals to p (see part (a) of Proposition
6.3.4), we conclude that = p, as claimed.
Conversely, let
= p where is an invariant measure for p (and we set ({}) =
0). Since is -finite, there exist An S such that (An ) < for all n and
necessarily also q(An ) > 0 for all n large enough (by monotonicity from below
of the probability measure q()). Further, the invariance of implies that
m =
(S) = (S) so
inherits
(p)m = , i.e. the relation (6.3.5) holds. In particular,
the positivity of . Moreover, both
({}) = and
(An ) = contradict the
finiteness of (An ) for all n, so the measure
is -finite on (S, S). Next, start
the chain {Zn } at Z0 = X 0 S of initial measure . It yields the same measure
= m for Z1 = X0 , with measure
= p for Z2 = X 1 followed by
m = for
Z3 = X1 and
p for Z4 = X 2 . As the measure of X1 equals that of X0 , it follows
of X 1 , i.e.
is invariant for p.
that the measure
p of X 2 equals the measure
Finally, suppose the measure
satisfies
=
p. Iterating this identity we deduce
=
k for the transition probability
that
=
pn for all n 1, hence also
X
(6.3.6)
k(x, B) =
2n pn (x, B) .
n=1
Due to its strong H-irreducibility, the atom {} of the split chain is an accessible
set for the transition probability p (see part (b) of Exercise 6.3.10). So, from (6.3.6)
we deduce that k(x, {}) > 0 for all x S. Consequently, as n ,
Bn = {x S : k(x, {}) n1 } S ,
whereas by the identity

({}) = (
k)({}) also
({}) n1
(Bn ). This proves
the first claim of the lemma. Indeed, we have just shown that when
=
p it
follows that
is positive if and only if
({}) > 0 and -finite if and only if
({}) < .

Our next result shows that, similarly to Proposition 6.2.27, the recurrent atom
induces an invariant measure for the split chain (and hence also one for the original
chain).
Proposition 6.3.25. If {Xn } is H-recurrent of transition probability p(, ) then
(6.3.7)
(B) = E
1
TX
n=0
I{X n B}
267
is an invariant measure for p(, ).

Proof. Let ,n (B) = P (X n B, T > n), noting that
(6.3.8)
(B) =
,n (B)
n=0
B S
and ,n (g) = E [I{T >n} g(X n )] for all g bS. Since {T > n} FnX =
(X k , k n), we have by the tower and Markov properties that, for each n 0,
P (X n+1 B, T > n) = E [I{T >n} P (X n+1 B|FnX ) ]
,n p)(B) .
= E [I{T >n} p(X n , B) ] = ,n (p(, B)) = (
Hence,
(
p)(B) =
(
,n p)(B) =
n=0
= E
n=0
T
X
n=1
P (X n+1 B, T > n)

I{X n B} = (B)
since P (T < , X 0 = X T ) = 1. We thus established that p = and as

({}) = 1 we conclude from Lemma 6.3.24 that it is an invariant measure for
the split chain.

Building on Lemma 6.3.24 and Proposition 6.3.25 we proceed to the uniqueness of
the invariant measure for an H-recurrent chain, namely the extension of Proposition
6.2.30 to a typically uncountable state space.
Theorem 6.3.26. Up to a constant multiple, the unique invariant measure for
H-recurrent transition probability p(, ) is the restriction to (S, S) of m, where
is per (6.3.7).
Remark. As (S) = E T , it follows from the theorem that an H-recurrent
chain has an invariant probability measure if and only if E (T ) = Eq (T ) < .
In accordance with Definition 6.2.40 we call such chains positive H-recurrent. While
the value of E (T ) depends on the specific split chain one associates with {Xn }, it
follows from the preceding that positive H-recurrence, i.e. the finiteness of E (T ),
is determined by the original chain. Further, in view of the relation (6.3.5) between
m and and the decomposition (6.3.8) of , the unique invariant probability
measure for {Xn } is then
(6.3.9)
(A) =
X
1
Pq (X n A, T > n)
Eq (T ) n=0
A S .
Proof. By Lemma 6.3.24, to any invariant measure for p (with ({}) = 0),
corresponds the invariant measure
= p for the split chain p. It is also shown
there that 0 <
({}) < . Hence, with no loss of generality we assume hereafter
that the given invariant measure for p has already been divided by this positive,
finite constant, and so
({}) = 1. Recall that while proving Lemma 6.3.24 we
further noted that =
m, due to the invariance of for p. Consequently, to prove
the theorem it suffices to show that
= (for then =
m = m).
268
6. MARKOV CHAINS
is also
To this end, fix B S and recall from the proof of Lemma 6.3.24 that
invariant for pn and any n 1. Using the latter invariance property and applying
Exercise 6.2.3 for y = and the split chain {X n }, we find that
Z
Z
(B) = (
pn )(B) =
(dx)Px (X n B)
(dx)Px (X n B, T n)
S
n1
X
k=0
(
pnk )({})P (X k B, T > k) =
n1
X
,k (B) ,
k=0
with ,k () per the decomposition (6.3.8) of (). Taking n , we thus deduce

that
X
(6.3.10)
(B)
,k (B) = (B)
B S .
k=0
We proceed to show that this inequality actually holds with equality, namely, that
= . To this end, recall that while proving Lemma 6.3.24 we showed that
and are also invariant for the transition
invariant measures for p, such as
probability k(, ) of (6.3.6), and by strong H-irreducibility the measurable function
g() = k(, {}) is strictly positive on S. Therefore,
(g) = (
k)({}) =
({}) = 1 = ({}) = (
k)({}) = (g) .
Recall Exercise 4.1.13 that identity such as

(g) = (g) = 1 for a strictly positive
and
g mS, strengthens the inequality (6.3.10) between two -finite measures
= .

on (S, S) into the claimed equality
The next result is a natural extension of Theorem 6.2.57.
Theorem 6.3.27. Suppose {Xn } and {Yn } are independent copies of a strong
H-irreducible chain. Then, for any initial distribution of (X0 , Y0 ) and all n,
(6.3.11)
kPXn PYn ktv 2P( > n) ,
where k ktv denotes the total variation norm of Definition 3.2.22 and = min{
0 : X = Y = } is the time of the first joint visit of the atom by the corresponding
copies of the split chain under the coupling of Proposition 6.3.4.
Proof. Fixing g bS bounded by one, recall that the split mapping yields
g bS of the same bound, and by part (c) of Proposition 6.3.4
E g(Xn ) E g(Yn ) = E g(X n ) E g(Y n )
for any joint initial distribution of (X0 , Y0 ) on (S2 , S S) and all n 0. Further,
since X = Y in case n, following the proof of Theorem 6.2.57 one finds that
|E g(X n ) E g(Y n )| 2P( > n). Since this applies for all g bS bounded by
one, we are done.

Our goal is to extend the scope of the convergence result of Theorem 6.2.59 to the
setting of positive H-recurrent chains. To this end, we first adapt Definition 6.2.54
of an aperiodic chain.
Definition 6.3.28. The period of a strongly H-irreducible chain is the g.c.d. d
of the set I = {n 1 : P (X n = ) > 0}, of return times to its pseudo-atom and
such chain is called aperiodic if it has period one. For example, q(C) > 0 implies
aperiodicity of the chain.
269
Remark. Recall that being (strongly) H-irreducible amounts for a countable state
space to having exactly one non-empty equivalence class of intercommunicating
states (which is accessible from any other state). The preceding definition then
coincides with the common period of these intercommunicating states per Definition
6.2.54.
More generally, our definition of the period of the chain seems to depend on which
small set and regeneration measure one chooses. However, in analogy with Exercise
6.3.20, after some work it can be shown that any two split chains for the same strong
H-irreducible chain induce the same period.
Theorem 6.3.29. Let () denote the unique invariant probability measure of an
aperiodic positive H-recurrent Markov chain {Xn }. If x S is such that Px (T <
) = 1, then
lim kPx (Xn ) ()ktv = 0 .
(6.3.12)
Remark. It follows from (6.3.9) that () is absolutely continuous with respect

to () of Proposition 6.3.18. Hence, by parts (a) and (c) of Exercise 6.3.19, both
P (T < ) = 1 ,
(6.3.13)
and Px (T < ) = 1 for -a.e. x S. Consequently, the convergence result

(6.3.12) holds for -a.e. x S.
Proof. Consider independent copies X n and Y n of the split chain starting

at X 0 = x and at Y 0 whose law is the invariant probability measure = p of
the split chain. The corresponding Xn and Yn per Proposition 6.3.4 have the laws
Px (Xn ) and (), respectively. Hence, in view of Theorem 6.3.27, to establish
(6.3.12) it suffices to show that with probability one X n = Y n = for some finite,
possibly random value of n. Proceeding to prove the latter fact, recall (6.3.13)
and the H-recurrence of the chain, in view of which we have with probability one
that Y n = for infinitely many values of n, say at random times {Rk }. Similarly,
our assumption that Px (T < ) = 1 implies that with probability one X n =
ek }
for infinitely many values of n, say at another sequence of random times {R
and it remains to show that these two random subsets of {1, 2, . . .} intersect with
probability one. To this end, note that upon adapting the argument used in solving
e1 , rk = Rk+1 Rk and rek = R
ek+1 R
ek for
Exercise 6.2.11 you find that R1 , R
k 1 are mutually independent, with {rk , rek , k 1} identically distributed, each
fn+1 = W
fn + Zen ,
following the law of T under P . Let Wn+1 = Wn + Zn and W
f0 = 1, where the i.i.d. {Z, Z , Z
e } are independent of {X n }
starting at W0 = W
and {Y n } and such that P(Z = k) = 2k for k 1. It then follows by the strong
e f , n 0, is a random
Markov property of the split chains that Sn = RWn R
Wn
walk on Z, whose i.i.d. increments {n } have each the law of the difference between
two independent copies of TZ under P . As mentioned already, our thesis follows
from P(Sn = 0 i.o.) = 1, which in view of Corollary 6.2.12 and Theorem 6.2.13 is in
turn an immediate consequence of our claim that {Sn } is an irreducible, recurrent
Markov chain.
Turning to prove that {Sn } is irreducible, note that since Z is independent of
{Tk }, for any n 1
X
X
2k P (Nn () = k, X n = ) .
P (TZ = n) =
2k P (Tk = n) =
k=1
k=1
270
6. MARKOV CHAINS
Consequently, P (TZ = n) > 0 if and only if P (X n = ) > 0. That is, the

support of the law of TZ is the set I of possible return times to . By the
assumed aperiodicity of the chain, the g.c.d. of I is one (see Definition 6.3.28).
Further, by definition this subset of positive integers is closed under addition, hence
as we have seen in the course of proving Lemma 6.2.55, the set I contains all large
enough integers. As 1 is the difference between two independent copies of TZ , the
law of each of which is strictly positive for all large enough positive integers, clearly
P(1 = z) > 0 for all z Z, out of which the irreducibility of {Sn } follows.
As for the recurrence of {Sn }, note that by the assumed positive H-recurrence of
{Xn } and the independence of Z and this chain,
X
X
E (TZ ) =
E (Tk )P(Z = k) = E (T )
kP(Z = k) = E (T )E(Z) < .
k=1
k=1
Hence, the increments n of the irreducible random walk {Sn } on Z are integrable
p
and of zero mean. Consequently, n1 Sn 0 as n which by the Chung-Fuchs
theorem implies the recurrence of {Sn } (see Exercise 6.3.23).

Exercise 6.3.30. Suppose {Xk } is the first order auto-regressive process Xn =
Xn1 + n , n 1 with || < 1 and where the integrable i.i.d. {n } have a strictly
positive, continuous density f () with respect to Lebesgue measure on Rd .
(a) Show that {Xk } is
P
Pan strong H-irreducible chain.
(b) Show that Vn = k=0 k k converges a.s. to V = k0 k k whose
law () is an invariant probability measure for {Xk }.
(c) Show that {Xk } is positive H-recurrent.
(d) Explain why {Xk } is aperiodic and deduce that starting at any fixed x
Rd the law of Xn converges in total variation to ().
Exercise 6.3.31. Show that if {Xn } is an aperiodic, positive H-recurrent chain
and x, y S are such that Px (T < ) = Py (T < ) = 1, then for any A S,
lim |Px (Xn A) Py (Xn A)| = 0 .
Exercise 6.3.32. Suppose {n } are i.i.d. with P(1 = 1) = 1 P(1 = 1) = b

and {Un } are i.i.d. uniform on [5, 5] and independent of {n }. Consider the
Markov chain {Xn } with state space S = R such that Xn = Xn1 + n sign(Xn1 )
when |Xn1 | > 5 and Xn = Xn1 + Un otherwise.
(a) Show that this chain is strongly H-irreducible for any 0 b < 1.
(b) Show that it has a unique invariant measure (up to a constant multiple),
when 0 b 1/2.
(c) Show that if 0 b < 1/2 the chain has a unique invariant probability
measure () and that Px (Xn B) (B) as n for any x R
and every Borel set B.
CHAPTER 7
Continuous, Gaussian and stationary processes

A discrete parameter stochastic process (S.P.) is merely a sequence of random variables. We have encountered and constructed many such processes when considering
martingales and Markov chains in Sections 5.1 and 6.1, respectively. Our focus here
is on continuous time processes, each of which consists of an uncountable collection
of random variables (defined on the same probability space).
We have successfully constructed by an ad-hoc method one such process, namely
the Poisson process of Section 3.4. In contrast, Section 7.1 provides a canonical
construction of S.P., viewed as a collection of R.V.-s {Xt (), t T}. This construction, based on the specification of finite dimensional distributions, applies for
any index set T and any S.P. taking values in a B-isomorphic measurable space.
However, this approach ignores the sample function t 7 Xt () of the process.
Consequently, the resulting law of the S.P. provides no information about probabilities such as that of continuity of the sample function, or whether it is ever zero,
or the distribution of suptT Xt . We thus detail in Section 7.2 a way to circumvent
this difficulty, whereby we guarantee, under suitable conditions, the continuity of
the sample function for almost all outcomes , or at the very least, its (Borel)
measurability.
We conclude this chapter by studying in Section 7.3 the concept of stationary (of
processes and their increments), and the class of Gaussian (stochastic) processes,
culminating with the definition and construction of the Brownian motion.
7.1. Definition, canonical construction and law
We start with the definition of a stochastic process.
Definition 7.1.1. Given (, F , P), a stochastic process, denoted {Xt }, is a collection {Xt : t T} of R.V.-s. In case the index set T is an interval in R we call it
a continuous time S.P. The function t 7 Xt () is called the sample function (or
sample path, realization, or trajectory), of the S.P. at .
We shall follow the approach we have taken in constructing product measures (in
Section 1.4.2) and repeated for dealing with Markov chains (in Section 6.1). To
this end, we start with the finite dimensional distributions associated with the S.P.
Definition 7.1.2. By finite dimensional distributions (f.d.d.) of a S.P. {Xt , t
T} we refer to the collection of probability measures t1 ,t2 , ,tn () on B n , indexed
by n and distinct tk T, k = 1, . . . , n, where
t1 ,t2 , ,tn (B) = P((Xt1 , Xt2 , , Xtn ) B) ,
for any Borel subset B of Rn .
271
272
7. CONTINUOUS, GAUSSIAN AND STATIONARY PROCESSES
1.5
0.5
Xt()
t Xt(1)
0.5
t X ( )
1.5
0.5
1.5
t
2.5
Figure 1. Sample functions of a continuous time stochastic process, corresponding to two outcomes 1 and 2 .
Not all f.d.d. are relevant here, for you should convince yourself that the f.d.d. of
any S.P. should be consistent, as specified next.
Definition 7.1.3. We say that a collection of finite dimensional distributions is
consistent if for any Bk B, distinct tk T and finite n,
(7.1.1)
t1 , ,tn (B1 Bn ) = t(1) , ,t(n) (B(1) B(n) ) ,
for any permutation of {1, 2, , n} and

(7.1.2)
t1 , ,tn1 (B1 Bn1 ) = t1 , ,tn1 ,tn (B1 Bn1 R) .
Here is a simpler, equivalent definition of consistent f.d.d. in case T is linearly (i.e.

totally) ordered.
Lemma 7.1.4. In case T is a linearly ordered set (for example, T countable, or T
R), it suffices to define as f.d.d. the collection of probability measures s1 ,...,sn ()
running over s1 < s2 < < sn in T and finite n, where such collection is
consistent if and only if for any Ai B and k = 1, . . . , n,
(7.1.3)
s1 , ,sn (A1 Ak1 R Ak+1 An )
= s1 , ,sk1 ,sk+1 , ,sn (A1 Ak1 Ak+1 An )
Proof. Since the set T is linearly ordered, for any distinct ti T, i = 1, . . . , n

there exists a unique permutation on {1, . . . , n} such that si = t(i) are in increasing order and taking the random vector (Xs1 , , Xsn ) of (joint) distribution
s1 , ,sn (), we set t1 , ,tn as the distribution of the vector (Xt1 , , Xtn ) of permuted coordinates. This unambiguously extends the definition of the f.d.d. from
the ordered s1 < < sn to all distinct ti T. Proceeding to verify the consistency of these f.d.d. note that by our definition, the identity (7.1.1) holds whenever
{t(i) } are in increasing order. Permutations of {1, . . . , n} form a group with respect to composition, so (7.1.1) extends to {t(i) } of arbitrary order. Next suppose
that in the permutation of {1, . . . , n} such that si = t(i) are in increasing order
7.1. DEFINITION, CANONICAL CONSTRUCTION AND LAW
273
we have n = (k) for some 1 k n. Then, setting Bn = R and Ai = B(i) leads

to Ak = R and from (7.1.1) and (7.1.3) it follows that
t1 , ,tn (B1 Bn1 R) = s1 , ,sk1 ,sk+1 , ,sn (A1 Ak1 Ak+1 An ) .
Further, (t1 , . . . , tn1 ) is the image of (s1 , . . . , sk1 , sk+1 , . . . , sn ) under the permutation 1 restricted to {1, . . . , k 1, k + 1, . . . , n} so a second application of (7.1.1)
results with the consistency condition (7.1.2).

Our goal is to establish the existence and uniqueness (in law) of the S.P. associated with any given consistent collection of f.d.d. We shall do so via a canonical
construction, whereby we set = RT and F = B T as follows.
Definition 7.1.5. Let RT denote the collection of all functions x(t) : T 7 R. A
finite dimensional measurable rectangle in RT is any set of the form {x() : x(ti )
Bi , i = 1, . . . , n} for a positive integer n, Bi B and ti T, i = 1, . . . , n. The
cylindrical -algebra B T is the -algebra generated by the collection of all finite
dimensional measurable rectangles.
Note that in case T = {1, 2, . . .}, the -algebra B T is precisely the product algebra Bc used in stating and proving Kolmogorovs extension theorem. Further,
enumerating C = {tk }, it is not hard to see that B C is in one to one correspondence
with Bc for any infinite, countable C T.
The next concept is handy in studying the structure of B T for uncountable T.
Definition 7.1.6. We say that A RT has a countable representation if
A = {x() RT : (x(t1 ), x(t2 ), . . .) D} ,
for some D Bc and C = {tk } T. The set C is then called the (countable) base
of the (countable) representation (C, D) of A.
Indeed, B T consists of the sets in RT having a countable representation and is
further image of F X = (Xt , t T) via the mapping X : 7 RT .
Lemma 7.1.7. The -algebra B T is the collection C of all subsets of RT that have
a countable representation. Further, for any S.P. {Xt , t T}, the -algebra F X is
the collection G of sets of the form { : X () A} with A B T .
Proof. First note that for any subsets T1 T2 of T, the restriction to T1 of
functions on T2 induces a measurable projection p : (RT2 , B T2 ) 7 (RT1 , B T1 ). Further, enumerating over a countable C maps the corresponding cylindrical -algebra
B C in a one to one manner into the product -algebra Bc . Thus, if A C has the
countable representation (C, D) then A = p1 (D) for the measurable projection p
from RT to RC , hence A B T . Having just shown that C B T we turn to show that
conversely B T C. Since each finite dimensional measurable rectangle has a countable representation (of a finite base), this is an immediate consequence of the fact
that C is a -algebra. Indeed, RT has a countable representation (of empty base),
and if A C has the countable representation (C, D) then Ac has the countable
representation (C, Dc ). Finally, if Ak C has a countable representation (Ck , Dk )
for k = 1, 2, . . . then the subset C = k Ck of T serves as a common countable base
e k ), for k = 1, 2, . . .
for these sets. That is, Ak has the countable representation (C, D
1
e
and Dk = pk (Dk ) Bc , where pk denotes the measurable projection from RC to
274
RCk . Consequently, as claimed k Ak C for it has the countable representation

e k ).
(C, k D
As for the second part of the lemma, temporarily imposing on the -algebra 2
makes X : 7 RT an (S, S)-valued R.V. for S = RT and S = B T . From Exercises
1.2.10 and 1.2.11 we thus deduce that G is the -algebra generated by the sets of
the form { : Xti () Bi , i = 1, . . . , n} for Bi B, ti T and finite n, which
is precisely the -algebra F X .

Combining Lemma 7.1.7 and Kolmogorovs extension theorem, we proceed with
the promised canonical construction, yielding the following conclusion.
Proposition 7.1.8. For any consistent collection of f.d.d., there exists a probability space (, F , P) and a stochastic process 7 {Xt (), t T} on it, whose f.d.d.
are in agreement with the given collection. Further, the restriction of the probability
measure P to the -algebra F X is uniquely determined by the specified f.d.d.
Proof. Starting with the existence of the probability space, suppose first that
T = C is countable. In this case, enumerating over C = {sj } we further have
from the consistency condition (7.1.3) of Lemma 7.1.4 that it suffices to consider
the sequence of f.d.d. s1 ,...,sn for n = 1, 2, . . . and the existence of a probability
measure PC on (RC , Bc ) that agrees with the given f.d.d. follows by Kolmogorovs
extension theorem (i.e. Theorem 1.4.22). Moving to deal with uncountable T, take
= RT and F = B T with Xt () = t . Recall Lemma 7.1.7, that any A B T has
a countable representation (C, D) so we can assign P(A) = PC (D), where PC is
defined through Kolmogorovs extension theorem for the countable subset C of T.
We proceed to show that P() is well defined. That is, PC1 (D1 ) = PC2 (D2 ) for any
two countable representations (C1 , D1 ) and (C2 , D2 ) of the same set A B T . Since
C = C1 C2 is then also a countable base for A, we may and shall assume that
C1 C2 in which case necessarily D2 = p1
21 (D1 ) for the measurable projection
p21 from RC2 to RC1 . By their construction, PCi for i = 1, 2 coincide on all finite
dimensional measurable rectangles with a base in C1 . Hence, PC1 = PC2 p1
21 and
in particular PC2 (D2 ) = PC1 (D1 ). By construction the non-negative set function
P on (RT , B T ) has the specified f.d.d. for Xt () = t so we complete the proof
of existence by showing that P is countably additive. To this end, as shown in
the proof of Lemma 7.1.7, any sequence of disjoint sets Ak B T admits countable
e k ), k = 1, 2, . . . with a common base C and disjoint D
e k Bc .
representations (C, D
Hence, by the countable additivity of PC ,
X
X
ek) =
ek) =
P(k Ak ) = PC (k D
PC (D
P(Ak ) .
k
As for uniqueness, recall Lemma 7.1.7 that every set in F X is of the form { :
(Xt1 (), Xt2 (), . . .) D} for some D Bc and C = {tj } a countable subset of T.
Fixing such C, recall Kolmogorovs extension theorem, that the law of (Xt1 , Xt2 , . . .)
on Bc is uniquely determined by the specified laws of (Xt1 , . . . , Xtn ) for n = 1, 2, . . ..
Since this applies for any countable C, we see that the whole restriction of P to
F X is uniquely determined by the given collection of f.d.d.
Remark. Recall Corollary 1.4.25 that Kolmogorovs extension theorem holds

when (R, B) is replaced by any B-isomorphic measurable space (S, S). Check that
thus, the same applies for the preceding proof, hence Proposition 7.1.8 holds for
7.1. DEFINITION, CANONICAL CONSTRUCTION AND LAW
275
any (S, S)-valued S.P. {Xt } provided (S, S) is B-isomorphic (c.f. [Dud89, Theorem
12.1.2] for an even more general setting in which the same applies).
Motivated by Proposition 7.1.8 our definition of the law of the S.P. is as follows.
Definition 7.1.9. The law (or distribution) of a S.P. is the probability measure
PX on B T such that for all A B T ,
PX (A) = P({ : X () A}) .
Proposition 7.1.8 tells us that the f.d.d. uniquely determine the law of any S.P.
and provide the probability of any event in F X . However, for our construction to
be considered a success story, we want most events of interest be in F X . That is,
their image via the sample function should be in B T . Unfortunately, as we show
next, this is certainly not the case for uncountable T.
Lemma 7.1.10. Fixing R and I = [a, b) for some a < b, the following sets
A = {x RI : x(t) for all t I} ,
C(I) = {x RI : t 7 x(t) is continuous on I} ,

are not in B I .
Proof. In view of Lemma 7.1.7, if A B I then A has a countable base
C = {tk } and in particular the values of x(tk ) determine whether x() A or
not. But C is a strict subset of the uncountable index set I, so fixing some values
x(tk ) for all tk C, the function x() on I still may or may not be in A , as
by definition the latter further requires that x(t) for all t I \ C. Similarly, if
C(I) B I then it has a countable base C = {tk } and the values of x(tk ) determine
whether x() C(I). However, since C 6= I, fixing x() continuous on I \ {t} with
t I \ C, the function x() may or may not be continuous on I, depending on the
value of x(t).

Remark. With A
/ B I , the canonical construction provides minimal information
about MI = suptI Xt , which typically is not even measurable with respect to
F X . However, note that A F X in case all sample functions of {Xt } are rightcontinuous. That is, for such S.P. the law of MI is uniquely determined by the f.d.d.
We return to this point in Section 7.2 when considering separable modifications.
Similarly, since C(I)
/ B I , the canonical construction does not assign a probability
for continuity of the sample function. To further demonstrate that this type of
difficulty is generic, recall that by our ad-hoc construction of the Poisson process
out of its jump times, all sample functions of this process are in
Z = {x ZI+ : t 7 x(t) is non-decreasing} ,
where I = [0, ). However, convince yourself that Z
/ B I , so had we applied the
canonical construction starting from the f.d.d. of the Poisson process, we would
not have had any probability assigned to this key property of its sample functions.
You are next to extend the phenomena illustrated by Lemma 7.1.10, providing a
host of relevant subsets of RI which are not in B I .
Exercise 7.1.11. Let I R denote an interval of positive length.
276
(a) Show that none of the following collections of functions is in B I : all linear
functions, all polynomials, all constants, all non-decreasing functions, all
functions of bounded variation, all differentiable functions, all analytic
functions, all functions continuous at a fixed t I.
(b) Show that B I fails to contain the collection of functions that vanish somewhere in I, the collection of functions such that x(s) < x(t) for some
s < t, and the collection of functions with at least one local maximum.
(c) Show that C(I) has no non-empty subset A B I , but the complement of
C(I) in RI has a non-empty subset A B I .
I
(d) Show that the completion B of B I with respect to any probability measure
P on B I fails to contain the set A = B(I) of all Borel measurable functions
x : I 7 R.
Hint: Consider A and Ac .
In contrast to the preceding exercise, independence of the increments of a S.P. is
determined by its f.d.d.
Exercise 7.1.12. A continuous time S.P. {Xt , t 0} has independent increments
if Xt+h Xt is independent of FtX = (Xs , 0 s t) for any h > 0 and all t 0.
Show that if Xt1 , Xt2 Xt1 , . . . , Xtn Xtn1 are mutually independent, for all
n < and 0 t1 < t2 < < tn < , then {Xt } has independent increments.
Hence, this property is determined by the f.d.d. of {Xt }.
Here is the canonical construction for Poisson random measures, where T is not a
subset of R (for example, the Poisson point processes where T = BRd ).
Exercise 7.1.13. Let T = {A X : (A) < } for a given measure space
(X, X , ). Construct a S.P. {NA : A T} such that NA has the Poisson((A)) law
for each A T and NAk , k = 1, . . . , n are P-mutually independent whenever Ak ,
k = 1, . . . , n are disjoint sets.
c
Hint: Given Aj T, j = 1, 2, let Bj1 = Aj = Bj0
and Nb1 ,b2 , for b1 , b2 {0, 1}
such that (b1 , b2 ) 6= (0, 0), be independent R.V. of Poisson((B1b1 B2b2 )) law. As
the distribution of (NA1 , NA2 ) take the joint law of (N1,1 + N1,0 , N1,1 + N0,1 ).
bt of rate one is merely the restriction to sets A =
Remark. The Poisson process N
[0, t], t 0, of the Poisson random measure {NA } in case () is Lebesgues measure
on [0, ). More generally, in case () has density f () with respect to Lebesgues
measure on [0, ), we call such restriction Xt = N[0,t] the inhomogeneous Poisson
process of rate function f (t) 0, t 0. It is a counting process of independent
b([0,t]) of a Poisson process
increments, which is a non-random time change Xt = N
of rate one, but in general the gaps between jump times of {Xt } are neither i.i.d.
nor of exponential distribution.
7.2. Continuous and separable modifications
The canonical construction of Section 7.1 determines the law of a S.P. {Xt } on the
image B T of F X . Recall that F X is inadequate as far as properties of the sample
functions t 7 Xt () are concerned. Nevertheless, a typical patch of this approach
is to choose among S.P. with the given f.d.d. one that has regular enough sample
functions. To illustrate this, we start with a simple explicit example in which path
properties are not entirely determined by the f.d.d.
7.2. CONTINUOUS AND SEPARABLE MODIFICATIONS
Example 7.2.1. consider the S.P.

Yt () = 0,
t,
Xt () =
277
1, t =
0, otherwise
on the probability space ([0, 1], B[0,1] , U ), with U the uniform measure on I = [0, 1].
Since At = {S: Xt () 6= Yt ()} = {t}, clearly P(Xt = Yt ) = 1 for each fixed t I.
Moreover, P( ni=1 Ati ) = 0 for any t1 , . . . , tn I, hence {Xt } has the same f.d.d.
as {Yt }. However, P({ : suptI Xt () 6= 0}) = 1, whereas P({ : suptI Yt () 6=
0}) = 0. Similarly, P({ : X () C(I)}) = 0, whereas P({ : Y () C(I)}) = 1.
While the two S.P. of Example 7.2.1 have different maximal value and differ in
their sample path continuity, we would typically consider one to be merely a (small)
modification of the other, motivating our next definition.
Definition 7.2.2. Stochastic processes {Xt } and {Yt } are called versions of one
another if they have the same f.d.d. A S.P. {Yt , t T} is further called a modification of {Xt , t T} if P(Xt 6= Yt ) = 0 for all t T and two such S.P. are
called indistinguishable if { : Xt () 6= Yt () for some t T} is a P-null set
(hence, upon completing the space, P(Xt 6= Yt for some t T) = 0). Similarly
to Definition 1.2.8, throughout we consider two indistinguishable S.P.-s to be the
same process, hence often omit the qualifier a.s. in reference to sample function
properties that apply for all t T.
For example, {Yt } is the continuous modification of {Xt } in Example 7.2.1 but
these two processes are clearly distinguishable. In contrast, modifications with a.s.
right-continuous sample functions are indistinguishable.
Exercise 7.2.3. Show that continuous time S.P.-s {Xt } and {Yt } which are modifications of each other and have w.p.1. right-continuous sample functions, must
also be indistinguishable.
You should also convince yourself at this point that as we have implied, if {Yt } is
a modification of {Xt }, then {Yt } is also a version of {Xt }. The converse fails, for
while a modification has to be defined on the same probability space as the original
S.P. this is not required of versions. Even on the same probability space it is easy
to find a pair of versions which are not modifications of each other.
Example 7.2.4. For the uniform probability measure on the finite set = {H, T },
the constant in time S.P.-s Xt () = IH () and Yt () = 1 Xt () are clearly
versions of each other but not modifications of each other.
We proceed to derive a relatively easy to check sufficient condition for the existence
of a (continuous) modification of the S.P. which has H
older continuous sample
functions, as defined next.
Definition 7.2.5. Recall that a function f (t) on a metric space (T, d(, )) is
locally -H
older continuous if
|f (t) f (s)|
cu ,
sup
d(t, s)
{t6=s,d(t,u)d(s,u)<hu }
for > 0, some c : T 7 [0, ) and h : T 7 (0, ], and is uniformly -Holder continuous if the same applies for constant c < h = . In case = 1 such functions
are also called locally (or uniformly) Lipschitz continuous, respectively. We say
278
that a S.P. {Yt , t T} is locally/uniformly -H

older/Lipschitz continuous, with
respect to a metric d(, ) on T if its sample functions t 7 Yt () have the corresponding property (for some S.P cu () < and hu () > 0, further requiring c to
be a non-random constant for uniform continuity). Since local H
older continuity
implies continuity, clearly then P({ : Yt () C(T)}) = 1. That is, such processes
have continuous sample functions. We also use the term continuous modification
et } of a given S.P. {Xt } such that {X
et } has continuto denote a modification {X
ous sample functions (and similarly define locally/uniformly -Holder continuous
modifications).
Remark. The Euclidean norm d(t, s) = kt sk is used for sample path continuity
of a random field, namely, where T Rr for some finite r, taking d(t, s) = |t s|
for a continuous time S.P. Also, recall that for compact metric space (T, d) there is
no difference between local and uniform H
older continuity of f : T 7 R, so in this
case local -H
older continuity of S.P. {Yt , t T} is equivalent to
|Yt () Ys ()|
c()}) = 1 ,
d(t, s)
s6=tT
P({ : sup
for some finite R.V. c().
Theorem 7.2.6 (Kolmogorov-Centsov continuity theorem). Suppose {Xt } is a

S.P. indexed on T = Ir , with I a compact interval. If there exist positive constants
, and finite c such that
(7.2.1)
E[|Xt Xs | ] ckt skr+ ,
for all
s, t T ,
then there exists a continuous modification of {Xt , t T} which is also locally

-H
older continuous for any 0 < < /.
Remark. Since condition (7.2.1) involves only the joint distribution of (Xs , Xt ),
it is determined by the f.d.d. of the process. Consequently, either all versions of
the given S.P. satisfy (7.2.1) or none of them does.
Proof. We consider hereafter the case of r = 1, assuming with no loss of
generality that T = [0, 1], and leave to the reader the adaptation of the proof to
r 2 (to this end, see [KaS97, Solution of Problem 2.2.9]).
Our starting point is the bound
(7.2.2)
P(|Xt Xs | ) E[|Xt Xs | ] c |t s|1+ ,
which holds for any > 0, t, s I, where the first inequality follows from Markovs
inequality and the second from (7.2.1). From this bound we establish the a.s.
(2)
local H
older continuity of the sample function of {Xt } over the collection Q1 =
S
(2,)
(2,)
of dyadic rationals in [0, 1], where QT
= {j2 T, j Z+ }. To
1 Q1
this end, fixing < / and considering (7.2.2) for = 2 , we have by finite
sub-additivity that
2 1
P(max |X(j+1)2 Xj2 | 2 ) c2 ,

j=0
P
for = > 0. Since 2 is finite, it then follows by Borel-Cantelli I that
2 1
max |X(j+1)2 Xj2 | < 2 ,

j=0
n () ,
where n () is finite for all

/ N and N F has zero probability.
279
As you show in Exercise 7.2.7 this implies the local -Holder continuity of t 7
Xt () over the dyadic rationals. That is,
(7.2.3)
|Xt () Xs ()| c()|t s| ,

(2)
for c() = 2/(1 2 ) finite and any t, s Q1 such that |t s| < h () = 2n () .

et , t T}, we fix k / and set N = k N .
Turning to construct the S.P. {X
k
es () = Xs ()IN c () for s Q(2) , we further set X
et =
Considering the R.V. X
1
esn for some non-random {sn } Q(2) such that sn t [0, 1] \ Q(2) .
limn X
1
1
es () over Q(2) , the
Indeed, in view of (7.2.3), by the uniform continuity of s 7 X
1
esn () is Cauchy, hence convergent, per . By construction,
sequence n 7 X
et , t [0, 1]} is such that
the S.P. {X
et () X
es ()| ck |t s|k ,
|X
for any k and t, s [0, 1] such that |t s| < e

hk (), where e
hk = IN + INc hk is
et } is locally -Holder
positive for all and ck = c(k ) is finite. That is, {X
continuous on T = [0, 1] for any k , hence also for all < / (and in particular,
et , t [0, 1]} has continuous sample functions).
{X
et } is a modification of {Xt }. To this end,
It thus remains only to verify that {X
es () =
observe first that since P(Nk ) = 0 for all k, also P(N ) = 0. Further, X
(2)
Xs () for all s Q1 and
/ N . Next, from (7.2.2) we have that P(|Xt Xsn |
p
) 0 for any fixed > 0 and sn t, that is, Xsn Xt . Hence, recall Theorem
a.s.
2.2.10, also Xsn(k) Xt along some subsequence k 7 n(k). Considering an
(2)
(2)
et },
arbitrary t [0, 1] \ Q1 and the sequence sn Q1 as in the construction of {X
e
e
e
we have in addition that Xsn(k) Xt . Consequently, P(Xt 6= Xt ) P(N ) = 0
et } is a modification of {Xt } on T = [0, 1].
from which we conclude that {X

Exercise 7.2.7. Fixing x R[0,1] , let
2 r
,r (x) = max |x((j + r)2 ) x(j2 )| .

j=0
(a) Show that for any integers k > m 0,

sup
(2,k)
t,sQ
1
|ts|<2m
|x(t) x(s)| 2
k
X
,1 (x) .
=m+1
Hint: Applying induction on k consider s < t and s s t t, where

(2,k1)
(2,k1)
s = min{u Q1
: u s} and t = max{u Q1
: u t}.
(b) Fixing > 0, let c = 2/(1 2 ) and deduce that if ,1 (x) 2 for
all n, then
|x(t) x(s)| c |t s|
(2)
for all t, s Q1
such that |t s| < 2n .
Hint: Apply part (a) for m n such that 2(m+1) |t s| < 2m .

We next identify the restriction of the cylindrical -algebra of RT to C(T) as the
Borel -algebra on the space of continuous functions, starting with T = Ir for I a
compact interval.
280
Lemma 7.2.8. For T = Ir and I R a compact interval, consider the topological

space (C(T), kk ) of continuous functions on T, equipped with the topology induced
by the supremum norm kxk = suptT |x(t)|. The corresponding Borel -algebra,
denoted hereafter BC(T) coincides with {A C(T) : A B T }.
Proof. Recall that for any z C(T),
kzk = sup |z(t)| .

tTQr
Hence, each open ball

B(x, r) = {y C(T) : ky xk < r}
in S = (C(T), k k ) is the countable intersection of Rt C(T) for the corresponding one dimensional measurable rectangles Rt B T indexed by t T Qr .
Consequently, each open ball B(x, r) is in the -algebra C = {A C(T) : A B T }.
With denoting a countable dense subset of the separable metric space S, it readily
follows that S has a countable base U, consisting of the balls B(x, 1/n) for positive
integers n and centers x . With every open set thus being a countable union of
elements from U, it follows that BS = (U). Further, U C, hence also BS C.
Conversely, recall that C = (O) for the collection O of sets of the form
O = {x C(T) : x(ti ) Oi , i = 1, . . . , n} ,
with n finite, ti T and open Oi R, i = 1, . . . , n. Clearly, each O O is an open

subset of S and it follows that C BS .

In the next exercise, you adapt the proof of Lemma 7.2.8 for T = [0, ) (and the
same would apply for T Rr which is the product of one-dimensional intervals).
Exercise 7.2.9. For T = [0, ), equip the set C(T) of continuous functions on T
with the topology of uniform convergence on compact subsets of T. Show that the
corresponding Borel -algebra BC(T) coincides with {A C(T) : A B T }.
Hint: Uniform convergence on compacts is equivalent to convergence
comP in the
j
plete, separable metric space S = (C([0, )), (, )), where (x, y) =
(kx
j=1 2
ykj ) for kxkt = sups[0,t] |x(s)| and (r) = r/(1 + r) (c.f. [Dud89, Page 355]).
Combining Proposition 7.1.8, Exercise 7.2.3, Theorem 7.2.6 and Lemma 7.2.8,
yields the following useful canonical construction for continuous-time processes of
a.s. continuous sample path.
Corollary 7.2.10. Given a consistent collection of f.d.d. indexed on T = Ir (with
I R a compact interval), such that (7.2.1) holds (for some positive , and finite
e () : 7 (C(T), k k ), measurable with respect to BC(T) ,
c), there exists a S.P. X
which has the specified f.d.d. and is indistinguishable from any of its continuous
modifications.
Remark. An alternative approach is to directly construct the sample functions
of stochastic processes of interest. That is, to view the process from the start as a
random variable 7 X () taking values in certain topological space of functions
equipped with its Borel -algebra (for example, the space C(T) with a suitable
metric). In dealing with the Brownian motion, we pursue both approaches, first
relying on the canonical construction (namely, Corollary 7.2.10), and then proving
instead an invariance principle via weak convergence in C(T) (c.f. Section 9.2).
281
In contrast with Theorem 7.2.6, here is an example of a S.P. with no continuous

modification, for which (7.2.1) holds with = 0.
Example 7.2.11. Consider the S.P. Xt () = I{>t} , for t [0, 1] and the uniform
probability measure on = (0, 1]. Then, E[|Xt Xs | ] = U ((s, t]) = |t s| for
all 0 < s < t 1, so {Xt , t [0, 1]} satisfies (7.2.1) with c = 1, = 0 and any
et } is a modification of {Xt } then a.s. X
et () = Xt () at all
> 0. However, if {X
e
t (0, 1] Q, from which it follows that s 7 Xs () is discontinuous at s = .
While direct application of Theorem 7.2.6 is limited to (locally -Holder) continuous modifications on compact intervals, say [0, T ], it is easy to combine these to
one (locally -H
older) continuous modification, valid on [0, ).
Lemma 7.2.12. Suppose there exist Tn such that the continuous time S.P.
et(n) , t [0, Tn ]}.
{Xt , t 0} has (locally -H
older) continuous modifications {X
Then, the S.P. {Xt , t 0} also has such modification on [0, ).
Proof. By assumption, for each positive integer n, the event
(n)
e () = Xt (),
An = { : X
t
t Q [0, Tn ]} ,
has probability one. The event A = n An of probability one is then such that
et(n) () = X
et(m) () for all A , positive integers n, m and any t Q[0, Tn Tm ].
X
e (n) () and t 7 X
e (m) () it follows that for all A ,
By continuity of t 7 X
t
t
et(n) () = X
et(m) () ,
X
n, m, t [0, Tn Tm ] .
et () on [0, ) that coinConsequently, for such there exists a function t 7 X

(n)
e
cides with each of the functions Xt () on its interval of definition [0, Tn ]. By
assumption the latter are (locally -Holder) continuous, so the same applies for the
et () on [0, ). Setting X
et () 0 in case
sample function t 7 X
/ A completes
et , t 0} with (locally -Holder) continuous sample
the construction of the S.P. {X
functions, such that for any t [0, Tn ],
et ) P(Ac ) + P(Xt 6= X
e (n) ) = 0 .
P(Xt 6= X
t
Since Tn , we conclude that this S.P. is a (locally -Holder) continuous modification of {Xt , t 0}.

The following application of Kolmogorov-Centsov theorem demonstrates the importance of its free parameter .
Exercise 7.2.13. Suppose {Xt , t I} is a continuous time S.P. such that E(Xt ) =
0 and E(Xt2 ) = 1 for all t I, a compact interval on the line.
(a) Show that if for some finite c, p > 1 and h > 0,
(7.2.4)
E[Xt Xs ] 1 c(t s)p
for all s < t s + h, t, s I ,
then there exists a continuous modification of {Xt , t I} which is also

locally -H
older continuous, for < (p 1)/2.
(b) Show that if (Xs , Xt ) is a multivariate normal for each t > s, then it
suffices for the conclusion of part (a) to have E[Xt Xs ] 1 c(t s)p1
instead of (7.2.4).
Hint: In part (a) use = 2 while for part (b) try = 2k and k 1.
282
Example 7.2.14. There exist S.P.-s satisfying (7.2.4) with p = 1 for which there
is no continuous modification. One such process is the random telegraph signal
Rt = (1)Nt R0 , where P(R0 = 1) = P(R0 = 1) = 1/2 and R0 is independent of
the Poisson process {Nt } of rate one. The process {Rt } alternately jumps between
1 and +1 at the random jump times {Tk } of the Poisson process {Nt }. Hence, by
the same argument as in Example 7.2.11 it does not have a continuous modification.
Further, for any t > s 0,
E[Rs Rt ] = 1 2P(Rs 6= Rt ) 1 2P(Ns < Nt ) 1 2(t s) ,
so {Rt } satisfies (7.2.4) with p = 1 and c = 2.
Remark. The S.P. {Rt } of Example 7.2.14 is a special instance of the continuoustime Markov jump processes, which we study in Section 8.3.3. Though the sample
function of this process is a.s. discontinuous, it has the following RCLL property,
as is the case for all continuous-time Markov jump processes.
Definition 7.2.15. Given a countable C I we say that a function x RI is
C-separable at t if there exists a sequence sk C that converges to t such that
x(sk ) x(t). If this holds at all t I, we call x() a C-separable function. A
continuous time S.P. {Xt , t I} is separable if there exists a non-random, countable
C I such that all sample functions t 7 Xt () are C-separable. Such a process is
further right-continuous with left-limits (in short, RCLL), if the sample function
t 7 Xt () is right-continuous and of left-limits at any t I (that is, for h 0
both Xt+h () Xt () and the limit of Xth () exists). Similarly, a modification
which is a separable S.P. or one having RCLL sample functions is called a separable
modification, or RCLL modification of the S.P., respectively. As usual, suffices to
have any of these properties w.p.1 (for we do not differentiate between a pair of
indistinguishable S.P.).
Remark. Clearly, a S.P. of continuous sample functions is also RCLL and a S.P.
having right-continuous sample functions (in particular, any RCLL process), is
further separable. To summarize,
H
older continuity Continuity RCLL Separable
But, the S.P. {Xt } of Example 7.2.1 is non-separable. Indeed, C-separability of

t 7 Xt () at t = requires that C, so for any countable subset C of [0, 1] we
have that P(t 7 Xt is C-separable) P(C) = 0.
One motivation for the notion of separability is its prevalence. Namely, to any
consistent collection of f.d.d. indexed on an interval I, corresponds a separable
S.P. with these f.d.d. This is achieved at the small cost of possibly moving from
real-valued variables to R-valued variables (each of which is nevertheless a.s. realvalued).
Proposition 7.2.16. Any continuous time S.P. {Xt , t I} admits a separable
modification (consisting possibly of R-valued variables). Hence, to any consistent
I
collection of f.d.d. indexed on I corresponds an (R , (BR )I )-valued separable S.P.
with these f.d.d.
We prove this proposition following [Bil95, Theorem 38.1], but leave its technical
engine (i.e. [Bil95, Lemma 1, Page 529]), as your next exercise.
Exercise 7.2.17. Suppose {Yt , t I} is a continuous time S.P.
283
(a) Fixing B B, consider the probabilities p(D) = P(Ys B for all s D),
for countable D I. Show that for any A I there exists a countable
subset D = D (A, B) of A such that p(D ) = inf{p(D) : countable
D A}.
Hint: Let D = k Dk where p(Dk ) k 1 +inf{p(D) : countable D A}.
(b) Deduce that if t A then Nt (A, B) = { : Ys () B for all s D (A, B)
and Yt ()
/ B} has zero probability.
(c) Let C denote the union of D (A, B) over all A = I (q1 , q2 ) and B =
(q3 , q4 )c , with qi Q. Show that at any t I there exists Nt F such
that P(Nt ) = 0 and the sample functions t 7 Yt () are C-separable at t
for every
/ Nt .
Hint: Let Nt denote the union of Nt (A, B) over the sets (A, B) as in the
definition of C, such that further t A.
Proof. Assuming first that {Yt , t I} is a (0, 1)-valued S.P. set Ye = Y on

the countable, dense C I of part (c) of Exercise 7.2.17. Then, fixing non-random
{sn } C such that sn t I \ C we define the R.V.-s
Yet = Yt INtc + INt lim sup Ysn ,
n
for the events Nt of zero probability from part (c) of Exercise 7.2.17. The resulting
S.P. {Yet , t I} is a [0, 1]-valued modification of {Yt } (since P(Yet 6= Yt ) P(Nt ) = 0
for each t I). It clearly suffices to check C-separability of t 7 Yet () at each fixed
t
/ C and this holds by our construction if Nt and by part (c) of Exercise
7.2.17 in case Ntc . For any (0, 1)-valued S.P. {Yt } we have thus constructed a
separable [0, 1]-valued modification {Yet }. To handle an R-valued S.P. {Xt , t I},
let {Yet , t I} denote the [0, 1]-valued, separable modification of the (0, 1)-valued
S.P. Yt = FG (Xt ), with FG () denoting the standard normal distribution function.
Since FG () has a continuous inverse FG1 : [0, 1] 7 R (where FG1 (0) = and
et = F 1 (Yet ) is an R-valued separable
FG1 (1) = ), it directly follows that X
G
modification of the S.P. {Xt }.

Here are few elementary and useful consequences of separability.
Exercise 7.2.18. Suppose S.P. {Xt , t I} is C-separable and J I with J an

open interval.
(a) Show that
sup Xt = sup Xt
tJ
tJC
is in mF , hence its law is determined by the f.d.d.

(b) Similarly, show that for any h > 0 and s I,
sup
t[s,s+h)
|Xt Xs | =
sup
t[s,s+h)C
|Xt Xs |
is in mF X with its law determined by the f.d.d.

The joint measurability of sample functions is an important property to have.
Definition 7.2.19. A continuous time S.P. {Xt , t I} is measurable if Xt () :
I 7 R is measurable with respect to B I F (that is, for any B B, the
subset {(t, ) : Xt () B} of I is in B I F , where as usual B I denotes the
284
completion of the Borel -algebra with respect to Lebesgues measure on I and F is

the completion of F with respect to P).
As we show in Proposition 8.1.8, any right-continuous S.P. (and in particular,
RCLL), is also measurable. While separability does not imply measurability, building on the obvious measurability of (simple) RCLL processes, following the proof
of [Doo53, Theorem II.2.6] we show next that to any consistent and continuous
in probability collection of f.d.d. corresponds a both separable and measurable S.P.
having the specified f.d.d.
Definition 7.2.20. A S.P. {Xt , t I} is continuous in probability if for any t I
and > 0,
lim P(|Xs Xt | > ) = 0 .
st
Remark. Continuity in probability is a very mild property, which is completely

determined by the f.d.d. and has little to do with the sample functions of the
process. For example, note that the Poisson process is continuous in probability, as
are the random telegraph noise {Rt } of Example 7.2.14 and even the non-separable
S.P. {Xt } of Example 7.2.1.
Proposition 7.2.21. Any continuous in probability process {Xt , t I} has an
I
(R , (BR )I )-valued separable modification which is further a measurable process.
Proof. It suffices to consider I = [0, 1]. Indeed, by an affine time change the
same proof then applies for any compact interval I, and if I is unbounded, simply
decompose it to countably many disjoint bounded intervals and glue together the
corresponding separable and measurable modifications of the given process.
Further, in view of Proposition 7.2.16 and the transformation via FG () we have
utilized in its proof, we consider with no loss of generality a (0, 1)-valued {sk }separable, continuous in probability S.P. {Yt , t [0, 1]} and provide a [0, 1]-valued
measurable modification {Yet } of {Yt }, which we then verify to be also a separable
process. To this end, with no loss of generality, assume further that s1 = 0. Then,
for any n N set tn+1 = 2 and with 0 = t1 < < tn the monotone increasing
rearrangement of {sk , k = 1, . . . , n}, consider the [0, 1]-valued, RCLL stochastic
process
n
X
(n)
Ytj I[tj ,tj+1 ) (t) ,
Yt =
j=1
which is clearly also a measurable S.P. By the denseness of {sk } in [0, 1], it follows
(n) p
from the continuity in probability of {Yt } that Yt Yt as n , for any fixed
(n)
(m)
t [0, 1]. Hence, by bounded convergence E[|Yt Yt |] 0 as n, m for
each t [0, 1]. Then, by yet another application of bounded convergence
(n)
lim E[|YT
m,n
(m)
YT
|] = 0 ,
where the R.V. T [0, 1] is chosen independently of P, according to the uniform probability measure U corresponding to Lebesgues measure () restricted
(n)
to ([0, 1], B[0,1] ). By Fubinis theorem, this amounts to {Yt ()} being a Cauchy,
hence convergent, sequence in L1 ([0, 1] , B [0,1] F , U P) (recall Proposition
4.3.7 that the latter is a Banach space). In view of Theorem 2.2.10, upon passing
(n )
to a suitable subsequence nj we thus have that (t, ) 7 Yt j () converges to some
285
()
B [0,1] F -measurable function (t, ) 7 Yt () for all (t, )

/ N , where we may
and shall assume that {sk } N B [0,1] F and U P(N ) = 0. Taking now
()
Yet () = IN c (t, )Yt () + IN (t, )Yt () ,
()
()
note that Yet () = Yt () for a.e. (t, ), so with Yt () a measurable process, by

the completeness of our product -algebra, the S.P. {Yet , t [0, 1]} is also measur(n )
able. Further, fixing t [0, 1], if {Yet () 6= Yt ()} then At = { : Yt j ()
(n ) p
()
Yt () 6= Yt ()}. But, recall that Yt j Yt for all t [0, 1], hence P(At ) = 0,
i.e. {Yet , t [0, 1]} is a modification of the given process {Yt , t [0, 1]}.
Finally, since {Yet } coincides with the {sk }-separable S.P. {Yt } on the set {sk }, the
sample function t 7 Yet () is, by our construction, {sk }-separable at any t [0, 1]
(n )
such that (t, ) N . Moreover, Yt j = Ysk = Yesk for some k = k(j, t), with
sk(j,t) t by the denseness of {sk } in [0, 1]. Hence, if (t, )
/ N then
(n )
Yet () = lim Yt j () = lim Yesk(j,t) () .
j
Thus, {Yet , t [0, 1]} is {sk }-separable and as claimed, it is a separable, measurable
modification of {Yt , t [0, 1]}.
Recall (1.4.7) that the measurability of the process, namely of (t, ) 7 Xt (),
implies that all its sample functions t 7 Xt () are Lebesgue measurable functions
on I. Measurability of a S.P. also results with well defined integrals
R of its sample
function. For example, if a Borel function h(t, x) is such that I E[|h(t, Xt )|]dt
1
is
R finite, then by Fubinis theorem t 7 E[h(t, Xt )] is in L (I, BI , ), the integral
h(s, Xs )ds is an a.s. finite R.V. and
I
Z
Z
E[h(s, Xs )] ds = E[ h(s, Xs ) ds] .
I
Conversely, as you are to show next, under mild conditions the differentiability of
sample functions t 7 Xt implies the differentiability of t 7 E[Xt ].
Exercise 7.2.22. Suppose each sample function t 7 Xt () of a continuous time

S.P. {Xt , t I} is differentiable at any t I.
(a) Verify that t

Xt is a random variable for each fixed t I.
(b) Show that if |Xt Xs | |t s|Y for some integrable random variable Y ,
a.e. and all t, s I, then t 7 E[Xt ] has a finite derivative and for
any t I,

d
E(Xt ) = E
Xt .
dt
t
We next generalize the lack of correlation of independent R.V. to the setting of
continuous time S.P.-s.
Exercise 7.2.23. Suppose square-integrable, continuous time S.P.-s {Xt , t I}
and {Yt , t I} are P-independent. That is, both processes are defined on the same
probability space and the -algebras F X and F Y are P-independent. Show that in
this case,
E[Xt Yt |FsZ ] = E[Xt |FsX ]E[Yt |FsY ] ,
for any s t I, where Zt = (Xt , Yt ) R2 , FsX = (Xu , u I, u s) and FsY ,
FsZ are similarly defined.
286
7.3. Gaussian and stationary processes

Building on Definition 3.5.13 of Gaussian random vectors, we have the following
important class of (centered) Gaussian (stochastic) processes, which plays a key
role in our construction of the Brownian motion.
Definition 7.3.1. A S.P. {Xt , t T} is a Gaussian process (or Gaussian S.P.),
if (Xt1 , . . . , Xtn ) is a Gaussian random vector for any n finite and tk T, k =
1, . . . , n.
Alternatively, a S.P. is Gaussian if and only if it has multivariate
normal f.d.d. We further say that a Gaussian S.P. is centered if its mean function
m(t) = E[Xt ] is zero.
Recall the following notion of non-negative definiteness, based on Definition 3.5.12.
Definition 7.3.2. A symmetric function c(t, s) = c(s, t) on a product set T T is
called non-negative definite (or positive semidefinite) if for any finite n and tk T,
k = 1, . . . , n, the n n matrix of entries c(tj , tk ) is non-negative definite. That is,
for any ak R, k = 1, . . . , n,
n X
n
X
(7.3.1)
aj c(tj , tk )ak 0.
j=1 k=1
Example 7.3.3. Note that the auto-covariance function c(t, s) = Cov(Xt , Xs ) of

a square-integrable S.P. {Xt , t T} is non-negative definite.
Indeed, the left side
Pn
of (7.3.1) is in this case precisely the non-negative Var( j=1 aj Xtj ).
Convince yourself that non-negative definiteness is the only property that the autocovariance function of a Gaussian S.P. must have and further that the following is
an immediate corollary of the canonical construction and the definitions of Gaussian
random vectors and stochastic processes.
Exercise 7.3.4.
(a) Show that for any index set T, the law of a Gaussian S.P. is uniquely
determined by its mean and auto-covariance functions.
(b) Show that a Gaussian S.P. exists for any mean function and any nonnegative definite auto-covariance function.
Remark. An interesting consequence of Exercise 7.3.4 is the existence of an
isonormal process on any vector space H equipped with an inner product as in Definition 4.3.5. That is, a centered Gaussian process {Xh , h H} indexed by elements
of H whose auto-covariance function is given by the inner product (hP
1 , h2 ) : HH 7
n
R. Indeed, the latter is non-negative definite on HH since for h = j=1 aj hj H,
n X
n
X
j=1 k=1
aj (hj , hk )ak = (h, h) 0 .
One of the useful properties of Gaussian processes is their closure with respect to
L2 -convergence (as a consequence of Proposition 3.5.15).
(k)
Proposition 7.3.5. If the S.P. {Xt , t T} and the Gaussian S.P. {Xt , t T}
(k)
are such that E[(Xt Xt )2 ] 0 as k , for each fixed t T, then {Xt , t T}
is a Gaussian S.P. whose mean and auto-covariance functions are the pointwise
(k)
limits of those for the processes {Xt , t T}.
7.3. GAUSSIAN AND STATIONARY PROCESSES
287
Proof. Fix n finite and tk T, k = 1, . . . , n. Applying Proposition 3.5.15 for

(k)
(k)
the sequence of Gaussian random vectors X k = (Xt1 , . . . , Xtn ), we deduce that
X = (Xt1 , . . . , Xtn ) is also a Gaussian random vector whose mean and covariance
parameters (, V) are the element-wise limits of the parameters of the sequence of
random vectors {X k }. With this holding for all f.d.d. of the S.P. {Xt , t T},
by Definition 7.3.1 the latter is a Gaussian S.P. (of the stated mean and autocorrelation functions).

Recall Exercise 5.1.9, that for a Gaussian random vector (Yt2 Yt1 , . . . , Ytn Ytn1 ),
with n finite and t1 < t2 < < tn , having independent coordinates is equivalent
to having uncorrelated coordinates. Hence, from Exercise 7.1.12 we deduce that
Corollary 7.3.6. A continuous time, Gaussian S.P. {Yt , t I} has independent
increments if and only if Cov(Yt Yu , Ys ) = 0 for all s u < t I.
Remark. Check that the zero covariance condition in this corollary is equivalent
to the Gaussian process having auto-covariance function of the form c(t, s) = g(ts).
Recall Definition 6.1.20 that a discrete time stochastic process {Xn } with a Bisomorphic state space (S, S), is (strictly) stationary if its law PX is shift invariant,
namely, PX 1 = PX for the shift operator ()k = k+1 on S . This concept
of invariance of the law of the process to translation of time, extends naturally to
continuous time S.P.
Definition 7.3.7. The (time) shifts s : S[0,) S[0,) are defined for s 0 via
s (x)() = x( + s) and a continuous time S.P. {Xt , t 0} is called stationary (or
strictly stationary), if its law PX is invariant under any time shift s , s 0. That
is, PX (s )1 = PX for all s 0. For two-sided continuous time S.P. {Xt , t R}
the definition of time shifts extends to s R and stationarity is then the invariance
of the law under s for any s R.
Recall Proposition 7.1.8 that the law of a continuous time S.P. is uniquely determined by its f.d.d. Consequently, such process is (strictly) stationary if and only if
its f.d.d. are invariant to translation of time. That is, if and only if
(7.3.2)
(Xt1 , . . . , Xtn ) = (Xt1 +s , . . . , Xtn +s )
for any n finite and s, ti 0 (or for any s, ti R in case of a two-sided continuous
time S.P.). In contrast, here is a much weaker concept of stationarity.
Definition 7.3.8. A square-integrable continuous time S.P. of constant mean
function and auto-covariance function of the form c(t, s) = r(|ts|) is called weakly
stationary (or L2 -stationary).
Indeed, considering (7.3.2) for n = 1 and n = 2, clearly any square-integrable
stationary S.P. is also weakly stationary. As you show next, the converse fails in
general, but applies for all Gaussian S.P.
Exercise 7.3.9. Show that any weakly stationary Gaussian S.P. is also (strictly)
stationary. In contrast, provide an example of a (non-Gaussian) weakly stationary
process which is not stationary.
To gain more insight about stationary processes solve the following exercise.
288
Exercise 7.3.10. Suppose {Xt , t 0} is a centered weakly stationary S.P. of

auto-covariance function r(t).
(a) Show that |r(t)| r(0) for all t > 0 and further, if r(h) = r(0) for some
a.s.
h > 0 then Xt+h = Xt for each t 0.
(b) Deduce that any centered, weakly stationary process of independent increments must be a modification of the trivial process having constant
sample functions Xt () = X0 () for all t 0 and .
Definition 7.3.11. We say that a continuous time S.P. {Xt , t I} has stationary
increments if for t, s I the law of the increment Xt Xs depends only on t s.
We conclude this chapter with the definition and construction of the celebrated
Brownian motion which is the most fundamental continuous time stochastic process.
Definition 7.3.12. A S.P. {Wt , t 0} is called a Brownian motion (or a Wiener
process) starting at x R, if it is a Gaussian process of mean function m(t) = x and
auto-covariance c(t, s) = Cov(Wt , Ws ) = t s, whose sample functions t 7 Wt ()
are continuous. The case of x = 0 is called the standard Brownian motion (or
standard Wiener process).
In addition to constructing the Brownian motion, you are to show next that it has
stationary, independent increments.
Exercise 7.3.13.
(a) Construct a continuous time Gaussian S.P. {Bt , t 0} of the mean and
auto-covariance functions of Definition 7.3.12
Hint: Look for f.d.d. such that B0 = x and having independent increments Bt Bs of zero mean and variance t s.
(b) Show that there exists a Wiener process, namely a continuous modification {Wt , t 0} of {Bt , t 0}.
Hint: Try Kolmogorov-Centsov theorem for = 4.
(c) Deduce that for any T finite, the S.P. {Wt , t [0, T ]} can be viewed as
the random variable W : (, F ) 7 (C([0, T ]), kk ), which is measurable
with respect to the Borel -algebra on C([0, T ]) and is further a.s. locally
-H
older continuous for any < 1/2.
Hint: As in Exercise 7.2.13 try = 2k with k 1 in (7.2.1).
(d) Show that the S.P. {Bt , t 0} is non-stationary, but it is a process of
stationary, independent increments.
Example 7.3.14. Convince yourself that every stationary process has stationary
increments while the Brownian motion of Exercise 7.3.13 is an example of a nonstationary process with stationary (independent) increments. The same phenomena
applies for discrete time S.P. (in which case the symmetric srw serves as an example of a non-stationary process with stationary, independent increments).
An alternative construction of the Wiener process on I = [0, T ] is as the infinite
series
X
Wt = x +
ak (t)Gk ,
k=0
7.3. GAUSSIAN AND STATIONARY PROCESSES
289
Sample paths of Brownian motion

2.5
2
1.5
1
Wt
0.5
0
0.5
1
1.5
2
2.5
0
0.5
1.5
t
2.5
Figure 2. Three sample functions of Brownian motion. The density curves illustrate that the random variable W1 has a N (0, 1)
law, while W2 has a N (0, 2) law.
with {Gk } i.i.d. standard normal random variables and ak () continuous functions
on I such that
X
1
(7.3.3)
ak (t)ak (s) = t s = (|t + s| |t s|) .
2
k=0
For example, taking T = 1/2 and expanding f (x) = |x| for |x| 1 into a Fourier
series, one finds that
1 X
4
cos((2k + 1)x) .
|x| =
2
(2k + 1)2 2
k=0
Hence, by the trigonometric identity cos(ab)cos(a+b) = 2 sin(a) sin(b) it follows

that (7.3.3) holds for
2
sin((2k + 1)t) .
ak (t) =
(2k + 1)
Though we shall not do so, the continuity w.p.1. of t 7 Wt is then obtained by
showing that for any > 0
X
P(k
ak (t)Gk k ) 0
k=n
as n (see [Bry95, Theorem 8.1.3]).
We turn to explore some interesting Gaussian processes of continuous sample

functions that are derived out of the Wiener process {Wt , t 0}.
Exercise 7.3.15. With {Wt , t 0} a standard Wiener process, show that each
of the following is a Gaussian S.P. of continuous sample functions, compute its
290
mean and auto-covariance functions and determine whether or not it is a stationary

process.
bt = Wt min(t, 1)W1 .
(a) The standard Brownian bridge B
(b) The Ornstein-Uhlenbeck process Ut = et/2 Wet .
(r,)
(c) The Brownian motion with drift Zt
= Wt + rt + x, with non-random
drift r R and diffusion coefficient >R 0.
t
(d) The integrated Brownian motion It = 0 Ws ds.
Exercise 7.3.16. Suppose {Wt , t 0} is a standard Wiener process.
(a) Compute E(Ws |Wt ) and Var(Ws |Wt ), first for s > t, then for s < t.
a.s.
(b) Show that t1 Wt 0 when t .
Hint: As we show in the sequel, the martingale {Wt , t 0} satisfies
Doobs L2 maximal inequality.
et = (1 t)Wt/(1t) (with B
e1 = 0), has
(c) Show that for t [0, 1] the S.P. B
the same law as the standard Brownian bridge and its sample functions
are continuous w.p.1.
(d) Show that restricted to [0, 1], the law of the standard Brownian bridge
matches that of {Wt , t [0, 1], conditioned upon W1 = 0} (hence the
name Brownian bridge).
The fractional Brownian motion is another Gaussian S.P. of considerable interest

in financial mathematics and in the analysis of computer and queuing networks.
Exercise 7.3.17. For H (0, 1), the fractional Brownian motion (or in short,
fBM), of Hurst parameter H is the centered Gaussian S.P. {Xt , t 0}, of autocovariance function
1
c(t, s) = [|t|2H + |s|2H |t s|2H ] , s, t 0.
2
(a) Show that the square-integrability with respect to Lebesgues measure of
H1/2
g(u) = |1 u|H1/2 sgn(1
sgn(u) (which you need not verR u) + |u|
H1/2
ify), implies that c(t, s) = gt (x)gs (x)dx for gt (x) = kgk1
g(x/t)
2 |t|
in case t > 0 and g0 (x) = 0.
Hint: gt (s + x) gs (s + x) = gts (x), hence kgt gs k22 = kgts k22 .
(b) Deduce that the fBM {Xt , t 0} exists and has a continuous modification
which is also locally -H
older continuous for any 0 < < H.
(c) Verify that for H = 12 this modification is the standard Wiener process.
(d) Show that for any non-random b > 0, the S.P. {bH Xbt , t 0} is an
fBM of the same Hurst parameter H.
(e) For which values of H are the increments of the fBM stationary and for
which values are they independent?
Exercise 7.3.18. Let S denote the unit circle on the plane endowed with the
Borel -algebra and uniform probability measure Q (which is just the image of
[0, 1] equipped with Lebesgues measure, under t 7 ei2t S).
Construct a centered Gaussian stochastic process G(), indexed on the collection A
of sub-arcs of S, of auto-covariance function E[G(A)G(B)] = Q(AB)Q(A)Q(B)
such that its sample functions (s, u) 7 G((s, u))() are continuous with respect to
the Euclidean topology of T = [0, 1]2 \ {(s, s) : s [0, 1]}, where (s, u) denotes the
sub-arc A = {ei2t : s < t u} in case s < u, while (u, s) stands for Ac := S \ A.
bu B
bs for A = (s, u) such that 0 s < u 1.
Hint: Try G(A) = G(Ac ) = B
CHAPTER 8
Continuous time martingales and Markov

processes
Continuous time filtrations and stopping times are introduced in Section 8.1, emphasizing the differences with the corresponding notions for discrete time processes
and the connections to sample path continuity. Building upon it and Chapter 5
about discrete time martingales, we review in Section 8.2 the theory of continuous time martingales. Similarly, Section 8.3 builds upon Chapter 6 about Markov
chains, in providing a short introduction to the rich theory of strong Markov processes.
8.1. Continuous time filtrations and stopping times
We start with the definitions of continuous time filtrations and S.P. adapted to
them (compare with Definitions 5.1.1 and 5.1.2, respectively).
Definition 8.1.1. A (continuous time) filtration is a non-decreasing family of
sub--algebras {Ft } of the measurable space (, F ), indexed by t 0. By FtS F
we denote such filtration {Ft } and the associated minimal -algebra F = ( t Ft )
such that Fs Ft for all 0 s t .
Definition 8.1.2. A (continuous time) S.P. {Xt , t 0} is adapted to a (continuous time) filtration {Ft }, or in short Ft -adapted, if Xt mFt for each t 0 or
equivalently, if FtX Ft for all t 0.
Remark 8.1.3. To avoid cumbersome technical difficulties, we assume throughout
that the filtration is augmented so that every P-null set is in F0 . That is, if N A
for some A F with P(A) = 0 then N F0 (which is a somewhat stronger
assumption than the completion of both F and F0 ). In particular, this assures that
any modification of an Ft -adapted continuous time S.P. remains Ft -adapted.
When dealing with continuous time processes it helps if each new piece of information has a definite first time of arrival, as captured mathematically by the concept
of right-continuous filtration.
Definition 8.1.4. To any continuous time filtration {Ft } we associate the corresponding left-filtration Ft = (Fs , s < t) at time t, consisting
of all events prior
T
to t (where we set F0 = F0 ), and right-filtration Ft+ = >0 Ft+ at time t, consisting of all events immediately after t. A filtration {Ft } is called right-continuous
if it coincides with its right-filtration, that is Ft = Ft+ for all t 0.
The next example ties the preceding definitions to those in the much simpler
setting of Chapter 5.
291
292
8. CONTINUOUS TIME MARTINGALES AND MARKOV PROCESSES
Example 8.1.5. To each discrete time filtration {Gn , n Z+ } corresponds the

interpolated (continuous time) filtration Ft = G[t] , where [t] denotes the integer part
of t 0. Convince yourself that any interpolated filtration is right-continuous, but
usually not left-continuous. That is, Ft 6= Ft (at any t = n integer in which Gn 6=
Gn1 ), with each jump in the filtration accounting for a new piece of information
arriving at that time.
Similarly, we associate with any Gn -adapted discrete time S.P. {Yn } an interpolated continuous time S.P. Xt = Y[t] , t 0, noting that {Xt , t 0} is then
Ft -adapted if and only if {Yn } is Gn -adapted.
Example 8.1.6. In analogy with Definition 5.1.3, another generic continuous time
filtration is the canonical filtration FtX = (Xs , 0 s t) associated with each
continuous time S.P. {Xt , t 0}.
Unfortunately, sample path continuity of a S.P. {Xt } does not guarantee the rightcontinuity of its canonical filtration {FtX }. Indeed, considering the uniform probability measure on = {1, 1} note that the canonical filtration {FtX } of the
S.P. Xt () = t, which has continuous sample functions, is evidently not rightcontinuous at t = 0 (as F0X = {, } while FtX = F = 2 for all t > 0).
When S.P. {Xs , s 0} is Ft -adapted, we can view {Xs , s [0, t]} as a S.P. on the
smaller measurable space (, Ft ), for each t 0. However, as seen in Section 7.2,
more is required in order to have Borel sample functions, prompting the following
extension of Definition 7.2.19 (and refinement of Definition 8.1.2).
Definition 8.1.7. An Ft -adapted S.P. {Xt , t 0} is called Ft -progressively measurable if Xs () : [0, t] 7 R is measurable with respect to B[0,t] Ft , for each
t 0.
Remark. In contrast to Definition 7.2.19, we have dropped the completion of the
relevant -algebras in the preceding definition. Indeed, the standing assumption of
Remark 8.1.3 guarantees the completeness of each -algebra of the filtration Ft and
as we see next, progressive measurability is in any case equivalent to adaptedness
for all RCLL processes.
Proposition 8.1.8. An Ft -adapted S.P. {Xs , s 0} of right-continuous sample
functions is also Ft -progressively measurable.
(2,)
Proof. Fixing t > 0, let Qt+ denote the finite set of dyadic rationals of the
form j2 [0, t] augmented by {t} and arranged in increasing order 0 = t0 < t1 <
< tk = t (where k = t2 ). The -th approximation of the sample function
Xs () for s [0, t], is then
Xs() () = X0 I{0} (s) +
k
X
Xtj ()I(tj1 ,tj ] (s) .
j=1
Note that per positive integer and B B,
{(s, ) [0, t] : Xs() () B} = {0} X01 (B)
k
[
j=1
(B) ,
(tj1 , tj ] Xt1
j
(B) is
which is in the product -algebra B[0,t] Ft , since each of the sets Xt1
j
in Ft (recall that {Xs , s 0} is Ft -adapted and tj [0, t]). Consequently, each
8.1. CONTINUOUS TIME FILTRATIONS AND STOPPING TIMES
293
()
of the maps (s, ) 7 Xs () is a real-valued R.V. on the product measurable

space ([0, t] , B[0,t] Ft ). Further, by right-continuity of the sample functions
()
s 7 Xs (), for each fixed (s, ) [0, t] the sequence Xs () converges as

to Xs (), which is thus a R.V. on the same (product) measurable space
(recall Corollary 1.2.23).
Associated with any filtration {Ft } is the collection of all Ft -stopping times and
the corresponding stopped -algebras (compare with Definitions 5.1.11 and 5.1.34).
Definition 8.1.9. A random variable : 7 [0, ] is called a stopping time for
the (continuous time) filtration {Ft }, or in short Ft -stopping time, if { : ()
t} Ft for all t 0. Associated with each Ft -stopping time is the stopped
-algebra F = {A F : A { t} Ft for all t 0} (which quantifies the
information in the filtration at the stopping time ).
The Ft+ -stopping times are also called Ft -Markov times (or Ft -optional times),
with the corresponding Markov -algebras F + = {A F : A { t} Ft+
for all t 0}.
Remark. As their name suggest, Markov/optional times appear both in the context of Doobs optional stopping theorem (in Section 8.2.3), and in that of the
strong Markov property (see Section 8.3.2).
Obviously, any non-random constant t 0 is a stopping time. Further, by definition, every Ft -stopping time is also an Ft -Markov time and the two concepts
coincide for right-continuous filtrations. Similarly, the Markov -algebra F + contains the stopped -algebra F for any Ft -stopping time (and they coincide in case
of right-continuous filtrations).
Your next exercise provides more explicit characterization of Markov times and
closure properties of Markov and stopping times (some of which you saw before in
Exercise 5.1.12).
Exercise 8.1.10.
(a) Show that is an Ft -Markov time if and only if { : () < t} Ft for
all t 0.
(b) Show that if {n , n Z+ } are Ft -stopping times, then so are 1 2 ,
1 + 2 and supn n .
(c) Show that if {n , n Z+ } are Ft -Markov times, then in addition to 1 +
2 and supn n , also inf n n , lim inf n n and lim supn n are Ft -Markov
times.
(d) In the setting of part (c) show that 1 + 2 is an Ft -stopping time when
either both 1 and 2 are strictly positive, or alternatively, when 1 is a
strictly positive Ft -stopping time.
Similarly, here are some of the basic properties of stopped -algebras (compare
with Exercise 5.1.35), followed by additional properties of Markov -algebras.
Exercise 8.1.11. Suppose and are Ft -stopping times.
(a) Verify that ( ) F , that F is a -algebra, and if () = t is nonrandom then F = Ft .
(b) Show that F = F F and deduce that each of the events { < },
{ }, { = } belongs to F .
Hint: Show first that if A F then A { } F .
294
(c) Show that for any integrable R.V. Z,

E[Z|F ]I = E[Z|F ]I ,
and deduce that
E[E(Z|F )|F ] = E[Z|F ] .
(d) Show that if and mF then is an Ft -stopping time.
Exercise 8.1.12. Suppose , n are Ft -Markov times.
(a) Verify that F + = {A F : A { < t} Ft for all t 0}.

(b) Suppose 1 is further Ft -stopping time and 1 with a strict inequality
whenever is finite. Show that then F +T F1 .
(c) Setting = inf n n , show that F + = n Fn+ . Deduce that if n are
T
Ft -stopping times and < n whenever is finite, then F + = n Fn .
In contrast to adaptedness, progressive measurability transfers to stopped processes (i.e. the continuous time extension of Definition 5.1.31), which is essential
when dealing in Section 8.2 with stopped sub-martingales (i.e. the continuous time
extension of Theorem 5.1.32).
Proposition 8.1.13. Given Ft -progressively measurable S.P. {Xs , s 0}, the
stopped at (Ft -stopping time) S.P. {Xs ()(), s 0} is also Ft -progressively
measurable. In particular, if either < or there exists X mF , then
X mF .
Proof. Fixing t > 0, denote by S the product -algebra B[0,t] Ft on the
product space S = [0, t] . The assumed Ft -progressive measurability of {Xs , s
0} amounts to the measurability of g1 : (S, S) 7 (R, B) such that g1 (s, ) = Xs ().
Further, as (s, ) 7 Xs () () is the composition g1 (g2 (s, )) for the mapping
g2 (s, ) = (s (), ) from (S, S) to itself, by Proposition 1.2.18 the Ft -progressive
measurability of the stopped S.P. follows form our claim that g2 is measurable.
Indeed, recall that is an Ft -stopping time, so { : () > u} Ft for any
u [0, t]. Hence, for any fixed u [0, t] and A Ft ,
g21 ((u, t] A) = (u, t] (A { : () > u}) S ,
which suffices for measurability of g2 (since the product -algebra S is generated

by the collection {(u, t] A : u [0, t], A Ft }).
Turning to the second claim, since {Xs , s 0} is Ft -progressively measurable,
we have that for any fixed B B and finite t 0,
X1 (B) 1 ([0, t]) = { : Xt () () B} { : () t} Ft
(recall (1.4.7) that { : (t, ) A} is in Ft for any set A S). Moreover, by

1
our assumptions X
(B) 1 ({}) is in F , hence so is its union with the sets
1
1
X (B) ([0, n]), n Z+ , which is precisely X1 (B). We have thus shown
that X1 (B) F for any B B, namely, that X mF .

Recall Exercise 5.1.13 that for discrete time S.P. and filtrations, the first hitting
time B of a Borel set B by Fn -adapted process is an Fn -stopping time. Unfortunately, this may fail in the continuous time setting, even when considering an open
set B and the canonical filtration FtX of a S.P. of continuous sample functions.
8.1. CONTINUOUS TIME FILTRATIONS AND STOPPING TIMES
295
Example 8.1.14. Indeed, consider B = (0, ) and the S.P. Xt () = t of Example 8.1.6. In this case, B (1) = 0 while B (1) = , so the event { : B ()
0} = {1} is not in F0X = {, } (hence B is not an FtX -stopping time). As shown
next, this problem is only due to the lack of right-continuity in the filtration {FtX }.
Proposition 8.1.15. Consider an Ft -adapted, right-continuous S.P. {Xs , s 0}.
Then, the first hitting time B () = inf{t 0 : Xt () B} is an Ft -Markov
time for an open set B and further an Ft -stopping time when B is a closed set and
{Xs , s 0} has continuous sample functions.
Proof. Fixing t > 0, by definition of B the set B1 ([0, t)) is the union of
over all s [0, t). Further, if the right-continuous function s 7 Xs ()
intersects an open set B at some s [0, t) then necessarily Xq () B at some
q Qt = Q [0, t). Consequently,
[
(8.1.1)
B1 ([0, t)) =
Xs1 (B) .
Xs1 (B)
sQt
Now, the Ft -adaptedness of {Xs } implies that Xs1 (B) Fs Ft for all s t,
and in particular for any s in the countable collection Qt . We thus deduce from
(8.1.1) that {B < t} Ft for all t 0, and in view of part (a) of Exercise 8.1.10,
conclude that B is an Ft -Markov time in case B is open.
Assuming hereafter that B is closed and u 7 Xu continuous, we claim that for
any t > 0,
[
\
(8.1.2)
{B t} =
Xs1 (B) =
{Bk < t} := At ,
0st
k=1
where Bk = {x R : |x y| < k , for some y B}, and that the left identity
in (8.1.2) further holds for t = 0. Clearly, X01 (B) F0 and for Bk open, by the
preceding proof {Bk < t} Ft . Hence, (8.1.2) implies that {B t} Ft for all
t 0, namely, that B is an Ft -stopping time.
Turning to verify (8.1.2), fix t > 0 and recall that if At then |Xsk () yk | <
k 1 for some sk [0, t) and yk B. Upon passing to a sub-sequence, sk s [0, t],
hence by continuity of the sample function Xsk () Xs (). This in turn implies
that yk Xs () B (because B is a closed set). Conversely, if Xs () B for some
s [0, t) then also Bk s < t for all k 1, whereas even if only Xt () = y B,
by continuity of the sample function also Xs () y for 0 s t (and once again
Bk < t for all k 1). To summarize, At if and only if there exists s [0, t]
such that Xs () B, as claimed. Considering hereafter t 0 (possibly t = 0),
the existence of s [0, t] such that Xs B results with {B t}. Conversely,
if B () t then Xsn () B for some sn () t + n1 and all n. But then
snk s t along some sub-sequence nk , so for B closed, by continuity of

the sample function also Xsnk () Xs () B.
We conclude with a technical result on which we shall later rely, for example,
in proving the optional stopping theorem and in the study of the strong Markov
property.
Lemma 8.1.16. Given an Ft -Markov time , let = 2 ([2 ] + 1) for 1.
Then, are Ft -stopping times and A { : () = q} Fq for any A F + ,
1 and q Q(2,) = {k2, k Z+ }.
296
Proof. By its construction, takes values in the discrete set Q(2,) {}.
Moreover, with { : () < t} Ft for any t 0 (see part (a) of Exercise 8.1.10),
it follows that for any q Q(2,) ,
{ : () = q} = { : () [q 2 , q)} Fq .
Hence, is an Ft -stopping time, as claimed. Next, fixing A F + , in view of

Definitions 8.1.4 and 8.1.9, the sets At,m = A { : () t m1 } are in Ft for
any m 1. Further, by the preceding, fixing q Q(2,) and q = q 2 you have
that
A { : () = q} = A { : () [q , q)} = (m1 Aq,m ) \ (m1 Aq ,m )
is the difference between an element of Fq and one of Fq Fq . Consequently,
A { : () = q} is in Fq , as claimed.

8.2. Continuous time martingales
As we show in this section, once the technical challenges involved with the continuity of time are taken care off, the results of Chapter 5 extend in a natural way
to the collection of continuous time (sub and super) martingales. Similar to the
break-up of Chapter 5, we devote Subsection 8.2.1 to the definition, examples and
closure properties of this collection of S.P. (compare with Section 5.1), followed by
Subsection 8.2.2 about tail (and upcrossing) inequalities and convergence properties
of such processes (compare with Sections 5.2.2 and 5.3, respectively). The statement, proof and applications of Doobs optional stopping theorem are explored in
Subsection 8.2.3 (compare with Section 5.4), with martingale representations being
the focus of Subsection 8.2.4 (compare with Sections 5.2.1 and 5.3.2).
8.2.1. Definition, examples and closure properties. For a continuous
filtration, it is not enough to consider the martingale property one step ahead, so
we replace Definitions 5.1.4 and 5.1.16 by the following continuous time analog of
Proposition 5.1.20.
Definition 8.2.1. The pair (Xt , Ft , t 0) is called a continuous time martingale
(in short MG), if the integrable (continuous time) S.P. {Xt , t 0} is adapted to
the (continuous time) filtration {Ft , t 0} and for any fixed t s 0, the identity
E[Xt |Fs ] = Xs holds a.s. Replacing the preceding identity with E[Xt |Fs ] Xs a.s.
for each t s 0, or with E[Xt |Fs ] Xs a.s. for each t s 0, defines the
continuous time sub-MG and continuous time sup-MG, respectively. These three
classes of continuous time S.P. are related in the same manner as in the discrete
time setting (c.f. Remark 5.1.17).
It immediately follows from the preceding definition that t 7 EXt is non-decreasing
for a sub-MG, non-increasing for a sup-MG, and constant (in time), for a MG.
Further, unless explicitly stated otherwise, one uses the canonical filtration when
studying MGs (or sub/sup-MGs).
Exercise 8.2.2. Suppose (Xt , Ft , t 0) is a continuous time sub-MG.
(a) Show that (Xt , FtX , t 0) is also a sub-MG.

(b) Show that if EXt = EX0 for all t 0, then (Xt , Ft , t 0) is also a
martingale.
8.2. CONTINUOUS TIME MARTINGALES
297
The decomposition of conditional second moments, as in part (b) of Exercise 5.1.8,

applies for all continuous time square-integrable MGs.
Exercise 8.2.3. Suppose (Xt , Ft , t 0) is a square-integrable MG. Verify that
(8.2.1)
E[Xt2 |Fs ] Xs2 = E[(Xt Xs )2 |Fs ]
and deduce that t 7
EXt2
for any t s 0 ,
is non-decreasing.
As you see next, the Wiener process and the compensated Poisson process play
the same role that the random walk of zero-mean increments plays in the discrete
time setting (with Wiener process being the prototypical MG of continuous sample
functions, and compensated Poisson process the prototypical MG of discontinuous
RCLL sample functions).
Proposition 8.2.4. Any integrable S.P. {Xt , t 0} of independent increments
(see Exercise 7.1.12), and constant mean function is a MG.
Proof. Recall that a S.P. Xt has independent increments if Xt+h Xt is
independent of FtX , for all h > 0 and t 0. We have also assumed that E|Xt | <
and EXt = EX0 for all t 0. Therefore, E[Xt+h Xt |FtX ] = E[Xt+h Xt ] = 0.
Further, Xt mFtX and hence E[Xt+h |FtX ] = Xt . That is, {Xt , FtX , t 0} is a
MG, as claimed.

Example 8.2.5. In view of Exercise 7.3.13 and Proposition 8.2.4 we have that the
Wiener process/ Brownian motion (Wt , t 0) of Definition 7.3.12 is a martingale.
Combining Proposition 3.4.9 and Exercise 7.1.12, we see that the Poisson process
Nt of rate has independent increments and mean function ENt = t. Consequently, by Proposition 8.2.4 the compensated Poisson process Mt = Nt t is
also a martingale (and FtM = FtN ).
Similarly to Exercise 5.1.9, as you check next, a Gaussian martingale {Xt , t 0}
is necessarily square-integrable and of independent increments, in which case Mt =
Xt2 hXit is also a martingale.
Exercise 8.2.6.
(a) Show that if {Xt , t 0} is a square-integrable S.P. having zero-mean
independent increments, then (Xt2 hXit , FtX , t 0) is a MG with hXit =
EXt2 EX02 a non-random, non-decreasing function.
(b) Prove that the conclusion of part (a) applies to any martingale {Xt , t 0}
which is a Gaussian S.P.
(c) Deduce that if {Xt , t 0} is square-integrable, with X0 = 0 and zeromean, stationary independent increments, then (Xt2 tEX12 , FtX , t 0)
is a MG.
In the context of the Brownian motion {Bt , t 0}, we deduce from part (b) of
Exercise 8.2.6 that {Bt2 t, t 0} is a MG. This is merely a special case of the
following collection of MGs associated with the standard Brownian motion.
uk (t, y, ) for k 0 and u0 (t, y, ) =

Exercise 8.2.7. Let uk+1 (t, y, ) =
2
exp(y t/2).
(a) Show that for any R the S.P. (u0 (t, Bt , ), t 0) is a martingale with
respect to FtB .
298
(b) Check that for k = 1, 2, . . .,

[k/2]
uk (t, y, 0) =
X
r=0
k!
y k2r (t/2)r .
(k 2r)!r!
(c) Deduce that the S.P. (uk (t, Bt , ), t 0), k = 1, 2, . . . are also MGs
with respect to FtB , as are Bt2 t, Bt3 3tBt , Bt4 6tBt2 + 3t2 and
Bt6 15tBt4 + 45t2 Bt2 15t3 .
(d) Verify that for each k Z+ and R the function uk (t, y, ) solves the
heat equation ut (t, y) + 21 uyy (t, y) = 0.
The collection of sub-MG (equivalently, sup-MG or MG), is closed under the
addition of S.P. (compare with Exercise 5.1.19).
Exercise 8.2.8. Suppose (Xt , Ft ) and (Yt , Ft ) are sub-MGs and t 7 f (t) a nondecreasing, non-random function.
(a) Verify that (Xt + Yt , Ft ) is a sub-MG and hence so is (Xt + f (t), Ft ).
(b) Rewrite this, first for sup-MGs Xt and Yt , then in case of MGs.
With the same proof as in Proposition 5.1.22, you are next to verify that the
collection of sub-MGs (and that of sup-MGs), is also closed under the application
of a non-decreasing convex (concave, respectively), function (c.f. Example 5.1.23
for the most common choices of this function).
Exercise 8.2.9. Suppose the integrable S.P. {Xt , t 0} and convex function
: R 7 R are such that E[|(Xt )|] < for all t 0. Show that if (Xt , Ft ) is a
MG then ((Xt ), Ft ) is a sub-MG and the same applies even when (Xt , Ft ) is only
a sub-MG, provided () is also non-decreasing.
As you show next, the martingale Bayes rule of Exercise 5.5.16 applies also for a
positive, continuous time martingale (Zt , Ft , t 0).
Exercise 8.2.10. Suppose (Zt , Ft , t 0) is a (strictly) positive MG on (, F , P),
normalized so that EZ0 = 1. For each t > 0, let Pt = PFt and consider the equivalent probability measure Qt on (, Ft ) of Radon-Nikodym derivative dQt /dPt = Zt .

(a) Show that Qs = Qt F for any s [0, t].
s
(b) Fixing u s [0, t] and Y L1 (, Fs , Qt ) show that Qt -a.s. (hence
also P-a.s.), EQt [Y |Fu ] = E[Y Zs |Fu ]/Zu .
e > 0 and Nt is a Poisson Process of rate > 0 then
(c) Verify that if
e
()t
e Nt is a strictly positive martingale with EZ0 = 1 and
Zt = e
(/)
e under the measure
show that {Nt , t [0, T ]} is a Poisson process of rate
QT , for any finite T .
e
Remark. Up to the re-parametrization = log(/),
the martingale Zt of part (c)
of the preceding exercise is of the form Zt = u0 (t, Nt , ) for u0 (t, y, ) = exp(y
t(e 1)). Building on it and following the line of reasoning of Exercise 8.2.7
yields the analogous collection of martingales for the Poisson process {Nt , t 0}.
For example, here the functions uk (t, y, ) on (t, y) R+ Z+ solve the equation
ut (t, y) + [u(t, y) u(t, y + 1)] = 0, with Mt = u1 (t, Nt , 0) being the compensated
Poisson process of Example 8.2.5 while u2 (t, Nt , 0) is the martingale Mt2 t.
299
Remark. While beyond our scope, we note in passing that in continuous time the
martingale
transform of Definition 5.1.27 is replaced by the stochastic integral Yt =
Rt
V
dX
.
This
stochastic integral results with stochastic differential equations and
s
0 s
is the main object of study of stochastic calculus (to which many texts are devoted,
among them [KaS97]). In case Vs = Xs is the Wiener process Ws , the analog
Rt
of Example 5.1.29 is Yt = 0 Ws dWs , which for the appropriate definition of the
stochastic integral (due to It
o), is merely the martingale Yt = 21 (Wt2 t). Indeed,
It
os stochastic integral is defined via martingale theory, at the cost of deviating
from the standard integration by parts formula. The latter would have applied if
the sample functions t 7 Wt () were differentiable w.p.1., which is definitely not
the case (as we shall see in Section 9.3).
Exercise 8.2.11. Suppose S.P. {Xt , t 0} is integrable and Ft -adapted. Show
that if E[Xu ] E[X ] for any u 0 and Ft -stopping time whose range () is
a finite subset of [0, u], then (Xt , Ft , t 0) is a sub-MG.
Hint: Consider = sIA + uIAc with s [0, u] and A Fs .
We conclude this sub-section with the relations between continuous and discrete
time (sub/super) martingales.
Example 8.2.12. Convince yourself that to any discrete time sub-MG (Yn , Gn , n
Z+ ) corresponds the interpolated continuous time sub-MG (Xt , Ft , t 0) of the interpolated right-continuous filtration Ft = G[t] and RCLL S.P. Xt = Y[t] of Example
8.1.5.
Remark 8.2.13. In proving results about continuous time MGs (or sub-MGs/supMGs), we often rely on the converse of Example 8.2.12. Namely, for any nonrandom, non-decreasing sequence {sk } [0, ), if (Xt , Ft ) is a continuous time
MG (or sub-MG/sup-MG), then clearly (Xsk , Fsk , k Z+ ) is a discrete time MG
(or sub-MG/sup-MG, respectively), while (Xsk , Fsk , k Z ) is a RMG (or reversed
subMG/supMG, respectively), where s0 s1 sk .
8.2.2. Inequalities and convergence. In this section we extend the tail
inequalities and convergence properties of discrete time sub-MGs (or sup-MGs), to
the corresponding results for sub-MGs (and sup-MGs) of right-continuous sample
functions, which we call hereafter in short right-continuous sub-MGs (or sup-MGs).
We start with Doobs inequality (compare with Theorem 5.2.6).
Theorem 8.2.14 (Doobs inequality). If {Xs , s 0} is a right-continuous
sub-MG, then for t 0 finite, Mt = sup0st Xs , and any x > 0
(8.2.2)
P(Mt x) x1 E[Xt I{Mt x} ] x1 E[(Xt )+ ] .
Proof. It suffices to show that for any y > 0

(8.2.3)
yP(Mt > y) E[Xt I{Mt >y} ] .
Indeed, by dominated convergence, taking y x yields the left inequality in (8.2.2),

and the proof is then complete since E[ZIA ] E[(Z)+ ] for any event A and integrable R.V. Z.
(2,)
Turning to prove (8.2.3), fix hereafter t 0 and let Qt+ denote the finite set
of dyadic rationals of the form j2 [0, t] augmented by {t}. Recall Remark
(2,)
8.2.13 that enumerating Qt+ in a non-decreasing order produces a discrete time
300
sub-MG {Xsk }. Applying Doobs inequality (5.2.1) for this sub-MG, we find that
xP(M () x) E[Xt I{M()x} ] for
M () = max Xs ,
(2,)
t+
sQ
and any x > 0. Considering x y, it then follows by dominated convergence that

yP(M () > y) E[Xt I{M()>y} ] ,
for any 1. Next note that as ,
M () M () =
sup
Xs ,
(2)
sQt {t}
and moreover, M () = Mt by the right-continuity of the sample function t 7 Xt

(compare with part (a) of Exercise 7.2.18). Consequently, for both P(M () >
y) P(Mt > y) and E[Xt I{M()>y} ] E[Xt I{Mt >y} ], thus completing the proof of
(8.2.3).

With (y)p+ denoting hereafter the function (max(y, 0))p , we proceed with the refinement of Doobs inequality for MGs or when the positive part of a sub-MG has
finite p-th moment for some p > 1 (compare to Exercise 5.2.11).
Exercise 8.2.15.
(a) Show that in the setting of Theorem 8.2.14, for any p 1, finite t 0
and x > 0,

P(Mt x) xp E (Xt )p+ ,
(b) Show that if {Ys , s 0} is a right-continuous MG, then
i
h
P( sup |Ys | y) y p E |Yt |p .
0st
By integrating Doobs inequality (8.2.2) you bound the moments of the supremum
of a right-continuous sub-MG over a compact time interval.
Corollary 8.2.16 (Lp maximal inequalities). With q = q(p) = p/(p 1), for
any p > 1, t 0 and a right-continuous sub-MG {Xs , s 0},

(8.2.4)
E ( sup Xu )p+ q p E[(Xt )p+ ] ,
0ut
and if {Ys , s 0} is a right-continuous MG then also

(8.2.5)
E ( sup |Yu |)p q p E[|Yt |p ] .
0ut
Proof. Adapting the proof of Corollary 5.2.13, the bound (8.2.4) is just the
conclusion of part (b) of Lemma 1.4.31 for the non-negative variables X = (Xt )+
and Y = (Mt )+ , with the left inequality in (8.2.2) providing its hypothesis. We
are thus done, as the bound (8.2.5) is merely (8.2.4) in case of the non-negative
sub-MG Xt = |Yt |.

In case p = 1 we have the following extension of Exercise 5.2.15.
Exercise 8.2.17. Suppose {Xs , s 0} is a non-negative, right-continuous subMG. Show that for any t 0,

E sup Xu (1 e1 )1 {1 + E[Xt (log Xt )+ ]} .
0ut
Hint: Relying on Exercise 5.2.15, interpolate as in our derivation of (8.2.2).
301
Doobs fundamental up-crossing inequality (see Lemma 5.2.18), extends to the

number of up-crossings in dyadic-times, as defined next.
Definition 8.2.18. The number of up-crossings (in dyadic-times), of the interval
[a, b] by the continuous time S.P. {Xu , u [0, t]}, is the random variable Ut [a, b] =
sup Ut, [a, b], where Ut, [a, b]() denotes the number of up-crossings of [a, b] by the
(2,)
finite sequence {Xsk (), sk Qt+ }, as in Definition 5.2.17.
Remark. It is easy to check that for any right-continuous S.P. the number of
up-crossings in dyadic-times coincides with the natural definition of the number of
up-crossings Ut [a, b] = sup{UF [a, b] : F a finite subset of [0, t]}, where UF [a, b] is the
number of up-crossings of [a, b] by {Xs (), s F } However, for example the nonrandom S.P. Xt = ItQ has zero up-crossings in dyadic-times, while Ut [a, b] =
for any 1 > b > a > 0 and t > 0. Also, Ut [a, b] may be non-measurable on (, F )
in the absence of right continuity of the underlying S.P.
Lemma 8.2.19 (Doobs up-crossing inequality). If {Xs , s 0} is a sup-MG,
then for any t 0,
(8.2.6)
(b a)E(Ut [a, b]) E[(Xt a) ] E[(X0 a) ]

(2,)
a a and t 0. Since {Qt+ } is a non-decreasing sequence of

finite sets, by definition 7 Ut, is non-decreasing and by monotone convergence
it suffices to show that for all ,
(b a)E( Ut, [a, b] ) E[(Xt a) ] E[(X0 a) ] .
(2,)
Recall Remark 8.2.13 that enumerating sk Qt+ in a non-decreasing order produces a discrete time sup-MG {Xsk , k = 0, . . . , n} with s0 = 0 and sn = t, so this
is merely Doobs up-crossing inequality (5.2.6).

Since Doobs maximal and up-crossing inequalities apply for any right-continuous
sub-MG (and sup-MG), so do most convergence results we have deduced from them
in Section 5.3. For completeness, we provide a short summary of these results (and
briefly outline how to adapt their proofs), starting with Doobs a.s. convergence
theorem.
Theorem 8.2.20 (Doobs convergence theorem). Suppose right-continuous
a.s.
sup-MG {Xt , t 0} is such that supt {E[(Xt ) ]} < . Then, Xt X and
E|X | lim inf t E|Xt | is finite.
Proof. Let U [a, b] = supnZ+ Un [a, b]. Paralleling the proof of Theorem
5.3.2, in view of our assumption that supt {E[(Xt ) ]} is finite, it follows from Lemma
8.2.19 and monotone convergence that E(U [a, b]) is finite for each b > a. Hence,
w.p.1. the variables U [a, b]() are finite for all a, b Q, a < b. By sample path
right-continuity and diagonal selection, in the set
a,b = { : lim inf Xt () < a < b < lim sup Xt ()} ,
t
it suffices to consider t Q
(2)
, hence a,b F . Further, if a,b , then
Xq2k1 () < a < b < Xq2k () ,
for some dyadic rationals qk , hence U [a, b]() = . Consequently, the a.s.
convergence of Xt to X follows as in the proof of Lemma 5.3.1. Finally, the stated
bound on E|X | is then derived exactly as in the proof of Theorem 5.3.2.
302
Remark. Similarly to Exercise 5.3.3, for right-continuous sub-MG {Xt } the finiteness of supt E|Xt |, of supt E[(Xt )+ ] and of lim inf t E|Xt | are equivalent to each other
and to the existence of a finite limit for E|Xt | (or equivalently, for limt E[(Xt )+ ]),
a.s.
each of which further implies that Xt X integrable. Replacing (Xt )+ by (Xt )
the same applies for sup-MGs. In particular, any non-negative, right-continuous,
sup-MG {Xt , t 0} converges a.s. to integrable X such that EX EX0 .
Note that Doobs convergence
p theorem does not apply for the Wiener process
{Wt , t 0} (as E[(Wt )+ ] = t/(2) is unbounded). Indeed, as we see in Exercise
8.2.35, almost surely, lim supt Wt = and lim inf t Wt = . That is, the
magnitude of oscillations of the Brownian sample path grows indefinitely.
In contrast, Doobs convergence theorem allows you to extend Doobs inequality
(8.2.2) to the maximal value of a U.I. right-continuous sub-MG over all t 0.
Exercise 8.2.21. Let M = sups0 Xs for a U.I. right-continuous sub-MG
a.s.
{Xt , t 0}. Show that Xt X integrable and for any x > 0,
(8.2.7)
P(M x) x1 E[X I{M x} ] x1 E[(X )+ ] .
Hint: Start with (8.2.3) and adapt the proof of Corollary 5.3.4.
The following integrability condition is closely related to L1 convergence of rightcontinuous sub-MGs (and sup-MGs).
Definition 8.2.22. We say that a sub-MG (Xt , Ft , t 0) is right closable, or
has a last element (X , F ) if Ft F and X L1 (, F , P) is such that
for any t 0, almost surely E[X |Ft ] Xt . A similar definition applies for a
sup-MG, but with E[X |Ft ] Xt and for a MG, in which case we require that
E[X |Ft ] = Xt , namely, that {Xt } is a Doobs martingale of X with respect to
{Ft } (see Definition 5.3.13).
Building upon Doobs convergence theorem, we extend Theorem 5.3.12 and Corollary 5.3.14, showing that for right-continuous MGs the properties of having a last
element, uniform integrability and L1 convergence, are equivalent to each other.
Proposition 8.2.23. The following conditions are equivalent for a right-continuous
non-negative sub-MG {Xt , t 0}:
(a) {Xt } is U.I.;
L1
(b) Xt X ;
a.s.
(c) Xt X a last element of {Xt }.
Further, even without non-negativity (a) (b) = (c) and a right-continuous
MG has any, hence all, of these properties, if and only if it is a Doob martingale.
Remark. By definition, any non-negative sup-MG has a last element X = 0

(and obviously, the same applies for any non-positive sub-MG), but many nonnegative sup-MGs are not U.I. (for example, any non-degenerate critical branching
process is such, as explained in the remark following the proof of Proposition 5.5.5).
So, whereas a MG with a last element is U.I. this is not always the case for sub-MGs
(and sup-MGs).
Proof. (a) (b): U.I. implies L1 -boundedness which for a right-continuous
sub-MG yields by Doobs convergence theorem the convergence a.s., and hence in
probability of Xt to integrable X . Clearly, the L1 convergence of Xt to X
303
also implies such convergence in probability. Either way, recall Vitalis convergence
theorem (i.e. Theorem 1.3.49), that U.I. is equivalent to L1 convergence when
p
Xt X . We thus deduce the equivalence of (a) and (b) for right-continuous
sub-MGs, where either (a) or (b) yields the corresponding a.s. convergence.
(a) and (b) yield a last element: With X denoting the a.s. and L1 limit of the
U.I. collection {Xt }, it is left to show that E[X |Fs ] Xs for any s 0. Fixing
t > s and A Fs , by the definition of sub-MG we have E[Xt IA ] E[Xs IA ].
Further, E[Xt IA ] E[X IA ] (recall part (c) of Exercise 1.3.55). Consequently,
E[X IA ] E[Xs IA ] for all A Fs . That is, E[X |Fs ] Xs .
Last element and non-negative = (a): Since Xt 0 and EXt EX finite, it
follows that for any finite t 0 and M > 0, by Markovs inequality P(Xt > M )
M 1 EXt M 1 EX 0 as M . It then follows that E[X I{Xt >M} ] converges to zero as M , uniformly in t (recall part (b) of Exercise 1.3.43). Further,
by definition of the last element we have that E[Xt I{Xt >M} ] E[X I{Xt >M} ].
Therefore, E[Xt I{Xt >M} ] also converges to zero as M , uniformly in t, i.e.
{Xt } is U.I.
Equivalence for MGs: For a right-continuous MG the equivalent properties (a)
and (b) imply the a.s. convergence to X such that for any fixed t 0, a.s.
Xt E[X |Ft ]. Applying this also for the right-continuous MG {Xt } we deduce
that X is a last element of the Doobs martingale Xt = E[X |Ft ]. To complete
the proof recall that any Doobs martingale is U.I. (see Proposition 4.2.33).

Finally, paralleling the proof of Proposition 5.3.22, upon combining Doobs convergence theorem 8.2.20 with Doobs Lp maximal inequality (8.2.5) we arrive at
Doobs Lp MG convergence.
Proposition 8.2.24 (Doobs Lp martingale convergence).
If right-continuous MG {Xt , t 0} is Lp -bounded for some p > 1, then Xt X
a.s. and in Lp (in particular, kXt kp kX kp ).
Throughout we rely on right continuity of the sample functions to control the tails
of continuous time sub/sup-MGs and thereby deduce convergence properties. Of
course, the interpolated MGs of Example 8.2.12 and the MGs derived in Exercise
8.2.7 out of the Wiener process are right-continuous. More generally, as shown
next, for any MG the right-continuity of the filtration translates (after a modification) into RCLL sample functions, and only a little more is required for an RCLL
modification in case of a sup-MG (or a sub-MG).
Theorem 8.2.25. Suppose (Xt , Ft , t 0) is a sup-MG with right-continuous filtration {Ft , t 0} and t 7 EXt is right-continuous. Then, there exists an RCLL
et , t 0} of {Xt , t 0} such that (X
et , Ft , t 0) is a sup-MG.
modification {X
et , t 0} recall Lemma 8.2.19 that any supProof. Step 1. To construct {X

MG {Xt , t 0} has a finite expected number of up-crossings E(Un [a, b]) for each
b > a and n Z+ . Hence, P() = 0, where
= { : Un [a, b]() = , for some n Z+ , a, b Q, b > a} .
Further, if is such that for some 0 t < n,
lim inf Xq () < lim sup Xq (),
qt,qQ(2)
qt,qQ(2)
304

(2)
then there exist a, b Q, b > a, and a decreasing sequence qk Qn such that

Xq2k () < a < b < Xq2k1 (), which in turn implies that Un [a, b]() is infinite.
Thus, if
/ then the limits Xt+ () of Xq () over dyadic rationals q t exist at
(2)
all t 0. Considering the R.V.-s Mn = sup{(Xq ) : q Qn } and the event
[
= { : Mn () = , for some n Z+ } ,
observe that if
/ then Xt+ () are finite valued for all t 0. Further, setting
et () = Xt+ ()Ic (), note that 7 X
et () is measurable and finite for each
X
t 0. We conclude the construction by verifying that P( ) = 0. Indeed, recall

that right-continuity was applied only at the end of the proof of Doobs inequality
(8.2.2), so using only the sub-MG property of (Xt ) we have that for all y > 0,
P(Mn > y) y 1 E[(Xn ) ] .
Hence, Mn is a.s. finite. Starting with Doobs second inequality (5.2.3) for the
sub-MG {Xt }, by the same reasoning P(Mn+ > y) y 1 (E[(Xn ) ] + E[X0 ]) for
all y > 0. Thus, Mn+ is also a.s. finite and as claimed P( ) = 0.
Step 2. Recall that our convention, as in Remark 8.1.3, implies that the P-null
event F0 . It then follows by the Ft -adaptedness of {Xt } and the preceding
et } is Ft+ -adapted, namely Ft -adapted (by the assumed
construction of Xt+ , that {X
right-continuity of {Ft , t 0}). Clearly, our construction of Xt+ yields rightet (). Further, a re-run of part of Step 1 yields
continuous sample functions t 7 X
the RCLL property, by showing that for any c the sample function t 7 Xt+ ()
has finite left limits at each t > 0. Indeed, otherwise there exist a, b Q, b > a and
sk t such that Xs+ () < a < b < Xs+ (). By construction of Xt+ this implies
2k
2k1
(2)
the existence of qk Qn such that qk t and Xq2k1 () < a < b < Xq2k ().
Consequently, in this case Un [a, b]() = , in contradiction with c .
Step 3. Fixing s 0, we show that Xs+ = Xs for a.e.
/ , hence the Ft et , t 0} is a modification of the sup-MG (Xt , Ft , t 0) and as
adapted S.P. {X
et , Ft , t 0) is also a sup-MG. Turning to show that Xs+ a.s.
such, (X
= Xs , fix
non-random dyadic rationals qk s as k and recall Remark 8.2.13 that
(Xqk , Fqk , k Z ) is a reversed sup-MG. Further, from the sup-MG property, for
any A Fs ,
sup E[Xqk IA ] E[Xs IA ] < .
k
Considering A = , we deduce by Exercise 5.5.21 that the collection {Xqk } is U.I.

and thus, the a.s. convergence of Xqk to Xs+ yields that E[Xqk IA ] E[Xs+ IA ]
(recall part (c) of Exercise 1.3.55). Moreover, E[Xqk ] E[Xs ] in view of the
assumed right-continuity of t 7 E[Xt ]. Consequently, taking k we deduce
that E[Xs+ IA ] E[Xs IA ] for all A Fs , with equality in case A = . With both
Xs+ and Xs measurable on Fs , it thus follows that a.s. Xs+ = Xs , as claimed.
8.2.3. The optional stopping theorem. We are ready to extend the very
useful Doobs optional stopping theorem (see Theorem 5.4.1), to the setting of
right-continuous sub-MGs.
Theorem 8.2.26 (Doobs optional stopping).
If (Xt , Ft , t [0, ]) is a right-continuous sub-MG with a last element (X , F )
in the sense of Definition 8.2.22, then for any Ft -Markov times , the integrable
X and X are such that EX EX , with equality in case of a MG.
305
Remark. Recall Proposition 8.1.8 that right-continuous, Ft -adapted {Xs , s 0}

is Ft -progressively measurable, hence also Ft+ -progressively measurable. With the
existence of X mF , it then follows from Proposition 8.1.13 that X mF+
is a R.V. for any Ft -Markov time (and by the same argument X mF in case
is an Ft -stopping time).
Proof. Fixing 1 and setting sk = k2 for k Z+ {}, recall Remark
8.2.13 that (Xsk , Fsk , k Z+ ) is a discrete time sub-MG. Further, the assumed
existence of a last element (X , F ) for the sub-MG (Xt , Ft , t 0) implies that
a.s. E[Xs |Fsk ] Xsk for any k Z+ . With a slight abuse of notations we
call {sk }-valued R.V. an Fsk -stopping time if { sk } Fsk for all k Z+ .
Then, as explained in Remark 5.4.2, it thus follows from Theorem 5.4.1 that for
any Fsk -stopping times , the R.V. X and X are integrable, with
(8.2.8)
E[X ] E[X ] E[X0 ] .
In Lemma 8.1.16 we have constructed an Fsk -stopping time = 2 ([2 ] + 1)

for the given Ft -Markov time . Similarly, we have the Fsk -stopping time =
2 ([2 ] + 1) corresponding to the Ft -Markov time . Our assumption that
translates to , hence the inequality (8.2.8) holds for any positive integer
. By their construction () () and () () as . Thus, by the
assumed right-continuity of t 7 Xt (), we have the a.s. convergence of X to X
and of X to X (when ).
We claim that (Xn , Fn , n Z ) is a discrete time reversed sub-MG. Indeed,
fixing 2, note that Q(2,1) {} is a subset of Q(2,) {} = {sk }. Appealing
once more to Remark 5.4.2, we can thus apply Lemma 5.4.3 for the pair 1
of Fsk -stopping times and deduce that a.s.
E[X1 |F ] X .
The latter inequality holds for all 2, amounting to the claimed reversed subMG property. Since in addition inf n EXn EX0 is finite (see (8.2.8)), we deduce
from Exercise 5.5.21 that the sequence {X }
=1 is U.I. The same argument shows
that {X }
is
U.I.
Hence,
both
sequences
converge
in L1 to their respective limits
=1
X and X . In particular, both variables are integrable and in view of (8.2.8) they
further satisfy the stated inequality EX EX .

We proceed with a few of the consequences of Doobs optional theorem, starting
with the extension of Lemma 5.4.3 to our setting.
Corollary 8.2.27. If (Xt , Ft , t [0, ]) is a right-continuous sub-MG with a last
element, then E[X |F+ ] X w.p.1. for any Ft -Markov times (with equality
in case of a MG), and if is an Ft -stopping time, then further E[X |F ] X
w.p.1. (again with equality in case of a MG).
Proof. Fixing A F+ , it follows as in the proof of Lemma 5.4.3 that =
IA + IAc is an Ft+ -stopping time. Thus, applying Theorem 8.2.26 for we
deduce that E[X ] E[X ]. Further, X and X are integrable, so proceeding as
in the proof of Lemma 5.4.3 we get that E[(Z + X )IA ] 0 for Z + = E[X |F+ ]
and all A F+ . Recall as noted just after the statement Theorem 8.2.26, that
X mF+ for the Ft+ -stopping time , and consequently, a.s. Z + X , as
claimed.
306
In case is an Ft -stopping time, note that by the tower property (and taking out
the known IA ), also E[(Z X )IA ] 0 for Z = E[Z + |F ] = E[X |F ] and all
A F . Here, as noted before, we further have that X mF and consequently,
in this case, a.s. Z X as well. Finally, if (Xt , Ft , t 0) is further a MG, combine
the statement of the corollary for sub-MGs (Xt , Ft ) and (Xt , Ft ) to find that a.s.
X = Z + (and X = Z for an Ft -stopping time ).

Remark 8.2.28. We refer hereafter to both Theorem 8.2.26 and its refinement
in Corollary 8.2.27 as Doobs optional stopping. Clearly, both apply if the rightcontinuous sub-MG (Xt , Ft , t 0) is such that a.s. E[Y |Ft ] Xt for some integrable R.V. Y and each t 0 (for by the tower property, such sub-MG has the
last element X = E[Y |F ]). Further, note that if is a bounded Ft -Markov
time, namely [0, T ] for some non-random finite T , then you dispense of the requirement of a last element by considering these results for Yt = XtT (whose last
element Y is the integrable XT mFT mF and where Y = X , Y = X ).
As this applies whenever both and are non-random, we deduce from Corollary
8.2.27 that if Xt is a right-continuous sub-MG (or MG) for some filtration {Ft },
then it is also a right-continuous sub-MG (or MG, respectively), for the corresponding filtration {Ft+ }.
The latter observation leads to the following result about the stopped continuous
time sub-MG (compare to Theorem 5.1.32).
Corollary 8.2.29. If is an Ft -stopping time and (Xt , Ft , t 0) is a rightcontinuous subMG (or supMG or a MG), then Xt = Xt() () is also a rightcontinuous subMG (or supMG or MG, respectively), for this filtration.
Proof. Recall part (b) of Exercise 8.1.10, that = u is a bounded Ft stopping time for each u [0, ). Further, fixing s u, note that for any A Fs ,
= (s )IA + (u )IAc ,
is an Ft -stopping time such that . Indeed, as Fs Ft when s t, clearly

{ t} = { t} (A {s t}) (Ac {s u t}) Ft ,
for all t 0. In view of Remark 8.2.28 we thus deduce, upon applying Theorem
8.2.26, that E[IA Xu ] E[IA Xs ] for all A Fs . From this we conclude that
the sub-MG condition E[Xu | Fs ] Xs holds a.s. whereas the right-continuity
of t 7 Xt is an immediate consequence of right-continuity of t 7 Xt .

In the discrete time setting we have derived Theorem 5.4.1 also for U.I. {Xn }
and mostly used it in this form (see Remark 5.4.2). Similarly, you now prove
Doobs optional stopping theorem for right-continuous sub-MG (Xt , Ft , t 0) and
Ft -stopping time such that {Xt } is U.I.
Exercise 8.2.30. Suppose (Xt , Ft , t 0) is a right-continuous sub-MG.
(a) Fixing finite, non-random u 0, show that for any Ft -stopping times
, a.s. E[Xu |F ] Xu (with equality in case of a MG).
Hint: Apply Corollary 8.2.27 for the stopped sub-MG (Xtu , Ft , t 0).
(b) Show that if (Xu , u 0) is U.I. then further X and X (defined
as lim supt Xt in case = ), are integrable and E[X |F ] X a.s.
(again with equality for a MG).
Hint: Show that Yu = Xu has a last element.
307
Relying on Corollary 8.2.27 you can now also extend Corollary 5.4.5.
Exercise 8.2.31. Suppose (Xt , Ft , t 0) is a right-continuous sub-MG and {k }
is a non-decreasing sequence of Ft -stopping times. Show that if (Xt , Ft , t 0) has a
last element or supk k T for some non-random finite T , then (Xk , Fk , k Z+ )
is a discrete time sub-MG.
Next, restarting a right-continuous sub-MG at a stopping time yields another
sub-MG and an interesting formula for the distribution of the supremum of certain
non-negative MGs.
Exercise 8.2.32. Suppose (Xt , Ft , t 0) is a right-continuous sub-MG and that
is a bounded Ft -stopping time.
(a) Verify that if Ft is a right-continuous filtration, then so is Gt = Ft+ .

(b) Taking Yt = X +t X , show that (Yt , Gt , t 0) is a right-continuous
sub-MG.
Exercise 8.2.33. Consider a non-negative MG {Zt , t 0} of continuous sample

a.s.
functions, such that Z0 = 1 and Zt 0 as t . Show that for any x > 1,
P(sup Zt x) = x1 .
t>0
Exercise 8.2.34. Using Doobs optional stopping theorem re-derive Doobs inequality. Namely, show that for t, x > 0 and right-continuous sub-MG {Xs , s 0},
P( sup Xs > x) x1 E[(Xt )+ ] .
0st
Hint: Consider the sub-MG ((Xut )+ , FuX ), the FuX -Markov time = inf{s 0 :
Xs > x}, and = .
We conclude this sub-section with concrete applications of Doobs optional stopping theorem in the context of first hitting times for the Wiener process (Wt , t 0)
of Definition 7.3.12.
(r)
Exercise 8.2.35. For r 0 consider Zt = Wt + rt, the Brownian motion with

(r)
drift (and continuous sample functions), starting at Z0 = 0.
(r)
(r)
(a) Check that the first hitting time b = inf{t 0 : Zt

b > 0, is an FtW -stopping
time.
(b) For s > 0 set (r, s) = r2 + 2s r and show that
b} of level
(r)
E[exp(sb )] = exp((r, s)b) .

Hint: Check that 12 2 + r s = 0 at = (r, s), then stop the martingale
(r)
u0 (t, Wt , (r, s)) of Exercise 8.2.7 at b .
(r)
(c) Letting s 0 deduce that P(b < ) = exp(2rb).
(d) Considering now r = 0 and b , deduce that a.s. lim supt Wt =
and lim inf t Wt = .
(r)
(r)
Exercise 8.2.36. Consider the exit time a,b = inf{t 0 : Zt

(r)
Zt
/ (a, b)} of an
interval, for the S.P.

= Wt +rt of continuous sample functions, where W0 = 0,
r R and a, b > 0 are finite non-random.
308

(r)
(a) Check that a,b is a.s. finite FtW -stopping time and show that for any
r 6= 0,
P(Z
(r)
(r)
a,b
= a) = 1 P(Z
(r)
(r)
a,b
while P(W (0) = a) = b/(a + b).
= b) =
1 e2rb
,
e2ra e2rb
a,b
(r)
Hint: For r 6= 0 consider u0 (t, Wt , 2r) of Exercise 8.2.7 stopped at a,b .

(b) Show that for all s 0
(0)
sinh(a 2s) + sinh(b 2s)
sa,b
)=
E(e
.
sinh((a + b) 2s)
(0)
Hint: Stop the MGs u0 (t, Wt , 2s) of Exercise 8.2.7 at a,b = a,b .
2
2
(c) Deduce that Ea,b = ab and Var(a,b ) = ab
3 (a + b ).
Hint: Recall part (b) of Exercise 3.2.40.
Here is a related result about first hitting time of spheres by a standard ddimensional Brownian motion.
Definition 8.2.37. The standard d-dimensional Brownian motion is the Rd valued S.P. {W (t), t 0} such that W (t) = (W1 (t), . . . , Wd (t)) with {Wi (t), t 0},
i = 1, 2, . . . , d mutually independent, standard (one-dimensional) Wiener processes.
It is clearly a MG and a centered Rd -valued Gaussian S.P. of continuous sample
functions and stationary, independent increments.
Exercise 8.2.38. Let FtW = (W (s), s t) denote the canonical filtration of
a standard k-dimensional Brownian motion, Rt = kW (t)k2 its Euclidean distance
from the origin and b = inf{t 0 : Rt b} the corresponding first hitting time of
a sphere of radius b > 0 centered at the origin.
(a) Show that Mt = Rt2 kt is an FtW -martingale of continuous sample
functions and that b is an a.s. finite FtW -stopping time.
(b) Deduce that E[b ] = b2 /k.
Remark. The S.P. {Rt , t 0} of the preceding exercise is called the Bessel
process with dimension
R t 1k. Though we shall not do so, it can be shown that the
S.P. Bt = Rt 0 Rs ds is well-defined and in fact is a standard Wiener process
(c.f. [KaS97, Proposition 3.3.21]), with = (k 1)/2 the corresponding index of
the Bessel process. The Bessel process is thus defined for all 1/2 (and starting
at R0 = r > 0, also for 0 < < 1/2). One can then further show that if R0 = r > 0
then Pr (inf t0 Rt > 0) = I{>1/2} (hence the k-dimensional Brownian motion is
O-transient for k 3, see Definition 6.3.21), and Pr (Rt > 0, for all t 0) = 1 even
for the critical case of = 1/2 (so by translation, for any given point z R2 , the
two-dimensional Brownian path, starting at any position other than z R2 w.p.1.
enters every disc of positive radius centered at z but never reaches the point z).
8.2.4. Doob-Meyer decomposition and square-integrable martingales.
In this section we study the structure of square-integrable martingales and in
particular the roles of the corresponding predictable compensator and quadratic
variation. In doing so, we fix throughout the probability space (, F , P) and a
right-continuous filtration {Ft } on it, augmented so that every P-null set is in F0
(see Remark 8.1.3).
309
Definition 8.2.39. We denote by M2 the vector space of all square-integrable

martingales {Xt , t 0} for the fixed right-continuous filtration, which start at
X0 = 0 and have right-continuous sample functions. We further denote by Mc2
the linear subspace of M2 consisting of those square-integrable martingales whose
sample functions are continuous (and as before X0 = 0).
As in the discrete time setting of Section 5.3.2, the key to the study of a squareintegrable martingale X M2 is the Doob-Meyer decomposition of Xt2 to the sum
of a martingale and the predictable quadratic variation hXit . More generally, the
Doob-Meyer decomposition is the continuous time analog of Doobs decomposition
of any discrete time integrable process as the sum of a martingale and a predictable
sequence. The extension of the concept of predictable S.P. to the continuous time
setting is quite subtle and outside our scope, but recall Exercise 5.2.2 that when
decomposing a sub-MG, the non-martingale component should be an increasing
process, as defined next.
Definition 8.2.40. An Ft -adapted, integrable S.P. {At , t 0} of right-continuous,
non-decreasing sample functions starting at A0 = 0, is called an increasing process
(or more precisely, an Ft -increasing process).
Remark. An increasing process is obviously a non-negative, right-continuous,
sub-MG. By monotonicity, A = limt At is a well defined random variable,
and due to Proposition 8.2.23, integrability of A is equivalent to {At } being U.I.
which in turn is equivalent to this sub-MG having a last-element (i.e. being right
closable).
Recall the notion of q-th variation of a function f : [a, b] 7 R, with q > 0 a
parameter, which we next extend to the q-th variation of continuous time S.P.-s.
()
()
()
Definition 8.2.41. For any finite partition = {a = s0 < s1 < < sk =

()
()
b} of [a, b], let kk = maxki=1 {si si1 } denote the length of the longest interval
in and
k
X
()
()
(q)
|f (si ) f (si1 )|q
V() (f ) =
i=1
denote the q-th variation of the function f () on the partition . The q-th variation
of f () on [a, b] is then the [0, ]-valued
(q)
V (q) (f ) = lim V() (f ) ,
(8.2.9)
kk0
provided such limit exists (namely, the same R-valued limit exists along each sequence {n , n 1} such that kn k 0). Similarly, the q-th variation on [a, b]
(q)
of a S.P. {Xt , t 0}, denoted V (q) (X) is the limit in probability of V() (X ())
per (8.2.9), if such a limit exists, and when this occurs for any compact interval
[0, t], we have the q-th variation, denoted V (q) (X)t , as a stochastic process with
non-negative, non-decreasing sample functions, such that V (q) (X)0 = 0.
Remark. As you are soon to find out, of most relevance here is the case of qth variation for q = 2, which is also called the quadratic variation.
Note also
(1)
that V() (f ) is bounded above by the total variation of the function f , namely
(1)
V (f ) = sup{V() (f ) : a finite partition of [a, b]} (which induces at each interval a
310
norm on the linear subspace of functions of finite total variation, see also the related
Definition 3.2.22 of total variation norm for finite signed measures). Further, as
you show next, if V (1) (f ) exists then it equals to V (f ) (but beware that V (1) (f )
may not exist, for example, in case f (t) = 1Q (t)).
Exercise 8.2.42.
(1)
(a) Show that if f : [a, b] 7 R is monotone then V() (f ) = maxt[a,b] {f (t)}
mint[a,b] {f (t)} for any finite partition , so in this case V (1) (f ) = V (f )

is finite.
(1)
(b) Show that 7 V() () is non-decreasing with respect to a refinement
of the finite partition of [a, b] and hence, for each f there exist finite
(1)
partitions n such that kn k 0 and V(n ) (f ) V (f ).
(c) For f : [0, ) 7 R and t 0 let V (f )t denote the value of V (f ) for
the interval [0, t]. Show that if f () is left-continuous, then so is the
non-decreasing function t 7 V (f )t .
(d) Show that if t 7 Xt is left-continuous, then V (X)t is FtX -progressively
measurable, and n = inf{t 0 : V (X)t n} are non-decreasing FtX Markov times such that V (X)tn n for all n and t.
Hint: Show that enough to consider for V (X)t the countable collection
(2)
()
of finite partitions for which si Qt+ , then note that {n < t} =
k1 {V (X)tk1 n}.
From the preceding exercise we see that any increasing process At has finite total
variation, with V (A)t = V (1) (A)t = At for all t. This is certainly not the case for
non-constant continuous martingales, as shown in the next lemma (which is also
key to the uniqueness of the Doob-Meyer decomposition for sub-MGs of continuous
sample path).
Lemma 8.2.43. A martingale Mt of continuous sample functions and finite total
variation on each compact interval, is indistinguishable from a constant.
Remark. Sample path continuity is necessary here, for in its absence we have
the compensated Poisson process Mt = Nt t which is a martingale (see Example 8.2.5), of finite total variation on compact intervals (since V (M )t V (N )t +
V (t)t = Nt + t by part (a) of Exercise 8.2.42).
ft = Mt M0 such that V (M
f)t = V (M )t
Proof. Considering the martingale M
for all t, we may and shall assume hereafter that M0 = 0. Suppose first that
V (M )t K is bounded, uniformly in t and by a non-random finite constant. In
particular, |Mt | K for all t 0 and fixing a finite partition = {0 = s0 < s1 <
< sk = t}, the discrete time martingale Msi is square integrable and as shown
in part (b) of Exercise 5.1.8
E[Mt2 ] = E[
k
X
i=1
(2)
(Msi Msi1 )2 ] = E[V() (M )] .
By the definition of the q-th variation and our assumption that V (M )t K, it

follows that
(2)
V() (M ) K sup |Msi Msi1 | =: KD .

i=1
311
Taking expectation on both sides we deduce in view of the preceding identity that
E[Mt2 ] KED where 0 D V (M )t K for all finite partitions of [0, t].
Further, by the uniform continuity of t 7 Mt () on [0, T ] we have that D () 0
when kk 0, hence E[D ] 0 as kk 0 and consequently E[Mt2 ] = 0.
We have thus shown that if the continuous martingale Mt is such that supt V (M )t
is bounded by a non-random constant, then Mt () = 0 for any t 0 and a.e.
. To deal with the general case, recall Remark 8.2.28 that (Mt , FtM
+ , t 0) is
a continuous martingale, hence by Corollary 8.2.29 and part (d) of Exercise 8.2.42
so is (Mtn , FtM
+ , t 0), where n = inf{t 0 : V (M )t n} are non-decreasing
and V (M )tn n for all n and t. Consequently, for any t 0, w.p.1. Mtn = 0
for n = 1, 2, . . .. The assumed finiteness of V (M )t () implies that n , hence
Mtn Mt as n , resulting with Mt () = 0 for a.e. . Finally, by the
continuity of t 7 Mt (), the martingale M must then be indistinguishable from
the zero stochastic process (see Exercise 7.2.3).

Considering a bounded, continuous martingale Xt , the next lemma allows us to
(2)
conclude in the sequel that V() (X) converges in L2 as kk 0 and its limit can be
set to be an increasing process.
Lemma 8.2.44. Suppose X Mc2 . For any partition = {0 = s0 < s1 < }
()
of [0, ) with a finite number of points on each compact interval, the S.P. Mt =
()
Xt2 Vt (X) is an Ft -martingale of continuous sample path, where
(8.2.10)
()
Vt
(X) =
k
X
i=1
(Xsi Xsi1 )2 + (Xt Xsk )2 ,
t [sk , sk+1 ) .
(2)
If in addition supt |Xt | K for some finite, non-random constant K, then V(n ) (X)
is a Cauchy sequence in L2 (, F , P) for any fixed b and finite partitions n of [0, b]
such that kn k 0.
()
Proof. (a). With FtX Ft , the Ft -adapted process Mt = Mt of continuous

sample paths is integrable (by the assumed square integrability of Xt ). Noting that
for any k 0 and all sk s < t sk+1 ,
Mt Ms = Xt2 Xs2 (Xt Xsk )2 + (Xs Xsk )2 = 2Xsk (Xt Xs ) ,
clearly then E[Mt Ms |Fs ] = 2Xsk E[Xt Xs |Fs ] = 0, by the martingale property
of (Xt , Ft ), which suffices for verifying that {Mt , Ft , t 0} is a martingale.
(b). Utilizing these martingales, we now turn to prove the second claim of the
lemma. To this end, fix two finite partitions and of [0, b] and let
b denote
( )
the partition based on the collection of points . With Ut = Vt (X) and

()
()
Ut = Vt (X), applying part (a) of the proof for the martingale Zt = Mt
( )
Mt
= Ut Ut (which is square-integrable by the assumed boundedness of Xt ),
(b
)
we deduce that Zt2 Vt (Z), t [0, b] is a martingale whose value at t = 0 is zero.
(2)
(2)
Noting that Zb = V( ) (X) V() (X), it then follows that
h
2 i
(b
)
(2)
(2)
= E[Zb2 ] = E[Vb (Z)] .
E V( ) (X) V() (X)
(b
)
Next, recall (8.2.10) that Vb (Z) is a finite sum of terms of the form (Uu Us Uu +
Us )2 2(Uu Us )2 + 2(Uu Us )2 . Consequently, V (b) (Z) 2V (b) (U ) + 2V (b) (U )
312

(2)
and to conclude that V(n ) (X) is a Cauchy sequence in L2 (, F , P) for any finite
(b
)
partitions n of [0, b] such that kn k 0, it suffices to show that E[Vb (U )] 0

as k k kk 0.
To establish the latter claim, note first that since
b is a refinement of , each
interval [tj , tj+1 ] of
b is contained within some interval [si , si+1 ] of , and then
Utj+1 Utj = (Xtj+1 Xsi )2 (Xtj Xsi )2 = (Xtj+1 Xtj )(Xtj+1 + Xtj 2Xsi )
(see (8.2.10)). Since tj+1 si kk, this implies in turn that
(Utj+1 Utj )2 4(Xtj+1 Xtj )2 [osckk (X)]2 ,
where osc (X) = sup{|Xt Xs | : |t s| , t, s [0, b]}. Consequently,

(b
)
Vb
(b
)
(U ) 4Vb
(X)[osckk (X)]2
and by the Cauchy-Schwarz inequality

h
i2
h
2 i h
4 i
(b
)
(b
)
EVb (U ) 16E Vb (X) E osckk (X)
.
The random variables osc (X) are uniformly (in and ) bounded (by 2K) and
converge to zero as 0 (in view of the uniform continuity of t 7 Xt on [0, b]).
Thus, by bounded convergence the right-most expectation in the preceding inequal(b
)
ity goes to zero as kk 0. To complete the proof simply note that Vb (X) is of
P
the form j=1 Dj2 for the differences Dj = Xtj Xtj1 of the uniformly bounded
2
(b
)
discrete time martingale {Xtj }, hence E[ Vb (X) ] 6K 4 by part (c) of Exercise
5.1.8.

Building on the preceding lemma, the following decomposition is an important
special case of the more general Doob-Meyer decomposition and a key ingredient
in the theory of stochastic integration.
Theorem 8.2.45. For X Mc2 , the continuous modification of V (2) (X)t is the
unique Ft -increasing process At = hXit of continuous sample functions, such that
Mt = Xt2 At is an Ft -martingale (also of continuous sample functions), and any
two such decompositions of Xt2 as the sum of a martingale and increasing process
are indistinguishable.
Proof. Step 1. Uniqueness. If Xt2 = Mt +At = Nt +Bt with At , Bt increasing
processes of continuous sample paths and Mt , Nt martingales, then Yt = Nt Mt =
At Bt is a martingale of continuous sample paths, starting at Y0 = A0 B0 = 0,
such that V (Y )t V (A)t + V (B)t = At + Bt is finite for any t finite. From Lemma
8.2.43 we then deduce that w.p.1. Yt = 0 for all t 0 (i.e. {At } is indistinguishable
of {Bt }), proving the stated uniqueness of the decomposition.
Step 2. Existence of V (2) (X)t when X is uniformly bounded.
Turning to construct such a decomposition, assume first that X Mc2 is uniformly
( )
(in t and ) bounded by a non-random finite constant. Let V (t) = Vt (X) of
(8.2.10) for the partitions of [0, ) whose elements are the dyadic Q(2,) =
(2)
{k2, k Z+ }. By definition, V (t) = V( ) (X) for the partitions of [0, t] whose
(2,)
elements are the finite collections Qt+ of dyadic from [0, t] augmented by
{t}. Since k k k k = 2 , we deduce from Lemma 8.2.44 that per t 0 fixed,
{V (t), 1} is a Cauchy sequence in L2 (, F , P). Recall Proposition 4.3.7 that
313
any Cauchy sequence in L2 (, F , P) has a limit. So, in particular V (t) converges

in L2 , for , to some U (t, ). For any (other) sequence
e2n of finite partitions
of [0, t] such that ke
2n k 0, upon interlacing
e2n+1 = n we further have by
(2)
Lemma 8.2.44 that V(en ) (X) is a Cauchy, hence convergent in L2 , sequence. Its
limit coincides with the sub-sequential limit U (t, ) along n = 2 + 1, which also
(2)
e2n
matches the L2 limit of V(e2n ) (X). As this applies for any finite partitions
of [0, t] such that ke
2n k 0, we conclude that (t, ) 7 U (t, ) is the quadratic
variation of {Xt }.
Step 3. Constructing At . Turning to produce a continuous modification At () of
U (t, ), recall Lemma 8.2.44 that for each the process M,t = Xt2 V (t) is an
Ft -martingale of continuous sample path. The same applies for Vn (t) Vm (t) =
Mm,t Mn,t , so fixing an integer j 1 we deduce by Doobs L2 maximal inequality
(see (8.2.5) of Corollary 8.2.16), that
E[kVn Vm k2j ] 4E[(Vn (j) Vm (j))2 ] ,
where kf kj = sup{|f (t)| : t [0, j]} makes Y = C([0, j]) into a Banach space (see
part (b) of Exercise 4.3.8). In view of the L2 convergence of Vn (j) we have that
E[(Vn (j) Vm (j))2 ] 0 as n, m , hence Vn : 7 Y is a Cauchy sequence
in L2 (, F , P; Y), which by part (a) of Exercise 4.3.8 converges in this space to
some Uj (, ) C([0, j]). By the preceding we further deduce that Uj (t, ) is a
continuous modification on [0, j] of the pointwise L2 limit function U (t, ). In view
of Exercise 7.2.3, for any j > j the S.P.-s Uj and Uj are indistinguishable on [0, j],
so there exists one square-integrable, continuous modification A : 7 C([0, ))
of U (t, ) whose restriction to each [0, j] coincides with Uj (up to one P-null set).
Step 4. The decomposition: At increasing and Mt = Xt2 At martingale.
First, as V (0) = 0 for all , also A0 = U (0, ) = 0. We saw in Step 3 that
kV Akj 0 in L2 , hence also in L1 and consequently E[(kV Akj )] 0 when
, for (r) = r/(1 + r) r and any fixed positive integer j. Hence,
E[(V , A)] =
X
j=1
2j E[(kV Akj )] 0 ,
as , where (, ) is a metric on C([0, )) for the topology of uniform

convergence on compact intervals (see Exercise 7.2.9). To verify that At is an
a.s.
Ft -increasing process, recall Theorem 2.2.10 that (Vnk , A) 0 along some nonrandom subsequence nk . That is, with F0 augmented as usual by all P-null sets,
Vnk (t, ) At () as k , for all t 0 and
/ N , where N F0 is such that
P(N ) = 0. Setting At 0 when N , the Ft -adaptedness of Vnk (t) transfers to
()
At . Also, by construction t 7 Vt is non-decreasing when restricted to the times
(2)
in . Moreover, if q < q Q then for all k large enough q, q nk implying
that Vnk (q) Vnk (q ). Taking k it follows that Aq () Aq () for all ,
thus by sample path continuity, At is an Ft -increasing process.
Finally, since the Ft -martingales M,t converge in L1 for (and t 0 fixed),
to the Ft -adapted process Mt = Xt2 At , it is easy to check that {Mt , Ft , t 0} is
a martingale.
Step 5. Localization. Having established the stated decomposition in case X Mc2
is uniformly bounded by a non-random constant, we remove the latter condition
314
by localizing via the stopping times r = inf{t 0 : |Xt | r} for positive integers r. Indeed, note that since Xt () is bounded on any compact time interval,
(r)
r when r (for each ). Further, with Xt = Xtr a uniformly
bounded (by r), continuous martingale (see Corollary 8.2.29), by the preceding
(r)
proof we have Ft -increasing processes At , each of which is the continuous mod(r)
(r)
(2)
2
At
ification of the quadratic variation V (X (r) )t , such that Mt = Xt
r
are continuous Ft -martingales. Since E[(V ( ) (X (r) ), A(r) )] 0 for and
each positive integer r, in view of Theorem 2.2.10 we get by diagonal selection the
existence of a non-random sub-sequence nk and a P-null set N such that
/ N . From (8.2.10) we note
(V (nk ) (X (r) ), A(r) ) 0 for k , all r and
()
()
/ N then
that Vt (X (r) ) = Vtr (X) for any t, , r and . Consequently, if
(r)
At
(r)
(r)
= Atr for all t 0 and At
(r )
= At
(r)
as long as r r and t r . Since

(r )
(r)
t 7 At () are non-decreasing, necessarily At At for all t 0. We thus

(r)
(r)
deduce that At At for any
/ N and all t 0. Further, with At independent of r as soon as r () t, the non-decreasing sample function t 7 At ()
(r)
inherits the continuity of t 7 At (). Taking At () 0 for N we proceed
to show that At is integrable, hence an Ft -increasing process of continuous sample
2
, by monotone confunctions. To this end, fixing u 0 and setting Zr = Xu
r
(r)
(r)
(r)
vergence EZr = EMu + EAu EAu when r (as Mt are martingales,
(r)
starting at M0 = 0). Since u r u and the sample functions t 7 Xt are continuous, clearly Zr Xu2 . Moreover, supr |Zr | (sup0su |Xs |)2 is integrable (by
Doobs L2 maximal inequality (8.2.5)), so by dominated convergence EZr EXu2 .
Consequently, EAu = EXu2 is finite, as claimed.
(2)
Next, fixing t 0, > 0, r Z+ and a finite partition of [0, t], since V() (X) =
(2)
V() (X (r) ) whenever r t, clearly,
(r)
(2)
(2)
(r)
{|V() (X) At | 2} {r < t} {|V() (X (r) ) At | } {|At At | } .

(2)
(r)
We have shown already that V() (X (r) ) At
as kk 0. Hence,
(r)
(2)
lim sup P(|V() (X) At | 2) P(r < t) + P(|At At | )

kk0
(2)
and considering r we deduce that V() (X) At . That is, the process {At }
is a modification of the quadratic variation of {Xt }.
We complete the proof by verifying that the integrable, Ft -adapted process Mt =
Xt2 At of continuous sample functions satisfies the martingale condition. Indeed,
(r)
since Mt are Ft -martingales, we have for each s u and all r that w.p.1
(r)
2
|Fs ] = E[A(r)
E[Xu
u |Fs ] + Ms .
r
(r)
2
Xu2 and a.s. Au
Considering r we have already seen that Xu
r
(r) a.s.
2
} integrable, we get by dominated
Au , hence also Ms Ms . With supr {Xu
r
2
convergence of C.E. that E[Xur |Fs ] E[Xu2 |Fs ] (see Theorem 4.2.26). Similarly,
(r)
E[Au |Fs ] E[Au |Fs ] by monotone convergence of C.E. hence w.p.1 E[Xu2 |Fs ] =
E[Au |Fs ] + Ms for each s u, namely, (Mt , Ft ) is a martingale.
315
The following exercise shows that X Mc2 has zero q-th variation for all q > 2.
Moreover, unless Xt Mc2 is zero throughout an interval of positive length, its q-th
variation for 0 < q < 2 is infinite with positive probability and its sample path are
then not locally -H
older continuous for any > 1/2.
Exercise 8.2.46.
(a) Suppose S.P. {Xt , t 0} of continuous sample functions has an a.s.
finite r-th variation V (r) (X)t for each fixed t > 0. Show that then for
each t > 0 and q > r a.s. V (q) (X)t = 0 whereas if 0 < q < r, then
V (q) (X)t = for a.e. for which V (r) (X)t > 0.
et is a S.P. of continuous sample path and
(b) Show that if X Mc2 and A
finite total variation on compact intervals, then the quadratic variation
et is hXit .
of Xt + A
(c) Suppose X Mc2 and Ft -stopping time are such that hXi = 0. Show
that P(Xt = 0 for all t 0) = 1.
(d) Show that if a S.P. {Xt , t 0} is locally -H
older continuous on [0, T ]
for some > 1/2, then its quadratic variation on this interval is zero.
Remark. You may have noticed that so far we did not need the assumed rightcontinuity of Ft . In contrast, the latter assumption plays a key role in our proof of
the more general Doob-Meyer decomposition, which is to follow next.
We start by stating the necessary and sufficient condition under which a subMG has a Doob-Meyer decomposition, namely, it is the sum of a martingale and
increasing part.
Definition 8.2.47. An Ft -progressively measurable (and in particular Ft -adapted,
right-continuous), S.P. {Yt , t 0} is of class DL if the collection {Yu , an Ft stopping time} is U.I. for each finite, non-random u.
Theorem 8.2.48 (Doob-Meyer decomposition). A right-continuous, sub-MG
{Yt , t 0} for {Ft } admits the decomposition Yt = Mt + At with Mt a rightcontinuous Ft -martingale and At an Ft -increasing process, if and only if {Yt , t 0}
is of class DL.
Remark 8.2.49. To extend the uniqueness of Doob-Meyer decomposition beyond
sub-MGs with continuous sample functions, one has to require At to be a natural
process. While we do not define this concept here, we note in passing that every
continuous increasing process is a natural process and a natural process is also
an increasing process (c.f. [KaS97, Definition 1.4.5]), whereas the uniqueness is
attained since if a finite linear combination of natural processes is a martingale,
then it is indistinguishable from zero (c.f. proof of [KaS97, Theorem 1.4.10]).
Proof outline. We focus on constructing the Doob-Meyer decomposition
for {Yt , t I} in case I = [0, 1]. To this end, start with the right-continuous
modification of the non-positive Ft -sub-martingale Zt = Yt E[Y1 |Ft ], which exists
since t 7 EZt is right-continuous (see Theorem 8.2.25). Suppose you can find
A1 L1 (, F1 , P) such that
(8.2.11)
At = Zt + E[A1 |Ft ] ,
is Ft -increasing on I. Then, Mt = Yt At must be right-continuous, integrable and

Ft -adapted. Moreover, for any t I,
Mt = Yt At = Yt Zt E[A1 |Ft ] = E[M1 |Ft ] .
316
So, by the tower property (Mt , Ft , t I) satisfies the martingale condition and we
are done.
(2,)
Proceeding to construct such A1 , fix 1 and for the (ordered) finite set Q1
of dyadic rationals recall Doobs decomposition (in Theorem 5.2.1), of the discrete
(2,)
time sub-MG {Zsj , Fsj , sj Q1 } as the sum of a discrete time U.I. martingale
()
(2,)
{Msj , sj Q1 } and the predictable, non-decreasing (in view of Exercise 5.2.2),
(2,)
()
()
finite sequence {Asj , sj Q1 }, starting with A0 = 0. Noting that Z1 = 0, or
()
()
(2,)
equivalently M1 = A1 , it follows that for any q Q1
()
()
()
A()
q = Zq Mq = Zq E[M1 |Fq ] = Zq + E[A1 |Fq ] .
(8.2.12)
Relying on the fact that the sub-MG {Yt , t I} is of class DL, this representation
()
allows one to deduce that the collection {A1 , 1} is U.I. (for details see [KaS97,
proof of Theorem 1.4.10]). This in turn implies by the Dunford-Pettis compactness
criterion that there exists an integrable A1 and a non-random sub-sequence nk
(n ) wL1
such that A1 k A1 , as in Definition 4.2.31. Now consider the Ft -adapted,

integrable S.P. defined via (8.2.11), where by Theorem 8.2.25 (and the assumed
right continuity of the filtration {Ft }), we may and shall assume that the U.I. MG
E[A1 |Ft ] has right-continuous sample functions (and hence, so does t 7 At ). Since
(2,)
(2)
(2)
Q1
Q1 , upon comparing (8.2.11) and (8.2.12) we find that for any q Q1
and all large enough
()
Aq() Aq = E[A1 A1 |Fq ] .

(n ) wL1
(2)
Consequently, Aq k Aq for all q Q1 (see Exercise 4.2.32). In particular,

(2)
A0 = 0 and setting q < q Q1 , V = I{Aq >Aq } we deduce by the monotonicity
()
of j 7 Asj for each and , that
(n )
E[(Aq Aq )V ] = lim E[(Aq(nk ) Aq k )V ] 0 .

k
So, by our choice of V necessarily P(Aq > Aq ) = 0 and consequently, w.p.1. the
(2)
sample functions t 7 At () are non-decreasing over Q1 . By right-continuity the
same applies over I and we are done, for {At , t I} of (8.2.11) is thus indistinguishable from an Ft -increasing process.
The same argument applies for I = [0, r] and any r Z+ . While we do not
do so here, the Ft -increasing process {At , t I} can be further shown to be a
natural process. By the uniqueness of such decompositions, as alluded to in Remark 8.2.49, it then follows that the restriction of the process {At } constructed on
[0, r ] to a smaller interval [0, r] is indistinguishable from the increasing process one
constructed directly on [0, r]. Thus, concatenating the processes {At , t r} and
{Mt , t r} yields the stated Doob-Meyer decomposition on [0, ).
As for the much easier converse, fixing non-random u R, by monotonicity of
t 7 At the collection {Au , an Ft -stopping time} is dominated by the integrable
Au hence U.I. Applying Doobs optional stopping theorem for the right-continuous
MG (Mt , Ft ), you further have that Mu = E[Mu |F ] for any Ft -stopping time
(see part (a) of Exercise 8.2.30), so by Proposition 4.2.33 the collection {Mu ,
an Ft -stopping time} is also U.I. In conclusion, the existence of such Doob-Meyer
decomposition Yt = Mt + At implies that the right-continuous sub-MG {Yt , t 0}
is of class DL (recall part (b) of Exercise 1.3.55).
317
Your next exercise provides a concrete instance in which Doob-Meyer decomposition applies, connecting it with the decomposition in Theorem 8.2.45 of the nonnegative sub-MG Yt = Xt2 of continuous sample path, as the sum of the quadratic
variation hXit and the continuous martingale Xt2 hXit .
Exercise 8.2.50. Suppose {Yt , t 0} is a non-negative, right-continuous, sub-MG
for {Ft }.
(a) Show that Yt is in class DL.
(b) Show that if Yt further has continuous sample functions then the processes
Mt and At in its Doob-Meyer decomposition also have continuous sample
functions (and are thus unique).
Remark. From the preceding exercise and Remark 8.2.49, we associate to each
X M2 a unique natural process, denoted hXit and called the predictable quadratic
variation of X, such that Xt2 hXit is a right-continuous martingale. However,
when X
/ Mc2 , it is no longer the case that the predictable quadratic variation
matches the quadratic variation of Definition 8.2.41 (as a matter of fact, the latter
may not exist).
Example 8.2.51. A standard Brownian Markov process consists of a standard
Wiener process {Wt , t 0} and filtration {Ft , t 0} such that FsW Fs for any
s 0 while (Wt Ws , t s) is independent of Fs (see also Definition 8.3.7 for its
Markov property). For right-continuous augmented filtration Ft , such process Wt
is in Mc2 and further, Mt = Wt2 t is a martingale of continuous sample path. We
thus deduce from Theorem 8.2.45 that its (predictable) quadratic variation is the
non-random hW it = t, which by Exercise 8.2.46 implies that the total variation of
the Brownian sample path is a.s. infinite on any interval of positive length. More
generally, recall part (b) of Exercise 8.2.6 that hXit is non-random for any Gaussian martingale, hence so is the quadratic variation of any Gaussian martingale of
continuous sample functions.
As you show next, the type of convergence to the quadratic variation may be
strengthened (e.g. to convergence in L2 or a.s.) for certain S.P. by imposing some
restrictions on the partitions considered.
(2)
Exercise 8.2.52. Let V(n ) (W ) denote the quadratic variations of the Wiener
process on a sequence of finite partitions n of [0, t] such that kn k 0 as n .
(2)
L2
(2)
a.s.
(a) Show that V(n ) (W ) t.
(b) Show that V(n ) (W ) t whenever
n=1
kn k < .
Remark. However, beware that for a.e. there exist random finite partitions
(2)
n of [0, 1] such that kn k 0 and V(n ) (W ) (see [Fre71, Page 48]).
Example 8.2.53. While we shall not prove it, Levys martingale characterization
of the Brownian motion states the converse of Example 8.2.51, that any X Mc2
of quadratic variation hXit = t must be a standard Brownian Markov process (c.f.
[KaS97, Theorem 3.3.16]). However, recall Example 8.2.5 that for a Poisson process Nt of rate , the compensated process Mt = Nt t is in M2 and you can easily
check that Mt2 t is then a right-continuous martingale. Since the continuous increasing process t is natural, we deduce from the uniqueness of the Doob-Meyer
decomposition that hM it = t. More generally, by the same argument we deduce
318
from part (c) of Exercise 8.2.6 that hXit = tE(X12 ) for any square-integrable S.P.
with X0 = 0 and zero-mean, stationary independent increments. In particular,
this shows that sample path continuity is necessary for Levys characterization of
the Brownian motion and that the standard Wiener process is the only zero-mean,
square-integrable stochastic process Xt of continuous sample path and stationary
independent increments, such that X0 = 0.
Building upon Levys characterization, you can now prove the following special
case of the extremely useful Girsanovs theorem.
Exercise 8.2.54. Suppose (Wt , Ft , t 0) is a standard Brownian Markov process
on a probability space (, F , P) and fixing a non-random parameters R and
T > 0 consider the exponential Ft -martingale Zt = exp(Wt 2 t/2) and the
corresponding probability measure QT (A) = E(IA ZT ) on (, FT ).
Rt
(a) Show that V (2) (Z)t = 2 0 Zu2 du.
fu = Wu u is for u [0, T ] an Fu -martingale on the
(b) Show that W
probability space (, FT , QT ).
ft , Ft , t T ) is a standard Brownian Markov process on
(c) Deduce that (W
(, FT , QT ).
Here is the extension to the continuous time setting of Lemma 5.2.7 and Proposition 5.3.31.
Exercise 8.2.55. Let Vt = sups[0,t] Ys and At be the increasing process of continuous sample functions in the Doob-Meyer decomposition of a non-negative, continuous, Ft -submartingale {Yt , t 0} with Y0 = 0.
(a) Show that P(V x, A < y) x1 E(A y) for all x, y > 0 and any
Ft -stopping time .
(b) Setting c1 = 4 and cq = (2 q)/(1 q) for q (0, 1), conclude that
E[sups |Xs |2q ] cq E[hXiq ] for any X Mc2 and q (0, 1], hence
{|Xt |2q , t 0} is U.I. when hXiq is integrable.
Taking X, Y M2 we deduce from the Doob-Meyer decomposition of X Y M2

that (X Y )2t hX Y it are right-continuous Ft -martingales. Considering their
difference we deduce that XY hX, Y i is a martingale for
i
1h
hX, Y it = hX + Y it hX Y it
4
(this is an instance of the more general polarization technique). In particular, XY is
a right-continuous martingale whenever hX, Y i = 0, prompting our next definition.
Definition 8.2.56. For any pair X, Y M2 , we call the S.P. hX, Y it the bracket
of X and Y and say that X, Y M2 are orthogonal if for any t 0 the bracket
hX, Y it is a.s. zero.
Remark. It is easy to check that hX, Xi = hXi for any X M2 . Further, for
any s [0, t], w.p.1.
E[(Xt Xs )(Yt Ys )|Fs ] = E[Xt Yt Xs Ys |Fs ] = E[hX, Y it hX, Y is |Fs ],
so the orthogonality of X, Y M2 amounts to X and Y having uncorrelated

increments over [s, t], conditionally on Fs . Here is more on the structure of the
bracket as a bi-linear form on M2 , which on Mc2 coincides with the cross variation
of X and Y .
8.3. MARKOV AND STRONG MARKOV PROCESSES
319
Exercise 8.2.57. Show that for all X, Xi , Y M2 :
(a) hc1 X1 + c2 X2 , Y i = c1 hX1 , Y i + c2 hX2 , Y i for any ci R, i = 1, 2.

P
Hint: Recall Remark 8.2.49 that a martingale of the form j=1 hUj i
for Uj M2 and finite, is zero.
(b) hX, Y i = hY, Xi.
(c) |hX, Y i|2 hXihY i.
(d) With Zt = V (hX, Y i)t , for a.e. and all 0 s < t < ,
1
[hXit hXis + hY it hY is ] .
2
(e) Show that for X, Y Mc2 the bracket hX, Y it is also the limit in probability as kk 0 of
Zt Zs
k
X
i=1
[Xt() Xt() ][Yt() Yt() ] ,
()
where = {0 = t0
i1
()
< t1
i1
()
< < tk
= t} is a finite partition of [0, t].
We conclude with a brief introduction to stochastic integration (for more on this

topic, see [KaS97, Section 3.2]). Following our general approach to integration, the
Rt
It
o stochastic integral It = 0 Xs dWs is constructed first for simple processes Xt ,
i.e. those having sample path that are piecewise constant on non-random intervals,
as you are to do next.
Exercise 8.2.58. Suppose (Wt , Ft ) is a standard Brownian Markov process and
Xt is a bounded, Ft -adapted, left-continuous simple process. That is,
Xt () = 0 ()1{0} (t) +
i ()1(ti ,ti+1 ] (t) ,
i=0
where 0 = t0 < t1 < < tk < is a non-random unbounded sequence and the
Ftn -adapted sequence {n ()} is bounded uniformly in n and .
Rt
(a) With At = 0 Xu2 du, show that both
It =
k1
X
j=0
j (Wtj+1 Wtj ) + k (Wt Wtk ), when t [tk , tk+1 ) ,
and It2 At are martingales with respect to Ft .

(b) Deduce that It Mc2 with At = hIit being its quadratic variation, and in
Rt
particular EIt2 = 0 E[Xu2 ]du.
8.3. Markov and Strong Markov processes
In Subsection 8.3.1 we define Markov semi-groups and the corresponding Markov

processes. We also extend the construction of Markov chains from Subsection 6.1
to deal with these S.P. This is followed in Subsection 8.3.2 with the study of the
strong Markov property and the related Feller property, showing in particular that
both the Brownian motion and the Poisson process are strong Markov processes.
We then devote Subsection 8.3.3 to the study of Markov jump processes, which are
the natural extension of both Markov chains and (compound) Poisson processes.
320
8.3.1. Markov semi-groups, processes and the Markov property. We

start with the definition of a Markov process, focusing on (time) homogeneous
processes having (stationary, regular) transition probabilities (compare with Definitions 6.1.1 and 6.1.2).
Definition 8.3.1 (Markov processes). A collection {ps,t (, ), t s 0} of
transition probabilities on a measurable space (S, S) (as in Definition 6.1.2), is
consistent if it satisfies the Chapman-Kolmogorov equations
(8.3.1)
pt1 ,t3 (x, B) = pt1 ,t2 pt2 ,t3 (x, B) ,
x S, B S,
for any t3 t2 t1 0 (c.f. Corollary 6.3.3 for the composition of transition

probabilities). In particular, pt,t (x, B) = IB (x) = x (B) for any t 0. Such
collection is called a Markov semi-group (of stationary transition probabilities), if
in addition ps,t = pts for all t s 0. The Chapman-Kolmogorov equations are
then
(8.3.2)
ps+u (x, B) = ps pu (x, B) ,
x S, B S, u, s 0 ,
with p0 (x, B) = IB (x) = x (B) being the semi-group identity element.

An Ft -adapted S.P. {Xt , t 0} taking values in (S, S) is an Ft -Markov process
of (consistent) transition probabilities {ps,t , t s 0} and state space (S, S) if for
any t s 0 and B S, almost surely
(8.3.3)
P(Xt B|Fs ) = ps,t (Xs , B) .
It is further a (time) homogeneous Ft -Markov process of semi-group {pu , u 0} if

for any u, s 0 and B S, almost surely
(8.3.4)
P(Xs+u B|Fs ) = pu (Xs , B) .
Remark. Recall that a Gn -Markov chain {Yn } is a discrete time S.P. Hence, in this
case one considers only t, s Z+ and (8.3.1) is automatically satisfied by setting
ps,t = ps,s+1 ps+1,s+2 pt1,t to be the composition of the (one-step) transition
probabilities of the Markov chain (see Definition 6.1.2, with ps,s+1 = p independent
of s when the chain is homogeneous). Further, the interpolated process Xt = Y[t]
is then a right-continuous Ft -Markov process for the right-continuous interpolated
filtration Ft = G[t] of Example 8.1.5, but {Xt } is in general an inhomogeneous
Markov process, even in case the Markov chain {Yn } is homogeneous.
Similarly, if Ft -adapted S.P. (Xt , t 0) satisfies (8.3.4) for pt (x, B) = P(Xt
B|X0 = x) and x 7 pt (x, B) is measurable per fixed t 0 and B S, then
considering the tower property for IB (Xs+u )I{x} (X0 ) and (X0 ) Fs , one easily
verifies that (8.3.2) holds, hence (Xt , t 0) is a homogeneous Ft -Markov process.
More generally, in analogy with our definition of Markov chains via (6.1.1), one
may opt to say that Ft -adapted S.P. (Xt , t 0) is an Ft -Markov process provided
for each B S and t s 0,
a.s.
P(Xt B|Fs ) = P(Xt B|Xs ) .
Indeed, as noted in Remark 6.1.6 (in view of Exercise 4.4.5), for B-isomorphic
(S, S) this suffices for the existence of transition probabilities which satisfy (8.3.3).
However, this simpler to verify plausible definition of Markov processes results with
Chapman-Kolmogorov equations holding only up to a null set per fixed t3 t2
t1 0. The study of such processes is consequently made more cumbersome, which
is precisely why we, like most texts, do not take this route.
321
By Lemma 6.1.3 we deduce from Definition 8.3.1 that for any f bS and all
t s 0,
(8.3.5)
E[f (Xt )|Fs ] = (ps,t f )(Xs ) ,

R
where f 7 (ps,t f ) : bS 7 bS and (ps,t f )(x) = ps,t (x, dy)f (y) denotes the
Lebesgue integral of f () under the probability measure ps,t (x, ) per fixed x S.
The Chapman-Kolmogorov equations are necessary and sufficient for generating
consistent Markovian f.d.d. out of a given collection of transition probabilities and
a specified initial probability distribution. As outlined next, we thus canonically
construct the Markov process, following the same approach as in proving Theorem
6.1.8 (for Markov chain), and Proposition 7.1.8 (for continuous time S.P.).
Theorem 8.3.2. Suppose (S, S) is B-isomorphic. Given any (S, S)-valued consistent transition probabilities {ps,t , t s 0}, the probability distribution on (S, S)
uniquely determines the linearly ordered consistent f.d.d.
(8.3.6)
0,s1 ,...,sn = p0,s1 psn1 ,sn
for 0 = s0 < s1 < < sn , and there exists a Markov process of state space (S, S)
having these f.d.d. Conversely, the f.d.d. of any Markov process having initial
probability distribution (B) = P(X0 B) and satisfying (8.3.3), are given by
(8.3.6).
Proof. Recall Proposition 6.1.5 that p0,s1 psn1 ,sn denotes the
Markov-product-like measures, whose evaluation on product sets is by iterated integrations over the transition probabilities psk1 ,sk , in reverse order k = n, . . . , 1,
followed by a final integration over the initial measure . As shown in this proposition, given any transition probabilities {ps,t , t s 0}, the probability distribution
on (S, S) uniquely determines p0,s1 psn1 ,sn , namely, the f.d.d. specified in (8.3.6). We then uniquely specify the remaining f.d.d. as the probability
measures s1 ,...,sn (D) = s0 ,s1 ,...,sn (S D). Proceeding to check the consistency
of these f.d.d. note that ps,u pu,t (, S ) = ps,t (, ) for any s < u < t (by the
Chapman-Kolmogorov identity (8.3.1)). Thus, considering s = sk1 , u = sk and
t = sk+1 we deduce that if D = A0 An with Ak = S for some k = 1, . . . , n1,
then
ps0 ,s1 psn1 ,sn (D) = psk1 ,sk+1 psn1 ,sn (Dk )
for Dk = A0 Ak1 Ak+1 An , which are precisely the consistency
conditions of (7.1.3) for the f.d.d. {s0 ,...,sn }. These consistency requirements are
further handled in case of a product set D with An = S by observing that for all
x S and any transition probability psn1 ,sn (x, S) = 1, whereas our definition of
s1 ,...,sn already dealt with A0 = S. Having shown that this collection of f.d.d. is
consistent, recall that Proposition 7.1.8 applies even with (R, B) replaced by the Bisomorphic measurable space (S, S). Setting T = [0, ), it provides the construction
of a S.P. {Yt () = (t), t T} via the coordinate maps on the canonical probability
space (ST , S T , P ) with the f.d.d. of (8.3.6). Turning next to verify that (Yt , FtY , t
T) satisfies the Markov condition (8.3.3), fix t s 0, B S and recall that, for
t > s as in the proof of Theorem 6.1.8, and by definition in case t = s,
(8.3.7)
E[I{Y A} IB (Yt )] = E[I{Y A} ps,t (Ys , B)]
322
for any finite dimensional measurable rectangle A = {x() : x(ti ) Bi , i = 1, . . . , n}

such that ti [0, s] and Bi S. Thus, the collection
L = {A S [0,s] : (8.3.7) holds for A} ,
contains the -system of finite dimensional measurable rectangles which generates

S [0,s] , and in particular, S L. Further, by linearity of the expectation L is closed
under proper difference and by monotone convergence if An L is such that An A
then A L as well. Consequently, L is a -system and by Dynkins theorem,
(8.3.7) holds for every set in S [0,s] = FsY (see Lemma 7.1.7). It then follows that
P(Yt B|FsY ) = ps,t (Ys , B) a.s. for each t s 0 and B S. That is, {Yt , t 0}
is an FtY -Markov process.
Conversely, suppose {Xt , t 0} satisfies (8.3.3) and has initial probability distribution (). Then, for any t0 > > tn s 0, and f bS, = 0, . . . , n, almost
surely,
Z
Z
n
Y
(8.3.8) E[
f (Xt )|Fs ] = ps,tn (Xs , dyn )fn (yn ) pt1 ,t0 (y1 , dy0 )f0 (y0 ) .
=0
The latter identity is proved by induction on n, where denoting its right side by
gn+1,s (Xs ), we see that gn+1,s = ps,tn (fn gn,tn ) and the case n = 0 is merely (8.3.5).
In the induction step we have from the tower property and Ft -adaptedness of {Xt }
that
n1
n
Y
Y
f (Xt )|Ftn ]|Fs ]
E[
f (Xt )|Fs ] = E[fn (Xtn )E[
=0
=0
= E[fn (Xtn )gn,tn (Xtn )|Fs ] = gn+1,s (Xs ) ,
where the induction hypothesis is used in the second equality and (8.3.5) in the
third. In particular, considering the expected value of (8.3.8) for s = 0 and indicator
functions f () it follows that the f.d.d. of this process are given by (8.3.6), as
claimed.

Remark 8.3.3. As in Lemma 7.1.7, for B-isomorphic state space (S, S) any F
F X is of the form F = (X )1 (A) for some A S T , where X () : 7 ST denote
the collection of sample functions of the given Markov process {Xt , t 0}. Then,
P(F ) = P (A), so while proving Theorem 8.3.2 we have defined the law P () of
Markov process {Xt , t 0} as the unique probability measure on S [0,) such that
P ({ : (s ) B , = 0, . . . , n}) = P(Xs0 B0 , . . . , Xsn Bn ) ,
for B S and distinct s 0 (compare with Definition 6.1.7 for the law of a
Markov chain). We denote by Px the law P inR case (B) = IxB , namely, when
X0 = x is non-random and note that P (A) = S Px (A)(dx) for any probability
measure on (S, S) and all A S T , with Px uniquely determined by the specified
(consistent) transition probabilities {ps,t , t s 0}.
The evaluation of the f.d.d. of aPMarkov process is more explicit when S is a
countable set, as then ps,t (x, B) = yB ps,t (x, y) for any B S (and all Lebesgue
integrals are merely sums). Likewise, in case S = Rd (equipped with S = BS ),
computations are relatively explicit if for each t > s 0 and x S the probability
measure ps,t (x, ) is absolutely
R continuous with respect to Lebesgue measure on S,
in which case (ps,t f )(x) = ps,t (x, y)f (y)dy and the right side of (8.3.8) amounts
323
to iterated integrations of the transition probability kernel ps,t (x, y) of the process
with respect to Lebesgue measure on S.
The next exercise is about the closure of the collection of Markov processes under
certain invertible non-random measurable mappings.
Exercise 8.3.4. Suppose (Xt , FtX , t 0) is a Markov process of state space (S, S),
u : [0, ) 7 [0, ) is an invertible, strictly increasing function and for each t 0
e is invertible, with 1
the measurable mapping t : (S, S) 7 (e
S, S)
measurable.
t
Y
X
(a) Setting Yt = t (Xu(t) ), verify that Ft = Fu(t) and that (Yt , FtY , t 0)
e
is a Markov process of state space (e
S, S).
X
(b) Show that if (Xt , Ft , t 0) is a homogeneous Markov process then so is
Zt = 0 (Xt ).
Of particular note is the following collection of Markov processes.
Proposition 8.3.5. If real-valued S.P. {Xt , t 0} has independent increments,
then (Xt , FtX , t 0) is a Markov process of transition probabilities ps,t (y, B) =
PXt Xs ({z : y + z B}), and if {Xt , t 0} further has stationary, independent
increments, then this Markov process is homogeneous.
Proof. Considering Exercise 4.2.2 for G = FsX , Y = Xs mG and the R.V.
Z = Zt,s = Xt Xs which is independent of G, you find that (8.3.3) holds for
ps,t (y, B) = P(y + Z B), which in case of stationary increments depends only on
t s. Clearly, B 7 P(y + Z B) is a probability measure on (R, B), for any t s
and y R. Further, if B = (, b] then ps,t (y, B) = FZ (b y) is a Borel function
of y (see Exercise 1.2.27). As the -system L = {B B : y 7 P(y + Z B)
is a Borel function} contains the -system {(, b] : b R} generating B, it
follows that L = B, hence ps,t (, ) is a transition probability for each t s 0.
To verify that the Chapman-Kolmogorov equations hold, fix u [s, t] noting that
Zs,t = Zs,u + Zu,t , with Zu,t = Xt Xu independent of Zs,u = Xu Xs . Hence,
by the tower property,
ps,t (y, B) = E[P(y + Zs,u + Zu,t B|Zs,u )]
= E[pu,t (y + Zs,u , B)] = (ps,u (pu,t IB ))(y) = ps,u pu,t (y, B) ,
and this relation, i.e. (8.3.1), holds for all y R and B B, as claimed.
Among the consequences of Proposition 8.3.5 is the fact that both the Brownian
motion and the Poisson process (potentially starting at N0 = x R), are homogeneous Markov processes of explicit Markov semi-groups.
Example 8.3.6. Recall Proposition 3.4.9 and Exercise 7.3.13 that both the Poisson process and the Brownian motion are processes of stationary independent increments. Further, this property clearly extends to the Brownian motion with drift
(r)
(r)
Zt = Wt + rt + x, and to the Poisson process with drift Nt = Nt + rt + x, where
the drift r R is a non-random constant, x R is the specified (under Px ), initial
(r)
(r)
value of N0 (or Z0 ), and Nt N0 is a Poisson process of rate . Consequently,
(r)
(r)
both {Zt , t 0} and {Nt , t 0} are real-valued homogeneous Markov processes.
Specifically, from the preceding proposition we have that the Markov semi-group of
the Brownian motion with drift is pt (x + rt, B), where for t > 0,
Z (yx)2 /2t
e
dy ,
(8.3.9)
pt (x, B) =
2t
B
324
having the transition probability kernel pt (x, y) = exp((y x)2 /2t)/ 2t. Similarly, the Markov semi-group of the Poisson process with drift is qt (x + rt, B),
where
X
(t)k
(8.3.10)
qt (x, B) = et
IB (x + k) .
k!
k=0
Remark. Homogeneous Markov chains are characterized by their (one-step) transition probabilities, whereas each homogeneous Markov process has a full semigroup pt (), t 0. While outside our scope, we note in passing that the semi-group
relation (8.3.2) can be rearranged as s1 (ps+t pt ) = s1 (ps p0 )pt , which subject
to the appropriate regularity conditions should yield for s 0 the celebrated backward Kolmogorov equation t pt = Lpt . The operator L = lims0 s1 (ps p0 ) is then
called the generator of the Markov process (or its semi-group). For example, the
transition probability kernel pt (x + rt, y) of the Brownian motion with drift solves
the partial differential equation (pde), ut = 12 uxx + rux and the generator of this
semi-group is Lu = 12 uxx + rux (c.f. [KaS97, Chapter 5]). For this reason, many
computations about Brownian motion can also be done by solving rather simple
elliptic or parabolic pde-s.
We saw in Proposition 8.3.5 that the Wiener process (Wt , t 0) is a homogeneous
FtW -Markov process of continuous sample functions and the Markov semi-group
of (8.3.9). This motivates the following definition of a Brownian Markov process
(Wt , Ft ), where our accommodation of possible enlargements of the filtration and
different initial distributions will be useful in future applications.
Definition 8.3.7 (Brownian Markov process). We call (Wt , Ft ) a Brownian
Markov process if {Wt , t 0} of continuous sample functions is a homogeneous Ft Markov process with the Brownian semi-group {pt , t 0} of (8.3.9). If in addition
W0 = 0, we call such process a standard Brownian Markov process.
Stationarity of Markov processes, in the sense of Definition 7.3.7, is related to the
important concept of invariant probability measures which we define next (compare
with Definition 6.1.20).
Definition 8.3.8. A probability measure on a B-isomorphic space (S, S) is
called an invariant (probability) measure for Ra semi-group of transition probabilities {pu , u 0}, if the induced law P () = S Px ()(dx) (see Remark 8.3.3), is
invariant under any time shift s , s 0.
You can easily check that if Markov process is also a stationary process under an
initial probability measure , then it is effectively a homogeneous Markov process,
in the sense that ps,t (x, ) = pts (x, ) for any t s 0 and -a.e. x S.
However, many homogeneous Markov processes are non-stationary (for example,
recall Examples 7.3.14 and 8.3.6, that the Brownian motion is non-stationary yet
homogeneous, Markov process).
Here is the explicit characterization of invariant measures for a given Markov
semi-group and their connection to stationary Markov processes.
Exercise 8.3.9. Adapting the proof of Proposition 6.1.23 show that a probability
measure on B-isomorphic (S, S) is an invariant measure for a Markov semi-group
{pu , u 0}, if and only if pt = for any t 0 (note that a homogeneous Markov

Brownian Bridge B
325
Geometric Brownian Motion Y
18
0.8
16
14
0.6
12
0.4
t
10
0.2
8
0
6
0.2
0.4
0.6
0.1
0.2
0.3
0.4
0.5
t
0.6
0.7
0.8
0.9
0.2
0.4
OrnsteinUhlenbeck process Ut
0.6
0.8
1
t
1.2
1.4
1.6
1.8
1.6
1.8
Brownian Motion with drift: = 1, = 2, x= 1
0.5
0.4
8
7
0.3
6
0.2
t
5
0.1
4
0
3
0.1
0.2
0.3
0.1
0.2
0.3
0.4
0.5
t
0.6
0.7
0.8
0.9
0.2
0.4
0.6
0.8
1
t
1.2
1.4
Figure 1. Illustration of sample paths for processes in Exercise 8.3.10.

process {Xt , t 0} is a stationary S.P. if and only if the initial distribution (B) =
P(X0 B) is an invariant probability measure for the corresponding Markov semigroup).
Pursuing similar themes, your next exercise examines some of the most fundamental S.P. one derives out of the Brownian motion.
Exercise 8.3.10. With {Wt , t 0} a Wiener process, consider the Geometric
Brownian motion Yt = eWt , Ornstein-Uhlenbeck process Ut = et/2 Wet , Brownian
(r,)
motion with drift Zt
= Wt + rt and the standard Brownian bridge on [0, 1] (as
in Exercises 7.3.15-7.3.16).
(a) Determine which of these four S.P. is a Markov process with respect to
its canonical filtration, and among those, which are also homogeneous.
(b) Find among these S.P. a homogeneous Markov process whose increments
are neither independent nor stationary.
(c) Find among these S.P. a Markov process of stationary increments, which
is not a homogeneous Markov process.
Homogeneous Markov processes possess the following Markov property, extending
the invariance (8.3.4) of the process under the time shifts s of Definition 7.3.7 to
any bounded S [0,) -measurable function of its sample path (compare to (6.1.8) in
case of homogeneous Markov chains).
Proposition 8.3.11 (Markov property). Suppose (Xt , t 0) is a homogeneous Ft -Markov process on a B-isomorphic state space (S, S) and let Px denote
the corresponding family of laws associated with its semi-group. Then, x 7 Ex [h]
is measurable on (S, S) for any h bS [0,) , and further for any s 0, almost
surely
(8.3.11)
E[h s (X ())|Fs ] = EXs [h] .
326
Remark 8.3.12. From Lemma 7.1.7 you can easily deduce that any V bF X
is of the form V = h(X ) with h bS [0,) . Further, in view of Exercises 1.2.32
and 7.2.9, any bounded Borel function h() on the space C([0, )) of continuous
functions equipped with the topology of uniform convergence on compact intervals
is the restriction to C([0, )) of some e
h bR[0,) . In particular, for a real-valued
Markov process {Xt , t 0} of continuous sample functions, Ex [e
h] = Ex [h] and
e
hs (X ) = hs (X ), hence (8.3.11) applies for any bounded, BC([0,)) -measurable
function h.
Qn
Proof. Fixing s 0, in case h(x()) = =0 f (x(u )) for finite n, f bS
and u0 > > un 0, we have by (8.3.8) for t = s + u and the semi-group
pr,t = ptr of (Xt , t 0), that
E[
n
Y
=0
f (Xt )|Fs ] = pun (fn pun1 un ( (f1 pu0 u1 f0 )))(Xs ) = EXs [
n
Y
f (Xu )] .
=0
The measurability of x 7 Ex [h] for such functionals h() is verified by induction on

n, where if n = 0 then for f0 bS by Lemma 6.1.3 also Ex h = g1 (x) = pu0 f0 (x)
is in bS and by the same argument, in the induction step gn+1 (x) = pun (fn gn )(x)
are also in bS.
To complete the proof consider the collection H of functionals h bS [0,) such
that x 7 Ex [h] is S-measurable and (8.3.11) holds. The linearity of the (conditional) expectation and the monotone convergence theorem result with H a vectorspace that is closed under monotone limits, respectively. Further, as already shown,
H contains the indicators h() = IA () with A = {x() : x(u ) B S, =
0, . . . , n} a finite dimensional measurable rectangle. Thus, H satisfies the conditions of the monotone class theorem. Consequently H = bS [0,) , that is, for each
h S [0,) both x 7 Ex [h] bS and (8.3.11) holds w.p.1.

8.3.2. Strong Markov processes and Feller semi-groups. Given a homogeneous Ft -Markov process (Xt , t 0), we seek to strengthen its Markov property
about the shift of the sample path by non-random s 0 (see Proposition 8.3.11),
to the strong Markov property, whereby shifting by any Ft -Markov time is accommodated (see Proposition 6.1.16 about Markov chains having this property).
Definition 8.3.13 (strong Markov process). We say that an Ft -progressively
measurable, homogeneous Markov process {Xt , t 0} on B-isomorphic state space
(S, S), has the strong Markov property (or that (Xt , Ft ) is a strong Markov process), if for any bounded h(s, x()) measurable on the product -algebra U = B[0,)
S [0,) , and any Ft -Markov time , almost surely
(8.3.12)
I{ <} E[h(, X + ())|F + ] = gh (, X )I{ <} ,
where gh (s, x) = Ex [h(s, )] is bounded and measurable on B[0,) S, x 7 Px are

the laws associated with the semi-group of (Xt , Ft ), F + is the Markov -algebra
associated with (c.f. Definition 8.1.9), and both sides of (8.3.12) are set to zero
when () = .
As noted in Remark 8.3.12, every V bF X is of the form V = h(X ), with h()
in the scope of the strong Markov property, which for a real-valued homogeneous
Markov process {Xt , t 0} of continuous sample functions, contains all bounded
327
Borel functions h(, ) on [0, ) C([0, )). In applications it is often handy to

further have a time varying functional h(s, x()) (for example, see our proof of the
reflection principle, in Proposition 9.1.10).
Remark. Recall that the Markov time is an Ft+ -stopping time (see Definition
8.1.9), hence the assumed Ft -progressive measurability of {Xt } guarantees that on
the event { < } the R.V. and X are measurable on F + (see Proposition
8.1.13), hence by our definition so is gh (, X ). While we have stated and proved
Proposition 8.1.13 only in case of real-valued S.P. {Xt , t 0}, the same proof (and
conclusion), applies for any state space (S, S). We lose nothing by assuming progressive measurability of {Xt } since for a right-continuous process this is equivalent
to its adaptedness (see Proposition 8.1.8, whose proof and conclusion extend to any
topological state space).
Here is an immediate consequence of Definition 8.3.13.
Corollary 8.3.14. If (Xt , Ft ) is a strong Markov process and is an Ft -stopping
time, then for any h bU, almost surely
(8.3.13)
I{ <} E[h(, X + ())|F ] = gh (, X )I{ <} .
In particular, if (Xt , Ft ) is a strong Markov process, then {Xt , t 0} is a homogeneous Ft+ -Markov process and for any s 0 and h bS [0,) , almost surely
(8.3.14)
E[h(X )|Fs+ ] = E[h(X )|Fs ] .
Proof. By the preceding remark, having an Ft -stopping time results with

gh (, X )I{ <} which is measurable on F . Thus, applying the tower property
for the expectation of (8.3.12) conditional on F F + , results with (8.3.13).
Comparing (8.3.13) and (8.3.12) for constant in time h(x()) and the non-random,
finite stopping time = s we deduce that (8.3.14) holds whenever h = h0 s
for some h0 bS [0,) . Since (Xt , Ft ) is a homogeneous Markov process, considering h0 (x()) = IB (x(u)) for u 0 and B S, it follows that (8.3.4) holds
also for (Xt , Ft+ ), namely, that {Xt , t 0} is a homogeneous Ft+ -Markov process. With H denoting the collection of functionals h bS [0,) for which (8.3.14)
holds, by the monotone class theorem it suffices to check that this is the case when
Qk
h(x()) = m=1 IBm (x(um )), with k finite, um 0 and B
Qm S. Representing
such functionals as h() = h1 ()h0 s () with h0 (x()) = um s IBm (x(um s))
Q
and h1 (x()) = um <s IBm (x(um )), we complete the proof by noting that h1 (X ))
is measurable with respect to Fs Fs+ , so can be taken out of both conditional
expectations in (8.3.14) and thus eliminated.

To make the most use of the strong Markov property, Definition 8.3.13 calls for
an arbitrary h bU. As we show next, for checking that a specific S.P. is a strong
Markov process, it suffices to verify (8.3.12) only for h(s, x()) = IB (x(u)) and
bounded Markov times (compare with the definition of a homogeneous Markov
process via (8.3.4)), which is way more manageable task.
Proposition 8.3.15. An Ft -progressively measurable, homogeneous Markov process {Xt , t 0} with a semi-group {pu , u 0} on B-isomorphic state space (S, S),
has the strong Markov property if for any u 0, B S and bounded Ft -Markov
times , almost surely
(8.3.15)
P[X +u B|F + ] = pu (X , B) .
328
Proof. Step 1. We start by extending the validity of (8.3.15) to any a.s.

finite Markov time. To this end, fixing u 0, B S, n Z+ and a [0, ]valued Ft -Markov time , recall that n = n is a bounded Ft+ -stopping time
(c.f. part (c) of Exercise 8.1.10). Further, the bounded I{ n} and pu (Xn , B) are
both measurable on Fn+ (see part (a) of Exercise 8.1.11 and Proposition 8.1.13,
respectively). Hence, multiplying the identity (8.3.15) in case of n by I{ n} , and
taking in, then out, what is known, we find that a.s.
0 = E[I{ n} (IB (Xn +u ) pu (Xn , B))|Fn+ ] = I{ n} E[Z|Fn+ ] ,
for the bounded R.V.
Z = I{ <} [IB (X +u ) pu (X , B)] .
By part (c) of Exercise 8.1.11 it then follows that w.p.1. I{ n} E[Z|F + ] = 0.
Taking n we deduce that a.s. E[Z|F + ] = 0. Upon taking out the known
I{ <} pu (X , B) we represent this as
(8.3.16)
E[I{ <} f (X +u )|F + ] = I{ <} (pu f )(X ) ,
almost surely
for f () = IB (). By linearity of the expectation and conditional expectation, this

identity extends from indicators to all S-measurable simple functions, whereby it
follows by monotone convergence that it holds for all f bS.
Step 2. We are ready
Q to prove that (8.3.12) holds for any Ft -Markov time , in case
h(s, x()) = f0 (s) n=1 f (x(u )), with bounded Borel f0 : [0, ) R, f bS and
u1 > > un 0 = un+1 . As f0 ( ) bF + , one can always take this (known)
part of h(, ) out of the conditional expectation in (8.3.12) and thereafter eliminate
it. Thus, setting f0 = 1 we proceed to prove by induction on n that (8.3.12) holds,
namely, that for any Ft -Markov time , f bS and u1 > > un 0, almost
surely,
n
Y
E[I{ <}
f (X +u )|F + ] = I{ <} gn (X ) ,
=1
for the bounded, S-measurable functions g1 = pu1 u2 f1 and

g = pu u+1 (f g1 ) ,
= 2, . . . , n.
The identity (8.3.16) is the n = 1 basis of the proof. To carry out the induction
step, recall part (c) of Exercise 8.1.10 that = + u is a decreasing sequence
of Ft -Markov times, which are finite if and only if is, and further, F + Fn+
(see part (b) of Exercise 8.1.11). It thus follows by the tower property and taking
out the known term fn (Xn+ ) bFn+ (when < , see Proposition 8.1.13), that
E[I{ <}
n
Y
=1
f (X )|F + ] = E[fn (Xn )E[I{n <}
n1
Y
f (X )|Fn+ ]|F + ]
=1
= E[I{ <} fn (Xn )gn1 (Xn )|F+ ] = I{ <} gn (X ) .
Indeed, since n = u un are non-random and positive, the induction hypothesis applies for the Ft -Markov time n to yield the second equality, whereas
the third equality is established by considering the identity (8.3.16) for f = fn gn1
and u = un .
Step 3. Similarly to the proof of Proposition 8.3.11, fixing A F + , yet another
application of the monotone class theorem shows that any h bU is in the collection
329
H bU for which gh (s, x) = Ex [h(s, )] is measurable on B[0,) S and

(8.3.17)
E[I{ <} IA h(, X + )] = E[gh (, X )I{ <} IA ] .
Indeed, in Step 2 we have shown that H contains the indicators on the -system
P = {B D : B B[0,) , D S [0,) a finite dimensional measurable rectangle} ,
such that U = (P). Further, constants are in H which by the linearity of the
expectation (and hence of h 7 gh ), is a vector space. Finally, if hn h bounded
and hn H are non-negative, then h bU and by monotone convergence ghn
gh bounded and measurable, with the pair (h, gh ) also satisfying (8.3.17). Since
gh (, X )I{ <} is in bF+ and the preceding argument applies for all A F + ,
we conclude that per and h the identity (8.3.12) holds w.p.1., as claimed.

As you are to show now, both Markov and strong Markov properties apply for
product laws of finitely many independent processes, each of which has the corresponding property.
Exercise 8.3.16. Suppose on some probability space (, F , P) we have homo(i)
(i)
geneous Markov processes (Xt , Ft ) of B-isomorphic state spaces (Si , Si ) and
(i)
(i)
Markov semi-groups pt (, ), such that F , i = 1, . . . , are P-mutually independent.
(1)
()
(1)
()
(a) Let X t = (Xt , . . . , Xt ) and Ft = (Ft , . . . , Ft ). Show that (X t , Ft )

is a homogeneous Markov process, of the Markov semi-group
pt (x, B1 B ) =
(i)
pt (xi , Bi )
i=1
on the B-isomorphic state space (S, S), where S = S1 S and

S = S1 S .
(i)
(b) Suppose in addition that for each i = 1, . . . , k the Ft -progressively mea(i)
surable process Xt has the strong Markov property. Show that the strong
Markov property then holds for the Ft -progressively measurable (S, S)valued stochastic process X t .
Recall Proposition 6.1.16 that every homogeneous Markov chain of a B-isomorphic
state space has the strong Markov property and that in this context every Markov
time is a stopping time and takes only countably many possible values. As expected,
you are to show next that any homogeneous Markov process has the strong Markov
property (8.3.13) for such stopping times.
Exercise 8.3.17. Suppose (Xt , Ft ) is a homogeneous Markov process, (S, S) its
B-isomorphic state space and : 7 C is an Ft -stopping time with countable
C = {sk } [0, ].
(a) Show that A { : () = sk } Fsk for any finite sk C and A F .
(b) Deduce that h(, X + )I{ <} is a R.V. and gh (, X )I{ <} bF provided h(sk , ) are S [0,) -measurable and uniformly bounded on CS[0,) .
(c) Conclude that (8.3.13) holds a.s. for any such and h.
For the Feller semi-groups we define next (compare with the strong Feller property
of Remark 6.3.12), the right-continuity of sample functions yields the strong Markov
property.
330
Definition 8.3.18. A Feller semi-group is a Markov semi-group {pu , u 0} on

(R, B) such that pt : Cb (R) 7 Cb (R) for any t 0. That is, x 7 (pt f )(x) is
continuous for any fixed bounded, continuous function f and t 0.
Proposition 8.3.19. Any right-continuous homogeneous Markov process (Xt , Ft )
with a Feller semi-group (of transition probabilities), is a strong Markov process.
Proof. Fixing u 0, a bounded Ft -Markov time , A F + and f Cb (R),
we proceed to show that
(8.3.18)
E[f (X +u )IA ] = E[(pu f )(X )IA ] .
Indeed, recall that in Lemma 8.1.16 we have constructed a sequence of finite Ft stopping times = 2 ([2 ]+1) taking values in the countable set of non-negative
dyadic rationals, such that . Further, for any we have that A F (see
part (b) of Exercise 8.1.12), hence as shown in Exercise 8.3.17,
E[f (X +u )IA ] = E[(pu f )(X )IA ] .
Due to the sample path right-continuity, both X +u X +u and X X .
Since f Cb (R) and pu f Cb (R) (by the assumed Feller property), as
both f (X +u ) f (X +u ) and (pu f )(X ) (pu f )(X ). We thus deduce by
bounded convergence that (8.3.18) holds.
Next, consider non-negative fk Cb (R) such that fk I(,b) (see Lemma 3.1.6
for an explicit construction of such). By monotone convergence pu fk pu I(,b)
and hence
(8.3.19)
E[IB (X +u )IA ] = E[pu (X , B)IA ] ,
for any B in the -system {(, b) : b R} which generates the Borel -algebra
B. The collection of L of sets B B for which the preceding identity holds is a system (by linearity of the expectation and monotone convergence), so by Dynkins
theorem it holds for any Borel set B. Since this applies for any A F + , the
strong Markov property of (Xt , Ft ) follows from Proposition 8.3.15, upon noting
that the right-continuity of t 7 Xt implies that Xt is Ft -progressively measurable,
with pu (X , B) mF + (see Propositions 8.1.8 and 8.1.13, respectively).

Taking advantage of the preceding result, you can now verify that any rightcontinuous S.P. of stationary, independent increments is a strong Markov process.
Exercise 8.3.20. Suppose {Xt , t 0} is a real-valued process of stationary, independent increments.
(a) Show that {Xt , t 0} has a Feller semi-group.
(b) Show that if {Xt , t 0} is also right-continuous, then it is a strong
Markov process. Deduce that this applies in particular for the Poisson
process (starting at N0 = x R as in Example 8.3.6), as well as for any
Brownian Markov process (Xt , Ft ).
(c) Suppose the right-continuous {Xt , t 0} is such that limt0 E|Xt | = 0
and X0 = 0. Show that Xt is integrable for all t 0 and Mt = Xt tEX1
is a martingale. Deduce that then E[X ] = E[ ]E[X1 ] for any integrable
FtX -stopping time .
Hint: Establish the last claim first for the FtX -stopping times = 2 ([2 ]+
1).
331
Our next example demonstrates that some regularity of the semi-group is needed
when aiming at the strong Markov property (i.e., merely considering the canonical
filtration of a homogeneous Markov process with continuous sample functions is
not enough).
Example 8.3.21. Suppose X0 is independent of the standard Wiener process
{Wt , t 0} and q = P(X0 = 0) (0, 1). The S.P. Xt = X0 + Wt I{X0 6=0} has
continuous sample functions and for any fixed s 0, a.s. I{X0 =0} = I{Xs =0} (as
the difference occurs on the event {Ws = X0 6= 0} which is of zero probability). Further, the independence of increments of {Wt } implies the same for {Xt }
conditioned on X0 , hence for any u 0 and Borel set B, almost surely,
P(Xs+u B|FsX ) = I0B I{X0 =0} + P(Ws+u Ws + Xs B|Xs )I{X0 6=0}
= pbu (Xs , B) ,
where pbu (x, B) = p0 (x, B)Ix=0 + pu (x, B)Ix6=0 for the Brownian semi-group pu ().
Clearly, per u fixed, pbu (, ) is a transition probability on (R, B) and pb0 (x, B) is
the identity element for the semi-group relation pbu+s = pbu pbs which is easily shown
to hold (but this is not a Feller semi-group, since x 7 (b
pt f )(x) is discontinuous
at x = 0 whenever f (0) 6= Ef (Wt )). In view of Definition 8.3.1, we have just
shown that pbu (, ) is the Markov semi-group associated with the FtX -progressively
measurable homogeneous Markov process {Xt , t 0} (regardless of the distribution
of X0 ). However, (Xt , FtX ) is not a strong Markov process. Indeed, note that
= inf{t 0 : Xt = 0} is an FtX -stopping time (see Proposition 8.1.15), which
(0)
is finite a.s. (since if X0 6= 0 then Xt = Wt + X0 and = X0 of Exercise
8.2.35, whereas for X0 = 0 obviously = 0). Further, by continuity of the sample
functions, X = 0 whenever < , so if (Xt , FtX ) was a strong Markov process,
then in particular, a.s.
P(X +1 > 0|FX ) = pb1 (0, (0, )) = 0
(this is merely (8.3.15) for the stopping time , u = 1 and B = (0, )). However,
the latter identity fails whenever X0 6= 0 (i.e. with probability 1 q > 0), for then
the left side is merely p1 (0, (0, )) = 1/2 (since {Wt , FtX } is a Brownian Markov
process, hence a strong Markov process, see Exercise 8.3.20).
Here is an alternative, martingale based, proof that any Brownian Markov process
is a strong Markov process.
Exercise 8.3.22. Suppose (Xt , Ft ) is a Brownian Markov process.
(a) Let Rt and It denote the real and imaginary parts of the complex-valued
S.P. Mt = exp(iXt + t2 /2). Show that both (Rt , Ft ) and (It , Ft ) are
MG-s.
(b) Fixing a bounded Ft -Markov time , show that E[M +u |F + ] = M
w.p.1.
(c) Deduce that w.p.1. the R.C.P.D. of X +u given F + matches the normal
distribution of mean X () and variance u.
(d) Conclude that the Ft -progressively measurable homogeneous Markov process {Xt , t 0} is a strong Markov process.
8.3.3. Markov jump processes. This section is about the following Markov
processes which in many respects are very close to Markov chains.
332
Definition 8.3.23. A function x : R+ 7 S is called a step function if it is

constant on each of the intervals [sk1 , sk ), for some countable (possibly finite), set
of isolated points 0 = s0 < s1 < s2 < . A continuous-time stochastic process
(Xt , t 0) taking values in some measurable space (S, S) is called a pure jump
process if its sample functions are step functions. A Markov pure jump process is
a homogeneous Markov process which, starting at any non-random X0 = x S, is
also a pure jump process on its B-isomorphic state space (S, S).
Remark. We often use Markov jump process for Markov pure jump process and
note in passing that these processes are sometimes also called continuous time
Markov chains.
The relatively explicit analysis of Markov jump processes, as provided here, owes
much to the fact that the jump times in their sample functions are isolated. Many
interesting, and harder to analyze Markov processes have piecewise constant sample
functions, but with accumulation points of jump times.
We start by showing that the strong Markov property applies for all Markov jump
processes.
Proposition 8.3.24. Any Markov jump process (Xt , Ft ) is a strong Markov process.
Proof. Though we did not even endow the state space (S, S) with a topology,
the sample functions t 7 Xt , being step functions, are trivially right continuous,
hence the Markov jump process is Ft -progressively measurable (see Proposition
8.1.8). Fixing u 0, a bounded Ft -Markov time , A F + and B S, as
in the proof of Proposition 8.3.19 the identity (8.3.19) holds for some sequence
{ } of Ft -stopping times such that . Since the right-continuous sample
functions t 7 Xt of a jump process are constant except possibly for isolated jump
times, both X = X and X +u = X +u for all large enough. Consequently,
IB (X +u ) = IB (X +u ) and pu (X , B) = pu (X , B) for all large enough, so by
bounded convergence the identity (8.3.19) also holds for the Ft -Markov time .
Since this applies for any A F + , as explained while proving Proposition 8.3.19,
the strong Markov property of (Xt , Ft ) then follows from Proposition 8.3.15.

Example 8.3.25. The semi-group of a Markov jump process is often not a Feller
semi-group (so Proposition 8.3.24 is not a special case of Proposition 8.3.19). For
example, setting sgn(0) = 0 it is easy to check that pt (x, A) = et I{xA} + (1
et )I{sgn(x)A} is a Markov semi-group on R, which is not a Feller semi-group (as
(p1 h)(x) = e1 h(x) + (1 e1 )1x6=0 is discontinuous for h(x) = x2 1 Cb (R)).
This semi-group corresponds to a Markov jump process {Xt } with at most one jump
per sample function, such that starting at any state X0 other than the (absorbing)
states 1, 0 and 1, it jumps to sgn(X0 ) {1, 0, 1} at a random time having the
exponential distribution of parameter one.
In view of Lemma 7.1.7, the law of a homogeneous Markov process does not tell us
directly whether or not it is a Markov jump process. In fact, a Markov jump process
corresponds to the piecewise constant RCLL modification of the given Markov law
(and such modification is essentially unique, see Exercise 7.2.3), so one of the central
issues here is to determine when such a modification exists.
With the Poisson process as our prototypical example of a Markov jump process,
we borrow from the treatment of the Poisson process (in Subsection 3.4.2), and
333
proceed to describe the jump parameters of Markov jump processes. These parameters then serve as a convenient alternative to the general characterization of a
homogeneous Markov process via its (Markov) semi-group.
Proposition 8.3.26. Suppose (Xt , t 0) is a right-continuous, homogeneous
Markov process.
(a) Under Py , the FtX -Markov time = inf{t 0 : Xt 6= X0 } has the exponential distribution of parameter y , for all y S and some measurable
: S 7 [0, ].
(b) If y > 0 then is Py -almost-surely finite and Py -independent of the
S-valued random variable X .
(c) If (Xt , t 0) is a strong Markov process and y > 0 is finite, then
Py -almost-surely X 6= y.
(d) If (Xt , t 0) is a Markov jump process, then is a strictly positive,
FtX -stopping time.
Proof. (a). From Proposition 8.1.15 we know that 0 is an FtX -Markov

time, as are u,z = inf{t u : Xt 6= z} for any z S and u 0. Under Py the
event { u + t} for t > 0 implies that Xu = y and = u,Xu . Thus, applying the
Markov property for h = I t (so h(Xu+ ) = I{u,Xu u+t} ), we have that
Py ( u + t) = Py (u,Xu u + t, Xu = y, u)
= Ey [E(u,Xu u + t|FuX )I{ u,Xu =y} ] = Py ( t)Py ( u, Xu = y) .
Considering this identity for t = s + n1 and u = v + n1 with n , we find

that the [0, 1]-valued function g(t) = Py ( > t) is such that g(s + v) = g(s)g(v)
for all s, v 0. Setting g(1) = exp(y ) for : S 7 [0, ] which is measurable,
by elementary algebra we have that g(q) = exp(y q) for any positive q Q.
Considering rational qn t it thus follows that Py ( > t) = exp(y t) for all t 0.
(b). If y = , then Py -a.s. both = 0 and X = X0 = y are non-random, hence
independent of each other. Suppose now that y > 0 is finite, in which case is
finite and positive Py -almost surely. Then, X is well defined and applying the
Markov property for h = IB (X )I t (so h(Xu+ ) = IB (Xu,Xu )I{u,Xu t+u} ), we
have that for any B S, u 0 and t > 0,
Py (X B, t + u) = Py (Xu,y B, u,y t + u, u, Xu = y)
= Ey [P(Xu,Xu B, u,Xu t + u|FuX )I{ u,Xu =y} ]

= Py (X B, t)Py ( u, Xu = y) .
Considering this identity for t = n1 and u = v + n1 with n , we find that

Py (X B, > v) = Py (X B, > 0)Py ( > v) = Py (X B)Py ( > v) .
Since { > v} and {X B, < } are Py -independent for any v 0 and B S,

it follows that the random variables and X are Py -independent, as claimed.
(c). With A = {qn Q, qn 0 : x(qn ) 6= x(0) = y} S [0,) , note that by
right-continuity of the sample functions t 7 Xt () the event {X A} is merely
{ = 0, X0 = y} and with y finite, further Pz (X A) = 0 for all z S. Since
y > 0, by the strong Markov property of (Xt , t 0) for h(s, x()) = IA (x())
and the Py -a.s. finite FtX -Markov time , we find that Py -a.s. X +
/ A. Since
the event {X = X0 } implies, by definition of and sample path right-continuity,
the existence of rational qn 0 such that X +qn 6= X0 , we conclude that Py -a.s.
334
X 6= y.
(d). Here t 7 Xt () is a step function, hence clearly, for each t 0
[
{ t} =
{Xq 6= X0 } FtX
(2)
t+
qQ
and is a strictly positive, FtX -stopping time.
Markov jump processes have the following parameters.

Definition 8.3.27. We call p(x, A) and {x } the jump transition probability and
jump rates of a Markov jump process {Xt , t 0}, if p(x, A) = Px (X A) for
A S and x S of positive jump rate x , while p(x, A) = IxA in case x = 0.
More generally, a pair (, p) with : S 7 R+ measurable and p(, ) a transition
probability on (S, S) such that p(x, {x}) = Ix =0 is called jump parameters.
The jump parameters provide the following canonical construction of Markov jump
processes.
Theorem 8.3.28. Suppose (, p) are jump parameters on a B-isomorphic space
(S, S). Let {Zn , n 0} be the homogeneous Markov chain of transition probability
p(, ) and initial state Z0 = x S. For each y S let {j (y), j 1} be i.i.d.
random variables, independent of {Zn } and having each the exponential distribution
Pk
of parameter y . Set T0 = 0, Tk = j=1 j (Zj1 ), k 1 and Xt = Zk for all
t [Tk , Tk+1 ), k 0. Assuming Px (T < ) = 0 for all x S, the process
{Xt , t 0} thus constructed is the unique Markov jump process with the given
jump parameters. Conversely, (, p) are the parameters of a Markov jump process
if and only if Px (T < ) = 0 for all x S.
Remark. The random time j (Zj1 ) is often called the holding time at state
Zj1 (or alternatively, the j-th holding time), along the sample path of the Markov
jump process. However, recall part (c) of Proposition 8.3.26, that strong Markov
processes of continuous sample path have trivial jump parameters, i.e. their holding
times are either zero or infinite.
Proof. Part I. Existence.
Starting from jump parameters (, p) and X0 = x, if Px (T < ) = 0 then our
construction produces {Xt , t 0} which is indistinguishable from a pure jump
process and whose parameters coincide with the specified (, p). So, assuming
hereafter with no loss of generality that T () = for all , we proceed to
show that {Xt , FtX } is a homogeneous Markov process. Indeed, since pt (x, B) =
Px (Xt B) is per t 0 a transition probability on (S, S), this follows as soon as
we show that Px (Xs+u B|FsX ) = PXs (Xu B) for any fixed s, u 0, x S and
B S.
Turning to prove the latter identity, fix s, u, x, B and note that
[
{Xu B} =
{Z B, T+1 > u T } ,
0
is of the form {Xu B} = {(Z , T ) Au } where Au (S [0, ])c . Hence, this

event is determined by the law of the homogeneous Markov chain {Zn , Tn , n 0}
on S [0, ]. With Yt = sup{k 0 : Tk t} counting the number of jumps in
the interval [0, t], we further have that if {Ys = k}, then Xs = Zk and {Xs+u
335
B} = {(Zk+ , Tk+ s) Au }. Moreover, since t 7 Xt () is a step function,

FsX = (Ys , Zk , Tk , k Ys ). Thus, decomposing as the union of disjoint events
{Ys = k} it suffices to show that under Px , the law of (Zk+ , Tk+ s) conditional
on (Zk , Tk ) and the event {Ys = k} = {k+1 (Zk ) > s Tk 0}, is the same
as the law of (Z , T ) under PZk . In our construction, given Zk S, the random
variable = k+1 (Zk ) = Tk+1 Tk is independent of Tk and follows the exponential
distribution of parameter Zk . Hence, setting = sTk 0, by the lack of memory
of this exponential distribution, for any t, s, k 0,
Px (Tk+1 > t + s|Tk , Zk , {Ys = k}) = Px ( > t + |Tk , Zk , { > }) = P( > t|Zk ) .
That is, under Px , the law of Tk+1 s conditional on FsX and the event {Ys = k},
is the same as the law of T1 under PZk . With {Zn , n 0} a homogeneous Markov
chain whose transition probabilities are independent of {Tn , n 0}, it follows that
further the joint law of (Zk+1 , Tk+1 s) conditional on FsX and the event {Ys = k}
is the same as the joint law of (Z1 , T1 ) under PZk . This completes our proof that
{Xt , FtX } is a homogeneous Markov process, since for any z S, conditional on
Zk+1 = z the value of (Zk+1+ , Tk+1+ Tk+1 ) is independent of Tk+1 and by the
Markov property has the same joint law as (Z1+ , T1+ T1 ) given Z1 = z.
bt , t 0)
Part II. Uniqueness. Start conversely with a Markov pure jump process (X
b
such that X0 = x and whose jump parameters per Definition 8.3.27 are (, p).
In the sequel we show that with probability one we can embed within its sample
bt () a realization of the Markov chain {Zn , n 0} of transition
function t 7 X
bt = Zk for all t [Tk , Tk+1 ),
probability p(, ), starting at Z0 = x, such that X
k 0 and with T0 = 0, show that for any k 0, conditionally on {Zj , Tj , j k},
the variables k+1 = Tk+1 Tk and Zk+1 are independent of each other, with k+1
having the exponential distribution of parameter Zk .
This of course implies that even conditionally on the infinite sequence {Zn , n 0},
the holding times {k+1 , k 0} are independent of each other, with k+1 maintainbt () is a
ing its exponential distribution of parameter Zk . Further, since t 7 X
step function (see Definition 8.3.23), necessarily here T () = for all .
This applies for any non-random x S, thus showing that any Markov pure jump
process can be constructed as in the statement of the theorem, provided (, p) are
such that Px (T < ) = 0 for all x S, with the latter condition also necessary
for (, p) to be the jump parameters of any Markov pure jump process (and a
moment thought will convince you that this completes the proof of the theorem).
b0 = x and T1 = T0 + 1 for
Turning to the promised embedding, let T0 = 0, Z0 = X
b
1 = inf{t 0 : Xt 6= Z0 }. Recall Proposition 8.3.26 that 1 has the exponential
b
distribution of parameter x and is an FtX -stopping time. In case x = 0 we
bt () = Z0 () = x for all t 0. Otherwise,
are done for then T1 = and X
recall Proposition 8.3.26 that T1 is finite w.p.1. in which case Z1 = XT1 is well
defined and independent of T1 , with the law of Z1 being p(Z0 , ). Further, since
b
bt , F X
(X
t ) is a strong Markov process (see Proposition 8.3.24) and excluding the
null set { : T1 () = }, upon applying the strong Markov property at the finite
b
bT1 +t , t 0}
stopping time T1 we deduce that conditional on FTX1 the process {X
is a Markov pure jump process, of the same jump parameters, but now starting
at Z1 . We can thus repeat this procedure and w.p.1. construct the sequence
b
bt+T 6= X
bT }, k 1, where Tk+1 = Tk + k+1 are FtX
k+1 = inf{t 0 : X
-stopping
k
k
336
bT (terminating at Tk+1 = if Z = 0). This is the embedding

times and Zk = X
k
k
bt = Zk for all t [Tk , Tk+1 ), the sequence {Zn , n 0}
described before, for indeed X
has the law of a homogeneous Markov chain of transition probability p(, ) (starting
b
at Z0 = x), and conditionally on (Zj , Tj , j k) FTXk , the variables k+1 and
Zk+1 are independent of each other, with k+1 having the exponential distribution
of parameter Zk .

Remark 8.3.29. When the jump rates x = are constant, the corresponding
jump times Tk are those of a Poisson process Nt of rate , which is independent
of the Markov chain {Zn }. Hence, in this case the Markov jump process has the
particularly simple structure Xt = ZNt .
Here is a more explicit, equivalent condition for existence of a Markov pure jump
process with the specified jump parameters (, p). It implies in particular that such
a process exists whenever the jump rates are bounded (i.e. supx x finite).
Exercise 8.3.30. Suppose (, p) are jump parameters on the B-isomorphic state
space (S, S).
P
(a) Show that Px (T < ) = 0 if and only if Px ( n 1
Zn < ) = 0.
Hint: Upon Conditioning on {Zn } consider part (d) of Exercise 2.3.25.
(b) Conclude that to any jump parameters p(, ) and bS corresponds
a well defined, unique Markov jump process, constructed as in Theorem
8.3.28.
Remark. The event { : T () < } is often called an explosion. A further
distinction can then be made between the pure (or non-explosive) Markov jump
processes we consider here, and the explosive Markov jump processes such that
Px (T < ) > 0 for some x S, which nevertheless can be constructed as in
Theorem 8.3.28 to have step sample functions, but only up to the time T of
explosion.
Example 8.3.31 (birth processes). Markov (jump) processes which are also
counting processes, are called birth processes. The state space of such processes is
S = {0, 1, 2, . . .} and in view of Theorem 8.3.28 they correspond to jump transitions
p(x, x + 1) = 1. Specifically, these processes are of the form Xt = sup{k 0 :
Pk1
j=X0 j t}, where the holding times j , j 1, are independent Exponential(j )
random variables.
In view of Exercise 8.3.30 such processes are non-explosive if
P
1
= for all k 0. For example, this is the case when

and only if
jk j
j = j + 0 with 0 0 and > 0, and such a process is then called simple
birth with immigration process if also 0 > 0, or merely simple birth process if
0 = 0 (in contrast, the Poisson process corresponds to = 0 and 0 > 0). The
latter processes serve in modeling the growth in time of a population composed of
individuals who independently give birth at rate (following an exponentially distributed holding time between consecutive birth events), with additional immigration
into the population at rate 0 , independently of birth events.
Px+k1
for the arrival
Remark. In the context of Example 8.3.31, Ex Tk = j=x 1
j
time Tk to state k + x, so taking for example j = j for some > 1 results
P with
an explosive Markov jump process. Indeed, then Ex Tk c for finite c = j1 j
and any x, k 1. By monotone convergence Ex T c, so within an integrable,
hence a.s. finite time T the sample function t 7 Xt escapes to infinity, hence
337
the name explosion given to such phenomena. But, observe that unbounded jump
rates do not necessarily imply an explosion (as for example, in case of simple birth
processes), and explosion may occur for one initial state but not for another (for
example here 0 = 0 so there is no explosion if starting at x = 0).
As you are to verify now, the jump parameters characterize the relatively explicit
generator for the semi-group of a Markov jump process, which in particular satisfies
Kolmogorovs forward (in case of bounded jump rates), and backward equations.
Definition
8.3.32. The linear operator L : bS 7 mS such that (Lh)(x) =
R
x (h(y) h(x))p(x, dy) for h bS is called the generator of the Markov jump
process corresponding to jump parameters (, p). In particular, (LI{x}c )(x) = x
and more generally (LIB )(x) = x p(x, B) for any B {x}c (so specifying the
generator is in this context equivalent to specifying the jump parameters).
Exercise 8.3.33. Consider a Markov jump process (Xt , t 0) of semi-group
Pk
pt (, ) and jump parameters (, p) as in Definition 8.3.32. Let Tk = j=1 j denote
P
the jump times of the sample function s 7 Xs () and Yt =
k1 I{Tk t} the
number of such jumps in the interval [0, t].
(a) Show that if x > 0 then
Z
Px (2 t|1 ) = (1 ey t )p(x, dy) ,
and deduce that t1 Px (Yt 2) 0 as t 0, for any x S.
(b) Fixing x S and h bS, show that
|(ps h)(x) (p0 h)(x) Ex [(h(X ) h(x))I s ]| 2khk Px (Ys 2) ,

and deduce that for L per Definition 8.3.32,
lim s1 ((ps h)(x) (p0 h)(x)) = (Lh)(x) ,
(8.3.20)
s0
where the convergence in (8.3.20) is uniform within {h bS : khk

K}, for any K finite.
(c) Verify that t 7 (Lpt h)(x) is continuous and t 7 (pt h)(x) is differentiable
for any x S, h bS, t 0 and conclude that the backward Kolmogorov
equation holds. Specifically, show that
(8.3.21)
t (pt h)(x) = (Lpt h)(x)
t 0, x S, h bS.
(d) Show that if supxS x is finite, then L : bS 7 bS, the convergence in

(8.3.20) is also uniform in x and Kolmogorovs forward equation (also
known as the Fokker-Planck equation), holds. That is,
(8.3.22)
t (pt h)(x) = (pt (Lh))(x)
t 0, x S, h bS.
Remark. Exercise 8.3.33 relates the Markov semi-group with the corresponding jump parameters, showing that a Markov semi-group pt (, ) corresponds to a
Markov jump process only if for any x S, the limit
lim t1 (1 pt (x, {x}) = x
(8.3.23)
t0
exists, is finite and S-measurable. Moreover, necessarily then also

(8.3.24)
lim t1 pt (x, B) = x p(x, B)

t0
B {x}c ,
338
for some transition probability p(, ). Recall Theorem 8.3.28 that with the exception of possible explosion, the converse applies, namely whenever (8.3.23) and
(8.3.24) hold, the semi-group pt (, ) corresponds to a (possibly explosive) Markov
jump process. We note in passing that while Kolmogorovs backward equation
(8.3.21) is well defined for any jump parameters, the existence of solution which is
a Markov semi-group, is equivalent to non-explosion of the corresponding Markov
jump process.
In particular, in case of bounded jump rates the conditions (8.3.23) and (8.3.24)
are equivalent to the Markov process being a Markov pure jump process and in this
setting you are now to characterize the invariant measures for the Markov jump
process in terms of its jump parameters (or equivalently, in terms of its generator).
Exercise 8.3.34. Suppose (, p) are jump parameters on B-isomorphic state space
(S, S) such that supxS x is finite.
(a) Show that probability measure is invariant for the corresponding Markov
jump process if and only if (Lh) = 0 for the generator L : bS 7 bS of
these jump parameters and all h bS.
Hint: Combine Exercises 8.3.9 and 8.3.33 (utilizing the boundedness of
x 7 (Lh)(x)).
(b) Deduce that is an invariant probability measure for (, p) if and only if
(p) = .
In particular, the invariant probability measures of a Markov jump process with
constant jump rates are precisely the invariant measures for its jump transition
probability.
Of particular interest is the following special family of Markov jump processes.
Definition 8.3.35. Real-valued Markov pure jump processes with a constant jump
rate whose jump transition probability is of the form p(x, B) = P ({z : x+z B})
for some law P on (R, B), are called compound Poisson processes. Recall Remark
8.3.29 thatP
a compound Poisson process is of the form Xt = SNt for a random walk
Sn = S0 + nk=1 k with i.i.d. {, k } which are independent of the Poisson process
Nt of rate .
Remark. The random telegraph signal Rt = (1)Nt R0 of Example 7.2.14 is a

Markov jump process on S = {1, 1} with constant jump rate , which is not a
compound Poisson process (as its transition probabilities p(1, 1) = p(1, 1) = 1
do not correspond to a random walk).
As we see next, compound Poisson processes retain many of the properties of the
Poisson process.
Proposition 8.3.36. A compound Poisson process {Xt , t 0} has stationary,
independent increments and the characteristic function of its Markov semi-group
pt (x, ) is
(8.3.25)
Ex [eiXt ] = eix+t( ()1) ,
where () denotes the characteristic function of the corresponding jump sizes k .

Proof. We start by proving that {Xt , t 0} has independent increments,
where by Exercise 7.1.12 it suffices to fix 0 = t0 < t1 < t2 < < tn and show that
the random variables Di = Xti Xti1 , i = 1, . . . , n, are mutually independent. To
339
Pi
this end, note that Nt0 = 0 and conditional on the event Nti = mi for mi = j=1 rj
P
i
and fixed r = (r1 , . . . , rn ) Zn+ , we have that Di = m
k=mi1 +1 k are mutually
independent with Di then having the same distribution as the random walk Sri
starting at S0 = 0. So, for any fi bB, by the tower property and the mutual
independence of {Nti Nti1 , 1 i n},
Ex [
n
Y
fi (Di )] = E[Ex (
n
Y
fi (Di )|F N )] =
rZn
+ i=1
i=1
i=1
n n X
Y
i=1
ri =0
n n
X Y
P(Nti Nti1 = ri )E0 [fi (Sri )]
n
o Y
Ex [fi (Di )] ,
P(Nti Nti1 = ri )E0 [fi (Sri )] =
i=1
yielding the mutual independence of Di , i = 1, . . . , n.

We have just seen that for each t > s the increment Xt Xs has under Px
D
the same law as SNt Ns has under P0 . Since Nt Ns = Nts , it follows by the
independence of the random walk {Sr } and the Poisson process {Nt , t 0} that
Px (Xt Xs ) = P0 (SNts ) depends only on t s, which by Definition 7.3.11
amounts to {Xt , t 0} having stationary increments.
Finally, the identity (3.3.3) extends to E[z Nt ] = exp(t(z 1)) for Nt having
a Poisson distribution with parameter t and any complex variable z. Thus, as
Ex [eiSr ] = eix ()r (see Lemma 3.3.8), utilizing the independence of {Sr } from
Nt , we conclude that
Ex [eiXt ] = E[Ex (eiSNt |Nt )] = eix E[ ()Nt ] = eix+t( ()1) ,
for all t 0 and x, R, as claimed.
Exercise 8.3.37. Let {Xt , t 0} be a compound Poisson process of jump rate .

(a) Show that if the corresponding jump sizes {k } are square integrable then
E0 Xt = tE1 and Var(Xt ) = tE12 .
(b) Show that if E1 = 0 then {Xt , t 0} is a martingale. More generally,
u0 (t, Xt , ) is a martingale for u0 (t, y, ) = exp(y t(M () 1)) and
any R for which the moment generating function M () = E[e1 ] is
finite.
Here is the analog for compound Poisson processes of the thinning of Poisson
variables.
Proposition 8.3.38. Suppose {Xt , t 0} is a compound Poisson process of jump
rate and jump size law P . Fixing a disjoint finite partition of R \ {0} to Borel
m
P
(j)
Xt in terms of
sets Bj , j = 1, . . . , m, consider the decomposition Xt = X0 +
j=1
the contributions
(j)
Xt
Nt
X
k IBj (k )
k=1
(j)
to Xt by jumps whose size belong to Bj . Then {Xt , t 0} for j = 1, . . . , m are

independent compound Poisson processes of jump rates (j) = P( Bj ) and
(j)
i.i.d. jump sizes { (j) , } such that P( (j) ) = P( | Bj ), starting at
(j)
X0 = 0.
340
Proof. While one can directly prove this result along the lines of Exercise
bt = X0 + Pm Y (j)
3.4.16, we resort to an indirect alternative, whereby we set X
j=1 t
(j)
for the independent compound Poisson processes Yt of jump rates (j) and i.i.d.
(j)
(j)
bt is a pure jump process
jump sizes {k }, starting at Y0 = 0. By construction, X
whose jump times {Tk ()} are contained in the union over j = 1, . . . , m of the
(j)
(j)
(j)
isolated jump times {Tk ()} of t 7 Yt (). Recall that each Tk has the gamma
density of parameters = k and (j) (see Exercise 1.4.46 and Definition 3.4.8).
(j)
Therefore, by the independence of {Yt , t 0} w.p.1. no two jump times among
(j)
bt(j) = Yt(j) for all j and t 0 (as the
{Tk , j, k 1} are the same, in which case X
(j)
jump sizes of each Yt are in the disjoint element Bj of the specified finite partition
b (1) , . . . , X
b (m) ) being indistinguishable
of R \ {0}). With the Rm -valued process (X
bt , t 0} is a compound
from (Y (1) , . . . , Y (m) ), it thus suffices to show that {X
Poisson process of the specified jump rate and jump size law P .
(j)
To this end, recall Proposition 8.3.36 that each of the processes Yt has stationary independent increments and due to their independence, the same applies for
bt , t 0}, which is thus a real-valued homogeneous Markov process (see Propo{X
Pm
sition 8.3.5). Next, note that since = j=1 (j) and for all R,
m
X
(j) (j) () =
m
X
E[ei IBj ()] = () ,
j=1
j=1
we have from (8.3.25) and Lemma 3.3.8 that for any R,

b
Ex [eiXt ] = eix
= eix
m
Y
j=1
m
Y
(j)
E[eiYt ]
e
(j) t((j) ()1)
= eix+t( ()1) = Ex [eiXt ] .
j=1
That is, denoting by pt (, ) and pbt (, ) the Markov semi-groups of {Xt , t 0} and
bt , t 0} respectively, we found that per fixed x R and t 0 the transi{X
tion probabilities pt (x, ) and pbt (x, ) have the same characteristic function. Consequently, by Levys inversion theorem pt (, ) = pbt (, ) for all t 0, i.e., these
two semi-groups are identical. Obviously, this implies that the Markov pure jump
bt have the same jump parameters (see (8.3.23) and (8.3.24)),
processes Xt and X
bt , t 0} is a compound Poisson process of jump rate and jump
so as claimed {X
size law P .

Exercise 8.3.39. Suppose {Yt , t 0} is a compound Poisson process of jump rate
> 0, integrable Y0 and jump size law P for some integrable > 0.
(a) Show that for any integrable FtY -Markov time , EY = EY0 + EE .
Hint: Consider the FtY -martingales Xtn , n 1, where Xt = Yt tE.
(b) Suppose that Y0 and are square integrable and let r = inf{t 0 :
(r)
(r)
Zt > Yt }, where Zt = Bt +rt for a standard Brownian Markov process
(Bt , t 0), independent of (Yt , t 0). Show that for any r R,
Er =
E(Y0 )+
(r E)+
341
(where trivially r = 0 w.p.1. in case E(Y0 )+ = 0).

Hint: Consider filtration Ft = (Ys , Bs , s t) and MG Mt = Xt Bt .
As in the case of Markov chains, the jump transition probability P
of a Markov
jump process with countable state space S is of the form p(x, A) = yA p(x, y).
In this case, accessibility and intercommunication of states, as well as irreducible,
transient and recurrent classes of states, are defined according to the transition
probability {p(x, y)} and obey the relations explored already in Subsection 6.2.1.
Moreover, as you are to check next, Kolmogorovs equations (8.3.21) and (8.3.22)
are more explicit in this setting.
Exercise 8.3.40. Suppose (, p) are the parameters of a Markov jump process on
a countable state space S.
(a) Check that ps (x, z) = Px (Xs = z) are then the solution of the countable
system of linear ODEs
dps (x, z) X
=
q(x, y)ps (y, z)
s 0, x, z S,
ds
yS
starting at p0 (x, z) = Ix=z , where q(x, x) = x and q(x, y) = x p(x, y)

for x 6= y.
(b) Show that if supx x is finite then ps (x, z) must also satisfy the corresponding forward equation
dps (x, z) X
=
ps (x, y)q(y, z)
s 0, x, z S.
ds
yS
(c) In case S is a finite set, show that the matrix Ps of entries ps (x, z) is
P sk k
given by Ps = esQ =
k=0 k! Q , where Q is the matrix of entries
q(x, y).
The formula Ps = esQ explains why Q, and more generally L, is called the generator of the semi-group Ps .
From Exercise 8.3.34 we further deduce that, at least for bounded jump rates, an
invariant probability measure for the Markov
jump process is uniquely determined
P
by the function : S 7 [0, 1] such that x (x) = 1 and
X
(8.3.26)
y (y) =
(x)x p(x, y)
y S .
xS
For constant positive jump rates this condition coincides with the characterization
(6.2.5) of invariant probability measures for the jump transition probability. Consequently, for such jump processes the invariant, reversible and excessive measures
as well as positive and null recurrent states are defined as the corresponding objects for the jump transition probability and obey the relations explored already in
Subsection 6.2.2.
Remark. While we do not pursue this further, we note in passing that more
generally, a measure () is reversible for a Markov jump process with countable
state space S if and only if y (y)p(y, x) = (x)x p(x, y) for any x, y S (so
any reversible probability measure is by (8.3.26) invariant for the Markov jump
process). Similarly, in general we call x S with x = 0 an absorbing, hence
positive recurrent, state and say that a non-absorbing state is positive recurrent
342
if it has finite mean return time. That is, if Ex Tx < for the first return time
Tx = inf{t : Xt = x} to state x. It can then be shown, in analogy with
Proposition 6.2.41, that any invariant probability measure () is zero outside the
positive recurrent states and if its support is an irreducible class R of non-absorbing
positive recurrent states, then (z) = 1/(z Ez [Tz ]) (see, [GS01, Section 6.9] for
more details).
To practice your understanding, the next exercise explores in more depth the
important family of birth and death Markov jump processes (or in short, birth and
death processes).
Exercise 8.3.41 (Birth and death processes). A birth and death process is a
Markov jump process {Xt } on S = {0, 1, 2, . . .} for which {Zn } is a birth and death
chain. That is, p(x, x + 1) = px = 1 p(x, x 1) for all x S (where of course
p0 = 1). Assuming x > 0 for all x and px (0, 1) for all x > 0, let
b(k) =
k
0 Y pi1
.
k i=1 1 pi
Show that
P {Xt } is irreducible and has an invariant probability measure if and only
b(k) is finite, in which case its invariant measure is (k) =
b(k)/c.
if c = k0
The next exercise deals with independent random sampling along the path of a
Markov pure jump process.
P
Exercise 8.3.42. Let Yk = XTek , k = 0, 1, . . ., where Tek = ki=1 ei and the i.i.d.
ei 0 are independent of the Markov pure jump process {Xt , t 0}.
(a) Show that {Yk } is a homogeneous Markov chain and verify that any invariant probability measure for {Xt } is also an invariant measure for
{Yk }.
(b) Show that in case of constant jump rates x = and each ei having the
e > 0, one has the representation
exponential distribution of parameter
Pk
Yk = ZLk of sampling the embedded chain {Zn } at Lk = i=1 (i 1) for
i.i.d. i 1, each having the Geometric distribution of success probability
e
e
p = /(
+ ).
(c) Conclude that if {Tek } are the jump times of a Poisson process of rate
e > 0 which is independent of the compound Poisson process {Xt }, then
P1
{Yk } is a random walk, the increment of which has the law of i=1
i .
Compare your next result with part (a) of Exercise 8.2.46.
Exercise 8.3.43. Suppose {Xt , t 0} is a real-valued Markov pure jump process,

with 0 = T0 < T1 < T2 < denoting the jump times of its sample function. Show
that for any q > 0 its finite q-th variation V (q) (X)t exists, and is given by
X
V (q) (X)t =
I{Tk t} |XTk XTk1 |q .
k1
CHAPTER 9
The Brownian motion

The Brownian motion is the most fundamental continuous time stochastic process.
We have seen already in Section 7.3 that it is a Gaussian process of continuous sample functions and independent, stationary increments. In addition, it is a martingale
of the type considered in Section 8.2 and has the strong Markov property of Section
8.3. Having all these beautiful properties allows for a rich mathematical theory. For
example, many probabilistic computations involving the Brownian motion can be
made explicit by solving partial differential equations. Further, the Brownian motion is the corner stone of diffusion theory and of stochastic integration. As such
it is the most fundamental object in applications to and modeling of natural and
man-made phenomena.
This chapter deals with some of the most interesting properties of the Brownian
motion. Specifically, in Section 9.1 we use stopping time, Markov and martingale
theory to study path properties of this process, focusing on passage times and
running maxima. Expressing in Section 9.2 random walks and discrete time MGs as
time-changed Brownian motion, we prove Donskers celebrated invariance principle.
It then provides fundamental results about these discrete time S.P.-s, such as the
law of the iterated logarithm (in short lil), and the martingale clt. Finally, the
fascinating aspects of the (lack of) regularity of the Brownian sample path are the
focus of Section 9.3.
9.1. Brownian transformations, hitting times and maxima
We start with a few elementary path transformations under which the Wiener
process of Definition 7.3.12 is invariant (see also Figure 2 illustrating its sample
functions).
Exercise 9.1.1. For {Wt , t 0} a standard Wiener process, show that the S.P.
f (i) , i = 1, . . . , 6 are also standard Wiener processes.
W
t
ft(1) = Wt , t 0.
(a) (Symmetry) W
ft(2) = WT +t WT , t 0 with T > 0 a non-random
(b) (Time-homogeneity) W
constant.
ft(3) = WT WT t , for t [0, T ], with T > 0 a non(c) (Time-reversal) W
random constant.
f (4) = 1/2 Wt , t 0, with > 0 a non-random constant.
(d) (Scaling) W
t
ft(5) = tW1/t for t > 0 and W
f (5) = 0.
(e) (Time-inversion) W
0
P
(k)
(6)
(k)
n
f
= k=1 ck Wt , t 0, where Wt arePindependent
( f ) (Averaging) W
t
copies of the Wiener process and ck non-random such that nk=1 c2k = 1.
f (2) and W
f (3) are independent Wiener processes and evaluate
(g) Show that W
t
t
qt = Px (WT > WT t > WT +t ), where t [0, T ].
343
344
9. THE BROWNIAN MOTION
Remark. These invariance transformations are extensively used in the study of

the Wiener process. As a token demonstration, note that since time-inversion maps
La,b = sup{t 0 : Wt
/ (at, bt)} to the stopping time a,b of Exercise 8.2.36, it
follows that La,b is a.s. finite and P(WLa,b = bLa,b ) = a/(a + b).
Recall Exercise 8.3.20 (or Exercise 8.3.22), that any Brownian Markov process
(Wt , Ft ) is a strong Markov process, yielding the following consequence of Corollary
8.3.14 (and of the identification of the Borel -algebra of C([0, )) as the restriction
of the cylindrical -algebra B [0,) to C([0, )), see Exercise 7.2.9).
Corollary 9.1.2. If (Wt , Ft ) is a Brownian Markov process, then {Wt , t 0} is

a homogeneous Ft+ -Markov process and further, for any s 0 and Borel measurable
functional h : C([0, )) 7 R, almost surely
(9.1.1)
E[h(W )|Fs+ ] = E[h(W )|Fs ] .
From this corollary and the Brownian time-inversion property we further deduce
both Blumenthals 0-1 law about the Px -triviality of the -algebra F0W
+ and its
analog about the Px -triviality of the tail -algebra of the Wiener process (compare
the latter with Kolmogorovs 0-1 law). To this end, we first extend the definition
of the tail -algebra, as in Definition 1.4.9, to continuous time S.P.-s.
Definition 9.1.3. Associate with any continuous time S.P. {Xt , t 0} the canonical future -algebras TtX\= (Xs , s t), with the corresponding tail -algebra of
the process being T X =
TtX .
t0
Proposition 9.1.4 (Blumenthals 0-1 law). Let Px denote the law of the
Wiener process {Wt , t 0} starting at W0 = x (identifying (, F W ) with C([0, ))
and its Borel -algebra). Then, Px (A) {0, 1} for each A F0W
+ and x R.
Further, if A T W then either Px (A) = 0 for all x or Px (A) = 1 for all x.
Proof. Applying Corollary 9.1.2 for the Wiener process starting at W0

and its canonical filtration, we have by the Px -triviality of F0W that for each
F0W
+,
W
Px a.s.
IA = Ex [IA |F0W
+ ] = Ex [IA |F0 ] = Px (A)
Hence, Px (A) {0, 1}. Proceeding to prove our second claim, set X0 = 0
Xt = tW1/t for t > 0, noting that {Xt , t 0} is a standard Wiener process
X
for any t > 0, hence
part (e) of Exercise 9.1.1). Further, TtW = F1/t
\
\
X
TW=
TtW =
= F0X+ .
F1/t
t>0
=x
A
and
(see
t>0
Consequently, applying our first claim for the canonical filtration of the standard
Wiener processes {Xt } we see that P0 (A) {0, 1} for any A F0X+ = T W .
Moreover, since A T1W , it is of the form IA = ID 1 for some D F W , so by
the tower and Markov properties,
Z
Px (A) = Ex [ID 1 (())] = Ex [PW1 (D)] = p1 (x, y)Py (D)dy ,
for the strictly positive Brownian transition kernel p1 (x, y) = exp((xy)2 /2)/ 2.
If P0 (A) = 0 then necessarily Py (D) = 0 for Lebesgue almost every y, hence also
Px (A) = 0 for all x R. Conversely, if P0 (A) = 1 then P0 (Ac ) = 0 and with
Ac T W , by the preceding argument 1 Px (A) = Px (Ac ) = 0 for all x R.
9.1. BROWNIAN TRANSFORMATIONS, HITTING TIMES AND MAXIMA
345
Blumenthals 0-1 law is very useful in determining properties of the Brownian

sample function in the limits t 0 and t . Here are few of its many consequences.
Corollary 9.1.5. Let 0+ = inf{t 0 : Wt > 0}, 0 = inf{t 0 : Wt < 0} and
T0 = inf{t > 0 : Wt = 0}. Then, P0 (0+ = 0) = P0 (0 = 0) = P0 (T0 = 0) = 1
and w.p.1. the standard Wiener process changes sign infinitely many times in any
time interval [0, ], > 0. Further, for any x R, with Px -probability one,
1
lim sup Wt = ,
t
t
1
lim inf Wt = ,
t
t
Wun = 0 for some un () .
Proof. Since P0 (0+ t) P0 (Wt > 0) = 1/2 for all t > 0, also P0 (0+ =
0) 1/2. Further, 0+ is an FtW -Markov time (see Proposition 8.1.15). Hence,
{0+ = 0} = {0+ 0} F0W
+ and from Blumenthals 0-1 law it follows that
P0 (0+ = 0) = 1. By the symmetry property of the standard Wiener process (see
part (a) of Exercise 9.1.1), also P0 (0 = 0) = 1. Combining these two facts we
deduce that P0 -a.s. there exist tn 0 and sn 0 such that Wtn > 0 > Wsn for
all n. By sample path continuity, this implies the existence of un 0 such that
Wun = 0 for all n. Hence, P0 (T0 = 0) = 1. As for the second claim, note that for
any r > 0,
P0 (Wn r n i.o.) lim sup P0 (Wn r n) = P0 (W1 r) > 0

n
where the first inequality is due to Exercise 2.2.2 and the equality holds
by the
scaling property of {Wt } (see part (d) of Exercise 9.1.1). Since {W
r
n i.o.}
n
T W we thus deduce from Blumenthals 0-1 law that Px (Wn r ni.o.) = 1 for
any x R. Considering rk this implies that lim supt Wt / t = with
Px -probability one. Further, by the symmetry property of the standard Wiener
process,
P0 (Wn r n, i.o.) = P0 (Wn r n, i.o.) > 0 ,
so the preceding argument leads to lim inf t Wt / t = with Px -probability

one. In particular, Px -a.s. there exist tn and sn such that Wtn > 0 > Wsn
which by sample path continuity implies the existence of un such that Wun = 0
for all n.

Combining the strong Markov property of the Brownian Markov process and the
independence of its increments, we deduce next that each a.s. finite Markov time
is a regeneration time for this process, where it starts afresh independently of
the path it took up to this (random) time.
Corollary 9.1.6. If (Wt , Ft ) is a Brownian Markov process and is an a.s.
finite Ft -Markov time, then the S.P. {W +t W , t 0} is a standard Wiener
process, which is independent of F + .
Proof. With a.s. finite, Ft -Markov time and {Wt , t 0} an Ft -progressively
measurable process, it follows that Bt = Wt+ W is a R.V. on our probability
space and {Bt , t 0} is a well defined S.P. whose sample functions inherit the contift = Wt W0 has the f.d.d. hence the
nuity of those of {Wt , t 0}. Since the S.P. W
law of the standard Wiener process, fixing h bB [0,) and e
h(x()) = h(x() x(0)),
f )] is independent of y. Consequently,
the value of geh (y) = Ey [e
h(W )] = E[h(W
346
fixing A F + , by the tower property and the strong Markov property (8.3.12) of
the Brownian Markov process (Wt , Ft ) we have that
f )] .
h(W + )] = E[IA geh (W )] = P(A)E[h(W
E[IA h(B )] = E[IA e
In particular, considering A = we deduce that the S.P. {Bt , t 0} has the f.d.d.
ft }. Further, recall Lemma
and hence the law of the standard Wiener process {W
7.1.7 that for any F F B , the indicator IF is of the form IF = h(B ) for some
h bB [0,) , in which case by the preceding P(A F ) = P(A)P(F ). Since this
applies for any F F B and A F + we have established the P-independence of
the two -algebras, namely, the stated independence of {Bt , t 0} and F + .

Beware that to get such a regeneration it is imperative to start with a Markov
time . To convince yourself, solve the following exercise.
Exercise 9.1.7. Suppose {Wt , t 0} is a standard Wiener process.
(a) Provide an example of a finite a.s. random variable 0 such that
{W +t W , t 0} does not have the law of a standard Brownian motion.
(b) Provide an example of a finite FtW -stopping time such that [ ] is not
an FtW -stopping time.
Combining Corollary 9.1.6 with the fact that w.p.1. 0+ = 0, you are next to
prove the somewhat surprising fact that w.p.1. a Brownian Markov process enters
(b, ) as soon as it exits (, b).
Exercise 9.1.8. Let b+ = inf{t 0 : Wt > b} for b 0 and a Brownian Markov
process (Wt , Ft ).
(a) Show that P0 (b 6= b+ ) = 0.
(b) Suppose W0 = 0 and a finite random-variable H 0 is independent of
F W . Show that {H 6= H + } F has probability zero.
The strong Markov property of the Wiener process also provides the probability
that starting at x (c, d) it reaches level d before level c (i.e., the event W (0) = b
a,b
of Exercise 8.2.36, with b = d x and a = x c).
Exercise 9.1.9. Consider the stopping time = inf{t 0 : Wt

/ (c, d)} for a
Wiener process {Wt , t 0} starting at x (c, d).
(a) Using the strong Markov property of Wt show that u(x) = Px (W = d)
is an harmonic function, namely, u(x) = (u(x + r) + u(x r))/2 for
any c x r < x < x + r d, with boundary conditions u(c) = 0 and
u(d) = 1.
(b) Check that v(x) = (x c)/(d c) is an harmonic function satisfying the
same boundary conditions as u(x).
Since boundary conditions at x = c and x = d uniquely determine the value of a
harmonic function in (c, d) (a fact you do not need to prove), you thus showed that
Px (W = d) = (x c)/(d c).
We proceed to derive some of the many classical explicit formulas involving Brownian hitting times, starting with the celebrated reflection principle, which provides
among other things the probability density functions of the passage times for a
standard Wiener process and of their dual, the running maxima of this process.
347
Reflection principle for Brownian motion with b=1, t=3

3.5
2.5
1.5
0.5
0.5
3
s
Figure 1. Illustration of the reflection principle for Brownian motion.

Proposition 9.1.10 (Reflection principle). With {Wt , t 0} the standard
Wiener process, let Mt = sups[0,t] Ws denote its running maxima and Tb = inf{t
0 : Wt = b} the corresponding passage times. Then, for any t, b > 0,
(9.1.2)
P(Mt b) = P(b t) = P(Tb t) = 2P(Wt b) .
Remark. The reflection principle was stated by P. Levy [Lev39] and first rigorously proved by Hunt [Hun56]. It is attributed to D. Andre [And1887] who solved
the ballot problem of Exercise 5.5.30 by a similar symmetry argument (leading also
to the reflection principle for symmetric random walks, as in Exercise 6.1.19).
Proof. Recall Proposition 8.1.15 that b is a stopping time for FtW . Further,
since b > 0 = W0 and s 7 Ws is continuous, clearly b = Tb and WTb = b
whenever Tb is finite. Heuristically, given that Tb = s < u we have that Ws = b
and by reflection symmetry of the Brownian motion, expect the conditional law of
Wu Ws to retain its symmetry around zero, as illustrated in Figure 1. This of
course leads to the prediction that for any u, b > 0,
1
(9.1.3)
P(Tb < u, Wu > b) = P(Tb < u) .
2
With W0 = 0, by sample path continuity {Wu > b} {Tb < u}, so the preceding
prediction implies that
P(Tb < u) = 2P(Tb < u, Wu > b) = 2P(Wu > b) .
The supremum Mt () of the continuous function s 7 Ws () over the compact
interval [0, t] is attained at some s [0, t], hence the identity {Mt b} = {b t}
holds for all t, b > 0. Thus, considering u t > 0 leads in view of the continuity of
(u, b) 7 P(Wu > b) to the statement (9.1.2) of the proposition. Turning to rigorously prove (9.1.3), we rely on the strong Markov property of the standard Wiener
process for the FtW -stopping time Tb and the functional h(s, x()) = IA (s, x()),
where A = {(s, x()) : x() C(R+ ), s [0, u) and x(u s) > b}. To this end,
note that Fy,a,a = {x C([0, )) : x(u s) y for all s [a, a ]} is closed (with
respect to uniform convergence on compact subsets of [0, )), and x(u s) > b
for some s [0, u) if and only if x() Fbk ,q,q for some bk = b + 1/k, k 1 and
(2)
q < q Qu . So, A is the countable union of closed sets [q, q ] Fbk ,q,q , hence
348
Borel measurable on [0, ) C([0, )). Next recall that by the definition of the
set A,
1
gh (s, b) = Eb [IA (s, W )] = I[0,u) (s)Pb (Wus > b) = I[0,u) (s) .
2
Further, h(s, x(s+)) = I[0,u) (s)Ix(u)>b and WTb = b whenever Tb is finite, so taking
the expectation of (8.3.13) yields (for our choices of h(, ) and ), the identity,
E[I{Tb <u} I{Wu >b} ] = E[h(Tb , WTb + )] = E[gh (Tb , WTb )]
1
= E[gh (Tb , b)] = E[I{Tb <u} ] ,
2
which is precisely (9.1.3).
Since t1/2 Wt = G, a standard normal variable of continuous distribution function, we deduce from the reflection principle that the distribution functions
of Tb
and Mt are continuous and such that FTb (t) = 1 FMt (b) = 2(1 FG (b/ t)). In
particular, P(Tb > t) 0 as t , hence Tb is a.s. finite. We further have the
corresponding explicit probability density functions on [0, ),
b
b2
(9.1.4)
fTb (t) = FTb (t) =
e 2t ,
t
2t3
2 b2
(9.1.5)
FMt (b) =
e 2t .
fMt (b) =
b
2t
Remark. From the preceding formula for the density of Tb = b you can easily
check that it has infinite expected value, in contrast with the exit times a,b of
bounded intervals (a, b), which have finite moments (see part (c) of Exercise 8.2.36
for finiteness of the second moment and note that the same method extends to all
moments). Recall that in part (b) of Exercise 8.2.35 you have already found that
the Laplace transform of the density of Tb is
Z
est fTb (t)dt = e 2sb

LfTb (s) =
0
(and for inverting Laplace transforms, see Exercise 2.2.15). Further, using the
density of passage times, you can now derive the well-known arc-sine law for the
last exit of the Brownian motion from zero by time one.
Exercise 9.1.11. For the standard Wiener process {Wt } and any t > 0, consider
the time Lt = sup{s [0, t] : Ws = 0} of last exit from zero by t, and the Markov
time Rt = inf{s > t : Ws = 0} of first return to zero after t.
(a) Verify that Px (Ty > u) = P(T|yx| > u) for any x, y R, and with
pt (x, y) denoting the Brownian transition probability kernel, show that
for u > 0 and 0 u)dy ,
P(Rt > t + u) =
Z
P(Lt u) =
pu (0, y)P(T|y| > t u)dy .
(b) Deduce from (9.1.4) that the probability density function of Rt t is
fRt t (u) = t/( u(t + u)).

Hint: Express P(Rt > t+u)/u as one integral over y R, then change
variables to z 2 = y 2 (1/u + 1/t).
349
p
(c) Show that Lt has the arc-sine law
p P(Lt u) = (2/) arcsin( u/t) and
hence the density fLt (u) = 1/( u(t u)) on [0, t].
(d) Find the joint probability density function of (Lt , Rt ).
Remark. Knowing the law of Lt is quite useful, for {Lt > u} is just the event
{Ws = 0 for some s (u, t]}. You have encountered the arc-sine law in Exercise
3.2.16 (where you proved the discrete reflection principle for the path of the symmetric srw). Indeed, as shown in Section 9.2 by Donskers invariance principle,
these two arc-sine laws are equivalent.
Here are a few additional results about passage times and running maxima.
Exercise 9.1.12. Generalizing the proof of (9.1.3), deduce that for a standard
Wiener process, any u > 0 and a1 < a2 b,
(9.1.6)
P(Tb < u, a1 < Wu < a2 ) = P(2b a2 < Wu < 2b a1 ) ,
and conclude that the joint density of (Mt , Wt ) is

(9.1.7)
fWt ,Mt (a, b) =
2
2(2b a) (2ba)
2t
e
,
3
2t
for b a 0 and zero otherwise.

Exercise 9.1.13. Let X(s, t) := inf u[s,t] {W (u)} for a standard Wiener process
W (t), and for any t > 1, > 0, denote by gt, (x) the probability density of W (1)
at x 0, conditioned on the event {X(0, t) > }.
(a) Express gt, (x) in terms of the standard normal distribution function
FG ().
(b) Taking both t p and 0, show that gt, (x) converges to the density
fR (x) of R := G21 + G22 + G23 , for i.i.d. standard normal variables Gi ,
i = 1, 2, 3.
Exercise 9.1.14. Let W (t) = (W1 (t), W2 (t)) denote the two-dimensional Brownian motion of Definition 8.2.37, starting at a non-random W (0) = (x1 , x2 ) with
x1 > 0 and x2 > 0.
(a) Find the density of = inf{t 0 : W1 (t) = 0 or W2 (t) = 0}.
(b) Find the joint density of (, W1 ( ), W2 ( )) with respect to Lebesgue measure on {(t, x, y) (0, )3 : x = 0 or y = 0}.
Hint: The identity (9.1.6) might be handy here.
Exercise 9.1.15. Consider a Brownian Markov process (Wt , Ft ) with W0 0 and
pt (x, B) = Px (Wt B) its Brownian semi-group of transition probabilities.
(a) Show that (WtT0 , Ft ) is a homogeneous Markov process on [0, ) whose
transition probabilities are: p,t (0, {0}) = 1, and if x > 0 then p,t (x, B) =
pt (x, B)pt (x, B) for B (0, ), while p,t (x, {0}) = 2pt (x, (, 0]).
(b) Show that (|Wt |, Ft ) is a homogeneous Markov process on [0, ) whose
transition probabilities are p+,t (x, B) = pt (x, B) + pt (x, B) (for x 0
and B [0, )).
Remark. We call (WtT0 , Ft ) the Brownian motion absorbed at zero and (|Wt |, Ft )
the reflected Brownian motion. These are the simplest possible ways of constraining
the Brownian motion to have state space [0, ).
Exercise 9.1.16. The Brownian Markov process (Wt , Ft ) starts at W0 = 0.
350
(a) Show that Yt = Mt Wt is an Ft -Markov process, of the same transition

probabilities {p+,t , t 0} on [0, ) as the reflected Brownian motion.
(b) Deduce that {Yt , t 0} has the same law as the reflected Brownian motion.
Exercise 9.1.17. For a Brownian Markov process (Wt , Ft ) starting at W0 = 0,
show that L,t = sup{s [0, t] : Ws = Mt } has the same arc-sine law as Lt .
Solving the next exercise you first show that {Tb , b 0} is a strictly increasing,
left-continuous process, whose sample path is a.s. purely discontinuous.
Exercise 9.1.18. Consider the passage times {Tb , b 0} for a Brownian Markov
process (Wt , Ft ) starting at W0 = 0.
(a) Show that b 7 Tb (), is left-continuous, strictly increasing and w.p.1.
purely discontinuous (i.e. there is no interval of positive length on which
b 7 Tb is continuous).
Hint: Check that P(t 7 Mt is strictly increasing on [0, ]) = 0 for any
> 0, and relying on the strong Markov property of Wt deduce that w.p.1.
b 7 Tb must be purely discontinuous.
(b) Show that for any h bB and 0 b < c,
E[h(Tc Tb )|FT + ] = Eb [h(Tc )] = E0 [h(Tcb )] ,
b
(c) Deduce that {Tb , b 0} is a S.P. of stationary, non-negative independent increments, whose Markov semi-group has the transition probability
kernel
t2
t
e 2(yx)+ ,
qbt (x, y) = q
2(y x)3+
corresponding to the one-sided 1/2-stable density of (9.1.4).

(d) Show that {b+ , b 0} of Exercise 9.1.8 is a right-continuous modification of {Tb , b 0}, hence a strong Markov process of same transition
probabilities.
D
(e) Show that Tc = c2 T1 for any c R.
Exercise 9.1.19. Suppose that for some b > 0 fixed, {k } are i.i.d. each having
the probability density function (9.1.4) of Tb .
Pn
D
(a) Show that n2 k=1 k = Tb (which is why we say that the law of Tb is
-stable for = 1/2).
p
(b) Show that P(n2 maxnk=1 k y) exp(b 2/(y)) for all y 0 (compare with part (b) of Exercise 3.2.13).
Exercise 9.1.20. Consider a standard Wiener process {Wt , t 0}.
(a) Fixing b, t > 0 let b,t = inf{s t : Vs b} for Vs = |Ws |/ s. Check

D
that b,t = tb,1 , then show that for b < 1,

1
Eb,1 =
E[(V12 b2 )I{V1 b} ] ,
1 b2
whereas Eb,1 = in case
R s b 1.
(b) Considering now Vs = 0 exp[c(Ws Wu ) c2 (s u)/2]du for c R
non-random, verify that Vs s is a martingale and deduce that in this
case Eb,0 = b for any b > 0.
9.2. WEAK CONVERGENCE AND INVARIANCE PRINCIPLES
First 20 steps
First 1000 steps
0.5
First 100 steps

2
0.5
First 5000 steps
351
0.5
0.5
Figure 2. Scaled srw for different values of n.

9.2. Weak convergence and invariance principles
Consider the linearly interpolated, time-space rescaled random walk Sbn (t) =
n1/2 S(nt) (as depicted in Figure 2, for the symmetric srw), where
(9.2.1)
S(t) =
[t]
X
k=1
k + (t [t])[t]+1 ,
and {k } are i.i.d. Recall Exercise 3.5.18 that by the clt, if E1 = 0 and E12 = 1,
then as n the f.d.d. of the S.P. Sbn () of continuous sample path, converge
weakly to those of the standard Wiener process. Since f.d.d. uniquely determine
the law of a S.P. it is thus natural to expect also to have the stronger, convergence
in distribution, as defined next.
Definition 9.2.1. We say that S.P. {Xn (t), t 0} of continuous sample functions
D
converge in distribution to a S.P. {X (t), t 0}, denoted Xn () X (), if the
corresponding laws converge weakly in the topological space S consisting of C([0, ))
equipped with the topology of uniform convergence on compact subsets of [0, ).
D
That is, if g(Xn ()) g(X ()) whenever g : C([0, )) 7 R Borel measurable,
is such that w.p.1. the sample function of X () is not in the set Dg of points
of discontinuity of g (with respect to uniform convergence on compact subsets of
[0, )).
As we state now and prove in the sequel, such functional clt, also known as
Donskers invariance principle, indeed holds.
352
Theorem 9.2.2 (Donskers invariance principle). If {k } are i.i.d. with

E1 = 0 and E12 = 1, then for S() of (9.2.1), the S.P. Sbn () = n1/2 S(n)
converge in distribution, as n , to the standard Wiener process.
Remark. The preceding theorem is called an invariance principle because the

limiting process does not depend on the law of the summands {k } of the random
walk. However, the condition E12 < is almost necessary for the n1/2 scaling
and for having a Brownian limit process. Indeed, note Remark 3.1.13 that both fail
as soon as E|1 | = for some 0 < < 2.
Since h(x()) = f (x(t1 ), . . . , x(tk )) is continuous and bounded on C([0, )) for any
f Cb (Rk ) and each finite subset {t1 , . . . , tk } of [0, ), convergence in distribution
of S.P. of continuous sample path implies the weak convergence of their f.d.d. But,
beware that the convergence of f.d.d. does not necessarily imply convergence in
distribution, even for S.P. of continuous sample functions.
Exercise 9.2.3. Give a counter-example to show that weak convergence of the
f.d.d. of S.P. {Xn ()} of continuous sample functions to those of S.P. {X ()} of
D
continuous sample functions, does not imply that Xn () X ().
Hint: Try Xn (t) = nt1[0,1/n] (t) + (2 nt)1(1/n,2/n] (t).
Nevertheless, with S = (C([0, ), ) a complete, separable metric space (c.f. Exercise 7.2.9), we have the following useful partial converse as an immediate consequence of Prohorovs theorem.
Proposition 9.2.4. If the laws of S.P. {Xn ()} of continuous sample functions
are uniformly tight in C([0, )) and for n the f.d.d. of {Xn ()} converge
D
weakly to the f.d.d. of {X ()}, then Xn () X ().
Proof. Recall part (e) of Theorem 3.5.2, that by the Portmanteau theorem
D
Xn () X () as in Definition 9.2.1, if and only if the corresponding laws
n = PXn converge weakly on the metric space S = C([0, )) (and its Borel
-algebra). That is, if and only if Eh(Xn ()) Eh(X ()) for each h continw
uous and bounded on S (also denoted by n , see Definition 3.2.17). Let
(m)
(m)
{n } be a subsequence of {n }. Since {n } is uniformly tight, so is {n }. Thus,
(mk )
by Prohorovs theorem, there exists a further sub-subsequence {n } such that
(m )
n k converges weakly to a probability measure e on S. Recall Proposition 7.1.8
that the f.d.d. uniquely determine the law of S.P. of continuous sample functions.
Hence, from the assumed convergence of f.d.d. of {Xn ()} to those of {X ()},
(m )
we deduce that e = PX = . Consequently, Eh(Xn k ()) Eh(X ()) for
each h Cb ([0, )) (see Exercise 7.2.9). Fixing h Cb ([0, )) note that we have
(m)
just shown that every subsequence yn of the sequence yn = Eh(Xn ()) has a
(mk )
further sub-subsequence yn
that converges to y . Hence, we deduce by Lemma
2.2.11 that yn y . Since this holds for all h Cb ([0, )), we conclude that
D
Xn () X ().
Having Proposition 9.2.4 and the convergence of f.d.d. of Sbn (), Donskers invariance principle is a consequence of the uniform tightness in S of the laws of these
S.P.-s. In view of Definition 3.2.31, we prove this uniform tightness by exhibiting
compact sets K such that supn P(Sbn
/ K ) 0 as . To this end, recall the
353
following classical result of functional analysis (for a proof see [KaS97, Theorem
2.4.9] or the more general version provided in [Dud89, Theorem 2.4.7]).
`-Ascoli theorem). A set K C([0, )) has compact
Theorem 9.2.5 (Arzela
closure with respect to uniform convergence on compact intervals, if and only if
supxK |x(0)| is finite and for t > 0 fixed, supxK osct, (x()) 0 as 0, where
(9.2.2)
osct, (x()) = sup
sup
0h 0ss+ht
|x(s + h) x(s)| ,
is just the maximal absolute increment of x() over all pairs of times [0, t] which
are within distance of each other.
The Arzel`
a-Ascoli theorem suggests the following strategy for proving uniform
tightness.
Exercise 9.2.6. Let S denote the set C([0, )) equipped with the topology of
uniform convergence on compact intervals, and consider its subsets Fr, = {x() :
x(0) = 0, oscr, (x()) 1/r} for > 0 and integer r 1.
(a) Verify that the functional x() 7 osct, (x()) is continuous on S per fixed
t and and further that per x() fixed, the function osct, (x()) is nondecreasing in t and in . Deduce that Fr, are closed sets and for any
r 0, the intersection r Fr,r is a compact subset of S.
(b) Show that if S.P.-s {Xn (t), t 0} of continuous sample functions are
such that Xn (0) = 0 for all n and for any r 1,
lim sup P(oscr, (Xn ()) > r1 ) = 0 ,
0 n1
then the corresponding laws are uniformly tight in S.

/ Fr,r ) 2r .
Hint: Let K = r Fr,r with r 0 such that P(Xn
Since Sbn () = n1/2 S(n) and S(0) = 0, by the preceding exercise the uniform
tightness of the laws of Sbn (), and hence Donskers invariance principle, is an immediate consequence of the following bound.
Proposition 9.2.7. If {k } are i.i.d. with E1 = 0 and E12 finite, then
lim sup P(oscnr,n (S()) > r1 n) = 0 ,

0 n1
for S(t) =
P[t]
k=1 k
+ (t [t])[t]+1 and any integer r 1.
Proof. Fixing r 1, let qn, = P(oscnr,n (S()) > r1 n). Since t 7 S(t)
is uniformly continuous on compacts, oscnr,n (S())() 0 when 0 (for each
). Consequently, qn, 0 for each fixed n, hence uniformly over n n0
and any fixed n0 . With 7 qn, non-decreasing, this implies that b = b(n0 ) =
inf >0 supnn0 qn, is independent of n0 , hence b(1) = 0 provided inf n0 1 b(n0 ) =
lim0 lim supk qk, = 0. To show the latter, observe that since the piecewise
linear S(t) changes slope only at integer values of t,
osckr,k (S()) osckr,m (S()) Mm, ,
for m = [k] + 1, = rk/m and
(9.2.3)
Mm, =
max
1im
0jm1
|S(i + j) S(j)| .
354
Thus, for any > 0,
lim sup qk, lim sup P(Mm,(v) > v m) ,

m
p
k/m 1/(r ) as k and (v) = r3 v 2 . Since v when
where v = r
0, we complete the proof by appealing to part (c) of Exercise 9.2.8.

1
As you have just seen, the key to the proof of Proposition 9.2.7 is the following
bound on maximal fluctuations of increments of the random walk.
Pm
Exercise 9.2.8. Suppose Sm =
k=1 k for i.i.d. k such that E1 = 0 and
E12 = 1. For integers m, 1, let S(m) = Sm and Mm, be as in (9.2.3), with
Mm,0 = maxm
i=1 |Si |.
(a) Show that for any m 1 and t 0,
P(Mm,0 t + 2m) 2P(|Sm | t) .

Hint: Use Ottavianis inequality (see part (a) of Exercise 5.2.16).
(b) Show that for any m, 1 and x 0,
P(Mm, > 2x) P(Mm,1 > 2x) P(M2m,0 > x) .
(c) Deduce that if v 2 log (v) 0, then
lim sup lim sup P(Mm,(v) > v m) = 0 .

v
Hint: Recall that m
1/2
Sm G by the clt.
Applying Donskers invariance principle, you can induce limiting results for random walks out of the corresponding facts about the standard Brownian motion,
which we have found already in Subsection 9.1.
Example 9.2.9. Recall the running maxima Mt = sups[0,t] Ws , whose density
we got in (9.1.5) out of the reflection principle. Since h0 (x()) = sup{x(s) : s
[0, 1]} is continuous with respect to uniform convergence on C([0, 1]), we have from
Donskers invariance principle that as n ,
1 n
D
h0 (Sbn ) = max Sk M1
n k=0
(where we have used the fact that the maximum of the linearly interpolated function S(t) must be obtained at some integer value of t). The functions h (x()) =
R1
0 x(s) ds for = 1, 2, . . . are also continuous on C([0, 1]), so by same reasoning,

Z n
Z 1
D
h (Sbn ) = n(1+/2)
S(u) du
(Wu ) du .
0
Similar limits can be obtained by considering h (|x()|).
Exercise 9.2.10.
(a) Building on Example 9.2.9, show that for any integer 1,
Z 1
n
X
D
n(1+/2)
(Wu ) du ,
(Sk )
k=1
E12
as soon as E1 = 0 and
= 1 (i.e. there is no need to assume finiteness
of the -th moment of 1 ), and in case = 1 the limit law is merely a
normal of zero mean and variance 1/3.
355
(b) The cardinality of the set {S0 , . . . , Sn } is called the range of the walk by
time n and denoted rngn . Show that for the symmetric srw on Z,
D
n1/2 rngn sup Ws inf Ws .

s1
s1
We continue in the spirit of Example 9.2.9, except for dealing with functionals
that are no longer continuous throughout C([0, )).
Example 9.2.11. Let bn = n1 inf{k 1 : Sk n}. As shown in Exercise

9.2.12, considering the function g1 (x()) = inf{t 0 : x(t) 1} we find that
D
bn T1 as n , where the density of T1 = inf{t 0 : Wt 1} is given in
(9.1.4).
Rt
Similarly, let At (b) = 0 IWs >b ds denote the occupation time of B = (b, )
by the standard Brownian motion up to time t. Then, considering g(x(), B) =
R1
I
ds, we find that for n and b R fixed,
0 {x(s)B}
n
X
D
bn (b) = 1
A
I{Sk >bn} A1 (b) ,
n
(9.2.4)
k=1
as you are also to justify upon solving Exercise 9.2.12. Of particular note is the
D
case of b = 0, where Levys arc-sine law tells us that At (0) = Lt of Exercise 9.1.11
(as shown for example in [KaS97, Proposition 4.4.11]).
Recall the arc-sine limiting law of Exercise 3.2.16 for n1 sup{ n : S1 S 0}
in case of the symmetric srw. In view of Exercise 9.1.11, working with g0 (x()) =
sup{s [0, 1] : x(s) = 0} one can extend the validity of this limit law to any random
walk with increments of mean zero and variance one (c.f. [Dur10, Example 8.6.3]).
Exercise 9.2.12.
(a) Let g1+ (x()) = inf{t 0 : x(t) > 1}. Show that P(W G) = 1
for the subset G = {x() : x(0) = 0 and g1 (x()) = g1+ (x()) < }
of C([0, )), and that g1 (xn ()) g1 (x()) for any sequence {xn ()}
C([0, )) which converges uniformly on compacts to x() G. Further,
D
show that bn n1 g1 (Sbn ) bn and deduce that bn T1 .
(b) To justify (9.2.4), first verify that the non-negative g(x(), (b, )) is continuous on any sequence whose limit is in G = {x() : g(x(), {b}) = 0}
D
and that E[g(W , {b})] = 0, hence g(Sbn , (b, )) A1 (b). Then, deduce
D
bn (b)
that A
A1 (b) by showing that for any > 0 and n 1,
bn (b) g(Sbn , (b , )) + n () ,
g(Sbn , (b + , )) n () A
P
n
with n () = n1 k=1 I{|k |>n} converging in probability to zero
when n .
Our next result is a refinement due to Kolmogorov and Smirnov, of the GlivenkoCantelli theorem (whichPstates that for i.i.d. {X, Xk } the empirical distribution
n
functions Fn (x) = n1 i=1 I(,x] (Xi ), converge w.p.1., uniformly in x, to the
distribution function of X, whichever it may be, see Theorem 2.3.6).
356
Corollary 9.2.13. Suppose {Xk , X} are i.i.d. and x 7 FX (x) is a continuous

function. Then, setting Dn = supxR |FX (x) Fn (x)|, as n ,
D
bt | ,
n1/2 Dn sup |B
(9.2.5)
t[0,1]
bt = Wt tW1 on [0, 1].

for the standard Brownian bridge B
Proof. Recall the Skorokhod construction Xk = X (Uk ) of Theorem 1.2.37,

with i.i.d. uniform variables {Uk } on (0, 1], such that {Xk x} = {Uk FX (x)}
for all x R (see (1.2.1)), by which it follows that
Dn =
sup
uFX (R)
|u n
n
X
I(0,u] (Ui )| .
i=1
The assumed continuity of the distribution function FX () further implies that

FX (R) = (0, 1). Throwing away the sample Un , let Vn1,k , k = 1, . . . , n 1,
denote the k-th smallest number in {U1 , . . . , Un1 }. It is not hard to check that
e n | 2n1 for
|Dn D
k
n1
e n = max
D
|Vn1,k | .
k=1
n
Now, from Exercise 3.4.11 we have the representation Vn1,k = Tk /Tn , where Tk =
Pk
j=1 j for i.i.d. {j }, each having the exponential distribution of parameter one.
Pk
Next, note that Sk = Tk k is a random walk Sk = j=1 k with i.i.d. j = j 1
such that E1 = 0 and E12 = 1. Hence, setting Zn = n/Tn one easily checks that
n
e n = Zn max |n1/2 Sk
n1/2 D
k=1
k 1/2
(n
Sn )| = Zn sup |Sbn (t) tSbn (1)| ,
n
t[0,1]
where Sbn (t) = n1/2 S(nt) for S() of (9.2.1), so t 7 Sbn (t) tSbn (1) is linear on
each of the intervals [k/n, (k + 1)/n], k 0. Consequently,
e n = Zn g(Sbn ()) ,
n1/2 D
a.s.
for g(x()) = supt[0,1] |x(t) tx(1)|. By the strong law of large numbers, Zn
1
E1 = 1. Moreover, g() is continuous on C([0, 1]), so by Donskers invariance
D
bt | (see Definition 9.2.1), and by Slutskys
principle g(Sbn ()) g(W ) = sup
|B
t[0,1]
e n = Zn g(Sbn ()) and then n1/2 Dn (which is within 2n1/2 of

lemma, first n1/2 D
1/2 e
n Dn ), have the same limit in distribution (see part (c), then part (b) of Exercise
3.2.8).
Remark. Defining Fn1 (t) = inf{x R : Fn (x) t}, with minor modifications
the preceeding proof also shows that in case FX () is continuous, n1/2 (FX (Fn1 (t))
D
D b
1/2
t) B
Dn
t on [0, 1]. Further, with little additional work one finds that n
bF (x) | even in case FX () is discontinuous (and which for continuous FX ()
supxR |B
X
coincides with (9.2.5)).
You are now to provide an explicit formula for the distribution function FKS () of
bt |.
supt[0,1] |B
bt on [0, 1], as in ExExercise 9.2.14. Consider the standard Brownian bridge B
ercises 7.3.15-7.3.16.
357
bt b) = exp(2b2 ) for any b > 0.

(a) Show that qb = P(supt[0,1] B
(b)
(r)
Hint: Argue that qb = P(b < ) for b of Exercise 8.2.35.

(b) Deduce that for any non-random a, c > 0,
X
bt a or sup B
bt c) =
P( inf B
(1)n1 (pn + rn ) ,
t[0,1]
t[0,1]
n1
where p2n = r2n = qna+nc , r2n+1 = qna+nc+c and p2n+1 = qna+nc+a .

Hint: Using inclusion-exclusion prove this expression for pn = P( for
bti = c for even
bti = a for odd i and B
some 0 < t1 < < tn 1, B
bti = a at
b
i) and rn similarly defined, just with Bti = c at odd i and B
even i. Then use the reflection principle for Brownian motion Wt such
that |W1 | to equate these with the relevant qb (in the limit 0).
(c) Conclude that for any non-random b > 0,
X
2 2
bt | b) = 1 2
(9.2.6)
FKS (b) = P( sup |B
(1)n1 e2n b .
t[0,1]
n=1
Remark. The typical approach to accepting/rejecting the hypothesis that i.i.d.

observations {Xk } have been generated according to a specified continuous distribution FX is by thresholding the value of FKS (b) at the observed Kolmogorov-Smirnov
statistic b = n1/2 Dn , per (9.2.6). To this end, while outside our scope, using the
so called Hungarian construction, which is a much sharper coupling alternative to
Corollary 9.2.21, one can further find the rate (in n) of the convergence in distribution in (9.2.5) (for details, see [SW86, Chapter 12.1]).
We conclude with a sufficient condition for convergence in distribution on C([0, )).
Exercise 9.2.15. Suppose C([0, ))-valued random variables Xn , 1 n ,
p
defined on the same probability space, are such that kXn X k 0 for n
D
and any 1 fixed (where kxkt = sups[0,t] |x(s)|). Show that Xn X in the
topological space S consisting of C([0, )) equipped with the topology of uniform
convergence on compact subsets of [0, ).
Hint: Consider Exercise 7.2.9 and Corollary 3.5.3.
9.2.1. Skorokhods representation and the martingale clt. We pursue

here an alternative approach for proving invariance principles, which is better suited
to deal with dependence, culminating with a Lindeberg type, martingale clt. We
first utilize the continuity of the Brownian path to deduce, in view of Corollary 3.5.3,
that linear interpolation of random discrete samples along the path converges in
distribution to the Brownian motion, provided the sample times approach a uniform
density.
Lemma 9.2.16. Suppose {W (t), t 0} is a standard Wiener process and k 7 Tn,k
p
are non-decreasing, such that Tn,[nt] t when n , for each fixed t [0, ].
p
Then, kSbn W k 0 for the norm kx()k = supt[0,] |x(t)| of C([0, ]) and Sbn (t) =
Sn (nt), where
(9.2.7)
Sn (t) = W (Tn,[t] ) + (t [t])(W (Tn,[t]+1 ) W (Tn,[t] )) .
Remark. In view of Exercise 9.2.15, the preceding lemma implies in particular

p
D
that if Tn,[nt] t for each fixed t 0, then Sbn () W () in C([0, )).
358
Proof. Recall that each sample function t 7 W (t)() is uniformly continuous on [0, ], hence osc, (W ())() 0 as 0 (see (9.2.2) for the definition of
osc, (x())). Fixing > 0 note that as r ,
Gr = { : osc,3/r (W ())() } ,
so by continuity from below of probability measures, P(Gr ) 1 for some integer

p
r. Setting sj = j/r for j = 0, 1, . . . , r + 1, our hypothesis that Tn,[nt] t per fixed
t 0, hence uniformly on any finite collection of times, implies that for some finite
n0 = n0 (, r) and all n n0 ,
r
P(max |Tn,[nsj ] sj | r1 ) 1 .
j=0
Further, by the monotonicity of k 7 Tn,k , if t [sj1 , sj ) and n r, then

Tn,[nsj1 ] sj Tn,[nt] t Tn,[nt]+1 t Tn,[nsj+1 ] sj1
and since sj+1 sj1 = 2/r, it follows that for any n max(n0 , r),
(9.2.8)
P(
sup
b{0,1},t[0,)
|Tn,[nt]+b t| 3r1 ) 1 .
Recall that for any n 1 and t 0,
|Sbn (t) W (t)| (1 )|W (Tn,[nt] ) W (t)| + |W (Tn,[nt]+1 ) W (t)| ,
where = nt [nt] [0, 1). Observe that if both Gr and the event in (9.2.8)
occur, then by definition of Gr each of the two terms on the right-side of the last
inequality is at most . We thus see that kSbn W k whenever both Gr and
the event in (9.2.8) occur. That is, P(kSbn W k ) 1 2. Since this applies
p
for all > 0, we have just shown that kSbn W k 0, as claimed.
The key tool in our program is an alternative Skorokhod representation of random variables. Whereas in Theorem 1.2.37 we applied the inverse of the desired
distribution function to a uniform random variable on [0, 1], here we construct a
stopping time of the form A,B = inf{t 0 : Wt
/ (A, B)} such that W has
the stated, mean zero law. To this end, your next exercise exhibits the appropriate
random levels (A, B) to be used in this construction.
Definition 9.2.17. Given a random variable V 0 of positive, finite mean, we
say that Z 0 is a size-biased sample of V if E[g(Z)] = E[V g(V )]/EV for all
g bB. Alternatively, the Radon-Nikodym derivative between the corresponding
dPZ
laws is dP
(v) = v/EV .
V
Exercise 9.2.18. Suppose X is an integrable random variable, such that EX = 0
(so EX+ = EX is finite). Consider the [0, )2 -valued random vector
(A, B) = (0, 0)I{X=0} + (Z, X)I{X>0} + (X, Y )I{X<0} ,
where Y and Z are size-biased samples of X+ and X , respectively, which are
further independent of X. Show that then for any f bB,
(9.2.9)
E[r(A, B)f (A) + (1 r(A, B))f (B)] = E[f (X)] ,
where r(a, b) = b/(a + b) for a > 0 and r(0, b) = 1.

Hint: It suffices to show that E[Xh(Z, X)I{X>0} ] = E[(X)h(X, Y )I{X<0} ] for
h(a, b) = (f (b) f (a))/(a + b).
359
Theorem 9.2.19 (Skorokhods representation). Suppose (Wt , Ft , t 0) is

a Brownian Markov process such that W0 = 0 and F is independent of the [0, 1]valued independent uniform variables Ui , i = 1, 2. With Gt = ((U1 , U2 ), Ft ),
given the law PX of an integrable X such that EX = 0, there exists an a.s. finite
D
Gt -stopping time such that W = X, E = EX 2 and E 2 2EX 4 .
Remark. The extra randomness in the form of Ui , i = 1, 2 is not needed when
X = {a, b} takes only two values (for you can easily check that then Exercise
9.2.18 simply sets A = a and B = b). In fact, one can eliminate it altogether, at
the cost of a more involved proof (see [Bil95, Theorem 37.6] or [MP09, Sections
5.3.1-5.3.2] for how this is done).
Proof. We first construct (X, Y, Z) of Exercise 9.2.18 from the independent
pair of uniformly distributed random variables (U1 , U2 ). That is, given the specified distribution function FX () of X, we set X() = sup{x : FX (x) < U1 ()}
as in Theorem 1.2.37 so that X m(U1 ). Noting
R y that FX () uniquely determines the distribution functions FY (y) = I{y>0} 0 vdFX (v)/EX+ and FZ (z) =
R0
I{z>0} z (v)dFX (v)/EX , we apply the same procedure to construct the strictly
positive Y and Z out of U2 . With the resulting pair (Y, Z) measurable on (U2 )
and independent of X, we proceed to have the [0, )2 -valued random vector (A, B)
measurable on (U1 , U2 ) as in Exercise 9.2.18.
We claim that = inf{t 0 : Wt
/ (A, B)} is an a.s. finite Gt -stopping time.
Indeed, considering the Ft -adapted Mt+ = sups[0,t] Ws and Mt = inf s[0,t] Ws ,
note that by continuity of the Brownian sample function, t if and only if either
Mt+ B or Mt A. With (A, B) measurable on (U1 , U2 ) G0 it follows that
{ : () t} Gt for each t 0. Further, recall part (d) of Exercise 8.2.35 that
w.p.1. Mt+ , in which case the Gt -stopping time () is finite.
Setting Y = [0, )2 , we deduce from the preceding analysis of the events {
t} that () = h(, (A(), B()) with h(, (a, b)) = a,b () measurable on the
product space (, F ) (Y, BY ). Since {Wt } is Ft -progressively measurable, the
same applies for V = I{ <} f (W , ) and any bounded below Borel function f on
R2 . With (A, B) (U1 , U2 ) independent of F , it thus follows from Fubinis
theorem that E[V ] = E[g(A, B)] for
g(a, b) = E[I{a,b <} f (Wa,b , a,b )] .
In particular, considering V = I{ <} f (W ), since {Wt } is a Brownian Markov
process starting at W0 = 0, we have from part (a) of Exercise 8.2.36 that
g(a, b) = r(a, b)f (a) + (1 r(a, b))f (b) ,
where r(a, b) = b/(a + b) for a > 0 and r(0, b) = 1. In view of the identity (9.2.9)
we thus deduce that for any f bB,
E[f (W )] = E[V ] = E[g(A, B)] = E[f (X)] .
That is, W = X, as claimed. Recall part (c) of Exercise 8.2.36 that E[a,b ] = ab.
Since ab = r(a, b)f (a) + (1 r(a, b))f (b) for f (x) = x2 and any (a, b) Y, we
deduce by the same reasoning that E[ ] = E[AB] = E[X 2 ]. Similarly, the bound
2
E 2 35 EX 4 follows from the identity E[a,b
] = (ab)2 + ab(a2 + b2 )/3 of part (c)
of Exercise 8.2.36, and the inequality
ab(a2 + b2 + 3ab) 5ab(a2 + b2 ab) = 5[r(a, b)a4 + (1 r(a, b))b4 ]
360
for all (a, b) Y.
Building on Theorem 9.2.19 we have the following representation, due to Strassen,

of any discrete time martingale via sampling along the path of the Brownian motion.
Theorem 9.2.20 (Strassens martingale representation). Suppose the probability space contains a martingale (M , F ) such that M0 = 0, the i.i.d. [0, 1]-valued
uniform {Ui } and a standard Wiener process {W (t), t 0}, both independent of
F and of each other. Then:
(a) The filtrations Fk,t = (Fk , (Ui , i 2k), FtW ) are such that (W (t), Fk,t )
is a Brownian Markov process for any 1 k .
(b) There exist non-decreasing a.s. finite Fk,t -stopping times {Tk }, starting
with T0 = 0, where k = Tk Tk1 and the filtration Hk = Fk,Tk are such
that w.p.1. E[k |Hk1 ] = E[Dk2 |Fk1 ] and E[k2 |Hk1 ] 2E[Dk4 |Fk1 ]
for the martingale differences Dk = Mk Mk1 and all k 1.
(c) The discrete time process {W (T )} has the same f.d.d. as {M }.
Remark. In case F = (Mk , k ), our proof constructs the martingale as
samples M = W (T ) of the standard Wiener process {W (t), t 0} at a.s. finite,
non-decreasing ((Ui , i 2k), FtW )-stopping times Tk . Upon eliminating the
extra randomness (U1 , U2 ) in Theorem 9.2.19, we thus get the embedding of {Mn }
inside the path t 7 Wt , where {Tk } are FtW -stopping times and part (b) of the
theorem applies for the corresponding stopped -algebras Hk = FTW
. Indeed, we
k
have stipulated the a-apriori existence of (M , F ) independently of the Wiener
process only in order to accommodate non-canonical filtrations {F }.
Proof. (a). Fixing 1 k , since H = Fk,0 is independent of F W , we
have from Proposition 4.2.3 that for any B B and u, s 0,
E[IB (W (u + s))|Fk,s ] = E[IB (W (u + s))|(H, FsW )] = E[IB (W (u + s))|FsW ] .
With (W (t), FtW ) a Brownian Markov process, it thus follows that so are (W (t), Fk,t ).
(b). Starting at T0 = 0 we sequentially construct the non-decreasing Fk,t -stopping
times Tk . Assuming Tk1 have been constructed already, consider Corollary 9.1.6
for the Brownian Markov process (W (t), Fk1,t ) and the Fk1,t -stopping time Tk1 ,
to deduce that Wk (t) = W (Tk1 +t)W (Tk1 ) is a standard Wiener process which
is independent of Hk1 . The pair (U2k1 , U2k ) of [0, 1]-valued independent uniform
variables is by assumption independent of Fk1, , hence of both Hk1 and the
standard Wiener process {Wk (t)}. Recall that Dk = Mk Mk1 is integrable and
by the martingale property E[Dk |Fk1 ] = 0. With Fk1 Hk1 and representing
our probability measure as a product measure on ( , F G ), we thus apply
Theorem 9.2.19 for the continuous filtration Gt = (U2k1 , U2k , Wk (s), s t) which
is independent of Hk1 and the random distribution function
b D |F ((, x], ) ,
FXk (x; ) = P
k
k1
corresponding to the zero-mean R.C.P.D. of Dk given Fk1 . The resulting a.s.

finite Gt -stopping time k is such that
Z
E[k |Hk1 ] = x2 dFXk (x; ) = E[Dk2 |Fk1 ]
(see Exercise 4.4.6 for the second identity), and by the same reasoning E[k2 |Hk1 ]
b D |F
2E[Dk4 |Fk1 ], while the R.C.P.D. of Wk (k ) given Hk1 matches the law P
k
k1
361
of Xk . Note that the threshold levels (Ak , Bk ) of Exercise 9.2.18 are measurable on
Fk,0 since by right-continuity of distribution functions their construction requires
only the values of (U2k1 , U2k ) and {FXk (q; ), q Q}. For example, for any x R,
{ : Xk () x} = { : U1 () FXk (q; ) for all q Q, q > x} ,
with similar identities for {Yk y} and {Z

R q k z}, whose distribution functions at
q Q are ratios of integrals of the type 0 vdFXk (v; ), each of which is the limit
as n of the Fk1 -measurable
X
(/n)(FXk (/n + 1/n; ) FXk (/n; )) .
qn
Further, setting Tk = Tk1 + k , from the proof of Theorem 9.2.19 we have that
{Tk t} if and only if {Tk1 t} and either supu[0,t] {W (u) W (u Tk1 )} Bk
or inf u[0,t] {W (u) W (u Tk1 )} Ak . Consequently, the event {Tk t} is in
(Ak , Bk , t Tk1 , I{Tk1 t} , W (s), s t) Fk,t ,
by the Fk,0 -measurability of (Ak , Bk ) and our hypothesis that Tk1 is an Fk1,t stopping time.
(c). With W (T0 ) = M0 = 0, it clearly suffices to show that the f.d.d. of {W ( ) =
W (T ) W (T1 )} match those of {D = M M1 }. To this end, recall that
Hk = Fk,Tk is a filtration (see part (b) of Exercise 8.1.11), and in part (b) we
saw that Wk (k ) is Hk -adapted such that its R.C.P.D. given Hk1 matches the
R.C.P.D. of the Fk -adapted Dk , given Fk1 . Hence, for any f bB, we have from
the tower property that
E[
n
Y
f (W ( ))] = E[E[fn (Wn (n ))|Hn1 ]
=1
n1
Y
f (W ( ))]
=1
= E[E[fn (Dn )|Fn1 ]
n1
Y
f (W ( ))]
=1
= E[E[fn (Dn )|Fn1 ]
n1
Y
=1
f (D )] = E[
n
Y
f (D )],
=1
n1
where the third equality is from the induction assumption that {D }=1
has the
n1
same law as {W ( )}=1 .
The following corollary of Strassens representation recovers Skorokhods representation of the random walk {Sn } as the samples of Brownian motion at a sequence
of stopping times with i.i.d. increments.
Corollary 9.2.21 (Skorokhods representation for random
Pnwalks). Suppose 1 is integrable and of zero mean. The random walk Sn = k=1 k of i.i.d.
{k } can be represented as Sn = W (Tn ) for T0 = 0, i.i.d. k = Tk Tk1 0 such
that E[1 ] = E[12 ] and standard Wiener process {W (t), t 0}. Further, each Tk is
a stopping time for Fk,t = ((Ui , i 2k), FtW ) (with i.i.d. [0, 1]-valued uniform
{Ui } that are independent of {Wt , t 0}).
Proof. The construction we provided in proving Theorem 9.2.20 is based on
inductively applying Theorem 9.2.19 for k = 1, 2, . . ., where Xk follows the R.C.P.D.
of the MG difference Dk given Fk1 . For a martingale of independent differences,
362
such as the random walk Sn , we can a-apriori produce the independent thresholds
(Ak , Bk ), k 1, out of the given pairs of {Ui }, independently of the Wiener process.
Then, in view of Corollary 9.1.6, for k = 1, 2, . . . both (Ak , Bk ) and the standard
Wiener process Wk () = W (Tk1 + ) W (Tk1 ) are independent of the stopped
at Tk1 element of the continuous time filtration (Ai , Bi , i < k, W (s), s t).
Consequently, so is the stopping time k = inf{t 0 : Wk (t)
/ (Ak , Bk )} with
respect to the continuous time filtration Gk,t = (Ak , Bk , Wk (s), s t), from which
we conclude that {k } are in this case i.i.d.

Combining Strassens martingale representation with Lemma 9.2.16, we are now
in position to prove a Lindeberg type martingale clt.
Theorem 9.2.22 (martingale clt, Lindebergs). Suppose that for any n 1
fixed, (Mn, , Fn, ) is a (discrete time) L2 -martingale, starting at Mn,0 = 0, and
the corresponding martingale differences Dn,k = Mn,k Mn,k1 and predictable
compensators
X
2
hMn i =
E[Dn,k
|Fn,k1 ] ,
k=1
are such that for any fixed t [0, 1], as n ,

p
hMn i[nt] t .
(9.2.10)
If in addition, for each > 0,

n
X
p
2
(9.2.11)
gn () =
E[Dn,k
I{|Dn,k |} |Fn,k1 ] 0 ,
k=1
then as n , the linearly interpolated, time-scaled S.P.

(9.2.12)
Sbn (t) = Mn,[nt] + (nt [nt])Dn,[nt]+1 ,
converges in distribution on C([0, 1]) to the standard Wiener process, {W (t), t

[0, 1]}.
Remark. For martingale differences {Dn,k , k = 1, 2, . . .} that are mutually independent per fixed n, our assumption (9.2.11) reduces to Lindebergs condition
(3.1.4) and the predictable compensators vn,t = hMn i[nt] are then non-random. In
particular, vn,t = [nt]/n t in case Dn,k = n1/2 k for i.i.d. {k } of zero mean and
unit variance, in which case gn () = E[12 ; |1 | n] 0 (see Remark 3.1.4), and
we recover Donskers invariance principle as a special case of Lindebergs martingale
clt.
Proof. Step 1. We first prove a somewhat stronger convergence statement for
martingales of uniformly bounded differences and predictable compensators. Specifically, using k k for the supremum norm kx()k = supt[0,1] |x(t)| on C([0, 1]), we
proceed under the additional assumption that for some non-random n 0,
hMn in 2 ,
max |Dn,k | 2n ,
k=1
to construct a coupling of Sbn () of (9.2.12) and the standard Wiener process W ()

p
in the same probability space, such that kSbn W k 0. To this end, apply Theorem 9.2.20 simultaneously for the martingales (Mn, , Fn, ) with the same standard
Wiener process {W (t), t 0} and auxiliary i.i.d. [0, 1]-valued uniform {Ui }, to get
363
P
the representation Mn, = W (Tn, ). Recall that Tn, = k=1 n,k where for each
n, the non-negative n,k are adapted to the filtration {Hn,k , k 1} and such that
w.p.1. for k = 1, . . . , n,
2
E[n,k |Hn,k1 ] = E[Dn,k
|Fn,k1 ] ,
(9.2.13)
2
4
E[n,k
|Hn,k1 ] 2E[Dn,k
|Fn,k1 ] .
(9.2.14)
Under this representation, the process Sbn () of (9.2.12) is of the form considered in
p
p
Lemma 9.2.16, and as shown there, kSbn W k 0 provided Tn,[nt] t for each
fixed t [0, 1].
To verify the latter convergence in probability, set Tbn, = Tn, hMn i and bn,k =
n,k E[n,k |Hn,k1 ]. Note that by the identities of (9.2.13),
hMn i =
k=1
2
E[Dn,k
|Fn,k1 ] =
k=1
E[n,k |Hn,k1 ] ,
P
hence Tbn, = k=1 bn,k is for each n, the Hn, -martingale part in Doobs decomposition of the integrable, Hn, -adapted sequence {Tn, , 0}. Further, considering the
expectation in both sides of (9.2.14), by our assumed uniform bound |Dn,k | 2n
it follows that for any k = 1, . . . , n,
2
2
4
2
E[b
n,k
] E[n,k
] 2E[Dn,k
] 82n E[Dn,k
],
where the left-most inequality is just the L2 -reduction of conditional centering

(namely, E[Var(X|H)] E[X 2 ], see part (a) of Exercise 4.2.16). Consequently,
the martingale {Tbn, , 0} is square-integrable and since its differences bn,k are
uncorrelated (see part (a) of Exercise 5.1.8), we deduce that for any n,
2
E[Tbn,
]=
k=1
2
E[b
n,k
] 82n
n
X
k=1
2
E[Dn,k
] = 82n E[hMn in ] .
Recall our assumption that hMn in is uniformly bounded, hence fixing t [0, 1], we
L2
conclude that Tbn,[nt] 0 as n . This of course implies the convergence to zero
in probability of Tbn,[nt] , and in view of assumption (9.2.10) and Slutskys lemma,
p
also Tn,[nt] = Tbn,[nt] + hMn i[nt] t as n .
Step 2. We next eliminate the superfluous assumption hMn in 2 via the strategy
employed in proving part (a) of Theorem 5.3.33. That is, consider the Fn, -stopped
fn, = Mn,n for stopping times n = nmin{ < n : hMn i+1 > 2},
martingales M
e n,k =
such that hMn in 2. As the corresponding martingale differences are D
Dn,k I{kn } , you can easily verify that for all n
fn i = hMn in .
hM
fn i[nt] = hMn i[nt]

fn in = hMn in 2. Further, if {n = n} then hM
In particular, hM
for all t [0, 1] and the function
fn,[nt] + (nt [nt])D
e n,[nt]+1 ,
Sen (t) = M
coincides on [0, 1] with Sbn () of (9.2.12). Due to the monotonicity of 7 hMn i we

p
have that {n < n} = {hMn in > 2}, so our assumption (9.2.10) that hMn in 1
p
fn i[nt] t and applying Step 1
implies that P(n < n) 0. Consequently, hM
364
fn, , n} we have the coupling of Sen () and the standard

for the martingales {M
p
Wiener process W () such that kSen W k 0. Combining it with the (natural)
coupling of Sen () and Sbn () such that P(Sbn () 6= Sen ()) P(n < n) 0, we arrive
p
at the coupling of Sbn () and W () such that kSbn W k 0.
Step 3. We establish the clt under the condition (9.2.11), by reducing the problem
to the setting we have already handled in Step 2. This is done by a truncation
argument similar to the one we used in proving Theorem 2.1.11, except that now
we need to re-center the martingale differences after truncation is done (to convince
yourself that some truncation is required, consider the special case of i.i.d. {Dn,k }
with infinite forth moment, and note that a stopping argument as in Step 2 is
not feasible here because unlike hMn i , the martingale differences are not Fn, predictable).
Specifically, our assumption (9.2.11) implies the existence of finite nj such
that P(gn (j 1 ) j 3 ) j 1 for all j 1 and n nj n1 = 1. Setting n = j 1
for n [nj , nj+1 ) it then follows that as n both n 0 and for > 0 fixed,
2
P(2
n gn (n ) ) P(n gn (n ) n ) 0 .
In conclusion, there exist non-random n 0 such that 2

n gn (n ) 0, which we
use hereafter as the truncation level for the martingale differences {Dn,k , k n}.
P
e n,k , where
fn, = D
That is, consider for each n, the new martingale M
k=1
e n,k = Dn,k E[Dn,k |Fn,k1 ] ,
D
Dn,k = Dn,k I{|Dn,k |<n } .
e n,k | 2n for all k n. Hence, with D

b n,k = Dn,k I{|D | } ,
By construction, |D
n
n,k
by Slutskys lemma and the preceding steps of the proof we have a coupling such
p
that kSen W k 0, as soon as we show that for all n,
fn i 2
0 hMn i hM
k=1
2
b n,k
E[D
|Fn,k1 ]
(for the right hand side is bounded by 2gn (n ) which by our choice of n converges
to zero in probability, so the convergence (9.2.10) of the predictable compensators
fn ). These inequalities are in turn a direct consequence of
transfers from Mn to M
the bounds
e 2 |F ] + 2E[D
b 2 |F ]
e 2 |F ] E[D2 |F ] E[D2 |F ] E[D
(9.2.15)
E[D
n,k
n,k
n,k
n,k
n,k
holding for F = Fn,k1 and all 1 k n. The left-most inequality in (9.2.15) is

merely an instance of the L2 -reduction of conditional centering, while the middle
b n,k upon realizing that by definition
one follows from the identity Dn,k = D n,k + D
b n,k = 0, so
Dn,k 6= 0 if and only if D
2
2
2
b n,k
|F ] .
E[Dn,k
|F ] = E[Dn,k |F ] + E[D
The latter identity also yields the right-most inequality in (9.2.15), for E[Dn,k |F ] =
b n,k |F ] (due to the martingale condition E[Dn,k |Fn,k1 ] = 0), hence
E[D
2

2
2
2
e n,k
b n,k |F ] 2 E[D
b n,k
E[Dn,k |F ] E[D
|F ] = E[Dn,k |F ] = E[D
|F ] .
p
p
Now that we have exhibited a coupling for which kSen W k 0, if kSbn Sen k 0
then by the triangle inequality for the supremum norm k k there also exists a
p
coupling with kSbn W k 0 (to construct the latter coupling, since there exist
365
non-random n 0 such that P(kSbn Sen k n ) 0, given the coupled Sen ()

and W () you simply construct Sbn () per n, conditional on the value of Sen () in
such a way that the joint law of (Sbn , Sen ) minimizes P(kSbn Sen k n ) subject to
the specified laws of Sbn () and of Sen ()). In view of Corollary 3.5.3 this implies the
convergence in distribution of Sbn () to W () (on the metric space (C([0, 1]), k k)).
p
b n,k | |D
b n,k |2 /n , hence
Turning to verify that kSbn Sen k 0, recall first that |D
for F = Fn,k1 ,
b n,k |F )| E( |D
b n,k | |F ) 1
b2
|E(D n,k |F )| = |E(D
n E[Dn,k |F ] .
Note also that if the event n = {|Dn,k | < n , for all k n} occurs, then Dn,k
e n,k = E[Dn,k |Fn,k1 ] for all k. Therefore,
D
In kSbn Sen k In
n
X
k=1
e n,k |
|Dn,k D
n
X
k=1
|E(Dn,k |Fn,k1 )| 1
n gn (n ) ,
p
and our choice of n 0 such that
0 implies that In kSbn Sen k 0.
c
We thus complete the proof by showing that P(n ) 0. Indeed, fixing n and
r > 0, we apply Exercise 5.3.40 for the events Ak = {|Dn,k | n } adapted to
the filtration {Fn,k , k 0}, and by Markovs inequality for C.E. (see part (b) of
Exercise 4.2.22), arrive at
n
n
X

[

P(cn ) = P
Ak er + P
P(|Dn,k | n |Fn,k1 ) > r
k=1
p
2
n gn (n )
k=1
er + P 2
n
n
X
k=1

2
b n,k
E[D
|Fn,k1 ] > r = er + P(2
n gn (n ) > r) .
Consequently, our choice of n implies that P(cn ) 3r for any r > 0 and all n
large enough. So, upon considering r 0 we deduce that P(cn ) 0. As explained
before this concludes our proof of the martingale clt.

Specializing Theorem 9.2.22 to the case of a single martingale (M , F ) leads to
the following corollary.
Corollary 9.2.23. Suppose an L2 -martingale (M , F ) starting at M0 = 0, is of
p
F -predictable compensators such that n1 hM in 1 and as n ,
n1
n
X
k=1
E[(Mk Mk1 )2 ; |Mk Mk1 | n] 0 ,
for any fixed > 0. Then, as n , the linearly interpolated, time-scaled S.P.
(9.2.16)
Sbn (t) = n1/2 {M[nt] + (nt [nt])(M[nt]+1 M[nt] )} ,
converge in distribution on C([0, 1]) to the standard Wiener process.
Proof. Simply consider Theorem 9.2.22 for Mn, = n1/2 M and Fn, = F .
p
In this case hMn i = n1 hM i so (9.2.10) amounts to n1 hM in 1 and in stating
the corollary we merely replaced the condition (9.2.11) by the stronger assumption
that E[gn ()] 0 as n .

Further specializing Theorem 9.2.22, you are to derive next the martingale extension of Lyapunovs clt.
366
Pk
Exercise 9.2.24. Let Zk =
i=1 Xi , where the Fk -adapted, square-integrable
{Xk } are such that w.p.1. E[Xk |Fk1 ] = for some non-random and all k 1.
Pn
p
Setting Vn,q = nq/2 k=1 E[|Xk |q |Fk1 ] suppose further that Vn,2 1, while
p
for some q > 2 non-random, Vn,q 0.
(a) Setting Mk = Zk k show that Sbn () of (9.2.16) converges in distribution
on C([0, 1]) to the standard Wiener process.
D
(b) Deduce that Ln L , where
Ln = n1/2 max {Zk
0kn
k
Zn }
n
and P(L b) = exp(2b ) for any b > 0.

p
(c) In case > 0, set Tb = inf{k 1 : Zk > b} and show that b1 Tb 1/
when b .
The following exercises present typical applications of the martingale clt, starting
with the least-squares parameter estimation for first order auto regressive processes
(see Exercises 6.1.15 and 6.3.30 for other aspects of these processes).
Exercise 9.2.25 (First order auto regressive process). Consider the Rvalued S.P. Y0 = 0 and Yn = Yn1 +Dn for n 1, with {Dn } a uniformly bounded
Fn -adapted martingale difference sequence such that a.s. E[Dk2 |Fk1 ] = 1 for all
k 1, and (1, 1) is a non-random parameter.
Pn
a.s.
(a) Check that {Yn } is uniformly bounded. Deduce that n1 k=1 Dk2 1
P
a.s.
and n1 Zn 0, where Zn = nk=1 Yk1 Dk .
Hint: See P
part (c) of Exercise 5.3.41.
Pn
n
2
. Considering the estimator
bn = V1n k=1 Yk Yk1
(b) Let Vn = k=1 Yk1
D
n ) N (0, 1) as n .
of the parameter , conclude that Vn (b
(c) Suppose now that = 1. Show that in this case (Yn , Fn ) is a martingale
of uniformly bounded differences and deduce from the martingale clt that
the two-dimensional random vectors (n1 Zn , n2 Vn ) converge in distriR1
bution to ( 12 (W12 1), 0 Wt2 dt) with {Wt , t 0} a standard Wiener
process. Conclude that in this case
p
W2 1
D
Vn (b
.
n ) qR1
1
2 0 Wt2 dt
(d) Show that the conclusion of part (c) applies in case = 1, except for
multiplying the limiting variable by 1.
Hint: Consider the sequence (1)k Yk with Yk corresponding to Dk =
(1)k Dk and = 1.
Pk
Exercise 9.2.26. Let Ln = n1/2 max0kn { i=1 ci Yi }, where the Fk -adapted,
square-integrable {Yk } are such that w.p.1. E[Yk |Fk1 ] = 0 and E[Yk2 |Fk1 ] = 1
for all k 1. Suppose further that supk1 E[|Yk |q |Fk1 ] is finite a.s. for some
Pn
p
D
q > 2 and ck mFk1 are such that n1 k=1 c2k 1. Show that Ln L with
P(L b) = 2P(G b) for a standard normal variable G and all b 0.
p
Hint: Show that k 1/2 ck 0, then consider part (a) of Exercise 9.2.24.
367
9.2.2. Law of the iterated logarithm. With Wt a centered normal

variable
of variance t, one expects the Brownian sample function to grow as t for t

and t 0. While this is true for fixed, non-random times, such reasoning ignores
the random fluctuations of the path (as we have discussed before in the context of
random walks, see Exercise 2.2.24). Accounting for these we obtain the following
law of the iterated logarithm (lil).
p
Theorem 9.2.27 (Kinchins lil). Set h(t) = 2t log log(1/t) for t < 1/e and
e
ft , t 0},
h(t) = th(1/t). Then, for standard Wiener processes {Wt , t 0} and {W
w.p.1. the following hold:
Wt
Wt
lim sup
(9.2.17)
= 1,
lim inf
= 1,
t0
h(t)
h(t)
t0
(9.2.18)
lim sup
t
ft
W
= 1,
e
h(t)
lim inf
t
ft
W
= 1.
e
h(t)
x2
Remark. To determine the scale h(t) recall the estimate P(G x) = e 2 (1+o(1))
which implies that for tn = 2n , (0, 1) the sequence P(Wtn bh(tn )) =
2
nb +o(1) is summable when b > 1 but not summable when b < 1. Indeed, using
such tail bounds we prove the lil in the form of (9.2.17) by the subsequence method
you have seen already in the proof of Proposition 2.3.1. Specifically, we consider
such time skeleton {tn } and apply Borel-Cantelli I for b > 1 and near one (where
Doobs inequality controls the fluctuations of t 7 Wt by those at {tn }), en-route
to the upper bound. To get a matching lower bound we use Borel-Cantelli II for
b < 1 and the independent increments Wtn Wtn+1 (which are near Wtn when
is small).
Proof. Since e
h(t) = th(1/t), by the time-inversion invariance of the standard
ft = tW1/t that (9.2.18) is equivalent
Wiener process, it follows upon considering W
to (9.2.17). Further, by the symmetry of this process, it suffices to prove the
statement about the lim sup in (9.2.17).
Proceeding to upper bound Wt /h(t), applying Doobs inequality (8.2.2) for the
non-negative martingale Xs = exp((Ws s/2)) (see part (a) of Exercise 8.2.7),
such that E[X0 ] = 1, we find that for any t, , y 0,
P( sup {Ws s/2} y) = P( sup Xs ey ) ey .
s[0,t]
s[0,t]
Fixing , (0, 1), consider this inequality for tn = 2n , yn = h(tn )/2, n =

(1 + 2)h(tn )/tn and n > n0 () = 1/(2 log(1/)). Since exp(h(t)2 /2t) = log(1/t),
it follows that en yn = (n0 /n)1+2 is summable. Thus, by Borel-Cantelli I, for
some n1 = n1 (, , ) n0 finite, w.p.1. sups[0,tn ] {Ws n s/2} yn for all
n n1 . With log log(1/t) non-increasing on [0, 1/e], we then have that for every
s (tn+1 , tn ] and n n1 ,
1+
h(s)
Ws yn + n tn /2 = (1 + )h(tn )
Therefore, w.p.1. lim sups0 Ws /h(s) (1 + )/. Considering k = 1/k = 1 k

and k we conclude that
Ws
1 , w.p.1.
(9.2.19)
lim sup
h(s)
s0
368
To bound below the left side of (9.2.19), consider the independent events An =
{Wtn Wtn+1 (1 2 )h(tn )}, where as before tn = 2n , n > n0 () and (0, 1)
is fixed. Setting xn = (1 2 )h(tn )/ tn tn+1 , we have by the time-homogeneity

and scaling properties of the standard Wiener process (see parts (b) and (d) of
Exercise 9.1.1), that P(An ) = P(W1 xn ). Further, noting that both x2n /2 =
2
(1 2 ) log log(1/tn ) and nx1
n exp(xn /2) as n , by the lower
bound on the tail of the standard normal distribution (see part (a) of Exercise
2.2.24), we have that for some > 0 and all n large enough,
2
1 x2
P(An ) = P(W1 xn ) n exn/2 n1 .
2xn
P
Now, by Borel-Cantelli II the divergence of the series n P(An ) implies that w.p.1.
Wtn Wtn+1 (1 2 )h(tn ) for infinitely many values of n. Further, applying
the bound (9.2.19) for the standard Wiener process {Ws , s 0}, we know that
w.p.1. Wtn+1 2h(tn+1 ) 4h(tn ) for all n large enough. Upon adding these
two bounds, we have that w.p.1. Wtn (1 4 2 )h(tn ) for infinitely many
values of n. Finally, considering k = 1/k and k , we conclude that w.p.1.
lim supt0 Wt /h(t) 1, which completes the proof.
As illustrated by the next exercise, restricted to a sufficiently sparsely spaced {tn },

the a.s. maximal
fluctuations of the Wiener process are closer to the fixed time clt
h(t).
scale of O( t), than the lil scale e
Exercise 9.2.28. Show that for tn = exp(exp(n)) and a Brownian Markov process
{Wt , t 0}, almost surely,
p
lim sup Wtn / 2tn log log log tn = 1 .
n
Combining Kinchins lil and the representation of the random walk as samples of
the Brownian motion at random times, we have the corresponding lil of HartmanWintner for the random walk.
Pn
Proposition 9.2.29 (Hartman-Wintners lil). Suppose Sn = k=1 k , where
k are i.i.d. with E1 = 0 and E12 = 1. Then, w.p.1.
lim sup Sn /e
h(n) = 1 ,
n
where e
h(n) = nh(1/n) = 2n log log n.
Remark. Recall part (c) of Exercise 2.3.4 that if E[|1 | ] = for some 0 < < 2
then w.p.1. n1/ |Sn | is unbounded, hence so is |Sn |/e
h(n) and in particular the
lil fails.
Proof. In Corollary 9.2.21 we represent the random

walk as Sn = WTn for the
Pn
standard Wiener process {Wt , t 0} and Tn = k=1 k with non-negative i.i.d.
a.s.
{k } of mean one. By the strong law of large numbers, n1 Tn 1. Thus, fixing
> 0, w.p.1. there exists n0 () finite such that t/(1 + ) Tt t(1 + ) for all
t n0 . With t = ee (1 + ) for 0, and V = sup{|Ws Wt | : s, t [t , t+3 ]},
h(t)
note that if t [t+1 , t+2 ] and t n0 (), then |WTt Wt | V . Further, t 7 e
is non-decreasing, so w.p.1.
S
Wt
V
t
lim sup
lim sup
e
t
e
h(t) e
h(t)
h(t )
9.3. BROWNIAN PATH: REGULARITY, LOCAL MAXIMA AND LEVEL SETS
369
and in view of (9.2.18), it suffices to show that for some non-random 0 as 0,

(9.2.20)
lim sup
V
,
e
h(t )
w.p.1.
To this end, by the triangle inequality,
V 2 sup{|Ws Wt | : s [t , t+3 ]} .
It thus follows that for = (t+3 t )/t = (1 + )3 1 and all 0,

p
p

P(V 8 e
h(t )) P
sup |Ws Wt | 2e
h(t )
s[t ,t+3 ]
= P( sup |Wu | x ) 4P(W1 x ) ,

u[0,1]
h(t )/ t+3 t = 4 log log t 2, the equality is by timewhere x = 2 e

homogeneity and scaling of the Brownian motion and the last inequality is a consequence of the symmetry of Brownian motion and the reflection principle (see
(9.1.2)). With 2 exp(x2 /2) = (/ log t )2 bounded above (in ), applying the upper bound of part (a) of Exercise 2.2.24 for the standard normal distribution of W1
we find that for some finite and all 0,
2
x /2
P(W1 x ) (2)1/2 x1
2 .
e
P
h(t )) is finite, we deduce by Borel-Cantelli
8e
Having just shown that P(V
I that (9.2.20) holds for = 8 , which as explained before, completes the
proof.
Remark. Strassens lil goes further than Hartman-Wintners lil, in characterizing the almost sure limit set (i.e., the collection of all limits of convergent
subsequences in C([0, 1])), for {S(n)/e
h(n)} and S() of (9.2.1), as
Z t
Z 1
K = {x() C([0, 1]) : x(t) =
y(s)ds :
y(s)2 ds 1} .
0
While Strassens lil is outside our scope, here is a small step in this direction.
Exercise 9.2.30. Show that w.p.1. [1, 1] is the limit set of the R-valued sequence
{Sn /e
h(n)}.
9.3. Brownian path: regularity, local maxima and level sets
Recall Exercise 7.3.13 that the Brownian sample function is a.s. locally -Holder
continuous for any < 1/2 and Kinchins lil tells us that it is not -Holder continuous for any 1/2 and any interval [0, t]. Generalizing the latter irregularity
property, we first state and prove the classical result of Paley, Wiener and Zygmund (see [PWZ33]), showing that a.s. a Brownian Markov process has nowhere
differentiable sample functions (not even at a random time t = t()).
Definition 9.3.1. For a continuous function f : [0, ) 7 R and (0, 1], the
upper and lower (right) -derivatives at s 0 are the R-valued
D f (s) = lim sup u [f (s + u) f (s)] and D f (s) = lim inf u [f (s + u) f (s)] ,

u0
u0
370
which always exist. The Dini derivatives correspond to = 1 and denoted by D1 f (s)
and D1 f (s). Indeed, a continuous function f is differentiable from the right at s
if D1 f (s) = D1 f (s) is finite.
Proposition 9.3.2 (Paley-Wiener-Zygmund). With probability one, the sample function of a Wiener process t 7 Wt () is nowhere differentiable. More precisely, for = 1 and any T ,
(9.3.1) P({ : < D Wt () D Wt () < for some t [0, T ]}) = 0 .
Proof. Fixing integers k, r 1, let
[
\
{ : |Ws+u () Ws ()| ku} .
Akr =
s[0,1] u[0,1/r]
Note that if c D1 Wt () D1 Wt () c for some t [0, 1] and c < , then by

definition of the Dini derivatives, Akr for k = [c] + 1 and some r 1. We thus
establish (9.3.1) for = 1 and T = 1, as soon as we show that Akr C for some
C F W such that P(C) = 0 (due to the uncountable union/intersection in the
definition of Akr , it is a-apriori not in F W , but recall Remark 8.1.3 that we add all
P-null sets to F0 F , hence a-posteriori Akr F and P(Akr ) = 0). To this end
we set
n
\ [
Cn,i
C=
n4r i=1
in F
, where for i = 1, . . . , n,
Cn,i = { : |W(i+j)/n () W(i+j1)/n ()| 8k/n for j = 1, 2, 3} .
To see that Akr C note that for any n 4r, if Akr then for some integer
1 i n there exists s [(i 1)/n, i/n] such that |Wt () Ws ()| k(t s)
for all t [s, s + 1/r]. This applies in particular for t = (i + j)/n, j = 0, 1, 2, 3,
in which case 0 t s 4/n 1/r and consequently, |Wt () Ws ()| 4k/n.
Then, by the triangle inequality necessarily also Cn,i .
We next
show that P(C) = 0. Indeed, note that for each i, n the random variables
Gj = n(W(i+j)/n W(i+j1)/n ), j = 1, 2, . . ., are independent, each having the
standard normal distribution. With their density bounded by 1/ 2 1/2, it

follows that P(|Gj | ) for all > 0 and consequently,
P(Cn,i ) =
3
Y
j=1
P(|Gj | 8kn1/2 ) (8k)3 n3/2 .
This in turn implies that P(C) in P(Cn,i ) (8k)3 / n for any n 4r and
upon taking n , results with P(C) = 0, as claimed.
Having established (9.3.1) for T = 1, we note that by the scaling property of
the Wiener process, the same applies for any finite T . Finally, the subset of
considered there in case T = is merely the increasing limit as n of such
subsets for T = n, hence also of zero probability.

You can even improve upon this negative result as follows.
Exercise 9.3.3. Adapting the proof of Proposition 9.3.2 show that for any fixed
> 12 , w.p.1. the sample function t 7 Wt () is nowhere -H
older continuous.
That is, (9.3.1) holds for any > 1/2.
371
Exercise 9.3.4. Let {Xt , t 0} be a stochastic process of stationary increments

D
which satisfies for some H (0, 1) the self-similarity property Xct = cH Xt , for
all t 0 and c > 0. Show that if P(X1 = 0) = 0 then for any t 0 fixed, a.s.
lim supu0 u1 |Xt+u Xt | = . Hence, w.p.1. the sample functions t 7 Xt are
not differentiable at any fixed t 0.
Almost surely the sample path of the Brownian motion is locally -Holder continuous for any < 12 but not for any > 12 . Further, by the lil its modulus of
p
continuity is at least h() = 2 log log(1/). The exact modulus of continuity
of the Brownian path is provided by the following theorem, due to Paul Levy (see
[Lev37]).
q
Theorem 9.3.5 (L
evys modulus of continuity). Setting g() = 2 log( 1 )
for (0, 1], with (Wt , t [0, T ]) a Wiener process, for any 0 < T < , almost
surely,
(9.3.2)
lim sup
0
oscT, (W )
= 1,
g()
where oscT, (x()) is defined in (9.2.2).

Remark. This result does not extend to T = , as by independence of the
unbounded Brownian increments, for any > 0, with probability one
osc, (W ) max{|Wk W(k1) |} = .
k1
Proof. Fixing 0 < T < , note that g(T )/( T g()) 1 as 0. Furf ) = T 1/2 oscT,T (W ) where W
fs = T 1/2 WT s is a standard Wiener
ther, osc1, (W
process on [0, 1]. Consequently, it suffices to prove (9.3.2) only for T = 1.
Setting hereafter T = 1, we start with the easier lower bound on the left side of
(9.3.2). To this end, fix (0, 1) and note that by independence of the increments
of the Wiener process,
P(,1 (W ) (1 )g(2 )) = (1 q )2 exp(2 q ) ,
where q = P(|W2 | > (1 )g(2 )) and

(9.3.3)
2 r
,r (x()) = max |x((j + r)2 ) x(j2 )| .

j=0
Further, by scaling and the lower bound of part (a) of Exercise 2.2.24, it is easy to
check that for some 0 = 0 () and all 0 ,
p
q = P(|G| > (1 ) 2 log 2) 2(1) .
By definition osc1,2 (x()) ,1 (x()) for any x : [0, 1] 7 R and with exp(2 q )
exp(2 ) summable, it follows by Borel-Cantelli I that w.p.1.
osc1,2 (W ) ,1 (W ) > (1 )g(2 )
for all 1 (, ) finite. In particular, for any > 0 fixed, w.p.1.
lim sup
0
osc1, (W )
1,
g()
372
and considering k = 1/k 0, we conclude that

lim sup
0
osc1, (W )
1 w.p.1.
g()
To show the matching upper bound, we fix (0, 1) and b = b() = (1+2)/(1)
and consider the events
\
A =
{,r (W ) < bg(r2 )} .
r2
By the sub-additivity of probabilities,

P(Ac )
r2
r
X 2X
P(,r (W ) bg(r2 ))
P(|Gr,j | x,r )
r2 j=0
p
where x,r = 2b log(2 /r) and Gr,j = (W(j+r)2 Wj2 )/ r2 have the standard normal distribution. Since > 0, clearly x,r is bounded away from zero for
r 2 and exp(x2,r /2) = (r2 )b . Hence, from the upper bound of part (a) of
Exercise 2.2.24 we deduce that the r-th term of the outer sum is at most 2 C(r2 )b
for some finite constant C = C(). Further, for some finite = () and all 1,
Z 2 +1
X
tb dt 2(b+1) .
rb
r2
Therefore, as (1 )b() (1 + ) = > 0,
P(Ac ) C2 2b 2(b+1) = C2 ,
c
and since
P(A ) is finite, by the first Borel-Cantelli lemma, on a set of
probability one, A for all 0 (, ) finite. As you show in Exercise 9.3.6, it
then follows from the continuity of t 7 Wt that on ,
|Ws+h () Ws ()| bg(h)[1 + (, 0 (, ), h)] ,
where (, , h) 0 as h 0. Consequently, for any ,

lim sup g()1
0
sup
0ss+h1,h=
|Ws+h () Ws ()|
b.
Since g() is non-decreasing on [0, 1/e], we can further replace the condition h =
in the preceding inequality by h [0, ] and deduce that on
osc1, (W ) p
b() .
lim sup
g()
0
Taking k = 1/k for which b(1/k) 1 we conclude that w.p.1. the same bound also
holds with b = 1.

Exercise 9.3.6. Suppose x C([0, 1]) and m,r (x) are as in (9.3.3).
(a) Show that for any m, r 0,
sup
r2m |ts|<(r+1)2m
|x(t) x(s)| 4
,1 (x) + m,r (x) .
=m+1
Hint: Deduce from part (a) of Exercise 7.2.7 that this holds if in addition
(2,k)
t, s Q1
for some k > m.
373
(b) Show that for some c finite, if 2(m+1)(1) h 1/e with m 0 and
(0, 1), then
c
2(m+1)/2 g(h) .
g(2 ) cg(2m1 )
1
=m+1
p
Hint: Recall that g(h) = 2h log(1/h) is non-decreasing on [0, 1/e].
(c) Conclude
that there exists (, 0 , h) 0 as h 0, such that if ,r (x)
bg(r2 ) for some (0, 1) and all 1 r 2 , 0 , then
sup
|x(s + h) x(s)| bg(h)[1 + (, 0 , h)] .
0ss+h1
We take up now the study of level sets of the standard Wiener process
(9.3.4)
Z (b) = {t 0 : Wt () = b} ,
for non-random b R, starting with its zero set Z = Z (0).

Proposition 9.3.7. For a.e. , the zero set Z of the standard Wiener
process, is closed, unbounded, of zero Lebesgue measure and having no isolated
points.
Remark. Recall that by Baires category theorem, any closed subset of R having
no isolated points, must be uncountable (c.f. [Dud89, Theorem 2.5.2]).
Proof. First note that (t, ) 7 Wt () is measurable with respect to B[0,) F
and hence so is the set Z = Z {}. Applying Fubinis theorem for the product
measure Leb P and h(t, ) = IZ (t, ) = I{Wt ()=0} we find that E[Leb(Z )] =
R
(Leb P)(Z) = 0 P(Wt = 0)dt = 0. Thus, the set Z is w.p.1. of zero Lebesgue
measure, as claimed. The set Z is closed since it is the inverse image of the closed
set {0} under the continuous mapping t 7 Wt . In Corollary 9.1.5 we have further
shown that w.p.1. Z is unbounded and that the continuous function t 7 Wt
changes sign infinitely many times in any interval [0, ], > 0, from which it follows
that zero is an accumulation point of Z .
Next, with As,t = { : Z (s, t) is a single point}, note that the event that Z
has an isolated point in (0, ) is the countable union of As,t over s, t Q such that
0 < s < t. Consequently, to show that w.p.1. Z has no isolated point, it suffices to
show that P(As,t ) = 0 for any 0 < s < t. To this end, consider the a.s. finite FtW Markov times Rr = inf{u > r : Wu = 0}, r 0. Fixing 0 < s < t, let = Rs noting
that As,t = { < t R } and consequently P(As,t ) P(R > ). By continuity
of t 7 Wt we know that W = 0, hence R = inf{u > 0 : W +u W = 0}.
Recall Corollary 9.1.6 that {W +u W , u 0} is a standard Wiener process
and therefore P(R > ) = P(R0 > 0) = P0 (T0 > 0) = 0 (as shown already in
Corollary 9.1.5).

In view of its strong Markov property, the level sets of the Wiener process inherit
the properties of its zero set.
Corollary 9.3.8. For any fixed b R and a.e. , the level set Z (b) is
closed, unbounded, of zero Lebesgue measure and having no isolated points.
Proof. Fixing b R, b 6= 0, consider the FtW -Markov time Tb = inf{s >
0 : Ws = b}. While proving the reflection principle we have seen that w.p.1. Tb
374
is finite and WTb = b, in which case it follows from (9.3.4) that t Z (b) if and
fu () = 0, where W
fu = WT +u WT
only if t = Tb + u for u 0 such that W
b
b
is, by Corollary 9.1.6, a standard Wiener process. That is, up to a translation by
ft and we conclude the proof by
Tb () the level set Z (b) is merely the zero set of W
applying Proposition 9.3.7 for the latter zero set.
Remark 9.3.9. Recall Example 8.2.51 that for a.e. the sample function Wt ()
is of unbounded total variation on each finite interval [s, t] with s < t. Thus, from
part (a) of Exercise 8.2.42 we deduce that on any such interval w.p.1. the sample
function W () is non-monotone. Since every nonempty interval includes one with
rational endpoints, of which there are only countably many, we conclude that for
a.e. , the sample path t 7 Wt () of the Wiener process is monotone in no
interval. Here is an alternative, direct proof of this fact.
Tn
Exercise 9.3.10. Let An = i=1 { : Wi/n () W(i1)/n () 0} and
A = { : t 7 Wt () is non-decreasing on [0, 1]}.
(a) Show that P(An ) = 2n for all n 1 and that A = n An F has zero
probability.
(b) Deduce that for any interval [s, t] with 0 s < t non-random the probability that W () is monotone on [s, t] is zero and conclude that the event
F F where t 7 Wt () is monotone on some non-empty open interval,
has zero probability.
Hint: Recall the invariance transformations of Exercise 9.1.1 and that F
can be expressed as a countable union of events indexed by s < t Q.
Our next objects of interest are the collections of local maxima and points of
increase along the Brownian sample path.
Definition 9.3.11. We say that t 0 is a point of local maximum of f : [0, ) 7

R if there exists > 0 such that f (t) f (s) for all s [(t )+ , t + ], s 6= t, and
a point of strict local maximum if further f (t) > f (s) for any such s. Similarly,
we say that t 0 is a point of increase of f : [0, ) 7 R if there exists > 0 such
that f ((t h)+ ) f (t) f (t + h) for all h (0, ].
The irregularity of the Brownian sample path suggests that it has many local
maxima, as we shall indeed show, based on the following exercise in real analysis.
Exercise 9.3.12. Fix f : [0, ) 7 R.
(a) Show that the set of strict local maxima of f is countable.
Hint: For any > 0, the points of M = {t 0 : f (t) > f (s), for all
s [(t )+ , t + ], s 6= t} are isolated.
(b) Suppose f is continuous but f is monotone on no interval. Show that
if f (b) > f (a) for b > a 0, then there exist b > u3 > u2 > u1 a
such that f (u2 ) > f (u3 ) > f (u1 ) = f (a), and deduce that f has a local
maximum in [u1 , u3 ].
Hint: Set u1 = sup{t [a, b) : f (t) = f (a)}.
(c) Conclude that for a continuous function f which is monotone on no interval, the set of local maxima of f is dense in [0, ).
Proposition 9.3.13. For a.e. , the set of points of local maximum for
the Wiener sample path Wt () is a countable, dense subset of [0, ) and all local
maxima are strict.
375
Remark. Recall that the upper Dini derivative D1 f (t) of Definition 9.3.1 is nonpositive, hence finite, at every point t of local maximum of f (). Thus, Proposition
9.3.13 provides a dense set of points t 0 where D1 Wt () < and by symmetry
of the Brownian motion, another dense set where D1 Wt () > , though as we
have seen in Proposition 9.3.2, w.p.1. there is no point t() 0 for which both
apply.
Proof. If a continuous function f has a non-strict local maximum then there
exist rational numbers 0 q1 < q4 such that the set M = {u (q1 , q4 ) : f (u) =
supt[q1 ,q4 ] f (t)} has an accumulation point in [q1 , q4 ). In particular, for some
rational numbers 0 q1 < q2 < q3 < q4 the set M intersects both intervals
(q1 , q2 ) and (q3 , q4 ). Thus, setting Ms,r = supt[s,r] Wt , if P(Ms3 ,s4 = Ms1 ,s2 ) = 0
for each 0 s1 < s2 < s3 < s4 , then w.p.1. every local maximum of Wt ()
is strict. This is all we need to show, since in view of Remark 9.3.9, Exercise 9.3.12 and the continuity of Brownian motion, w.p.1. the (countable) set
of (strict) local maxima of Wt () is dense on [0, ). Now, fixing 0 s1 < s2 <
s3 < s4 note that Ms3 ,s4 Ms1 ,s2 = Z Y + X for the mutually independent
Z = supt[s3 ,s4 ] {Wt Ws3 }, Y = supt[s1 ,s2 ] {Wt Ws2 } and X = Ws3 Ws2 .
Since g(x) = P(X = x) = 0 for all x R, we are done as by Fubinis theorem,

P(Ms3 ,s4 = Ms1 ,s2 ) = P(X Y + Z = 0) = E[g(Y Z)] = 0.
Remark. While proving Proposition 9.3.13 we have shown that for any countable collection of disjoint intervals {Ii }, w.p.1. the corresponding maximal values
suptIi Wt of the Brownian motion must all be distinct. In particular, P(Wq = Wq
for some q 6= q Q) = 0, which of course does not contradict the fact that
P(W0 = Wt for uncountably many t 0) = 1 (as implied by Proposition 9.3.7).
Here is a remarkable contrast with Proposition 9.3.13, showing that the Brownian
sample path has no point of increase (try to imagine such a path!).
s, Kakutani). Almost every sample path
Theorem 9.3.14 (Dvoretzky, Erdo
of the Wiener process has no point of increase (or decrease).
For the proof of this result, see [MP09, Theorem 5.14].
Bibliography
[And1887] D
esir
e Andr
e, Solution directe du probl
eme r
esolu par M. Bertrand, Comptes Rendus
Acad. Sci. Paris, 105, (1887), 436437.
[Bil95] Patrick Billingsley, Probability and measure, third edition, John Wiley and Sons, 1995.
[Bre92] Leo Breiman, Probability, Classics in Applied Mathematics, Society for Industrial and
Applied Mathematics, 1992.
[Bry95] Wlodzimierz Bryc, The normal distribution, Springer-Verlag, 1995.
[Doo53] Joseph Doob, Stochastic processes, Wiley, 1953.
[Dud89] Richard Dudley, Real analysis and probability, Chapman and Hall, 1989.
[Dur10] Rick Durrett, Probability: Theory and Examples, fourth edition, Cambridge U Press,
2010.
[DKW56] Aryeh Dvortzky, Jack Kiefer and Jacob Wolfowitz, Asymptotic minimax character of
the sample distribution function and of the classical multinomial estimator, Ann. Math.
Stat., 27, (1956), 642669.
[Dyn65] Eugene Dynkin, Markov processes, volumes 1-2, Springer-Verlag, 1965.
[Fel71] William Feller, An introduction to probability theory and its applications, volume II,
second edition, John Wiley and sons, 1971.
[Fel68] William Feller, An introduction to probability theory and its applications, volume I, third
edition, John Wiley and sons, 1968.
[Fre71] David Freedman, Brownian motion and diffusion, Holden-Day, 1971.
[GS01] Geoffrey Grimmett and David Stirzaker, Probability and random processes, 3rd ed., Oxford University Press, 2001.
[Hun56] Gilbert Hunt, Some theorems concerning Brownian motion, Trans. Amer. Math. Soc.,
81, (1956), 294319.
[KaS97] Ioannis Karatzas and Steven E. Shreve, Brownian motion and stochastic calculus,
Springer Verlag, third edition, 1997.
[Lev37] Paul L
evy, Th
eorie de laddition des variables al
eatoires, Gauthier-Villars, Paris, (1937).
[Lev39] Paul L
evy, Sur certains processus stochastiques homog
enes, Compositio Math., 7, (1939),
283339.
[KT75] Samuel Karlin and Howard M. Taylor, A first course in stochastic processes, 2nd ed.,
Academic Press, 1975.
[Mas90] Pascal Massart, The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality, Ann.
Probab. 18, (1990), 12691283.
[MP09] Peter M
orters and Yuval Peres, Brownian motion, Cambridge University Press, 2010.
[Num84] Esa Nummelin, General irreducible Markov chains and non-negative operators, Cambridge University Press, 1984.
[Oks03] Bernt Oksendal, Stochastic differential equations: An introduction with applications, 6th
ed., Universitext, Springer Verlag, 2003.
[PWZ33] Raymond E.A.C. Paley, Norbert Wiener and Antoni Zygmund, Note on random functions, Math. Z. 37, (1933), 647668.
[Pit56] E. J. G. Pitman, On the derivative of a characteristic function at the origin, Ann. Math.
Stat. 27 (1956), 11561160.
[SW86] Galen R. Shorak and Jon A. Wellner, Empirical processes with applications to statistics,
Wiley, 1986.
[Str93] Daniel W. Stroock, Probability theory: an analytic view, Cambridge university press,
1993.
[Wil91] David Williams, Probability with martingales, Cambridge university press, 1991.
377
Index
-system, 14
-integrable, 31
-system, 14
-algebra, 7, 177
-algebra, P-trivial, 30, 56, 159
-algebra, Borel, 10
-algebra, Markov, 293, 326
-algebra, completion, 14, 276, 277, 284,
291, 292
-algebra, countably generated, 10
-algebra, cylindrical, 273, 325, 344
-algebra, exchangeable, 223
-algebra, generated, 10, 20
-algebra, induced, 62
-algebra, invariant, 234
-algebra, optional, 293
-algebra, stopped, 185, 232, 293
-algebra, tail, 57, 257, 344
-algebra, trivial, 8, 160
-field, 7
0-1 law, 79
0-1 law, Blumenthals, 344
0-1 law, Hewitt-Savage, 224
0-1 law, Kolmogorovs, 57, 132, 199, 344
0-1 law, L
evys, 199
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
Brownian
371
Brownian
369
Brownian
Brownian
Brownian
Brownian
Brownian
bridge, 290, 325, 356

motion, 149, 288, 343
motion, d-dimensional, 308
motion, absorbed, 349
motion, drift, 290, 323
motion, fractional, 290
motion, geometric, 325
motion, integral, 290
motion, level set, 373
motion, local maximum, 374
motion, maxima, 347, 354
motion, modulus of continuity,
motion, nowhere differentiable,
motion,
motion,
motion,
motion,
motion,
nowhere monotone, 374

reflected, 349
standard, 288, 345, 362
total variation, 317
zero set, 373
Cantor set, 29, 121

Carath
eodorys extension theorem, 13
Carath
eodorys lemma, 16
Cauchy sequence, 167
central limit theorem, 96, 113, 351
central limit theorem, Donskers, 351
central limit theorem, functional, 351
central limit theorem, Lindebergs, 96, 357,
362
central limit theorem, Lyapunovs, 101, 365
central limit theorem, Markov additive
functional, 256
central limit theorem, multivariate, 148,
150
Ces
aro averages, 47, 251
change of variables, 50
Chapman-Kolmogorov equations, 235, 320,
324
characteristic function, 117, 143, 338
compensator, predictable, 202, 308, 309,
362
conditional expectation, 154, 174
cone, convex, 9, 244
absolutely continuous, 155, 231, 322

absolutely continuous, mutually, 221
algebra, 12
almost everywhere, 20
almost surely, 20
angle-brackets, 202
arc-sine law, 108, 349, 350
arc-sine law, L
evys, 355
Bessel process, 308
Bessel process, index, 308
birth processes, 336
Bonferronis inequality, 39
Borel function, 18
Borel set, 11
Borel-Cantelli I, 78, 168, 204
Borel-Cantelli II, 79, 204
branching process, 212, 231, 241
379
380
consistent, 61
continuous mapping, 80, 81, 109
continuous modification, 277
continuous, H
older, 370
convergence almost surely, 24
convergence in Lq , 39, 165
convergence in Lq , weak, 165
convergence in distribution, 104, 141, 351
convergence in measure, 39
convergence in probability, 39, 105, 142,
309
convergence weakly, 352
convergence, bounded, 45
convergence, dominated, 42, 164, 198, 314
convergence, monotone, 33, 41, 163, 314
convergence, of types, 128, 131
convergence, total-variation, 111, 253, 268
convergence, uniformly integrable, 46
convergence, vague, 114
convergence, Vitalis theorem, 46, 48, 222
convergence, weak, 104, 109, 352
convolution, 68
countable representation, 22, 273
counting process, 137, 336
coupling, 105, 360
coupling, Markovian, 253, 254
coupling, monotone, 100
coupon collectors problem, 73, 136
covariance, 71
Cram
er-Wold device, 146
DeMorgans law, 7, 77
density, Ces
aro, 17
derivative, Dini, 370, 375
diagonal selection, principle, 115, 301, 314
distribution function, 26, 88, 104, 143
distribution, Bernoulli, 53, 92, 100, 119, 134
distribution, beta, 202
distribution, Binomial, 100, 135
distribution, Cauchy, 122, 131
distribution, exponential, 28, 52, 53, 83,
105, 120, 121, 137, 174, 356
distribution, extreme value, 107
distribution, gamma, 70, 81, 137, 139, 340
distribution, geometric, 53, 73, 83, 105,
211, 342
distribution, multivariate normal, 147, 149,
175, 179, 232, 281, 286
distribution, multivariate normal,
non-degenerate, 149
distribution, normal, 29, 53, 95, 119, 289
distribution, Poisson, 53, 70, 100, 119, 134,
137
distribution, Poisson thinning, 140, 339
distribution, stable, 350
distribution, support, 30
distribution, triangular, 120
Doobs convergence theorem, 194, 301
INDEX
Doobs decomposition, 186, 309, 316

Doobs optional stopping, 196, 207, 304
Doob-Meyer decomposition, 309, 315
doubly stochastic, 245
Dynkins theorem, 15
event, 7
event space, 7
event, shift invariant, 234, 264
expectation, 31
extinction probability, 213
Fatous Lemma, 164
Fatous lemma, 42
Feller property, 330
Feller property, strong, 262
field, 12
filtration, 56, 177, 227, 320
filtration, augmented, 291
filtration, canonical, 178, 292
filtration, continuous time, 291, 296
filtration, interpolated, 292, 299, 320
filtration, left, 291
filtration, left-continuous, 292
filtration, right, 291
filtration, right-continuous, 291, 308
finite dimensional distributions, 149, 229,
271, 321
finite dimensional distributions, consistent,
272, 321
Fokker-Planck equation, 337
Fubinis theorem, 63, 285
function, absolutely continuous, 200
function, continuous, 109, 169, 262
function, H
older continuous, 277
function, harmonic, 236
function, indicator, 18
function, Lebesgue integrable, 28, 50, 285
function, Lebesgue singular, 29
function, Lipschitz continuous, 277
function, measurable, 17
function, non-negative definite, 286
function, Riemann integrable, 51
function, semi-continuous, 22, 110, 183
function, separable, 282
function, simple, 18, 169
function, slowly varying, 132
function, step, 332
function, sub-harmonic, 236
function, super-harmonic, 183, 236, 244
Galton-Watson trees, 213, 217
Girsanovs theorem, 221, 318
Glivenko-Cantelli theorem, 88, 355
graph, weighted, 246
Hahn decomposition, 157
harmonic function, 346
Hellys selection theorem, 114, 255
INDEX
hitting time, first, 180, 209, 295, 307

hitting time, last, 180
holding time, 334, 336
hypothesis testing, 90
i.i.d., 72
independence, 54
independence, P, 54
independence, mutual, 55, 146
independence, pairwise, 100
independence, stochastic processes, 285
inequality, Lp martingale, 191, 300, 313
inequality, Cauchy-Schwarz, 37
inequality, Chebyshevs, 36
inequality, Doobs, 188, 195, 299
inequality, Doobs up-crossing, 192, 301
inequality, Doobs, second, 189
inequality, H
olders, 37
inequality, Jensens, 36, 162, 182
inequality, Kolmogorovs, 91, 106
inequality, Markovs, 35
inequality, Minkowskis, 38
inequality, Ottavianis, 191, 354
inequality, Schwarzs, 167
inequality, up-crossing, 193
inner product, 167, 286
integration by parts, 65
invariance principle, Donsker, 351, 362
Kakutanis theorem, 218
Kesten-Stigum theorem, 216
Kochen-Stone lemma, 79
Kolmogorovs backward equation, 324, 337
Kolmogorovs cycle condition, 247
Kolmogorovs extension theorem, 61, 230,
274
Kolmogorovs forward equation, 337, 341
Kolmogorovs three series theorem, 102,
197, 205
Kolmogorov-Centsov theorem, 278
Kolmogorov-Smirnov statistic, 357
Kroneckers lemma, 92, 204, 205
L
evys characterization theorem, 317
L
evys continuity theorem, 126, 145
L
evys downward theorem, 221
L
evys inversion theorem, 121, 128, 144,
340
L
evys upward theorem, 198
Laplace transform, 81, 116, 348
law, 25, 60, 144
law of large numbers, strong, 71, 82, 87, 92,
191, 206, 222, 250, 251, 256
law of large numbers, strong, non-negative
variables, 85
law of large numbers, weak, 71, 75
law of large numbers, weak, in L2 , 72
law of the iterated logarithm, 84, 367
law, joint, 60, 139
381
law, size biased, 358

law, stochastic process, 229
Lebesgue decomposition, 156, 219
Lebesgue integral, 31, 174, 228, 321
Lebesgue measure, 12, 16, 28, 34, 155
Lenglart inequality, 188, 318
likelihood ratio, 220
lim inf, 77
lim sup, 77
local maximum, 374
Markov chain, 227
Markov chain, -irreducible, 260
Markov chain, aperiodic, 252, 268
Markov chain, atom, 258
Markov chain, birth and death, 231, 242,
247, 249
Markov chain, continuous time, 332
Markov chain, cyclic decomposition, 256
Markov chain, Ehrenfest, 237
Markov chain, Feller, 262
Markov chain, first entrance decomposition,
236
Markov chain, H-recurrent, 263
Markov chain, homogeneous, 227
Markov chain, irreducible, 240, 260
Markov chain, last entrance decomposition,
236
Markov chain, law, 229
Markov chain, minorization, 258
Markov chain, O-recurrent, 264
Markov chain, O-transient, 264
Markov chain, open set irreducible, 262
Markov chain, period, 252, 268
Markov chain, positive H-recurrent, 267
Markov chain, positive recurrent, 248
Markov chain, recurrent, 240, 245, 263
Markov chain, recurrent atom, 258
Markov chain, renewal, 231, 241, 247, 256
Markov chain, reversible, 246, 247
Markov chain, stationary, 234, 254
Markov chain, transient, 240, 245
Markov class, closed, 237, 260
Markov class, irreducible, 237, 260, 341
Markov occupation time, 238, 250
Markov process, 291, 320
Markov process, birth and death, 342
Markov process, Brownian, 317, 324
Markov process, generator, 324, 337, 341
Markov process, homogeneous, 320, 326
Markov process, jump parameters, 334
Markov process, jump rates, 334
Markov process, jump, explosive, 336
Markov process, jump, pure, 332
Markov process, law, 322
Markov process, O-recurrent, 308
Markov process, O-transient, 308
Markov process, stationary, 324
382
Markov process, strong, 326, 344

Markov property, 161, 232, 325
Markov property, strong, 232, 326
Markov semi-group, 235, 320
Markov semi-group, Feller, 330
Markov state, absorbing, 241
Markov state, accessible, 237, 260, 341
Markov state, aperiodic, 252, 260
Markov state, intercommunicate, 237, 341
Markov state, null recurrent, 247, 251, 341
Markov state, O-recurrent, 264
Markov state, O-transient, 264
Markov state, period, 252
Markov state, positive recurrent, 247, 260,
341
Markov state, reachable, 262, 264
Markov state, recurrent, 238, 243, 341
Markov state, transient, 238, 341
Markov time, 293, 304, 326, 345
Markov, accessible set, 260
Markov, additive functional, 256
Markov, Doeblin chain, 259
Markov, equivalence class property, 248
Markov, H-irreducible, 261
Markov, Harris chain, 263
Markov, initial distribution, 228, 321
Markov, jump process, 282, 284
Markov, meeting time, 253
Markov, minorization, 258
Markov, occupation ratios, 251
Markov, small function, 259
Markov, small set, 261
Markov, split chain, 258
martingale, 178, 236, 357
martingale difference, 178, 195
martingale differences, bounded, 366
martingale transform, 183, 205
martingale, L2 , 179, 308
martingale, Lp , 201
martingale, Lp , right-continuous, 303
martingale, backward, 221, 299
martingale, Bayes rule, 221, 298
martingale, continuous time, 296
martingale, cross variation, 318
martingale, differences, 362
martingale, Doobs, 198, 207, 302
martingale, Gaussian, 179
martingale, interpolated, 299, 303
martingale, local, 186, 314
martingale, orthogonal, 318
martingale, product, 180, 218
martingale, reversed, 221, 299
martingale, right closed, 302
martingale, square-integrable, 179, 308
martingale, square-integrable, bracket, 318
martingale, square-integrable, continuous,
309
martingale, sub, 181
INDEX
martingale, sub, continuous time, 296

martingale, sub, last element, 302, 304, 309
martingale, sub, reversed, 221, 222, 305
martingale, sub, right closed, 309
martingale, sub, right-continuous, 299
martingale, sup, reversed, 304
martingale, super, 181, 236
martingale, super, continuous time, 296
martingale, uniformly integrable, 197, 302
maximal inequalities, 188
mean, 52
measurable space, 8
measurable function, 18
measurable function, bounded, 227
measurable mapping, 18
measurable rectangles, 60
measurable set, 7
measurable space, isomorphic, 21, 62, 173,
229, 274, 321, 322
measurable space, product, 11
measure, 8
measure space, 8
measure space, complete, 14
measure, -finite, 16
measure, completion, 28, 50, 155
measure, counting, 112, 260
measure, excessive, 243, 244, 255, 341
measure, finite, 8
measure, invariant, 234, 243, 324
measure, invariant probability, 234, 247,
267, 324, 338
measure, invariant, unique, 244
measure, maximal irreducibility, 260
measure, mutually singular, 156, 220, 244,
257
measure, non-atomic, 9, 12
measure, outer, 16
measure, positive, 234
measure, probability, 8
measure, product, 59, 60
measure, regular, 11
measure, restricted, 49, 64
measure, reversible, 246, 341
measure, shift invariant, 234, 287
measure, signed, 8, 157
measure, support, 30, 156, 243
measure, surface of sphere, 69, 129
measure, tight, 113
measure, uniform, 12, 48, 70, 120, 127, 129,
139
measures, uniformly tight, 113, 125, 143,
145, 262, 352
memory-less property, 211
memoryless property, 138, 335
merge transition probability, 258
modification, 277
modification, continuous, 278
modification, RCLL, 282, 303, 315
INDEX
modification, separable, 282

moment, 52
moment generating function, 122
moment problem, 113, 122
monotone class, 17
monotone class theorem, 19, 64, 326, 327
monotone class theorem, Halmoss, 17
monotone function, 310
network, 246
non-negative definite matrix, 147
norm, 167
occupancy problem, 74, 135
occupation time, 355
optional time, 293, 304, 326
order statistics, 139, 140, 356
Ornstein-Uhlenbeck process, 290, 325
orthogonal projection, 169
parallelogram law, 167
passage time, 347, 373
point of increase, 374
point process, 137
point process, Poisson, 137, 276
Poisson approximation, 134
Poisson process, 137, 276, 297, 317
Poisson process, arrival times, 137
Poisson process, compensated, 297, 298,
310, 317
Poisson process, compound, 338
Poisson process, drift, 323
Poisson process, excess life time, 138
Poisson process, inhomogeneous, 137, 276
Poisson process, jump times, 137
Poisson process, rate, 137
Poisson process, superposition, 141
Poisson process, time change, 276
Poisson, thinning, 140
polarization, 318
Portmanteau theorem, 110, 141, 352
pre-visible, 183
predictable, 183, 186, 189, 309, 316
probability density function, 28, 144, 155,
171, 348
probability density function, conditional,
171
probability density function, joint, 58
probability space, 8
probability space, canonical, 230, 321
probability space, complete, 14
Prohorovs theorem, 114, 143, 352
quadratic variation, predictable, 308, 309
Radon-Nikodym derivative, 155, 218, 221,
231, 257, 323, 358
Radon-Nikodym theorem, 155
random field, 278
383
random variable, 18
random variable, P-degenerate, 30, 128,
131
random variable, P-trivial, 30
random variable, integrable, 32
random variable, lattice, 128
random variables, exchangeable, 223
random vector, 18, 143, 147, 286
random walk, 129, 178, 183, 187, 196,
209211, 231, 242, 247, 265, 351
random walk, simple, 108, 148, 179, 181,
210, 231, 241
random walk, simple, range, 355
random walk, symmetric, 178, 180, 210, 234
record values, 85, 92, 101, 212
reflection principle, 108, 180, 347, 354
regeneration measure, 258
regeneration times, 258, 345
regular conditional probability, 171, 172
regular conditional probability distribution,
172, 205
renewal theory, 89, 106, 137, 211, 231, 238,
251
renewal times, 89, 231, 238, 251
RMG, 221, 299
ruin probability, 209, 308
sample function, continuous, 278, 288, 309
sample function, RCLL, 282
sample function, right-continuous, 299
sample space, 7
Scheff
es lemma, 43
set function, countably additive, 8, 274
set function, finitely additive, 8
set, boundary, 110
set, continuity, 110
set, cylinder, 61
set, Lebesgue measurable, 50
set, negative, 157
set, negligible, 24
set, null, 14, 277, 291, 308
set, positive, 157
Skorokhod representation, 27, 105, 356,
358, 359
Slutskys lemma, 106, 128, 356, 363
space, Lq , 32, 165
split mapping, 258
square-integrable, 179
srw, 288
stable law, 131
stable law, domain of attraction, 131
stable law, index, 131
stable law, skewness, 132
stable law, symmetric, 131
standard machine, 33, 49, 50
state space, 227, 320
Stirlings formula, 108
stochastic integral, 299, 319
384
stochastic process, 177, 227, 271, 320

stochastic process, -stable, 350
stochastic process, adapted, 177, 227, 291,
296, 320
stochastic process, auto regressive, 366
stochastic process, auto-covariance, 286,
289
stochastic process, auto-regressive, 232, 270
stochastic process, Bessel, 308
stochastic process, canonical construction,
271, 273
stochastic process, continuous, 278
stochastic process, continuous in
probability, 284
stochastic process, continuous time, 271
stochastic process, DL, 315
stochastic process, Gaussian, 179, 286, 288
stochastic process, Gaussian, centered, 286
stochastic process, H
older continuous, 278,
315
stochastic process, increasing, 309
stochastic process, independent increments,
276, 287, 288, 297, 318, 323, 338
stochastic process, indistinguishable, 277,
310, 312
stochastic process, integrable, 178
stochastic process, interpolated, 292, 299,
320
stochastic process, isonormal, 286
stochastic process, law, 234, 275, 287, 290
stochastic process, Lipschitz continuous,
278
stochastic process, mean, 286
stochastic process, measurable, 283
stochastic process, progressively
measurable, 292, 310, 315, 326
stochastic process, pure jump, 332
stochastic process, sample function, 271
stochastic process, sample path, 137, 271
stochastic process, separable, 282, 284
stochastic process, simple, 319
stochastic process, stationary, 234, 287,
290, 324
stochastic process, stationary increments,
288, 323, 338
stochastic process, stopped, 185, 294, 306
stochastic process, supremum, 283
stochastic process, variation of, 309
stochastic process, weakly stationary, 287
stochastic process, Wiener, 288, 297
stochastic process, Wiener, standard, 288
stochastic processes, canonical
construction, 280
stopping time, 180, 232, 293, 304, 307, 327
sub-space, Hilbert, 167
subsequence method, 85, 367
symmetrization, 127
INDEX
take out what is known, 160

tower property, 160
transition probabilities, stationary, 320
transition probability, 173, 227, 320
transition probability, adjoint, 246
transition probability, Feller, 262
transition probability, jump, 334
transition probability, kernel, 231, 323, 324
transition probability, matrix, 230
truncation, 74, 101, 134
uncorrelated, 70, 71, 75, 179
uniformly integrable, 45, 165, 207, 302
up-crossings, 192, 301
urn, B. Friedman, 202, 210
urn, P
olya, 201
variance, 52
variation, 309
variation, quadratic, 309
variation, total, 111, 310, 374
vector space, Banach, 167
vector space, Hilbert, 167, 286
vector space, linear, 167
vector space, normed, 167
version, 154, 277
Walds identities, 210
Walds identity, 162
weak law, truncation, 74
Wiener process, maxima, 347
Wiener process, standard, 345, 362
with probability one, 20

Dembo Notes

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

Dembo Notes

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dembo Notes

Uploaded by

Copyright:

Available Formats

Probability Theory: STAT310/MATH230

August 27, 2013

Chapter 1. Probability, measure and integration

Chapter 2. Asymptotics: the law of large numbers

Chapter 3. Weak convergence, clt and Poisson approximation

Chapter 4. Conditional expectations and probabilities

Chapter 5. Discrete time martingales and stopping times

Chapter 6. Markov chains

Chapter 7. Continuous, Gaussian and stationary processes

7.3. Gaussian and stationary processes

Chapter 8. Continuous time martingales and Markov processes

Chapter 9. The Brownian motion

Probability, measure and integration

1. PROBABILITY, MEASURE AND INTEGRATION

Definition 1.1.2. A pair (, F ) with F a -algebra of subsets of is called a

must be countably additive if () < . Give an example that it is not necessarily

1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS

Exercise 1.1.9. Given any measures {n , n 1} on (, F ), verify that =

n=1 cn n is also a measure on this space, for any finite constants cn 0.

1. PROBABILITY, MEASURE AND INTEGRATION

Exercise 1.1.12. Suppose a random outcome N follows the Poisson probability

H = H and H = G. Deduce that HHH = HH HH for any pair

Example 1.1.15. Suppose = S is a topological space (that is, S is equipped with

1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS

Exercise 1.1.17. Verify the alternative definitions of the Borel -algebra B:

If A R is in B of Example 1.1.15, we say that A is a Borel set. In particular, all

Example 1.1.19. Another classical example of an uncountable is relevant for

1. PROBABILITY, MEASURE AND INTEGRATION

Exercise 1.1.25. Show that if is a finitely additive non-negative set function

then must be a probability measure.

1.1.3. Lebesgue measure and Carath

Remark. In view of the closure of algebra with respect to complements, we could

Exercise 1.1.29. The algebra generated by a given collection of subsets A, denoted

1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS

Theorem 1.1.30 (Carath

A = {(a1 , b1 ] (ar , br ] : 0 a1 < b1 < < ar < br 1 , r < }

Lemma 1.1.31. The set function U0 is countably additive on A. That is,

To complete the proof, we argue by contradiction, assuming that U0 (Hn ) 2 for

1. PROBABILITY, MEASURE AND INTEGRATION

As Hn H for all n, we have that

Hence, by finite additivity of U0 and our assumption that U0 (Hn ) 2, also

1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS

which is also closed under monotone increasing limits (i.e. if Ai L and Ai A,

so that A L. We conclude that L is a -system, hence by Dynkins theorem,

1. PROBABILITY, MEASURE AND INTEGRATION

(b) ( n An ) n (An ) for any countable collection of sets An F .

for E F = 2 , and prove that it is countably sub-additive, hence an outer measure

Definition 1.1.41. Let be a non-negative set function on a measurable space

1.1. PROBABILITY SPACES, MEASURES AND -ALGEBRAS

exists. Give an example of sets V1 CES and V2 CES for which V1 V2

(b) Provide an example of -algebras An for which n=1 An is not a algebra.

1. PROBABILITY, MEASURE AND INTEGRATION

1.2. Random variables and their distribution

Such a mapping is also called a measurable mapping.

We now provide two important generic examples of random variables.

1.2. RANDOM VARIABLES AND THEIR DISTRIBUTION

k2n 1(k2n ,(k+1)2n ] (x) ,

1. PROBABILITY, MEASURE AND INTEGRATION

two (bounded) non-negative functions each of which is a monotone increasing limit