M ATH 556: M ATHEMATICAL S TATISTICS I

P ROBABILITY P RIMER

8. R ANDOM VARIABLES & P ROBABILITY M ODELS

A random variable (r.v.) X is a function defined on a space (Ω, F) that associates a real number X(ω) =
x with each “event” A ∈ F . We regard X as a (possibly many-to-one) mapping from Ω to R

X : F −→ R
A 7−→ x

We note the equivalence of events in F and their images under X. Consider the space (R, B), that is
the real line R, and the sigma-algebra B of sets that are open, half-open or closed subsets of the reals,
that is, sets of the form
(a, b) (a, b] [a, b) [a, b]
and countable unions and intersections of these sets. Consider B ∈ B, and consider the pre-image

B −1 = {ω ∈ Ω : X(ω) ∈ B}

For X to be a valid random variable, we require that B −1 ∈ F . Strictly, when referring to random vari-
ables, we should make explicit the connection to original sample space Ω and write X(ω) for individual
sample outcomes, and
PX [X ∈ B] = P({ω : X(ω) ∈ B})
for events. However, generally, we will suppress this and merely refer to X.

In deducing the probability measure for X, PX , we must have

P(B −1 ) = PX [X ∈ B].

for all B ∈ B. The notation P(·) will be retained to refer to probabilities associated with events in Ω,
whereas PX [ · ] will be used when referring to events in R.

Probability Functions: Consider the real function of a real argument, FX , defined by

∫
FX (x) = PX ((−∞, x]) = PX (dx)
(−∞,x]

for real values of x. Note that X ∈ (−∞, x] is equivalent to X ≤ x, and, by definition,

∫ x
FX (x) = dFX (t).
−∞

FX defines the probability distribution of X. The nature of FX determines how we can manipulate
this function. There are (essentially) three cases to consider:

1. FX is a step-function, that is, FX changes only at a certain (countable) set of x values.

2. FX is a continuous function.
3. FX is a mixture of 1. and 2.

1
8.1. D ISCRETE R ANDOM VARIABLES
A random variable X is discrete if FX is a step-function, that is, if the set of all values at which FX
changes, denoted X, is countable

• X ≡ {x1 , x2 , . . . , xn } (that is, a finite list)

• X ≡ {x1 , x2 , . . .} (that is, a countably infinite list)

where, without loss of generality it is assumed that

x1 < x 2 < · · ·

If X is discrete, then it follows the probability of event (X ∈ B) can be decomposed

∑
P({ω : X(ω) ∈ B}) = PX [X ∈ {xi : xi ∈ B}] = PX (X = xi )
i:xi ∈B

so that FX can be represented as a sum of probabilities

∑ ∑
FX (x) = PX (X ∈ {xi }) = PX [X = xi ].
xi ≤x xi ≤x

8.1.1. P ROBABILITY M ASS F UNCTION

The function fX , defined on X by
fX (x) = PX [X = x] x∈X
that assigns probability to each x ∈ X is the (discrete) probability mass function, or pmf. For com-
pleteness, we define
fX (x) = 0 x∈/X
so that fX is defined for all x ∈ R. Thus X is the support of random variable X (or the pmf fX ): X is the
set of x ∈ R such that fX (x) > 0.

Properties: A function fX is a probability mass function for discrete random variable X with support
X of the form {x1 , x2 , . . .} if and only if
∑
(i) fX (xi ) ≥ 0 (ii) fX (xi ) = 1
i

Clearly as fX (x) = PX [X = x], we must have 0 ≤ fX (x) ≤ 1 for all x ∈ R.

8.1.2. D ISCRETE C UMULATIVE D ISTRIBUTION F UNCTION

The cumulative distribution function, or cdf, FX of a discrete r.v. X is defined by
FX (x) = PX [X ≤ x] x ∈ R.

Connection between FX and fX : Let X be a discrete random variable with support X ≡ {x1 , x2 , . . .},
pmf fX and cdf FX . For any real value x, if x < x1 , then FX (x) = 0, and for x ≥ x1 ,
∑
FX (x) = fX (xi )
xi ≤x

so that, for i = 2, 3, . . .,
fX (xi ) = FX (xi ) − FX (xi−1 )
with, for completeness, fX (x1 ) = FX (x1 ) .

2
Properties:
(i) In the limiting cases,
lim FX (x) = 0 lim FX (x) = 1.
x→−∞ x→∞

(ii) FX is continuous from the right (but not continuous) on R that is, for x ∈ R,

lim FX (x + h) = FX (x)
h→0+

but, if x ∈ X,
lim FX (x + h) 6= FX (x)
h→0−

that is, the “left limit” is not equal to the “right limit” at x values in X.
(iii) FX is non-decreasing, that is
a < b =⇒ FX (a) ≤ FX (b)

(iv) For a < b,

PX [a < X ≤ b] = FX (b) − FX (a)

Notes:
• The functions fX and/or FX can both be used to describe the probability distribution of random
variable X.

• The function fX is non-zero only at the elements of X.

• The function FX is a step-function, which takes the value zero at minus infinity, the value one at
infinity, and is non-decreasing with points of discontinuity at the elements of X.

• The right-continuity of FX is denoted in plots by the use of a filled circle, •, as in the example
below.

• In the discrete case, FX is not differentiable for all x ∈ R; at points of continuity (that is, for
x∈/ X), it is differentiable, and the derivative is zero.

Example 1 Consider a coin tossing experiment where a fair coin is tossed repeatedly under identical
experimental conditions, with the sequence of tosses independent, until a Head is obtained. For this
experiment, the sample space, Ω is then the set of sequences

({H} , {T H} , {T T H} , {T T T H} . . .)

with associated probabilities 1/2, 1/4, 1/8, 1/16, . . . .

Define discrete random variable X by X(ω) = x ⇐⇒ first H on toss x. Then

( )x
1
fX (x) = PX [X = x] = x = 1, 2, 3, . . .
2

and zero otherwise. For x ≥ 1, let bxc be the largest integer not greater than x. Then

∑ ⌊x⌋
∑ ( )⌊x⌋
1
FX (x) = fX (xi ) = fX (i) = 1 −
2
xi ≤x i=1

and FX (x) = 0 for x < 1.

3
Graphs of the probability mass function (top) and cumulative distribution function (bottom) are
shown in Figure 1. Note that the mass function is only non-zero at points that are elements of X, and
that the cdf is defined for all real values of x, but is only continuous from the right. FX is therefore a
step-function.

PMF CDF
1.0

1.0
0.8

0.8
0.6

0.6
FX(x)
fX(x)

0.4

0.4
0.2

0.2
0.0

0.0

0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11

x x

( )x ( )⌊x⌋
1 1
Figure 1: PMF fX (x) = , x = 1, 2, 3, . . . and CDF FX (x) = 1 −
2 2

8.2. C ONTINUOUS R ANDOM VARIABLES

A random variable X is termed continuous if the function FX defined on R by
FX (x) = PX [X ≤ x]
for x ∈ R is a continuous function on R , that is, for x ∈ R,
lim FX (x + h) = FX (x).
h→0

8.2.1. C ONTINUOUS C UMULATIVE D ISTRIBUTION F UNCTION

The cumulative distribution function, or cdf, FX of a continuous r.v. X is defined by
FX (x) = PX [X ≤ x] x ∈ R.

4
8.2.2. P ROBABILITY D ENSITY F UNCTION
A cumulative distribution function FX is absolutely continuous if it can be written
∫ x
FX (x) = fX (t) dt
−∞

for some function fX , termed the probability density function, or pdf, of X. For any suitable set B,
∫
PX [X ∈ B] = fX (x) dx.
B

Directly from the definition, at values of x where FX is differentiable x,

d
fX (x) = {FX (t)}t=x .
dt
It follows that a function fX is a pdf for a continuous random variable X if and only if
∫ ∞
(i) fX (x) ≥ 0 (ii) fX (x) dx = 1
−∞

This result follows direct from definitions and properties of FX . Note that in the continuous case, there
is no requirement that fX is bounded above; it can be defined to be zero at an arbitrary set of points,
or defined piecewise on intervals of R .

Properties:

(i) If X is continuous, fX (x) 6= PX [X = x], as

PX [X = x] = lim [FX (x + h) − FX (x)] = 0

h→0

as FX is continuous.
(ii) For the cdf of a continuous r.v.,

lim FX (x) = 0 lim FX (x) = 1

x→−∞ x→∞

(iii) For a < b,

PX [a < X ≤ b] = PX [a ≤ X < b] = PX [a ≤ X ≤ b] = PX [a < X < b] = FX (b) − FX (a)

Example 2 Consider an experiment to measure the length of time that an electrical component
functions before failure. The sample space of outcomes of the experiment, Ω is + , and if Ax is the
event that the component functions for longer than x > 0 time units, suppose that
{ }
P (Ax ) = exp −x2 .

Define random variable X by X(ω) = x ⇐⇒ component fails at time x ∈ R. Then, fX (x) = 0 for
x ≤ 0, and for x > 0,
{ }
FX (x) = PX [X ≤ x] = 1 − PX (Ax ) = 1 − exp −x2

5
 PDF CDF

1.0
0.8

0.8
0.6

0.6
F(x)
f(x)

0.4

0.4
0.2

0.2
0.0

0.0

0 1 2 3 4 5 0 1 2 3 4 5
{ } { }
Figure 2: PDF fXx(x) = 2x exp −x2 , x > 0, and CDF FX (x) = 1x − exp −x2 , x>0

and FX (x) = 0 if x ≤ 0. Hence if x > 0,

d { }
fX (x) = {FX (t)}t=x = 2x exp −x2 .
dt

Graphs of the probability density function (top) and cumulative distribution function (bottom) are
shown in Figure 2. Note that both the pdf and cdf are defined for all real values of x, and that both are
continuous functions. ∫ x ∫ x
FX (x) = fX (t)dt = fX (t) dt
−∞ 0

as fX (x) = 0 for x ≤ 0, and also that

∫ ∞ ∫ ∞
fX (x) dx = fX (x) dx = 1.
−∞ 0