Discrete Probability Distributions
Random Variables
• Random Variable (RV): A numeric outcome that results
from an experiment
• For each element of an experiment’s sample space, the
random variable can take on exactly one value
• Discrete Random Variable: An RV that can take on only
a finite or countably infinite set of outcomes
• Continuous Random Variable: An RV that can take on
any value along a continuum (but may be reported
“discretely”
• Random Variables are denoted by upper case letters (Y)
• Individual outcomes for RV are denoted by lower case
letters (y)
Probability Distributions
• Probability Distribution: Table, Graph, or Formula that
describes values a random variable can take on, and its
corresponding probability (discrete RV) or density
(continuous RV)
• Discrete Probability Distribution: Assigns probabilities
(masses) to the individual outcomes
• Continuous Probability Distribution: Assigns density at
individual points, probability of ranges can be obtained by
integrating density function
• Discrete Probabilities denoted by: p(y) = P(Y=y)
• Continuous Densities denoted by: f(y)
• Cumulative Distribution Function: F(y) = P(Y≤y)
Discrete Probability Distributions
Probability (Mass) Function :
p( y ) P (Y y )
p( y ) 0 y
p( y ) 1
all y
Cumulative Distribution Function (CDF) :
F ( y ) P(Y y )
b
F (b) P (Y b) p( y)
y
F () 0 F () 1
F ( y ) is monotonically increasing in y
Example – Rolling 2 Dice (Red/Green)
Y = Sum of the up faces of the two die. Table gives value of y for all elements in S
Red\Green 1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
Rolling 2 Dice – Probability Mass Function & CDF
y p(y) F(y)
2 1/36 1/36 # of ways 2 die can sum to y
p( y )
3 2/36 3/36 # of ways 2 die can result in
4 3/36 6/36
y
5 4/36 10/36
F ( y ) p (t )
6 5/36 15/36 t 2
7 6/36 21/36
8 5/36 26/36
9 4/36 30/36
10 3/36 33/36
11 2/36 35/36
12 1/36 36/36
Rolling 2 Dice – Probability Mass Function
Dice Rolling Probability Function
0.18
0.16
0.14
0.12
0.1
p(y)
0.08
0.06
0.04
0.02
0
2 3 4 5 6 7 8 9 10 11 12
y
Rolling 2 Dice – Cumulative Distribution Function
Dice Rolling - CDF
0.9
0.8
0.7
0.6
F(y)
0.5
0.4
0.3
0.2
0.1
0
1 2 3 4 5 6 7 8 9 10 11 12 13
y
Expected Values of Discrete RV’s
• Mean (aka Expected Value) – Long-Run average
value an RV (or function of RV) will take on
• Variance – Average squared deviation between a
realization of an RV (or function of RV) and its mean
• Standard Deviation – Positive Square Root of
Variance (in same units as the data)
• Notation:
– Mean: E(Y) =
– Variance: V(Y) = 2
– Standard Deviation:
Expected Values of Discrete RV’s
Mean : E (Y ) yp( y )
all y
Mean of a function g (Y ) : E g (Y ) g ( y ) p ( y )
all y
Variance : V (Y ) E (Y E (Y )) E (Y )
2 2
2
( y ) 2 p ( y ) y 2 2 y 2 p ( y )
all y all y
y 2 p ( y ) 2 yp( y ) 2 p ( y )
all y all y all y
E Y 2 2 ( ) 2 (1) E Y 2 2
Standard Deviation : 2
Expected Values of Linear Functions of Discrete RV’s
Linear Functions : g (Y ) aY b (a, b constants)
E[aY b] (ay b) p ( y )
all y
a yp( y ) b p ( y ) a b
all y all y
V [aY b] (ay b) (a b) p( y )
2
all y
ay a
2
p( y ) a y p( y )
2 2
all y all y
a 2 ( y ) 2 p ( y ) a 2 2
all y
aY b a
Example – Rolling 2 Dice
y p(y) yp(y) y2p(y)
2 1/36 2/36 4/36 12
3 2/36 6/36 18/36 E (Y ) yp( y ) 7.0
y 2
4 3/36 12/36 48/36
12
5 4/36 20/36 100/36
E Y y 2 p( y ) 2
2 2 2
6 5/36 30/36 180/36 y 2
7 6/36 42/36 294/36 54.8333 (7.0) 2 5.8333
8 5/36 40/36 320/36
5.8333 2.4152
9 4/36 36/36 324/36
10 3/36 30/36 300/36
11 2/36 22/36 242/36
12 1/36 12/36 144/36
Sum 36/36 252/36 1974/36=
=1.00 =7.00 54.833
Binomial Experiment
• Experiment consists of a series of n identical trials
• Each trial can end in one of 2 outcomes: Success
(S) or Failure (F)
• Trials are independent (outcome of one has no
bearing on outcomes of others)
• Probability of Success, p, is constant for all trials
• Random Variable Y, is the number of Successes in
the n trials is said to follow Binomial Distribution with
parameters n and p
• Y can take on the values y=0,1,…,n
• Notation: Y~Bin(n,p)
Binomial Distribution
Consider outcomes of an experiment with 3 Trials :
SSS y 3 P( SSS ) P (Y 3) p(3) p 3
SSF , SFS , FSS y 2 P ( SSF SFS FSS ) P (Y 2) p (2) 3 p 2 (1 p )
SFF , FSF , FFS y 1 P( SFF FSF FFS ) P(Y 1) p (1) 3 p (1 p ) 2
FFF y 0 P ( FFF ) P(Y 0) p (0) (1 p )3
In General :
n n!
1) # of ways of arranging y S (and ( n y ) F ) in a sequence of n positions
s s
y y!(n y )!
2) Probability of each arrangement of y S s (and (n y ) F s ) p y (1 p) n y
n
3) P(Y y ) p ( y ) p y (1 p ) n y y 0,1,..., n
y
EXCEL Functions :
p ( y ) is obtained by function : BINOMDIST( y, n, p,0)
F ( y ) P(Y y ) is obtained by function : BINOMDIST( y, n, p,1)
Binomial Distribution (n=10,p=0.10)
0.5
0.45
0.4
0.35
0.3
p(y)
0.25
0.2
0.15
0.1
0.05
0
0 1 2 3 4 5 6 7 8 9 10
y
Binomial Distribution (n=10, p=0.50)
0.5
0.45
0.4
0.35
0.3
p(y)
0.25
0.2
0.15
0.1
0.05
0
0 1 2 3 4 5 6 7 8 9 10
y
Binomial Distribution(n=10,p=0.8)
0.35
0.3
0.25
0.2
p(y)
0.15
0.1
0.05
0
0 1 2 3 4 5 6 7 8 9 10
y
Poisson Distribution
• Distribution often used to model the number of incidences
of some characteristic in time or space:
– Arrivals of customers in a queue
– Numbers of flaws in a roll of fabric
– Number of typos per page of text.
• Distribution obtained as follows:
– Break down the “area” into many small “pieces” (n pieces)
– Each “piece” can have only 0 or 1 occurrences (p=P(1))
– Let =np ≡ Average number of occurrences over “area”
– Y ≡ # occurrences in “area” is sum of 0s & 1s over “pieces”
– Y ~ Bin(n,p) with p = /n
– Take limit of Binomial Distribution as n with p = /n
e y
p( y ) P (Y y ) 0, y 0,1,2,...
y!
Negative Binomial Distribution
• Used to model the number of trials needed until the rth
Success (extension of Geometric distribution)
• Based on there being r-1 Successes in first y-1 trials,
followed by a Success
y 1 r y r ( y 1)!
p( y ) p (1 p) p r (1 p) y r
r 1 (r 1)!( y r )!
( y )
p r (1 p) y r y r , r 1,...
(r )( y r 1)!
r r (1 p)
E (Y ) V (Y )
p p2
Γ(a) z a 1e z / a dz Note : Γ(a) (a 1) Γ(a)
0
Negative Binomial Distribution (II)
Generalization to " domain" of y* 0,1,...
k y*
( y * k ) k
p ( y*) y* 0,1,...
(k )( y * 1) k k
where :
k
k r y* y r p 1 p
k k
2
E (Y *) V (Y *)
k
This model is widely used to model count data when the Poisson model does
not fit well due to over-dispersion: V(Y) > E(Y).
In this model, k is not assumed to be integer-valued and must be estimated via
maximum likelihood (or method of moments)
