2-1

Chapter 2

Probability
 2-2

2 Probability
⚫ Using Statistics
⚫ Basic Definitions: Events, Sample Space, and

Probabilities
⚫ Basic Rules for Probability

⚫ Conditional Probability

⚫ Independence of Events

⚫ Combinatorial Concepts

⚫ The Law of Total Probability and Bayes’ Theorem

⚫ Joint Probability Table

⚫ Using the Computer

 2-3

2 LEARNING OBJECTIVES

After studying this chapter, you should be able to:

⚫ Define probability, sample space, and event.
⚫ Distinguish between subjective and objective probability.
⚫ Describe the complement of an event, the intersection, and
the union of two events.
⚫ Compute probabilities of various types of events.
⚫ Explain the concept of conditional probability and how to
compute it.
⚫ Describe permutation and combination and their use in
certain probability computations.
⚫ Explain Bayes’ theorem and its applications.
 2-4

2-1 Probability is:

⚫ A quantitative measure of uncertainty

⚫ A measure of the strength of belief in the
occurrence of an uncertain event
⚫ A measure of the degree of chance or
likelihood of occurrence of an uncertain
event
⚫ Measured by a number between 0 and 1 (or
between 0% and 100%)
 2-5

Types of Probability

⚫ Objective or Classical Probability

✓ based on equally-likely events
✓ based on long-run relative frequency of events

✓ not based on personal beliefs

✓ is the same for all observers (objective)

✓ examples: toss a coin, throw a die, pick a card

 2-6

Types of Probability (Continued)

⚫ Subjective Probability
✓ based on personal beliefs, experiences, prejudices,
intuition - personal judgment
✓ different for all observers (subjective)

✓ examples: Super Bowl, elections, new product

introduction, snowfall
 2-7

2-2 Basic Definitions

⚫ Set - a collection of elements or objects of

interest
✓ Empty set (denoted by )
⚫ a set containing no elements
✓ Universal set (denoted by S)
⚫ a set containing all possible elements
✓ Complement (Not). The complement of A is ( A )
⚫ a set containing all elements of S not in A
 2-8

Complement of a Set

Venn Diagram illustrating the Complement of an event

 2-9

Basic Definitions (Continued)

✓ Intersection (And) ( A  B)
– a set containing all elements in both A and B
✓ Union (Or) ( A  B)
– a set containing all elements in A or B or both
 2-10

Sets: A Intersecting with B

A
B

A B
 2-11

Sets: A Union B

A
B

A B
 2-12

Basic Definitions (Continued)

• Mutually exclusive or disjoint sets

–sets having no elements in common, having no
intersection, whose intersection is the empty set
• Partition
–a collection of mutually exclusive sets which
together include all possible elements, whose
union is the universal set
 2-13

Mutually Exclusive or Disjoint Sets

Sets have nothing in common

B
A
 2-14

Sets: Partition

S
A3
A1

A2 A4

A5
 2-15

Experiment
• Process that leads to one of several possible
outcomes *, e.g.:
✓ Coin toss
• Heads, Tails
✓ Throw die
• 1, 2, 3, 4, 5, 6
✓ Pick a card
◼ AH, KH, QH, ...
✓ Introduce a new product
• Each trial of an experiment has a single observed
outcome.
• The precise outcome of a random experiment is
unknown before a trial.
* Also called a basic outcome, elementary event, or simple event
 2-16

Events : Definition
⚫ Sample Space or Event Set
✓ Set of all possible outcomes (universal set) for a given
experiment
⚫ E.g.: Roll a regular six-sided die
◼ S = {1,2,3,4,5,6}

⚫ Event
✓ Collection of outcomes having a common characteristic
⚫ E.g.: Even number
◼ A = {2,4,6}
◼ Event A occurs if an outcome in the set A occurs
⚫ Probability of an event
✓ Sum of the probabilities of the outcomes of which it consists
⚫ P(A) = P(2) + P(4) + P(6)
 2-17

Equally-likely Probabilities
(Hypothetical or Ideal Experiments)
• For example:
✓ Throw a die
• Six possible outcomes {1,2,3,4,5,6}
• If each is equally-likely, the probability of each is 1/6 = 0.1667 =
16.67%
1
◼ P ( e ) =
n( S )
• Probability of each equally-likely outcome is 1 divided by the number of
possible outcomes
✓ Event A (even number)
• P(A) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 1/2
• P( A) =  P( e) for e in A
n( A ) 3 1
= = =
n( S ) 6 2
 2-18

Pick a Card: Sample Space

Hearts Diamonds Clubs Spades
Union of A
K
A
K
A
K
A
K
Event ‘Ace’
Events ‘Heart’ Q Q Q Q
n ( Ace ) 4 1
and ‘Ace’ J J J J
P ( Ace ) = = =
10 10 10 10
P ( Heart  Ace ) = 9 9 9 9
n(S ) 52 13

8 8 8 8
n ( Heart  Ace ) 7 7 7 7
= 6 6 6 6
n(S ) 5 5 5 5
4 4 4 4
16 4 3 3 3 3
=
2 2 2 2
52 13

The intersection of the

events ‘Heart’ and ‘Ace’
Event ‘Heart’
comprises the single point
n ( Heart ) 13 1
P ( Heart ) = = = circled twice: the ace of hearts
n(S ) 52 4 n ( Heart  Ace ) 1
P ( Heart  Ace ) = =
n(S ) 52
 2-19

2-3 Basic Rules for Probability

⚫ Range of Values for P(A): 0  P( A) 1

⚫ Complements - Probability of not A

P( A) = 1− P( A)

⚫ Intersection - Probability of both A and B

P( A  B) = n( A  B)
n( S )
✓ Mutuallyexclusive events (A and C) :
P( A  C) = 0
 2-20

Basic Rules for Probability

(Continued)
• Union - Probability of A or B or both (rule of unions)
P( A  B) = n( A  B) = P( A) + P( B) − P( A  B)
n( S )

✓Mutually exclusive events: If A and B are mutually exclusive, then

P( A  B) = 0 so P( A  B) = P( A) + P( B)
 2-21

Sets: P(A Union B)

A
B

P( A  B)
 2-22

2-4 Conditional Probability

• Conditional Probability - Probability of A given B

P( A  B)
P( A B) = , where P( B)  0
P( B)

✓Independent events:
P( A B) = P( A)
P( B A) = P( B)
 2-23

Conditional Probability (continued)

Rules of conditional probability:

P( A B) = P( A  B) so P( A  B) = P( A B) P( B)
P( B)
= P( B A) P( A)

If events A and D are statistically independent:

P ( A D ) = P ( A)
so P( A  D) = P( A) P( D)
P ( D A) = P ( D )
 2-24

Contingency Table - Example 2-2

Counts
AT& T IBM Total

Telecommunication 40 10 50 Probability that a project

Computers 20 30 50
is undertaken by IBM
given it is a
Total 60 40 100
telecommunications
Probabilities project:
P ( IBM  T )
AT& T IBM Total P ( IBM T ) =
P (T )
Telecommunication .40 .10 .50
0.10
= = 0.2
Computers .20 .30 .50 0.50

Total .60 .40 1.00

 2-25

2-5 Independence of Events

Conditions for the statistical independence of events A and B:

P ( A B ) = P ( A)
P ( B A) = P ( B )
and
P ( A  B ) = P ( A) P ( B )
P ( Ace  Heart ) P ( Heart  Ace )
P ( Ace Heart ) = P ( Heart Ace ) =
P ( Heart ) P ( Ace )
1 1
1 1
= 52 = = P ( Ace ) = 52 = = P ( Heart )
13 13 4 4
52 52

4 13 1
P( Ace  Heart ) = * = = P( Ace) P( Heart )
52 52 52
 2-26

Independence of Events –
Example 2-5

Events Television (T) and Billboard (B) are

assumed to be independent.

a)P(T  B) = P(T ) P( B)
= 0.04 * 0.06 = 0.0024
b)P(T  B) = P(T ) + P( B) − P(T  B)
= 0.04 + 0.06 − 0.0024 = 0.0976
 2-27

Product Rules for Independent Events

The probability of the intersection of several independent events

is the product of their separate individual probabilities:
P( A  A  A  An ) = P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
The probability of the union of several independent events
is 1 minus the product of probabilities of their complements:
P( A  A  A  An ) = 1− P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3

Example 2-7:
P(Q  Q  Q   Q ) =1− P(Q )P(Q )P(Q ) P(Q )
1 2 3 10 1 2 3 10
=1− 0.9010 =1− 0.3487 = 0.6513
 2-28

2-6 Combinatorial Concepts

Consider a pair of six-sided dice. There are six possible outcomes
from throwing the first die {1,2,3,4,5,6} and six possible outcomes
from throwing the second die {1,2,3,4,5,6}. Altogether, there are
6*6 = 36 possible outcomes from throwing the two dice.
In general, if there are n events and the event i can happen in
Ni possible ways, then the number of ways in which the
sequence of n events may occur is N1N2...Nn.
⚫ Pick 5 cards from a deck of ⚫ Pick 5 cards from a deck of
52 - with replacement 52 - without replacement
✓ 52*52*52*52*52=525 ✓ 52*51*50*49*48 =
380,204,032 different 311,875,200 different
possible outcomes possible outcomes
 2-29

More on Combinatorial Concepts

(Tree Diagram)

.
Order the letters: A, B, and C
C
. ABC

. .. B
C B .. ACB

. . . A
B A C

.. BAC

. . C
C
A
A
B
BCA

. B
A
. CAB

CBA
 2-30

Factorial

How many ways can you order the 3 letters A, B, and C?

There are 3 choices for the first letter, 2 for the second, and 1 for
the last, so there are 3*2*1 = 6 possible ways to order the three
letters A, B, and C.

How many ways are there to order the 6 letters A, B, C, D, E,

and F? (6*5*4*3*2*1 = 720)

Factorial: For any positive integer n, we define n factorial as:

n(n-1)(n-2)...(1). We denote n factorial as n!.
The number n! is the number of ways in which n objects can
be ordered. By definition 1! = 1 and 0! = 1.
 2-31

Permutations (Order is important)

What if we chose only 3 out of the 6 letters A, B, C, D, E, and F?

There are 6 ways to choose the first letter, 5 ways to choose the
second letter, and 4 ways to choose the third letter (leaving 3
letters unchosen). That makes 6*5*4=120 possible orderings or
permutations.

Permutations are the possible ordered selections of r objects out

of a total of n objects. The number of permutations of n objects
taken r at a time is denoted by nPr, where
P = n!
n r (n − r )!
For example :
6! 6! 6 * 5 * 4 * 3 * 2 *1
6 P3 = = = = 6 * 5 * 4 = 120
(6 − 3)! 3! 3 * 2 *1
 2-32

Combinations (Order is not Important)

Suppose that when we pick 3 letters out of the 6 letters A, B, C, D, E, and F

we chose BCD, or BDC, or CBD, or CDB, or DBC, or DCB. (These are the
6 (3!) permutations or orderings of the 3 letters B, C, and D.) But these are
orderings of the same combination of 3 letters. How many combinations of 6
different letters, taking 3 at a time, are there?
Combinations are the possible selections of r items from a group of n items  n
regardless of the order of selection. The number of combinations is denoted  r
and is read as n choose r. An alternative notation is nCr. We define the number
of combinations of r out of n elements as:
 n n!
 = n C r =
r r! (n − r)!
For example :
 n 6! 6! 6 * 5 * 4 * 3 * 2 *1 6 * 5 * 4 120
 = 6 C3 = = = = = = 20
 
r 3! ( 6 − 3)! 3!3! (3 * 2 * 1)(3 * 2 * 1) 3 * 2 * 1 6
 2-33

Example: Template for Calculating

Permutations & Combinations
 2-34

2-7 The Law of Total Probability and

Bayes’ Theorem
The law of total probability:
P( A) = P( A  B) + P( A  B )

In terms of conditional probabilities:

P( A) = P( A  B) + P( A  B )
= P( A B) P( B) + P( A B ) P( B )

More generally (where Bi make up a partition):

P( A) =  P( A  B )
i
=  P( AB ) P( B )
i i
 2-35

The Law of Total Probability-

Example 2-9
Event U: Stock market will go up in the next year
Event W: Economy will do well in the next year
P(U W ) =.75
P(U W ) = 30
P(W ) =.80  P(W ) = 1−.8 =.2

P(U ) = P(U W ) + P(U W )

= P(U W ) P(W ) + P(U W ) P(W )
= (.75)(.80) + (.30)(.20)
=.60+.06 =.66
 2-36

Bayes’ Theorem
• Bayes’ theorem enables you, knowing just a little
more than the probability of A given B, to find the
probability of B given A.
• Based on the definition of conditional probability
and the law of total probability.
P ( A  B)
P ( B A) =
P ( A)
P ( A  B) Applying the law of total
= probability to the denominator
P ( A  B) + P ( A  B )
P ( A B) P ( B)
= Applying the definition of
P ( A B) P ( B) + P ( A B ) P ( B ) conditional probability throughout
 2-37

Bayes’ Theorem - Example 2-10

• A medical test for a rare disease (affecting 0.1% of the

population [ P( I ) = 0.001 ]) is imperfect:
✓When administered to an ill person, the test will indicate so
with probability 0.92 [ P( Z I ) = .92  P( Z I ) = .08 ]
◼ The event (Z I ) is a false negative
✓When administered to a person who is not ill, the test will
erroneously give a positive result (false positive) with
probability 0.04 [ P(Z I ) = 0.04  P(Z I ) = 0.96 ]
◼ The event (Z I ) is a false positive. .
 2-38

Example 2-10 (continued)

P ( I ) = 0.001 P( I Z ) =
P( I  Z )
P( Z )
P( I  Z )
=
P( I  Z ) + P( I  Z )
P ( I ) = 0.999
P( Z I ) P( I )
=
P( Z I ) P( I ) + P( Z I ) P( I )
P ( Z I ) = 0.92 =
(.92)( 0.001)
(.92)( 0.001) + ( 0.04)(.999)
0.00092 0.00092
= =
0.00092 + 0.03996
P ( Z I ) = 0.04 =.0225
.04088
 2-39

Example 2-10 (Tree Diagram)

Prior Conditional Joint
Probabilities Probabilities Probabilities

P( Z I ) = 0.92 P( Z  I ) = (0.001)(0.92) =.00092

P ( Z I ) = 0.08 P( Z  I ) = (0.001)(0.08) =.00008

P( I ) = 0.001

P( I ) = 0.999 P( Z I ) = 0.04 P( Z  I ) = (0.999)(0.04) =.03996

P ( Z I ) = 0.96

P( Z  I ) = (0.999)(0.96) =.95904
 2-40

Bayes’ Theorem Extended

• Given a partition of events B1,B2 ,...,Bn:

P( A  B )
P ( B A) =
1
1

P ( A)
Applying the law of total
P( A  B ) probability to the denominator
= 1

 P( A  B ) i
Applying the definition of
P( A B ) P( B ) conditional probability throughout
= 1 1

 P( A B ) P( B )
i i
 2-41

Bayes’ Theorem Extended -

Example 2-11
⚫ An economist believes that during periods of high economic growth, the U.S.
dollar appreciates with probability 0.70; in periods of moderate economic
growth, the dollar appreciates with probability 0.40; and during periods of
low economic growth, the dollar appreciates with probability 0.20.
⚫ During any period of time, the probability of high economic growth is 0.30,
the probability of moderate economic growth is 0.50, and the probability of
low economic growth is 0.50.
⚫ Suppose the dollar has been appreciating during the present period. What is
the probability we are experiencing a period of high economic growth?
Partition: Event A − Appreciation
H - High growth P(H) = 0.30 P ( A H ) = 0.70
M - Moderate growth P(M) = 0.50 P ( A M ) = 0.40
L - Low growth P(L) = 0.20 P ( A L) = 0.20
 2-42

Example 2-11 (continued)

P ( H  A)
P ( H A) =
P ( A)
P ( H  A)
=
P ( H  A) + P ( M  A) + P ( L  A)
P( A H ) P( H )
=
P ( A H ) P ( H ) + P ( A M ) P ( M ) + P ( A L) P ( L)
( 0.70)( 0.30)
=
( 0.70)( 0.30) + ( 0.40)( 0.50) + ( 0.20)( 0.20)
0.21 0.21
= =
0.21 +0.20 + 0.04 0.45
= 0.467
 2-43

Example 2-11 (Tree Diagram)

Prior Conditional Joint
Probabilities Probabilities Probabilities
P ( A H ) = 0.70 P ( A  H ) = ( 0.30)( 0.70) = 0.21

P ( A H ) = 0.30
P ( H ) = 0.30 P ( A  H ) = ( 0.30)( 0.30) = 0.09

P ( A M ) = 0.40 P ( A  M ) = ( 0.50)( 0.40) = 0.20

P ( M ) = 0.50

P ( A M ) = 0.60 P ( A  M ) = ( 0.50)( 0.60) = 0.30

P ( A L ) = 0.20
P ( L ) = 0.20 P ( A  L ) = ( 0.20)( 0.20) = 0.04

P ( A L ) = 0.80 P ( A  L) = ( 0.20)( 0.80) = 0.16

 2-44

2-8 The Joint Probability Table

⚫ A joint probability table is similar to a contingency

table , except that it has probabilities in place of
frequencies.
⚫ The joint probability for Example 2-11 is shown

below.
⚫ The row totals and column totals are called

marginal probabilities.
 2-45

The Joint Probability Table

⚫ A joint probability table is similar to a contingency

table , except that it has probabilities in place of
frequencies.
⚫ The joint probability for Example 2-11 is shown on

the next slide.

⚫ The row totals and column totals are called

marginal probabilities.
 2-46

The Joint Probability Table:

Example 2-11

⚫ The joint probability table for Example 2-11

is summarized below.
High Medium Low Total
$
Appreciates
0.21 0.2 0.04 0.45
$Depreciates
0.09 0.3 0.16 0.55

Total 0.30 0.5 0.20 1.00

Marginal probabilities are the row totals and the column totals.
 2-47

2-8 Using Computer: Template for Calculating

the Probability of at least one success
 2-48

2-8 Using Computer: Template for Calculating

the Probabilities from a Contingency
Table-Example 2-11
 2-49

2-8 Using Computer: Template for Bayesian

Revision of Probabilities-Example 2-11
 2-50

2-8 Using Computer: Template for Bayesian

Revision of Probabilities-Example 2-11

Continuation of output from previous slide.