DAO1704/DSC1007 Lecture 3

Basic Probability Theory

 Agenda

• Conditional Probability
• Bayes’Theorem
• Independence
• Discrete probability distribution
• Summary measures of probability distribution
 Lawyer vs Librarian

• David is an introvert
 Conditional Probability

• The probability of an event given that another

event has occurred is called a Conditional
probability.
 Misunderstanding 1 – Ignoring Condition

• “The probability of surviving for 5 years with

colorectal cancer is …”
 Notation of Conditional Probability

• A conditional probability is often expressed using the phrase

"given" to describe the condition.

• Examples
• A: Probability that David is a lawyer given that he is an introvert
• B: Probability that David is a librarian given that he is an introvert
 Notation

• Let A be an event with probability P(A).

Prior Probability

• Let B be an event that has occurred that

may change the probability of A.
• The conditional probability of A given B is
denoted by P(A|B).
Posterior Probability
Misunderstanding 2 – Confusing with
Reverse

Probability that David is a lawyer given that he is an

introvert

≠
Probability that David is an introvert given that he is a
lawyer
 Conditional Probability

Computing Conditional Probability

𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
𝑃(𝐵)
 Conditional Probability
Venn diagram

B
 Conditional Probability

Note that
𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
implies:
𝑃(𝐵)
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝐵 𝑃(𝐵)
𝑃 𝐴∩𝐵 =𝑃 𝐵∩𝐴

Common Mistake:

✗
 Conditional Probability
Example
Experiment : pick one card from a deck

C : Obtain a Spade D : Obtain a Queen

 Conditional Probability
P(C) = 13/52=1/4
Q: P(C|D) = 1/4
P(C  D) = 1/52
P(D) = 4/52=1/13 Q: P(D|C) = 1/13
C: Obtain a Spade D: Obtain a Queen
 Example
• A box contains seven red and thirteen blue balls. Two balls were selected at random and were
discarded without their colors being seen. What is the probability that both balls discarded were blue?
 Example
• A box contains seven red and thirteen blue balls. Two balls were selected at random and were
discarded without their colors being seen. Suppose the third ball drawn is a red ball, what is the
probability that both balls discarded were blue?
 Bayes’ Theorem

Often, we begin analysis with initial or prior probabilities for

certain events.

Then, we obtain additional info (from sample, product test, etc.)

Given this info, we calculate revised or posterior probabilities.

Bayes’ theorem

Prior New Bayes’ Posterior

Probabilities Information Theorem Probabilities
 Bayes’ Theorem

Consider n mutually exclusive and collectively exhaustive events

A1, A2 , . . , An with given prior probabilities P(A1), P(A2) , . . , P(An)

Given that an event B has occurred, we can use Bayes’ theorem

to find the posterior probabilities of Ai .

𝑃 𝑩 𝑨𝒊 𝑃(𝑨𝒊 )
𝑃 𝑨𝒊 𝑩 =
𝑃 𝑩 𝑨𝟏 𝑃 𝑨𝟏 + 𝑃 𝑩 𝑨𝟐 𝑃 𝑨𝟐 + . . . +𝑃 𝑩 𝑨𝒏 𝑃 𝑨𝒏
 Bayes’ Theorem

𝑃(𝑨𝒊 ∩ 𝑩) 𝑃 𝑩 𝑨𝒊 𝑃(𝑨𝒊 )
𝑃 𝑨𝒊 𝑩 =
𝑃(𝑩)

𝑃 𝑩 𝑨𝒊 𝑃(𝑨𝒊 )
𝑃 𝑨𝒊 𝑩 =
𝑃 𝑩 𝑨𝟏 𝑃 𝑨𝟏 + 𝑃 𝑩 𝑨𝟐 𝑃 𝑨𝟐 + . . . +𝑃 𝑩 𝑨𝒏 𝑃 𝑨𝒏
 Bayes’ Theorem

B Ç A1 B ÇBA2 B Ç A3
A1 A2 A3
Bayes’ Theorem on Diagnostic Test
Sensitivity = the probability that a person tests positive if the disease is present
= P(test positive | has the disease)

Positive Predictive Value

= the probability that a person has disease if tested positive
= P(has disease | test positive)

Specificity = the probability that a person tests negative if the disease is absent
= P(test negative| disease absent)
 Bayes’ Theorem

1% of women at age 40 have breast cancer.

80% of women with breast cancer will get positive mammographies.

9.6% of women without breast cancer will also get positive

mammographies.

Q : A woman in this age group had a positive mammography in a

routine screening. What is the probability that she actually has
breast cancer?
 Bayes’ Theorem
BC = woman (at 40) has breast cancer P(BC) = 0.01
NBC = woman (at 40) does not have breast cancer P(NBC) = 0.99

PM = woman has positive mammography

P(PM|BC) = 0.80 P(PM|NBC) = 0.096

 Probability Table
• Let A1, A2 mutually exclusive and collectively exhaustive
events. Likewise for B1, B2.

Joint Marginal
probabilities Probabilities
 Probability Table
• Let A1, A2 mutually exclusive and collectively exhaustive
events. Likewise for B1, B2.

+ =
+

+
= + =

=
+ =
 Bayes’ Theorem
BC = woman (at 40) has breast cancer P(BC ) = 0.01
NBC = woman (at 40) does not have breast cancer

BC NBC
PM

NPM

1
 Bayes’ Theorem
PM = woman has positive mammography P(PM|BC) = 0.80

P(PM|NBC) = 0.096

BC NBC
PM

NPM

1
 Bayes’ Theorem
Now that we have completed the probability
table below, we can answer the question: P(BC|PM) =

BC NBC
PM

NPM

1
 Lawyer vs Librarian

• David is an introvert

P(Lawyer | Int) P(Lib | Int)

𝑃 𝐼𝑛𝑡 𝐿𝑎𝑤𝑦𝑒𝑟 𝑃 𝐿𝑎𝑤𝑦𝑒𝑟 𝑃 𝐼𝑛𝑡 𝐿𝑖𝑏 𝑃 𝐿𝑖𝑏

= =
𝑃 𝐼𝑛𝑡 𝑃 𝐼𝑛𝑡

𝑃 𝐼𝑛𝑡 𝐿𝑎𝑤𝑦𝑒𝑟 < 𝑃 𝐼𝑛𝑡 𝐿𝑖𝑏

𝑃 𝐿𝑎𝑤𝑦𝑒𝑟 ≫ 𝑃 𝐿𝑖𝑏
Independent Events
 Independent Events

Two events A and B are independent if knowing whether one event

has occurred will not change our assessment of the other event.

Events A and B are independent if

P(A|B) = P(A) or P(B|A) = P(B)

Recall : P(AB) = P(A|B) P(B)

Thus, for independent events,

P(AB) = P(A) P(B)
 Independent Events
Experiment : pick one card from a deck

C : Obtain a Spade P(C) =1/4, P(D) = 1/13,

D : Obtain a Queen P(C  D) =1/52

P(C  D) = P(C) x P(D)

Q : are they independent?
Information of the card being a Spade does change the
probability that it would be a Queen.

P(D | C) = P(D)=1/13
 Independent Events
Q : Can mutually exclusive events be independent?
 Example
• A box contains seven red and thirteen blue balls. A ball is drawn and the color is noted before it is put
back again. The process is repeated. What is the probability that both colors noted were blue?
Discrete Probability Distribution
 A Primary School Math Question

• Some motorcycles and cars are parked at a car

park. Joe makes a count to find a total of 20
vehicles and 52 wheels. How many cars are there?

A variable is an
alphabetic character
representing a
number which is
unknown.
 Random Variables

A random variable is a rule that assigns a numerical value

to each possible outcome of a probabilistic experiment.

A random variable (r.v.) may be discrete or continuous.

Discrete : can only assume values that are distinct and separate
Example 1 : 0, 1, 2, 3, 4, 5, . . .
Example 2 : 2.0, 2.5, 3.0, 3.5

Continuous : can take on any value within some interval of

numbers Examples : [ 0 , 100 ], [ 2 , 4 ]
 Random Variables

A random variable is a rule that assigns a numerical value

to each possible outcome of a probabilistic experiment.

A random variable (r.v.) may be discrete or continuous.

Examples
1.Total number of points in a throw of 2 dice
2.Number of cups of cappuccino Edward will sell today
3.Number of B’s you will score in this semester
4.Time between this and the next slide
5.Your present weight in kg
 Random Variables
Example :
Let’s say that 30 % of BBA students are international
students. Suppose that each of the 3 BIZ1007 tutors randomly
selects one of his students to participate in a survey.

What are the possible outcomes of this experiment?

Outcome X
L L L 0 L : Local
L L I 1 I : International
L I L 1
I L L 1
L I I 2
I L I 2
I I L 2
I I I 3
 Random Variables
Example :
Let’s say that 30 % of BBA students are international
students. Suppose that each of the 3 BIZ1007 tutors randomly
selects one of his students to participate in a survey.

Let X = number of international students chosen

Outcome X
L L L 0 L : Local
L L I 1 I : International
L I L 1
I L L 1
L I I 2
I L I 2
I I L 2
I I I 3
Probability Distribution
 Application of Variable

• Some motorcycles and cars are parked at a car

park. Joe makes a count to find a total of 20
vehicles. Let x be the number of cars. If the parking
fee is $4.00 for a car and $1.00 for a motorcycle,
what is the total parking fee in terms of x?
A function is a
relation that
associates an input
to a single output
according to some
rule.
 Probability Distribution

Recall
A random variable is a rule that assigns a numerical value
to each possible outcome of a probabilistic experiment.

A probability distribution for a random variable

describes how probabilities are distributed over the
values of the random variable
 Discrete Probability Distribution
A probability distribution for a discrete random variable X
consists of
(i) possible values x1, x2, ..., xn
(ii) corresponding probabilities p1, p2, . . . , pn
with the interpretation that

P(X = x1) = p1, P(X = x2) = p2, . . . , P(X = xn) = pn

Note:
Probabilities must sum to 1 : p1 + p2 + . . . + pn = 1.0 ( pi  0 )
 Discrete Probability Distribution
Example : Return to this example.
Let’s say that 30 % of BBA students are international
students. Suppose that each of the 3 BIZ1007 tutors randomly
selects one of his students to participate in a survey.

Let X = number of international students chosen

Outcome X
We
L know
: LocalX is a r.v.
L L L 0
L L I 1 I : International
L I L 1 Q: What is its probability
distribution?
I L L 1
L I I 2
I L I 2
I I L 2
I I I 3
 Discrete Probability Distribution
Example :
Let’s say that 30 % of BBA students are international
students. Suppose that each of the 3 BIZ1007 tutors randomly
selects one of his students to participate in a survey.

Let X = number of international students chosen

Outcome X
Probability distribution
L L L 0
L L I 1 X P(X)
L I L 1
0 P(X=0)
I L L 1
1 P(X=1)
L I I 2
I L I 2 2 P(X=2)
I I L 2 3 P(X=3)
I I I 3
 Discrete Probability Distribution
Example :
Let’s say that 30 % of BBA students are international students.
Suppose that each of the 3 BIZ1007 tutors randomly selects one
of his students to participate in a survey.

Let X = number of international students chosen

Outcome X Probability of Outcome
L L L 0
L L I 1
L I L 1
I L L 1
L I I 2
I L I 2
I I L 2
I I I 3
 Discrete Probability Distribution
Example :
Let’s say that 30 % of BBA students are international students.
Suppose that each of the 3 BIZ1007 tutors randomly selects one
of his students to participate in a survey.

Let X = number of international students chosen

X P( X)
–0

3
 Discrete Probability Distribution
Example :
Let’s say that 30 % of BBA students are international students.
Suppose that each of the 3 BIZ1007 tutors randomly selects one
of his students to participate in a survey.

Let X = number of international students chosen

X P(X)
0 0.343 Probability distribution of X
1 0.441
2 0.189
3 0.027
 Discrete Probability Distribution
AAhistogram
histogram is
is aa display
displayof
ofprobabilities
probabilitiesas
asaabar
barchart
chart

Example 1:
X = number of international students chosen

0.50

0.40

0.30

0.20

0.10

0.00
0 1 2 3
 Discrete Probability Distribution
Example 2:
Let X be the random variable that denotes the number of orders
for Boeing 767 aircraft for next year.
Suppose that the number of orders for Boeing 767 aircraft for
next year is estimated to obey the following distribution:
Orders for Boeing 767
Aircraft next year Probability
xi pi
42 0.05
43 0.10
44 0.15
45 0.20
46 0.25
47 0.15
48 0.10
 Discrete Probability Distribution

A histogram is a display of probabilities as a bar chart

Probability Distribution of the

Number of Orders of Boeing 767 Aircraft Next Year
0.35

0.30

0.25
Probability

0.20

0.15

0.10

0.05

0.00
43 44 45 46 47 48
Number of Orders

Page 9
Summary Measures of Probability
Distributions
 Discrete Probability Distribution
X and Y denote the sales next year in the eastern division and
the western division of a company, respectively.

Eastern Division Probability Distribution Function of Eastern

Division Sales
Sales
($ million) Probability 0.40

3.0 0.05 0.35

0.30
4.0 0.20

Probability
0.25
5.0 0.35 0.20
6.0 0.30 0.15

7.0 0.10 0.10

0.05
8.0 0.00
0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)

Western Division Probability Distribution Function of Western

Division Sales
Sales
($ million) Probability 0.40
0.35
3.0 0.15
0.30
4.0 0.20

Probability
0.25
5.0 0.25 0.20
6.0 0.15 0.15
0.10
7.0 0.15
0.05
8.0 0.10 0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)
Summary Measures of Probability Distributions

Summary statistics are used to summarize a set of

observations, in order to communicate the largest amount as
simply as possible.

Statisticians commonly try to describe the observations in

– a measure of location, or central tendency, such as the mean,
median, mode etc
– a measure of statistical dispersion like the standard
deviation, variance, range etc
– a measure of the shape of the distribution like skewness or
kurtosis
 Discrete Probability Distribution
X and Y denote the sales next year in the eastern division and
the western division of a company, respectively.

Eastern Division Probability Distribution Function of Eastern

Division Sales
Sales
($ million) Probability 0.40

3.0 0.05 0.35

0.30
4.0 0.20

Probability
0.25
5.0 0.35 0.20
6.0 0.30 0.15

7.0 0.10 0.10

0.05
8.0 0.00
0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)

Western Division Probability Distribution Function of Western

Division Sales
Sales
($ million) Probability 0.40
0.35
3.0 0.15
0.30
4.0 0.20

Probability
0.25
5.0 0.25 0.20
6.0 0.15 0.15
0.10
7.0 0.15
0.05
8.0 0.10 0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)
Summary Measures of Probability Distributions

Suppose the discrete r.v. X has probability distribution

x1, x2, . . . , xn
p1, p2, . . . , pn

The mean (or expected value) of X is

𝑛

𝐸 𝑿 = 𝝁𝒙 = 𝑝𝑖 𝑥𝑖
𝑖=1
Summary Measures of Probability Distributions

Suppose the discrete r.v. X has probability distribution

x1, x2, . . . , xn
p1, p2, . . . , pn
𝑛

The variance of X is 𝑉𝑎𝑟 𝑿 = 𝝈𝟐𝒙 = 𝑝𝑖 𝑥𝑖 − 𝜇𝑥 2

𝑖=1
𝑛

= 𝑝𝑖 𝑥𝑖2 − 𝜇𝑥2
𝑖=1

Standard Deviation : 𝝈𝒙 = 𝝈2𝒙

Summary Measures of Probability Distributions
Example :
Let X be the number that comes up on a roll of a die.
Compute the mean, variance and standard deviation of X.
Outcome P(X = x)
1 1/6

2 1/6

3 1/6
4 1/6
5 1/6
6 1/6
Summary Measures of Probability Distributions
Example :
Let’s say that 30 % of BBA students are international students.
Suppose that each of the 3 BIZ1007 tutors randomly selects one
of his students to participate in a survey.

Let X = number of international students chosen

𝑛
X P(X)
𝐸 𝑋 = 𝜇𝑥 = 𝑝𝑖 𝑥𝑖
0 0.343
𝑖=1
1 0.441 = 0.343 × 0 + 0.441 × 1 + 0.189 × 2 + 0.027 × 3
2 0.189 = 𝟎. 𝟗𝟎𝟎
3 0.027
Summary Measures of Probability Distributions

Suppose the discrete r.v. X has probability distribution

x1, x2, . . . , xn
p1, p2, . . . , pn
𝑛

The variance of X is 𝑉𝑎𝑟 𝑿 = 𝝈𝟐𝒙 = 𝑝𝑖 𝑥𝑖 − 𝜇𝑥 2

𝑖=1
𝑛

= 𝑝𝑖 𝑥𝑖2 − 𝜇𝑥2
𝑖=1

Standard Deviation : 𝝈𝒙 = 𝝈2𝒙

Summary Measures of Probability Distributions
Example :
Let X be the number that comes up on a roll of a die.
Compute the mean, variance and standard deviation of X.
Outcome P(X = x)
1 1/6

2 1/6

3 1/6
4 1/6
5 1/6
6 1/6
Summary Measures of Probability Distributions
Example :
Let’s say that 30 % of BBA students are international students.
Suppose that each of the 3 BIZ1007 tutors randomly selects one
of his students to participate in a survey.

Let X = number of international students chosen

Compute the mean, variance and standard deviation of X.

X P(X) 𝐸 𝑋 = 𝜇𝑥 = 𝟎. 𝟗𝟎𝟎
0 0.343 𝑛

1 0.441 𝑉𝑎𝑟 𝑋 = 𝜎𝑥2 = 𝑝𝑖 𝑥𝑖 − 𝜇𝑥 2

𝑖=1
2 0.189 = 0.343 0 − 𝟎. 𝟗 2
+ 0.441 1 − 𝟎. 𝟗 2
+
3 0.027 2 2
0.189 2 − 𝟎. 𝟗 + 0.027 3 − 𝟎. 𝟗
= 𝟎. 𝟔𝟑
Summary Measures of Probability Distributions
X and Y denote the sales next year in the eastern division and
the western division of a company, respectively.

Eastern Division Probability Distribution Function of Eastern

Division Sales
Sales
($ million) Probability 0.40

3.0 0.05 0.35

0.30
4.0 0.20

Probability
0.25
5.0 0.35 0.20
6.0 0.30 0.15

7.0 0.10 0.10

0.05

Compute the mean, variance and standard deviation of

8.0 0.00
0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)

X (eastern division sales) and Y (western division sales)

Western Division Probability Distribution Function of Western
Division Sales
Sales
($ million) Probability 0.40
0.35
3.0 0.15
0.30
4.0 0.20

Probability
0.25
5.0 0.25 0.20
6.0 0.15 0.15
0.10
7.0 0.15
0.05
8.0 0.10 0.00
3.0 4.0 5.0 6.0 7.0 8.0
Sales ($ million)
Summary Measures of Probability Distributions

Compute the mean, variance and standard deviation of

X (eastern division sales) and Y (western division sales)

Use Excel
X (Eastern Division) Y (Western Division)
Sales ($m) Probability Sales ($m) Probability
3.0 0.05 3.0 0.15
4.0 0.20 4.0 0.20
5.0 0.35 5.0 0.25
6.0 0.30 6.0 0.15
7.0 0.10 7.0 0.15
8.0 0.00 8.0 0.10

Mean 5.20 5.25

Variance 1.06 2.3875
Std Dev 1.0296 1.5452