Probability Theory

and Statistics
Kasper K. Berthelsen, Dept. For Mathematical Sciences
kkb@math.aau.dk
Literature:
Walpole, Myers, Myers & Ye:
Probability and Statistics for Engineers and Scientists,
Prentice Hall, 8th ed.
Slides and lecture overview:
http://people.math.aau.dk/~kkb/Undervisning/ET610/
Lecture format:
2x45 min lecturing followed by exercises in group rooms
1 Lecture1
 STATISTICS
What is it good for?
90

80

70

60

50 A

40 B
C
30

20

10

0
Quality control:
1 2 3 4

• What is my rate of
Forecasting: defective products?
• How can I best manage
• Expectations for the Analysis of sales: my production?
future? • What is the best way to
• How much do we sell,
• How will the stock sample?
and when?
markets behave??
• Should we change or
sales strategy?
2 Lecture1
 Probability theory
Sample space and events
Consider an experiment
Sample space S: S
Example:
VENN DIAGRAM
S={1,2,…,6} rolling a dice
S={plat,krone} flipping a coin
Event A: S
A Example:
A={1,6} when rolling a dice

Complementary
event A´: S
A A´ Example:
A´={2,3,4,5} rolling a dice

3 lecture 1
 Probability theory
Events

Example: S
A 5
Rolling a dice B Intersection:
4 6 2 1 3 A∩B={2}
S={1,2,3,4,5,6}
A={2,4,6}
B={1,2,3} Union:
A∪B={1,2,3,4,6}
S
Disjoint events: C∩D = Ø
C D
C={1,3,5} and D={2,4,6} are disjoint

4 lecture 1
 Probability theory
Counting sample points

Ways of placing your bets: Guess the results of 13 matches

Possible outcomes:
1X2
3 possibilities
Home win
3 possibilities Draw
• Away win
•
•

13
Answer: 3·3·3· ··· ·3 = 3
The multiplication rule
3 possibilities

5 lecture 1
 Probability theory
Counting sample points

Ordering n different objects

Number of permutations ???

There are ”n factorial”

• n ways of selecting the first object
• n -1 ways of selecting second object
•
•
n · (n -1) · ··· · 1 = n ! ways
•

• 1 way of selecting the last object The multiplication rule

3!= 6
6 lecture 1
 Probability theory
Counting sample points
Multiplication rule:
If k independent operations can be performed in
n1, n2, … , nk ways, respectively, then the k operations can be
performed in
n1 · n2 · ··· · nk ways
Tree diagram: T
T H Flipping a coin three times
T H
T (Head/Tail)
H
23 = 8 possible outcomes
T
H T H
T
H
H
7 lecture 1
 Probability theory
Counting sample points

Number of possible ways of selecting r objects from a

set of n destinct elements:

Without With
replacement replacement

n!
n Pr =
Ordered r
(n − r )! n
n n!
Unordered   = -
 r  r !(n − r )!

8 lecture 1
 Probability theory
Counting sample points

Example:
Ann, Barry, Chris, and Dan should from a committee
consisting of two persons, i.e. unordered without replacement.

Number of possible combinations:

4 4!
=
  = 6
 2  2!2!
Writing it out : AB AC AD BC BD CD

9 lecture 1
 Probability theory
Counting sample points

Example:
Select 2 out of 4 different balls ordered and without
replacement

4!
Number of possible combinations:=
4 P2 = 12
Notice: Order matters!
(4 − 2)!

10 lecture 1
 Probability theory
Probability

Let A be an event, then we denote S

A
P(A) the probability for A
It always hold that 0 < P(A) < 1 P(Ø) = 0 P(S) = 1

Consider an experiment which has N equally Example:

Rolling a dice
likely outcomes, and let exactly n of these
P(even number)
events correspond to the event A. Then
3 1
n = =
P( A) = =
# successful outcomes
6 2
N # possible outcomes

11 lecture 1
 Probability theory
Probability
Example: Quality control
A batch of 20 units contains 8 defective units.
Select 6 units (unordered and without replacement).

Event A: no defective units in our random sample.

 20 
Number of possible samples: N =   (# possible)
 6
 12 
Number of samples without defective units: n =  
6
 12  (# successful)
 
= =
P(A)  6  12!6!14!
= = 0.024
77
 20  6!6!20! 3230
12   lecture 1
 6 
 Probability theory
Probability

Example: continued
Event B: exactly 2 defective units in our sample
Number of samples with exactly 2 defective units:
 12   8 
=n  ⋅ 
 12   8   4   2
 ⋅ 
=
P(B) =
4   2 12!8!6!14!
= 0.3576
(# successful)
 20  4!8!2!6!20!
 
 6 
13 lecture 1
 Probability theory
Rules for probabilities

A B

Intersection: Union:
A∩B A∪B
P(A ∪B) = P(A) + P(B) - P(A ∩ B)
P(B) = P(B ∩ A) + P(B ∩ A´ )

If A and B are disjoint: P(A ∪ B) = P(A) + P(B)

In particular: P( A ) + P(A´ ) = 1
14 lecture 1
 Probability theory
Conditional probability

Conditional probability for A given B:

P(A ∩ B) A B
P(A|B) = where P ( B ) > 0
P(B)

Bayes’ Rule: P(A∩B) = P(A|B)P(B) = P(B|A)P(A)

Rewriting Bayes’ rule:

P(B|A)P(A) P(B|A)P(A)
=
P(A|B) =
P(B) P(B|A)P(A)+P(B|A´)P(A´)

15 lecture 1
 Probability theory
Conditional probability

Example page 59: Employed Unemployed Total

The distribution of Man 460 40 500
employed/unemployed Woman 140 260 400
amongst men and women in
a small town. Total 600 300 900

P(man & employd) 460 / 900 460 23

P(man | employed) = = = = = 76.7%
P(employd) 600 / 900 600 30
P(man & unemployed) 40 / 900 40 2
P(man |unemployed) = = = = = 13.3%
P(unemployed) 300 / 900 300 15

16 lecture 1
 Probability theory
Bayes’ rule

Example: Lung disease & Smoking

According to ”The American Lung Association” 7% of the population
suffers from a lung disease, and 90% of these are smokers. Amongst
people without any lung disease 25.3% are smokers.
Events: Probabilities:
A: person has lung disease P(A) = 0.07
B: person is a smoker P(B|A) = 0.90
P(B|A´ ) = 0.253
What is the probability that at smoker suffers from a lung disease?
P( B | A) P( A) 0.9 ⋅ 0.07
P( A | B) = = = 0.211
P( B | A) P( A) + P( B | A´)P( A´) 0.9 ⋅ 0.07 + 0.253 ⋅ 0.93
17 lecture 1
 Probability theory
Bayes’ rule – extended version
A4 S
A1 , … , Ak are a A3 A5
partitioning of S A1
B A6

Law of total probability: A2

k
P( B) = ∑ P( B | Ai ) P( Ai )
i =1
Bayes’ formel udvidet:
P( B | Ar ) P( Ar )
P( Ar | B) = k

∑ P( B | A ) P( A )
i =1
i i

18 lecture 1
 Probability theory
Independence
Definition:
Two events A and B are said to be independent if and only if

P(B|A) = P(B) or P(A|B) = P(A)

Alternative Definition:
Two events A and B are said to be independent if and only if

P(A∩B) = P(A)P(B)

Notice: Disjoint event (mutually exclusive event) are

dependent!
19 lecture 1
 Probability theory
Conditional probability

Example: Employed Unemployed Total

Man 460 40 500
Woman 140 260 400
Total 600 300 900

460 / 900
P (man |employed) = = 76.7%
600 / 900
P (man) = 500 / 900 = 55.6%
Conclusion: the two events “man” and “employed” are
dependent.
20 lecture 1
 Probability theory
Rules for conditional probabilities
Probability of events A and B happening simultaneously
P( A ∩ B) = P( A | B) P( B)
Probability of events A, B and C happening simultaneously
P( A ∩ B ∩ C ) = P( A | B ∩ C ) P( B | C ) P(C )
Proof:
P( A ∩ B ∩ C ) = P( A | B ∩ C ) P( B ∩ C ) = P ( A | B ∩ C ) P ( B | C ) P (C )
General rule:
P( A1 ∩ A2 ∩  ∩ Ak ) = P( A1 | A2 ∩  ∩ Ak ) ⋅
P( A2 | A3 ∩  ∩ Ak ) ⋅
 P( Ak −1 | Ak ) ⋅ P( Ak )
21 lecture 1