2/14/2025
Basic Probability Concepts
Agenda
• Principles of Counting
• Basic Probability Concepts
1
2/14/2025
Principles of Counting
Why Counting?
• To perform probability and statistics calculations, it is important to know how to count
the size of a population or the size of a sample drawn from the population correctly.
• Knowing how to count in various situations is very crucial.
2
2/14/2025
Examples
Suppose your video streaming service has these types of movies:
• You want to watch three movies tonight, one of each type. How Type Movies
many different combinations of movies you can watch? action 674
romance 913
comedy 84
• In this case, order doesn’t matter: Your choice of movie of any one
type does not depend on your choice of any other type.
Examples
There are 26 letters in the alphabet. How many 1-letter sequences can you make?
• How many 2-letter sequences are there?
• How many 5-letter sequences are there?
• In this case, order does matter: AB is different from BA
k Number of sequences (N)
1 letter 261 = 26
2 letters 26 x 26 = 262 = 676
3 letters 26 x 26 x 26 = 263 = 17,576 Nk = 26k
4 letters 26 x 26 x 26 x 26 = 264 = 456,976
5 letters 26 x 26 x 26 x 26 x 26 = 265 = 11,881,376
6 letters 26 x 26 x 26 x 26 x 26 x 26 = 266 = 308,915,776
3
2/14/2025
Examples
Now, for the same 26 letters in the alphabet. What if there can be no repeated letters in
the sequences?
k Number of sequences (N)
1 letter 26
2 letters 26 x 25 = 650
3 letters 26 x 25 x 24 = 15,600
4 letters 26 x 25 x 24 x 23 = 358,800
5 letters 26 x 25 x 24 x 23 x 22 = 7,893,600
6 letters 26 x 25 x 24 x 23 x 22 x 21 = 165,765,600
• The # of ways for a sequence of 𝑘 objects chosen without repetition from a collection of
𝑛 objects:
𝑛∙ 𝑛−1 ∙ 𝑛−2 ∙ 𝑛−3 ∙ … ∙ 𝑛−𝑘+1
Factorial Notation
• The product 𝑛 ∙ 𝑛 − 1 ∙ 𝑛 − 2 ∙ 𝑛 − 3 ∙ … ∙ 3 ∙ 2 ∙ 1 is n factorial, denoted as 𝑛!
n n!
• Note that 𝑛! grows very rapidly: 0 1 by definition
1
• Factorial notation can simplify formulas. 2
1 =1
2 =2∙1
3 6 =3∙2∙1
4 24 = 4 ∙ 3 ∙ 2 ∙ 1
5 120 = 5 ∙ 4 ∙ 3 ∙ 2 ∙ 1
6 720
7 5,040
8 40,320
9 362,880
10 3,628,800
11 39,916,800
12 479,001,600
13 6,227,020,800
14 87,178,291,200
8
4
2/14/2025
Sequence without Repetition Revisited
• Recall that the # of ways for a sequence of 𝑘 objects chosen without repetition from a
collection of 𝑛 objects is
𝑛∙ 𝑛−1 ∙ 𝑛−2 ∙ 𝑛−3 ∙ … ∙ 𝑛−𝑘+1
• Using factorial notation, we can simplify this as:
𝑛!
𝑛 𝑃𝑘 = Permutation
𝑛−𝑘 !
• What about the # of permutations of 𝑛 objects in a circle?
Example: Counting Numbers
• How many 3-digit numbers are there using the digits 1 through 9 that have no repeated
digits?
• How many of these 3-digit numbers from above are odd? Here are some hints:
– How many have 1, 3, 5, 7, or 9 as their third digit?
– How many choices are there for the third digit?
– It depends on what we chose for the first two digits!
10
10
5
2/14/2025
Example: Counting Numbers
• Start by counting digits from left to right.
• There are 4 even digits: 2, 4, 6, and 8. There are 5 odd digits: 1, 3, 5, 7, and 9.
Possible digit Number in
sequences each sequence
even even odd 4 x 3 x 5 = 60
odd even odd 5 x 4 x 4 = 80
even odd odd 4 x 5 x 4 = 80
odd odd odd 5 x 4 x 3 = 60
TOTAL 60 + 80 + 80 + 60 = 280
• Is there a better way to compute this?
11
11
Example: Forming Baseball Teams
Suppose you need to choose a baseball team of 9 children out of 15 children. How many
possible teams are there?
Questions to ask:
• Does order matter?
• Is there repetition?
12
12
6
2/14/2025
Counting When Order Doesn’t Matter
How many different committees of 4 students can you make from a group of 15 students?
• In this case, order doesn’t matter.
𝑛! 15! 15!
• We can start with the permutation where order does matter: 𝑛−𝑘 !
= =
15 − 4 ! 11!
𝑛! 4!
• Then, we determine the # of possible committees if order matters: 𝑛−𝑘 !
=
4−4 !
= 4!
15! 1 15!
• Thus, the # of possible committees of 4 students from 15 students: ∙ =
11! 4! 4! ∙ 11!
• In general, the # of ways to make a combination of 𝑘 objects without repetition among 𝑛
objects, and order doesn’t matter is: 𝑛!
𝑛 𝐶𝑘 = Combination
𝑘! 𝑛 − 𝑘 !
13
13
Binomial Coefficients
• The combination formula can also be written as:
𝑛 𝑛!
𝑛 𝐶𝑘 =
𝑘
=
𝑘! 𝑛 − 𝑘 !
𝑛
• is also known as a binomial coefficient.
𝑘
𝑛
• One important property of 𝑘 :
𝑛 𝑛! 𝑛! 𝑛
= = =
𝑘 𝑘! 𝑛 − 𝑘 ! 𝑛 − 𝑘 ! 𝑘! 𝑛−𝑘
e.g. Counting the number of ways to make a committee of 4 students out of 15 students
is the same as counting the number of ways to leave 11 students off the committee.
14
14
7
2/14/2025
More Counting Situations
• For the case when you have 𝑛 objects and would like to 𝑟 different partitions with 𝑛1
elements in the first partition, 𝑛2 in the second, and so forth.
• The number of ways to partition is:
𝑛 𝑛!
𝑛1 , 𝑛2 , ⋯ , 𝑛𝑟 = 𝑛1 ! 𝑛2 ! ⋯ 𝑛𝑟 ! 𝑛 = 𝑛1 + 𝑛2 + ⋯ + 𝑛𝑟
Example: In how many ways can 7 graduate students be assigned to 1 triple and 2 double
hotel rooms during a conference?
7 7!
= = 210
3, 2, 2 3! 2! 2!
15
15
Counting Scenarios So Far
Repetitions allowed No Repetitions
(with replacement) (without replacement)
Sequences 𝑛!
𝑛𝑘 𝑛 𝑃𝑘 =
(order matters) 𝑛−𝑘 !
Combinations 𝑛 𝑛!
? 𝑛 𝐶𝑘 = =
(order doesn’t matter) 𝑘 𝑘! 𝑛 − 𝑘 !
What about combinations or collections
with repetitions or replacements?
• Permutations: Arrangements of items in a sequence where order does matter.
• Combinations: Arrangements of items in a collection where order does not matter.
16
16
8
2/14/2025
Combinations with Replacements or Repititions
Problem: Determine the # of combinations for choosing 𝑟 items from 𝑛 distinct items.
• Imagine you have:
– 𝑟 stars ( ), representing the items you choose.
– 𝑛 − 1 separators (|), representing the separations between different types of items.
• For simplicity, let’s consider 3 types of fruits (A, B, C) and choose 2 fruits. Here are the
different combinations with repetitions:
AA Choose A and A (AA) - ||
AB Choose A and B (AB) - | | Similar to the problem of: 𝑛+𝑘−1
AC Choose A and C (AC) - || • Total items = 𝑛 − 1 + 𝑘 𝑘
BB Choose B and B (BB) - | | • Select 𝑘 from them
BC Choose B and C (BC) - | |
CC Choose C and C (CC) - || 𝑛+𝑘−1 𝑛+𝑘−1 𝑛+𝑘−1 !
= =
𝑘 𝑛−1 𝑘! 𝑛 − 1 !
17
17
Summary of Different Counting Scenarios
Repetitions allowed No Repetitions
(with replacement) (without replacement)
Sequences 𝑛!
𝑛𝑘 𝑛 𝑃𝑘 =
(order matters) 𝑛−𝑘 !
Combinations 𝑛+𝑘−1 𝑛+𝑘−1 ! 𝑛 𝑛!
= 𝑛 𝐶𝑘 =
𝑘
=
(order doesn’t matter) 𝑘 𝑘! 𝑛 − 1 ! 𝑘! 𝑛 − 𝑘 !
18
18
9
2/14/2025
Basic Probability Concepts
19
What is Probability?
• Probability is a branch of mathematics that deals with the likelihood or chance of
different outcomes.
• It measures how likely it is for an event to occur, using a scale from 0 to 1 (0 means the
event cannot happen and 1 means it will certainly happen).
• It creates mathematical models to study chance or randomness.
• Originally arose from studying games of chance.
– The probability that a flipped fair coin will land heads is 1/2.
– The probability that a card drawn from a shuffled deck of 52 cards is an ace is 4/52.
20
20
10
2/14/2025
Role of Probability
• What’s the relation and distinction of probability and statistics?
• Study of randomness and uncertainty
• Predict likelihood of future events
• Science of collecting, analyzing, interpreting,
and presenting data
• Making inferences about a population based
on sample data
• Probability allow us to draw conclusions about characteristics of (hypothetical) data
taken from the population, based on known features of the population.
• Statistics allows us to draw conclusions about the population, with inferential statistics
making clear use of elements of probability.
21
21
Some Probability Definitions
• Experiment: A process or activity that generates measurable results (a set of data).
– E.g. Flipping a coin or rolling a dice.
• Outcome: Each distinct result of an experiment. Aka sample point.
– E.g. The outcome of flipping a coin is H or T.
• Sample Space (𝑆): The set of all possible outcomes of an experiment.
– E.g. Sample space of rolling a six-sided die is 𝑆 = 1, 2, 3, 4, 5, 6 .
• Event (𝐸 ): A collection of outcomes; a specific subset of outcomes from 𝑆.
– E.g. The event of rolling an even number of a six-sided die: 𝐸 = 2, 4, 6 .
• Complementary Event (𝐸ത ): The event that represents all outcomes in the sample space
that are not in 𝐸 .
– E.g. If 𝐸 is rolling a 3 on a die, 𝐸ത is rolling anything other than 3, i.e., {1, 2, 4, 5, 6}.
22
22
11
2/14/2025
Some Probability Definitions (cont)
Here are definitions based on two events 𝐴 and 𝐵:
• Intersection of 𝐴 and 𝐵 (𝐴 ∩ 𝐵): The event containing all elements that are common to A
and B.
– E.g. 𝐴 = 2, 4, 6 and 𝐵 = 4, 5, 6 , then 𝐴 ∩ 𝐵 = 2, 4, 5, 6 .
• Union of 𝐴 and 𝐵 (𝐴 ∪ 𝐵 ): The event containing all elements that belong to A or B or both.
– E.g. Let 𝐴 = 𝑎, 𝑏, 𝑐 and 𝐵 = 𝑑, 𝑒 , then 𝐴 ∪ 𝐵 = 𝑎, 𝑏, 𝑐, 𝑑, 𝑒 .
23
23
Some Probability Definitions (cont)
Here are definitions based on two events 𝐴 and 𝐵:
• Mutually Exclusive Events: Two events that cannot occur at the same time.
– E.g. When flipping a coin, the events “heads” and “tails” are mutually exclusive.
– Can be written as 𝐴 ∩ 𝐵 = 𝜙
• Independent Events: Two events are independent if the occurrence of one does not
affect the occurrence of the other.
– E.g. Rolling a die and flipping a coin are independent events.
24
24
12
2/14/2025
Example: Selection of Three Items
• Suppose that three items are selected at random from a manufacturing process. Each
item is inspected and classified defective, D, or nondefective, N.
• Here’s a list of all the elements of the sample space:
S = {DDD, DDN, DND, DNN, NDD, NDN, NND, NNN}
• A tree diagram can also be constructed.
25
25
Venn Diagrams
• We can represent the relationships of different sets using Venn diagrams.
A 𝐴∩𝐵 B A B
𝐴∩𝐵
26
26
13
2/14/2025
Example
Construct a Venn diagram that would capture the following events:
– A: the card is red,
– B: the card is the jack, queen, or king of diamonds,
– C: the card is an ace.
What is the region for 𝐴 ∩ 𝐶 ?
What would these expressions equal to?
▪ 𝐴 ∩ 𝐴ҧ
▪ 𝐴 ∩ 𝐴ҧ
▪ 𝐴∩𝐵
▪ 𝐴∪𝐵
27
27
Statement of a Sample Space
Sample spaces with a large or infinite number of sample points are best described by a
statement or rule method.
• If the possible outcomes of an experiment are the set of cities in the world with a
population over 1 million, our sample space is written as:
S = {x | x is a city with a population over 1 million}
• If S is the set of all points (x, y) on the boundary or the interior of a circle of radius 2 with
center at the origin, we write the rule as:
S = {(x, y) | x2 + y2 ≤ 4}
28
28
14
2/14/2025
Some Probability Definitions
• The probability of an event 𝐸 is the ratio of the number of outcomes 𝑁𝐸 favorable to
event 𝐸 to the total number of possible outcomes 𝑁.
𝑁𝐸
𝑃 𝐸 =
𝑁
• E.g: Let 𝑁𝐸 = 4 ace cards and 𝑁 = 52 total cards. Then, 𝑃 𝑑𝑟𝑎𝑤𝑖𝑛𝑔 𝑎𝑛 𝑎𝑐𝑒 𝑐𝑎𝑟𝑑 = 4/52.
29
29
Some Probability Definitions
An empirical approach that uses experiments to count the occurrences of event 𝐸 .
• Repeat an experiment a number of times. If event 𝐸 occurs 30% of the time, then 0.3
can be a good approximation to the probability of event 𝐸 .
• If 𝑛 is the number of trials of the experiment and event 𝐸 occurs on 𝑁𝐸 of those trials,
then
𝑁𝐸
𝑃 𝐸 ≈
𝑁
• The approximation improves as value of 𝑛 increases.
30
30
15
2/14/2025
Some Basic Probability Laws
• For any event 𝐴:, the probability 𝑃(𝐴) is:
0 ≤𝑃 𝐴 ≤1
– The probability of event 𝐴 occurring ranges from never (probability 0) to always (probability 1).
• The complement 𝐴ҧ of an event 𝐴 is the event that 𝐴 does not occur:
𝑃 𝐴ҧ = 1 − 𝑃(𝐴)
31
31
Probabilities of Two Events
Let 𝐴 be the set of outcomes favorable to some event 𝐸𝐴 and 𝐵 be the set of outcomes
favorable to some event 𝐸𝐵 . There may be some common outcomes in both 𝐴 and 𝐵.
• 𝐴 ∪ 𝐵 is “𝐴 union 𝐵 ”: event 𝐴 or event 𝐵 occurs
• 𝐴 ∩ 𝐵 is “𝐴 intersect 𝐵 ”: event 𝐵 and event 𝐵 occurs A 𝐴∩𝐵 B
𝑃 𝐴 or 𝐵 = 𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃(𝐴 ∩ 𝐵)
• What if event 𝐴 and event 𝐵 are mutually exclusive events? 𝑃 𝐴 and 𝐵 = 𝑃 𝐴 ∩ 𝐵 = 0
𝑃 𝐴 or 𝐵 = 𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃(𝐵)
• If event 𝐴 and event 𝐵 are independent, then:
𝑃 𝐴∩𝐵 =𝑃 𝐴 ∙𝑃 𝐵
32
32
16
2/14/2025
Example: Probabilities of Two Events
What is the probability that a roll of a die produces an even number or a number between
2 and 5?
5
• Let 𝐴 = the set of even number outcomes
6 4
Let 𝐵 = the set of between 2 and 5 outcomes 3
2
𝑃 𝐴∪𝐵 =𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴∩𝐵 1
= 𝑃 2, 4, 6 + 𝑃 3, 4 − 𝑃( 4 )
3 2 1 𝟒
= + − =
6 6 6 𝟔
33
33
Example: Probabilities of Two Events
What is the probability that a roll of a die produces an even number and a number
between 2 and 5?
5
• Let 𝐴 = the set of even number outcomes
6 4
Let 𝐵 = the set of between 2 and 5 outcomes 3
2
𝑃 𝐴∪𝐵 =𝑃 𝐴 ∙𝑃 𝐵 1
= 𝑃 2, 4, 6 ∙ 𝑃 3, 4
3 2 6 𝟏
= ∙ = =
6 6 36 𝟔
34
34
17
