Topic 1: Probability Theory
Dr. P. Rajendra, Professor, Department of mathematics,
CMRIT, Bengaluru.
Basic Probability Theory
▶ Probability is a measure of the likelihood of an event
occurring.
▶ For any event A, the probability of A is denoted by P(A),
where 0 ≤ P(A) ≤ 1.
▶ The sum of probabilities of all possible outcomes in a sample
space is 1.
▶ Conditional Probability: The probability of event A given
that event B has occurred is P(A | B) = P(A∩B)
P(B) , assuming
P(B) > 0.
▶ Independence: Two events A and B are independent if
P(A ∩ B) = P(A)P(B).
Example
Consider a spam detection model where A is the event ”email
contains the word ’free’” and B is the event ”email is spam”.
Understanding P(B | A) helps in improving the accuracy of the
model.
Random Variables
▶ A Random Variable is a function that assigns a numerical
value to each outcome in a sample space.
▶ Random variables can be discrete or continuous.
▶ Discrete Random Variables take on a countable number of
distinct values.
▶ Continuous Random Variables take on an uncountable
number of values within an interval.
Example
In a recommendation system, let X be a discrete random variable
representing the number of clicks on recommended items. A
continuous random variable Y might represent the time a user
spends on a page.
Probability Mass Function (PMF)
▶ The Probability Mass Function (PMF) is a function that
gives the probability that a discrete random variable is exactly
equal to some value.
▶ For a discrete random variable X , the PMF P(X = x)
satisfies:
X
P(X = xi ) = pi , where pi = 1
i
▶ The PMF provides a way to specify the probability
distribution of a discrete random variable.
Example
In a machine learning model predicting user actions, let X be the
number of actions a user takes. The PMF might show the
probability distribution of different action counts.
Probability Density Function (PDF)
▶ The Probability Density Function (PDF) describes the
likelihood of a continuous random variable taking on a
particular value.
▶ For a continuous random variable X , the PDF f (x) satisfies:
Z b
P(a ≤ X ≤ b) = f (x) dx
a
▶ The total area under the PDF curve is 1.
Example
In natural language processing (NLP), the time between word
occurrences in a document could be modeled as a continuous
random variable with a specific PDF.
Mathematical Expectation (Expected Value)
▶ The Expected Value or Expectation of a random variable is
the long-run average value of repetitions of the experiment it
represents.
▶ For a discrete random variable X with PMF P(X = xi ) = pi ,
the expectation is:
X
E [X ] = xi · pi
i
▶ For a continuous random variable X with PDF f (x), the
expectation is: Z ∞
E [X ] = x · f (x) dx
−∞
Example
In an e-commerce setting, X could represent the revenue generated
per user. The expected revenue, E [X ], is a crucial metric for
decision-making.
Mean
▶ The Mean of a random variable is another term for its
expected value and provides a measure of the central
tendency of the distribution.
▶ For a discrete random variable X :
X
µ = E [X ] = x i · pi
i
▶ For a continuous random variable X :
Z ∞
µ = E [X ] = x · f (x) dx
−∞
Example
In a user engagement model, the mean number of interactions per
session can inform interface design and content placement.
Variance
▶ The Variance of a random variable is a measure of how much
the values of the random variable differ from the mean.
▶ For a discrete random variable X :
X
Var(X ) = E [(X − µ)2 ] = (xi − µ)2 · pi
i
▶ For a continuous random variable X :
Z ∞
2
Var(X ) = E [(X − µ) ] = (x − µ)2 · f (x) dx
−∞
▶ The square root of the variance is the standard deviation,
which gives a measure of spread in the same units as the
random variable.
Example
In a predictive model, understanding the variance of predictions
(e.g., predicted sales) helps in assessing the reliability and
confidence in those predictions.
Probability Mass Function (PMF)
The Probability Mass Function(PMF) is the probability distribution
of a discrete random variable and provides the possible values and
their associated probabilities.
1. P(xi ) ≥ 0
Pn
2. i=1 P(X = xi ) = 1
3. 0 ≤ P(x) ≤ 1
4. Mean(µ):
n
X
µ = E [X ] = xi P(xi )
i=1
5. Variance(σ 2 ):
n
X
σ2 = xi2 P(xi ) − µ2 or σ 2 = E [X 2 ] − (E [X ])2
i=1
Probability Density Function (PDF)
The Probability Density Function (PDF) is the probability
distribution of a continuous random variable and provides the
possible values and their associated probabilities infinitely.
1. P(xi ) ≥ 0 or f (x) ≥ 0
R∞
2. −∞ f (x) dx = 1
3. Mean (µ): Z ∞
µ= xf (x) dx
−∞
4. Variance (σ 2 ):
Z ∞
2
σ = x 2 f (x) dx − µ2
−∞
5. Probability over an interval:
P(a ≤ x ≤ b) = P(a < x ≤ b)
Z b
= P(a ≤ x < b) = P(a < x < b) = f (x) dx
a
Question 1
Show that the following probabilities satisfy the properties of a
discrete random variable. Hence, find the mean and variance.
x 0 1 2 3
1 3 3 1
P(x) 8 8 8 8
Answer
Step 1: Verify that P(x) satisfies the properties of a discrete
random variable
To verify that P(x) satisfies the properties of a discrete random
variable, we need to check two conditions:
1. P(xi ) ≥ 0 for all i.
P
2. i P(xi ) = 1.
1. Non-negativity:
1 3 3 1
P(x) = , , , ≥0
8 8 8 8
2. Sum of probabilities equals 1:
X 1 3 3 1 8
P(x) = + + + = =1
x
8 8 8 8 8
Since both conditions are satisfied, P(x) represents a valid discrete
probability distribution.
Answer(Continued..)
Step 2: Find the Mean (µ) and Variance (σ 2 )
The mean (µ) of a discrete random variable is given by:
X
µ = E [X ] = x · P(x)
x
Calculating the mean:
1 3 3 1
µ=0· +1· +2· +3·
8 8 8 8
3 6 3 12
µ=0+ + + = = 1.5
8 8 8 8
The variance (σ 2 ) is given by:
σ 2 = E [X 2 ] − (E [X ])2
Answer(Continued..)
First, find E [X 2 ]:
X
E [X 2 ] = x 2 · P(x)
x
Calculating E [X 2 ]:
1 3 3 1
E [X 2 ] = 02 · + 12 · + 2 2 · + 32 ·
8 8 8 8
3 12 9 24
E [X 2 ] = 0 + + + = =3
8 8 8 8
Now, calculate the variance:
σ 2 = E [X 2 ] − (E [X ])2 = 3 − (1.5)2 = 3 − 2.25 = 0.75
Thus, the mean is µ = 1.5 , the variance is σ 2 = 0.75 and S.D is
σ = 0.86.
Question 2
Find the value of k such that the following distribution represents
a discrete probability distribution. Hence, find the Mean, Standard
Deviation, P(x ≤ 1), P(x > 1), and P(−1 < x ≤ 2).
x -3 -2 -1 0 1 2 3
P(x) k 2k 3k 4k 3k 2k k
Answer
Step 1: Find the value of k
The sum of all probabilities must equal 1 for it to be a valid
probability distribution:
k + 2k + 3k + 4k + 3k + 2k + k = 1
16k = 1
1
k=
16
Answer (Continued)
Step 2: Mean (Expected Value)
The mean µ is given by:
X
µ = E [X ] = xi · P(xi )
i
Substituting the values:
1 2 3 4 3 2 1
µ = (−3) · + (−2) · + (−1) · + 0 · + 1 · + 2 · + 3 ·
16 16 16 16 16 16 16
−3 − 4 − 3 + 0 + 3 + 4 + 3 0
µ= = =0
16 16
Answer (Continued)
Step 3: Standard Deviation
The variance σ 2 is given by:
σ 2 = E [X 2 ] − (E [X ])2
First, compute E [X 2 ]:
X
E [X 2 ] = xi2 · P(xi )
i
1 2 3 4 3 2 1
E [X 2 ] = (−3)2 · +(−2)2 · +(−1)2 · +02 · +12 · +22 · +32 ·
16 16 16 16 16 16 16
9 + 8 + 3 + 0 + 3 + 8 + 9 40
E [X 2 ] = = = 2.5
16 16
Since µ = 0:
σ 2 = 2.5 − 02 = 2.5
√
σ = 2.5 ≈ 1.58
Answer (Continued)
Step 4: Required Probabilities
▶ P(x ≤ 1) is calculated as:
P(x ≤ 1) = P(−3) + P(−2) + P(−1) + P(0) + P(1)
1 13
P(x ≤ 1) = k + 2k + 3k + 4k + 3k = 13k = 13 × =
16 16
▶ P(x > 1) is calculated as:
1 3
P(x > 1) = P(2) + P(3) = 2k + k = 3k = 3 × =
16 16
▶ P(−1 < x ≤ 2)
1 9
= P(0) + P(1) + P(2) = 4k + 3k + 2k = 9k = 9 × =
16 16
Question 3
A random variable X has a probability function for various values
of x as given below:
x 0 1 2 3 4 5 6 7
2 2 2
P(x) 0 k 2k 2k 3k k 2k 7k + k
Find:
i) k
ii) P(x < 6)
iii) P(x ≥ 6)
iv) P(3 < x ≤ 6)
v) The probability distribution and distribution function of X .
Answer
To find the value of k, we use the fact that the total probability
must equal 1:
0 + k + 2k + 2k + 3k + k 2 + 2k 2 + (7k 2 + k) = 1
Simplifying this, we have:
10k + 10k 2 − k = 1
This simplifies to:
10k 2 + 9k − 1 = 0
√
2
Using the quadratic formula k = −b± 2a
b −4ac
, where a = 10,
b = 9, and c = −1, we find:
p √
−9 ± 92 − 4 · 10 · (−1) −9 ± 81 + 40
k= =
2 · 10 20
√
−9 ± 121
k=
20
Answer (Continued..)
−9 ± 11
k=
20
So, the two possible values for k are:
1
k= = 0.1(we have k = -1 also, but k must be positive)
10
Now, calculate the probabilities:
ii) P(x < 6) = P(0) + P(1) + P(2) + P(3) + P(4) + P(5) =
0 + k + 2k + 2k + 3k + k 2 = 8k + k 2
iii) P(x ≥ 6) = P(6) + P(7) = 2k 2 + (7k 2 + k) = 9k 2 + k
iv) P(3 < x ≤ 6) = P(4)+P(5)+P(6) = 3k +k 2 +2k 2 = 3k +3k 2
Using k = 0.1:
P(x < 6) = 8(0.1) + (0.1)2 = 0.8 + 0.01 = 0.81
P(x ≥ 6) = 9(0.1)2 + 0.1 = 0.09 + 0.1 = 0.19
P(3 < x ≤ 6) = 3(0.1) + 3(0.1)2 = 0.3 + 0.03 = 0.33
Question 4
Show that the function
(
e −x , x ≥ 0
f (x) =
0, x <0
is a probability density function (PDF). Hence, find
P(1.5 < x < 2.5).
Answer
Step 1: Verify that f (x) is a PDF
To show that f (x) is a probability density function, it must satisfy
two conditions:
1. f (x) ≥ 0 for all x.
2. The total integral of f (x) over all x must equal 1.
1. Non-negativity:
f (x) = e −x ≥ 0 for all x ≥0
2. Integral equals 1:
Z ∞ Z ∞
f (x) dx = e −x dx
−∞ 0
Calculate the integral:
Z ∞
∞
e −x dx = −e −x 0 = (0 − (−1)) = 1
0
Since both conditions are satisfied, f (x) is a valid probability
density function.
Answer(Continued..)
Step 2: Find P(1.5 < x < 2.5)
Z 2.5
P(1.5 < x < 2.5) = e −x dx
1.5
Calculate the integral:
Z 2.5 2.5
e −x dx = −e −x 1.5
1.5
= −e −2.5 + e −1.5
= e −1.5 − e −2.5
1 1
= −
e 1.5 e 2.5
Thus,
1 1
P(1.5 < x < 2.5) = −
e 1.5 e 2.5
Question 5
A random variable X has a probability density function (PDF)
given by:
(
kx 2 , 0 ≤ x ≤ 3
f (x) =
0, otherwise
Evaluate:
i) k
ii) P(x ≤ 1)
iii) P(x > 1)
iv) P(1 ≤ x ≤ 2)
v) P(x ≤ 2)
vi) P(x ≥ 2)
Answer
i) Finding k:
The total probability must be equal to 1, so:
Z ∞ Z 3
f (x) dx = kx 2 dx = 1
−∞ 0
Calculating the integral:
3 3
x3
Z
2 27
k x dx = k =k − 0 = 9k
0 3 0 3
So,
1
9k = 1 =⇒ k =
9
ii) P(x ≤ 1):
Z 1 Z 1
1 2
P(x ≤ 1) = f (x) dx = x dx
0 0 9
Answer (Continued..)
Calculating the integral:
1
1 x3 1 1 1
= · =
9 3 0 9 3 27
iii) P(x > 1):
1 26
P(x > 1) = 1 − P(x ≤ 1) = 1 − =
27 27
iv) P(1 ≤ x ≤ 2):
Z 2 Z 2
1 2
P(1 ≤ x ≤ 2) = f (x) dx = x dx
1 1 9
Calculating the integral:
2
1 x3
1 8 1 1 7 7
= − = · =
9 3 1 9 3 3 9 3 27
Answer (Continued..)
v) P(x ≤ 2):
Z 2 Z 2
1 2
P(x ≤ 2) = f (x) dx = x dx
0 0 9
Calculating the integral:
2
1 x3 1 8 8
= · =
9 3 0 9 3 27
vi) P(x ≥ 2):
8 19
P(x ≥ 2) = 1 − P(x ≤ 2) = 1 − =
27 27
Question 6
The diameter of an electric cable is assumed to be a continuous
random variable with a probability density function (PDF) given by:
(
kx(1 − x), 0 ≤ x ≤ 1
f (x) =
0, otherwise
Find:
i) The value of k.
ii) The Mean and Variance.
Answer
i) Finding the value of k:
The total probability must be equal to 1:
Z ∞ Z 1
f (x) dx = kx(1 − x) dx = 1
−∞ 0
Calculating the integral:
Z 1 Z 1 Z 1
2
k x(1 − x) dx = k x dx − x dx
0 0 0
1 1 !
x2 x3
1 1
k − =k −
2 0 3 0 2 3
3 2 1
k − =k
6 6 6
k
= 1 =⇒ k = 6
6
Answer(Continued..)
ii) Finding the Mean (µ) and Variance (σ 2 ):
The mean µ is given by:
Z ∞ Z 1
µ = E [X ] = xf (x) dx = x · 6x(1 − x) dx
−∞ 0
Calculating the integral:
Z 1 Z 1 Z 1
2 2 3
6 x (1 − x) dx = 6 x dx − x dx
0 0 0
1 1 !
x3 x4
1 1
6 − =6 −
03 4 0 3 4
4 3 1 6 1
6 − =6 = =
12 12 12 12 2
Thus, the mean µ = 12 .
Answer(Continued..)
The variance σ 2 is given by:
σ 2 = E [X 2 ] − (E [X ])2
First, calculate E [X 2 ]:
Z ∞ Z 1
2 2
E [X ] = x f (x) dx = x 2 · 6x(1 − x) dx
−∞ 0
Z 1 Z 1 Z 1
3 3 4
6 x (1 − x) dx = 6 x dx − x dx
0 0 0
1 !
1
x4 x5
1 1
6 − =6 −
04 5 0 4 5
5 4 1 6 3
6 − =6 = =
20 20 20 20 10
3
So, E [X 2 ] = 10 .
Answer(Continued..)
Now, calculate the variance σ 2 :
2
2 3 1 3 1 3 2.5 0.5 1
σ = − = − = − = =
10 2 10 4 10 10 10 20
1
Thus, the variance σ 2 = 20 .
Assignment Problems (DRV)
1. Find the value of k such that the following distribution
represents a discrete probability distribution. Hence, find the
Mean, S.D, P(x ≥ 5), and P(3 < x ≤ 6). [Ans: (i) k=1/49,
Mean = 4.14, S.D = 1.64, (ii) 0.48 (iii) 0.67]
x 0 1 2 3 4 5 6
P(x) k 3k 5k 7k 9k 11k 13k
2. A random variable has the following probability function for
the various values of X = x. Find i) the value of k, ii)
P(x < 1), iii) P(x ≥ 1). [Ans: (i) k=0.1 (ii) 0.4 (iii) 0.9]
x -2 -1 0 1 2 3
P(x) 0.1 k 0.2 2k 0.3 k
3. A random variable has the following probability function for
the various values of X = x. Find i) the value of k, ii)
P(x ≤ 1), iii) P(0 ≤ x < 3). [Ans: (i) k=1/32 (ii) 0.18 (iii)
0.5]
x 0 1 2 3 4 5
P(x) k 5k 10k 10k 5k k
Assignment Problems (CRV)
1. A random variable X has the probability density function
(PDF):
(
kx 2 , −3 ≤ x ≤ 3
f (x) =
0, otherwise
Find:
i) k [Ans:1/18]
ii) P(x ≤ 2) [Ans:35/54]
iii) P(x ≥ 2) [Ans:19/54]
iv) P(x > 1) [Ans:26/54]
v) P(1 ≤ x ≤ 2) [Ans:7/54]
2. Find the constant k such that
(
kxe −x , 0 < x < 1
f (x) =
0, otherwise
is a valid probability density function (PDF). Find the mean.
[Ans: (i)k=e/(e-2) (ii)=(2e-5)/(e-2)]
