Distributions of Sampling Statistics
Mahbub Latif, PhD
February 2025
2
Plan
Sampling distribution of sample mean
Central limit theorem
Sampling distribution of sample variance
3
Introduction
The science of statistics deals with drawing conclusions from observed data,
which is often a sample from a population of interest
To use sample data to make inferences about an entire population, it is
necessary to make some assumptions between the two
There is an underlying probability distribution
The sample data are independent values drawn from this population
4
Introduction
If X , … , X are independent random variables having a common
1 n
distribution F , i.e.
X1 , … , Xn is a random sample from a distribution with distribution
function F
Two types of methods
F is specified up to some unknown parameters (parametric inference)
Nothing is known about F except the type of the associated variable
(nonparametric inference)
5
Example 6.1a
Suppose that a new process has just been installed to produce computer
chips, and the successive chips produced by this new process will have
lifetimes that are independent with a common unknown distribution F
Physical reasons sometimes suggest the parametric form of the distribution
F (e.g. F is a normal distribution, etc., i.e. parametric inference)
For normal distribution, only μ and σ need to be estimated
2
In other situations, there might not be any physical justification for
supposing that F has any particular form (nonparametric inference)
6
The Sample Mean
7
The Sample Mean
Let X , … , X be a random sample from a population with mean μ and
1 n
variance σ
2
For any i, E(X ) = μ and
i V ar(Xi ) = σ
2
The sample mean is defined as
n
1 1
¯
X = (X1 + ⋯ + Xn ) = ∑ Xi
n n
i=1
Sample mean X̄ is a random variable because it is a function of random
variables
8
Properties of X̄
The expected value
X1 + ⋯ + Xn
¯
E[X] = E[ ] = μ
n
μ → population mean
9
Properties of X̄
The variance
2
X1 + ⋯ + Xn σ
¯
V ar[X] = V ar[ ] =
n n
σ
2
→ population variance
n → sample size
10
X1 , … , Xn is a random sample from N (0, 1)
¯ 1
X ∼ N (0, )
n
11
Suppose X 1, … , Xn is a random sample from N (10, 5)
¯
X ∼ N (10, 1) when n = 5
What would be the distribution of X
¯
when the population is not normal? 12
Central Limit Theorem
13
Central Limit Theorem
Let X , … , X is a random sample from a distribution with mean μ and
1 n
variance σ , for a large n
2
2
Y = (X1 + ⋯ + Xn ) ∼ N (nμ, nσ )
n
1
¯ 2
Y = ∑ Xi ∼ N (μ, σ /n)
n
i=1
¯ − μ
Y
⇒ Z = ∼ N (0, 1)
σ/√n
14
Sampling distribution of a sample mean
15
Sampling distribution of a sample mean
16
Normal distribution
17
Summary of central limit theorem
Let X , … , X be a random sample from a population with mean μ and
1 n
variance σ , and the corresponding sample mean is
2
n
1
¯
X = ∑ Xi
n
i=1
If the population is normal then for any n
¯ 2
X ∼ N (μ, σ /n)
If the population is non-normal then only for a large n
2
X̄ ∼ N (μ, σ /n)
18
Application of central limit theorem to binomial distribution
Let X , … , X be a random sample from a Bernoulli distribution with
1 n
parameter p
Define X = X1 + ⋯ + Xn and X ∼ B(n, p)
Using central limit theorem, for a large n
X ∼ N (np, np(1 − p))
E(X) = np and V ar(X) = np(1 − p)
19
Example 6.3c
The ideal size of a first-year class at a particular college is 150 students.
From the past experience college knows that, on the average, only 30
percent of those accepted for admission will actually attend
The college uses a policy of approving the applications of 450 students.
Compute the probability that more than 150 first-year students attend this
college.
20
Example 6.3c
X denotes the number of students that attend and X ∼ B(450, .3)
Using binomial formula
450
450
i 450−i
P (X > 150) = ∑ ( )(.3) (1 − .3)
i
i=151
Using normal approximation
150.5 − (450)(.3)
P (X > 150) = P (X > 150.5) = 1 − Φ( )
(450)(.3)(1 − .3)
= 1 − Φ(1.59) = 1 − 0.9441
21
Example 6.3d
The weights of a population of workers have mean 167 and standard
deviation 27.0
If a sample of 36 workers is chosen, approximate the probability that the
sample mean of their weights lies between 163 and 170.
Repeat the above question when the sample is of size 144.
22
Sample variance
Let X , … , X is a random sample from a distribution with mean μ and
1 n
variance σ2
Sample variance
n
1
2 ¯ 2
S = ∑(Xi − X)
n − 1
i=1
Sample standard deviation (SD): S = √S
2
It can be shown that E(S 2
) = σ
2
23
Sampling distribution of sample variance
Let X , … , X is a random sample from a distribution with mean μ and
1 n
variance σ 2
2
(n − 1)S
¯ 2 2
X ∼ N (μ, σ /n) and ∼ χ
2 n−1
σ
χ
2
n−1
→ Chi-square distribution with (n − 1) degrees of freedom
24
Chi-square distribution
Let Z 1, … , Zn be n independent standard normal random variables
The distribution of X = ∑ follows a chi-square distribution with n
n 2
Z
i=1 i
degrees of freedom, i.e.
2 2 2
X = Z + ⋯ + Zn ∼ χ n
1
25
Chi-square distribution
26
27
The t-distribution
Let X , … , X be a random sample from a population with mean μ and
1 n
variance σ2
For a large n, X
¯ 2
∼ N (μ, σ /n)
¯
X − μ
Z = ∼ N (0, 1)
σ/√n
28
The t-distribution
If σ is unknown, it is replaced by sample standard deviation s in Z statistic
The resulting statistic follows a t-distribution with (n − 1) degrees of
freedom
¯
X − μ
t = ∼ tn−1
s/√n
29
Comparison between t and standard normal distributions
30
31
Problems
1, 2, 5, 8, 9, 10, 11, 12, 13, 14, 18
32
Problem 1
Plot the probability mass function of the sample mean of X 1, … , Xn when
(i) n = 2 and (ii) n = 3.
Assume
P (X = 0) = .2, P (X = 1) = 0.3, P (X = 3) = 0.5
Calculate E(X) = μ and V (X) = σ 2
In both cases, determine E(X)
¯
and V (X)
¯
33
Problem 1 (for n = 2 )
Probability distribution of
X1 X2 X̄ P (X̄ = x)
sample mean
0 0 0.0 0.04 x̄ P (X̄ = x̄)
0 1 0.5 0.06 0.0 0.04
0 3 1.5 0.10 0.5 0.12
1 0 0.5 0.06 1.0 0.09
1 1 1.0 0.09 1.5 0.20
1 3 2.0 0.15 2.0 0.30
3 0 1.5 0.10 3.0 0.25
3 1 2.0 0.15
3 3 3.0 0.25
Obtain E(X)
¯
and V ar(X)
¯
34
