MATH-361
Probability and Statistics
Lecture 21
Introduction to Statistics,
Random Sampling
A/P Kamran Aziz Bhatti | Assistant Professor | Dept. of Electrical Engg. | NUST College of Electrical & Mechanical Engineering | Pakistan
Probability and Statistics: Basics
• Probability and Statistics
• Are They Same? What is the relation between the two?
• Common
• Both analyze the relative frequencies of events
• Differences
• Probability deals with predicting the likelihood of future events,
while statistics involves the analysis of the frequency of past
events.
• Dice Example: Probabilistic vs Statistician
2
Probability and Statistics: Basics
Probability theory enables us to find the
consequences of a given ideal world
While
Statistical theory enables us to measure
the extent to which our world is ideal
3
Statistical Inference
• Statistics is based on Statistical Inference
• Statistical inference is the process through which inferences
about a population are made based on certain statistics
calculated from a sample of data drawn from that population.
• Drawing samples is called Random Sampling
• Population is a larger set of samples from which random
sampling is performed
4
Definition: Statistics
Statistics consists of methods for designing
and evaluating random experiments to
obtain information about practical
problems
5
Random Sampling
• Samples are selected from populations
• Entire population cannot be inspected
• To obtain meaningful conclusions, samples must be selected
randomly
• Each item has the same chance of being sampled
• Only then, sample mean will be good approx. of true mean
• Random Sampling or Random Selection can be achieved by
Random Number Generators
• Not truly random – Sometimes called pseudorandom
• Come through some tricky formulas
6
Graphical Representation of Data
• Stem-and-Leaf Plot
• Absolute frequency, cumulative absolute frequency
• Histogram
• Class intervals, class marks, relative class frequency
• Median, Quartiles
• Median, range, upper/lower/middle quartile, interquartile range
• Box Plot
• Outlier
• A value more than a distance of 1.5 IQR from either end
7
Point Estimation
• Sample Mean
• Sample Variance
• Sample Standard Deviation is the positive square root of
sample variance
8
Point Estimation
• Estimation of Parameters of Distributions
• A point estimate of a parameter is a number which is
computed from a given sample and serves as an
approximation of the unknown exact value of the parameter of
the population
• An interval estimate is an interval (“confidence interval”)
obtained from a sample
• Approximation (Estimation) of mean
9
Point Estimation
• Approximation (Estimation) of variance
• For some distributions, mean and variance directly gives
parameter estimates
• For other distributions, parameters can be found by using
mean and variance estimates
• kth moment is estimated as
10
Maximum Likelihood Method
• Consider n independent samples corresponding to random
variable X with pdf 𝑓(𝑥) having a single parameter 𝜃,
• For discrete case, the probability that a sample of size n consists
precisely of those n values is
• For continuous case, the probability that the sample consists of
values in the small intervals is
• The function 𝑙 depends on 𝑥1 , 𝑥2 , … , 𝑥𝑛 as well as 𝜃 (unknown
parameter)
11
Maximum Likelihood Method
• Basic idea of ML method:
• We choose that approximation for the unknown value of 𝜃 for
which 𝑙 is as large as possible. If 𝑙 is a differentiable function of
𝜃, the condition becomes
• Why partial derivative??
• A solution of above equation (depending on 𝑥1 , 𝑥2 , … , 𝑥𝑛 ) is
called Maximum Likelihood Estimate (MLE) of 𝜽
12
Maximum Likelihood Method
• A simpler condition is
• If more than on parameters need to be estimated, the
conditions become
• Example: Estimate the mean and variance of Normal
Distribution
13
Maximum Likelihood Estimate (MLE)
• Example: Estimate the mean and variance of Normal
Distribution
• DC Level in White Gaussian Noise
• Unbiased Estimator
• Computer simulation
• Asymptotically Performance
14
