Dealing with variables and hypotheses
Wojciech Fendler, M.D. Ph.D.
Department of Biostatistics and Translational Medicine
We formulate universal truths that will be
confirmed in the general population on the
basis of small samples
How to do it properly?
1. The cornerstone of experiment design – random selection
of samples
2. Good experimental design
The root of the problem?
The general concept of sampling
Population
Random sample
Unknown Relationship Yi ˆ0 ˆ1 X i ˆi
Yi 0 1X i i
Three things to remember in medical research
1. Anything, however unlikely, may happen due to chance
2. It is impossible to directly infer causation in the clinical setting
3. One rarely cares about the observed effect in the studied
sample, but rather about the potential to generalize the result
to the whole target population
Statistical hypotheses
• Parity
• Mutually complementary
imply that
• They cannot both occur
• They both include all possible outcomes
H0: x1 = x2 null hypothesis
HA: x1 x2 alternative hypothesis
Statistical hypotheses
• Hypothesis parity: „what is not false, it has to be true”
• Null hypothesis (‘straw man’ hypothesis, H0) most often is opposite to what
the researcher believes in (alternate hypothesis, HA)
H0: x1 = x2
HA: x1 x2
• Both H0 and HA must complement each other fully – no alternative options
Neyman, Pearson, Gosset
Simple example
• Is body height associated with gender?
H0 = height of men and women is equal
HA = height of men and women differs
• Why is it easier to reject the null hypothesis than
find universal evidence confirming the HA?
Why is it more convenient to formulate untrue
hypotheses as the starting point?
The truth
Guilty Not Guilty
Guilty Ok Innocent person
Our verdict sentenced
Not guilty Murderer goes Ok
free
H0 – Innocent until proven guilty
HA – Guilty as charged
How to formulate a statistical hypothesis ?
• in a majority of cases our null hypothesis is an opposite to
what is stated in research theory (Reject Support approach)
why is it so ?
• aimed at rejecting null hypothesis and accepting alternative
one – HA = research theory – H0 = opposite to research theory
Principles of hypothesis testing
• We can only reject H0 and never prove that it is true
• Non-significant result does not mean accepting H0, but merely its non-
rejecting at some particular conditions
• Even if we have no right to deny that
H0 is true, then we cannot reject it but it does not mean that it is true
When do we commit errors in hypothesis
testing?
The truth
Guilty Not Guilty
Guilty Ok type 1 error – H0
rejected
Our verdict incorrectly
Not guilty Type 2 error – false H0 Ok
not rejected
Type I statistical error – we reveal the non-existing (false) difference
Type II statistical error – we conceal the real (existing) difference
The meaning of errors in hypothesis testing
• Type 1 error (how to interpret):
– Rejection of a true null hypothesis – detection of a difference where there is
none
– False positive finding – discovery of an association which is purely by chance
• Type 2 error:
– Not rejecting a false null hypothesis – failure to detect a true effect
– False negative result – failure to discover the effet in a properly planned study
How do we combat the errors of hypothesis
testing?
• Type 1 error
– Plan the study properly
– Use adequate statistical tests
– …be lucky
• Type 2 error
– Plan a sufficiently large group
– Use adequate statistical tests
– …be lucky
The acceptable probability of errors (type 1 - a)
• For type 1 error – it is generally assumed that a
5% probability of this error is the maximum
tolerable margin allowing the researcher to reject
the null hypothesis
– Results of statistical tests that show the null
hypothesis be true with a probability <0.05 are
considered „statistically significant”
P value
• Post-test probability that we reject the null hypothesis
incorrectly
• A low p value means that the results are more likely to be
due to actual effects rather than a chance observation
• Typically we consider p values <0.05 as „statistically
significant” which translates to be a sufficient weight of
evidence to claim that the observed effect is an actual fact
What if we want the differences to be non-
existent?
„Absence of evidence is not
evidence of absence”
C. Sagan
The acceptable probability of errors (type 2 - )
• For type 2 error – it is generally assumed that the admissible probability of
this error is 20%
• The lower the better, but lowering it escalates the number of samples and
cost
• 1- is called statistical power – the probability that the study will be able to
reject a wrong null hypothesis (i.e. the probability of not making a type 2
error)
RS and AS approach to hypthesis testing
• Reject-support (RS) testing
– rejection of H0 supports research theory
• Accept-support (AS) testing
– non-rejecting H0 supports research theory
Comparison between RS and AS testing
RS testing AS testing
interested to reject H0 not interested to reject H0
care about low a care about low
large sample size large sample size
beneficial disadvantageous
high power important for society low a important
Statistical inference
The process of deducing the
underlying distribution by analysis
of data. Inferential analysis provides
the properties of a variable in the
population by testing hypotheses
and deriving estimates.
Basic statistical terms
Variables (also known as features, characteristics) – values
that are monitored or measured, being under control of or
manipulated by researcher in a course of study
May be classified according to a given criterion
Variables
• Independent variables – variables that may be controlled and/or
modified (manipulated) by a researcher in an experiment
• Dependent variables – variables that may be merely monitored
and measured by a researcher, cannot be manipulated or changed;
researcher does not affect these values
Discrete variables
• Example: responses to a five-point rating scale can only take on
the values 1, 2, 3, 4, and 5
• All qualitative variables are discrete; some quantitative variables
are discrete, such as performance rated as 1,2,3,4, or 5, or
temperature rounded to the nearest degree
• Sometimes, a variable that takes on enough discrete values can be
considered to be continuous for practical purposes
• Example: time rounded to the nearest millisecond
Continuous variables
• Examples: Length, weight, concentration, time, and the points on a
line are continuous variables
• The variable "Time to solve an anagram problem" is continuous since
it could take 2 minutes, 2.13 minutes etc. to finish a problem
• The variable "Number of correct answers on a 100 point multiple-
choice test" is not a continuous variable since it is not possible to get
54.12 problems correct
Variables
Independent variables
• Variables that may be controlled and/or modified (manipulated)
by a researcher in an experiment
Dependent variables
• Variables that may be merely monitored and measured by a
researcher, cannot be manipulated or changed; researcher does
not affect these values
Types of variables
What are the implications of such a
characteristics ?
Different tests are employed to manage with
different types of variables
Measurement scales
Measurement is the assignment of numbers to
objects or events in a systematic fashion
Four levels of measurement scales are commonly
distinguished:
• nominal
• ordinal
• interval
• ratio
Measurement scales - nominal
Nominal measurement - consists of assigning items to groups or categories
No quantitative information is conveyed and no ordering of the items is implied
Nominal scales are therefore qualitative rather than quantitative
Variables measured on a nominal scale are often referred to as categorical or
qualitative variables
Examples:
- religious preference
- race
- sex
- living in a village or a city
Measurement scales - ordinal
Ordinal measurements - are ordered in the sense that higher numbers represent
higher values, although the intervals between the numbers are not necessarily equal
Example:
- NYHA score of cardiac insufficiency
- A 4-grade rating scale:
- Grade I symptoms after exertion
- Grade II symptoms after moderate exertion
- Grade III symptoms after light exertion
- Grade IV symptoms at rest (indication for heart transplant)
- A change of 1 grade is an improvement but of different magnitude
Measurement scales - interval
Interval scale – the scale, on which the intervals between the numbers are
equal; one unit on the scale represents the same magnitude on the trait or
characteristic being measured across the whole range of the scale
Interval scale does not have a “true” zero point – it is not possible to make
statements on how many times one valus is higher from the other
Interval scales continued
Examples:
- The Fahrenheit scale for temperature; equal differences on this scale represent
equal differences in temperature, but a temperature of 30 degrees is not twice as
warm as one of 15 degrees
- Anxiety scale of behaviour; if anxiety were measured on an interval scale, then a
difference between a score of 10 and a score of 11 would represent the same
difference in anxiety as would a difference between a score of 50 and a score of 51
Measurement scales - ratio
Ratio scale – is like interval scales except it has a true zero point
Examples:
- Kelvin scale of temperature, which has an absolute zero; the temperature of
300 Kelvin is twice as high as a temperature of 150 Kelvin
The majority of continuous variables values represent either ratio or interval
scales
When monitoring variables …
… we take care of:
Precision – the degree of repeatability of measurements in a
series
Validity/accuracy – how far a measurement reflects what it
suppose to reflect (internal validity)
Precision and accuracy
High precision, Low accuracy,
high accuracy high precision
Low accuracy
and low
precision
Low precision,
high accuracy
Precision
(repeatability)
How estimated ? – by comparing measurements and estimating
their repeatability in a series
How significant it is for an investigation? – Increases the chance
to detect real differences between groups, because reduces within-
group variability (statistical power)
Why is reduced ? – random errors
Accuracy
(internal validity)
How is it estimated ? – by comparing values of measurements with
‘golden standard’ or reference values
How significant it is for an investigation? – Increases reliability of
the results
Why is reduced ? – systematic errors
Measures of central tendency
– which should be used and when ?
• mean (arithmetic)
• geometric mean
• harmonic mean
• median
• mode
• minimum, maximum
Mean
• The arithmetic mean - distinguished from the geometric mean or
harmonic mean
• The expected value of a random variable, which is also called the
population mean
Measures of location - Median
example
The median is the arithmetic mean of the two middle values in a
series with an even number of elements or the value of the
middle element in the ordered list of values
10 12 13 14 18 24 25 80 89 90 120 140 145
If the number of values is even, the median is the mathematical
average of the two middle values
10 12 13 14 18 24 25 26 80 89 90 120 140 145
Mode
• The value that occurs the most frequently
• Has to occur more frequently than other values
• There can be several modes, provided that they occur
more frequently than other values and equally frequently
with each other
• Examples:
– 1, 2, 3, 4, 4, 5, 6, 7, 10 (Mode 4)
– 1, 2, 3, 4, 4, 5, 5, 6, 7, 10 (Modes 4 AND 5)
Statistics of central tendency
- SD + SD
Mean/ Median / Mode Mode Mean
Median
Measures of dispersion
Statistical dispersion (also called statistical variability or variation) is variability or
spread in a variable or a probability distribution
used to express the variability of a given characteristic in the studied
population
• variance
• standard deviation (SD)
• standard error (of the mean) SE(M)
• variability coefficient
• agreement
• quantiles, quartiles
Variance – the total variability of
the variable
Continuous case
If the random variable X is continuous with probability density function p(x),
where
and where the integrals are definite integrals taken for x ranging over the range of X.
Sum of squared differences from the mean of a continuous variable
Standard deviation
The standard deviation of a statistical population, a data set, or a probability
distribution is the square root of its variance. Standard deviation is a widely
used measure of the variability or dispersion, being algebraically more tractable
though practically less robust than the expected deviation or average absolute
deviation
Value of SD represents the average difference from the mean in the group
Standard deviation
A data set with a mean of 50 (shown in blue) and a standard deviation (σ) of 20
Normal (Gaussian) distribution
0.5% 0.5%
0.1% 0.1%
Standard error (of mean)
Depends on sample size and SD
Used as a measure of precision for estimating the
true value of the mean
When SD and when SEM ?
• Standard deviation – measure of within-population
variability
• Standard error (of mean) – measure of imprecision in
technical replicates
The use of SEM for comparing groups (controls-cases) does not make a
sense and neither does using SD for calibration curves
What is the median?
What will you get if you divide the
upper and lower halves of an ordered
data series by their respective medians?
Interquartile range
The interquartile range (IQR), also called the midspread or middle fifty, is a
measure of statistical dispersion, being equal to the difference between the third
and first quartiles
IQR = Q3 − Q1
The median is the corresponding measure of central tendency
Used together with the median for describing non-normally distributed samples
Interquartile range
i x[i] Quartile
1 102
2 104
3 105 Q1
4 107
5 108
6 109 Q2 (median)
7 110
8 112
9 115 Q3
10 116
11 118
How do we read measures of central
tendency and dispersion ?
Middle line –
mean
Box – standard
deviation
Total cholesterol
How do we read measures of central
tendency and dispersion ?
Middle line –
median
Box – upper
and lower
quartiles
Thank you for your attention
