2 EM: transformation and Non

parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Today’s topics

1. Data distributions & transformation

Non-parametric & 2. Non-parametric tests:
 Mann-Whitney U test
Transformation  Wilcoxon matched pairs test
 Kruskal-Wallis test
Day 2
Transformation · Normality test · Tail · Distribution · Log · Sqrt · Ln · Zero
3. Graphs for non-normal data
values · Non-parametric test · Medians · Ranks · Tied data · 4. One-two sided testing
one/two-sided· Mann-Whitney U · Wilcoxon matched pairs ·
Kruskal Wallis · Groups · Chi-square table · Boxplots

1 2

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Adult body weight

100

80

Distribution type Distribution type Beetle Abundance, N

Frequency

60

40

Normal distribution: symmetric & continuous

Poisson distribution & Negative Binomial distribution:
20
Std. Dev = 5.24

(Gaussian distribution) Mean = 41.7

Frequency
skewed
0 N = 1011.00
26
28
30
32
34
36
38
40
42
44
46
48
50
52
54
56
58
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0

Weight (Kg)
not continuous: discrete,
counts, e.g., quadrats

Lognormal distribution: skewed & continuous Growth

 exponential growth, 1 2 3 4 5 6 7

biomass, concentrations
Binomial distribution: two outcomes:
Frequency

dead/alive
present/absent
smoking: y/n
not continuous, discrete

3 4

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Introduction

Mean/SD Unequal Hypothesis
P Equal
t-test uneq df 1-2 sided
P<0.05 Typet-test
I & IIeq Pairedt-test paired
Power

Adult body weight

100

Distribution type
80
Frequency

60

Normal distribution: symmetric & continuous 40

20
Std. Dev = 5.24
Mean = 41.7

0 N = 1011.00
26
28
30
32
34
36
38
40
42
44
46
48
50
52
54
56
58
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0
.0

Weight (Kg)

5 6

1
2 EM: transformation and Non
parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Elephant movement
Speed (m/min)
136.1
119.4
143.8
191.4
40.0
64.7
144.4
98.5
94.0
29.3
68.0
How to
104.3
60.5 make
42.1
161.0 this tail
36.6
27.2
55.2
shorter?
20.3
…….
N > 1000

7 8

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

tapply(elespeed$speed, elespeed$daynight, shapiro.test)

Normality test
Speed (m/min)
136.1
√
?Speed
11.7
x
Example in R tapply(sqrt(elespeed$speed), elespeed$daynight, shapiro.test)
119.4 10.9
143.8 12.0 R: sqrt(variable name)
Packages: leveneTest(sqrt(speed) ~ daynight,
191.4 13.8
40.0 6.3
 car data = elespeed,

 ggh4x center = median)

Variance test 64.7 8.0
144.4 12.0
 fitdistrplus 98.5 9.9
t.test(sqrt(elespeed$speed) ~ daynight, 94.0 9.7
data = elespeed, t-test 29.3 5.4
var.equal = TRUE) 68.0 8.2
104.3 10.2
60.5 7.8
Summary table 42.1 6.5
161.0 12.7
36.6 6.1
27.2 5.2
55.2 7.4
20.3 4.5
Bar graph + SD

9 10

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Transformation?

 Check first: variance (s2), mean, histogram

Distribution of the data is  Transformation: only rules of thumb, no hard guidelines
not distinguishable from a
normal distribution  if s2> x: Log, or ln for e.g., growth, biomass
 if s2 x : square root x for e.g., area, size;
 Double square root x : highly right skewed
 ln(p/(1-p)) for binomial, e.g. presence/absence data

Speed ( m/min) CHECK AGAIN! if normal

11 12

2
2 EM: transformation and Non
parametric

Introduction Transformation MWU Wilcoxon KW test Graphs Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Log (0), ln (0) = error

?
Transformation and 0-values:

Log or ln: log(x+1) or ln(x+1)

N
x: x
( +0.5) (+0.5 =expert advice)

Always transform whole variable!

Speed (classes)

13 14

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Today’s topics

1. Data distributions & transformation

Non-parametric 2. Non-parametric tests:
tests  Mann-Whitney U test H0: medians
 Wilcoxon matched pairs test
are equal
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

15 16

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Non-parametric test Mann-Whitney U test (test of medians)

Area 1 Area 2 Value Rank
Cup 1 8.0 2.7 2.7 1
4.3 2
 # beetles in pitfall traps Cup 2 12.3 4.3
5.3 3 Sum Rank area 1 = 83.5 = R1
Cup 3 15.2 5.5 8.0
 e.g., beetle abundance 4 Sum Rank area 2 = 36.5 = R2
….. 21.0 9.8 9.8 5
 Not-normal & 2 groups? 25.7 12.3 12.3 6.5
12.3 6.5
44.0 17.7
15.2 8 U1 = N1N2 + ([N2(N2+1)] / 2) - R2
44.0 19.1 17.7 9
 Transformation no good 60.3 - =56 + 28 - 36.5 = 47.5
19.1 10
 Mann-Whitney U test N1 = 8 N2 = 7 21.0 11 U2 = N1N2 + ([N1(N1+1)] / 2) - R1
25.7 12
= 8.5
± equal sample size 44.5 13.5
44.5 13.5
H0 = medians 60.3 15 Take the smallest U-value
are equal and compare with U-table

17 18

3
2 EM: transformation and Non
parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side
Mann-Whitney U test:
> wilcox.test(nrbeetles ~ area, data = beetles) # tied data:

This test: our test variable (U) should be < critical value
…there is a statistically significant difference between the medians (Mann
Whitney U -test, U=8.5, N1= 8 and N2=7, P<0.05)…..
Power of the Mann Whitney U test << t-test

19 20

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Area A Area B
50 50
% %

Median
H0: the two medians are equal

21 22

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side
Matched pairs? Not Normal?
Today’s topics
 Wilcoxon matched pairs test

1. Data distributions & transformation

2. Non-parametric tests:
 Mann-Whitney U test
 Wilcoxon matched pairs test
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

23 24

4
2 EM: transformation and Non
parametric

Introduction Transformation MWU Wilcoxon KW test Graphs Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Wilcoxon matched pairs test Wilcoxon matched pairs test

RANK
ID Mass week2 week3 A-B |A-B| RANK (with sign)
1 40.3 42.2 -1.9 1.9 8 -8 >wilcox.test (Pair(massw2,massw3)~1, data=nestlingmass)
2 41.4 42.1 -0.7 0.7 4 -4
3 40.9 43.1 -2.2 2.2 10 -10
4 42.0 41.9 0.1 0.1 1 1
5 40.0 42.0 -2 2 9 -9
6 41.9 42.9 -1 1 6 -6
7 42.2 41.4 0.8 0.8 5 5 Calculated V=47
8 42.3 42.1 0.2 0.2 2 2 P<0.002
9 41.7 43.5 -1.8 1.8 7 -7
So: the median nestling mass is
10 42.0 42.3 -0.3 0.3 3 -3
significantly larger in week 3 than in
+ ranks =1+5+2=8
week 2 (Wilcoxon matched pair test;
- ranks = 8+4+10+9+6+7+3=47 (absolute values) V=47, N=10, P<0.002)
Take the smallest value ranks = 8
Compare with statistical table (or use R)

25 26

Introduction
Mean/SD Unequal Hypothesis
P Equal
t-test uneq df 1-2 sided
P<0.05 Typet-test
I & IIeq Pairedt-test paired
Power Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Today’s topics

1. Data distributions & transformation

2. Non-parametric tests:
 Mann-Whitney U test
 Wilcoxon matched pairs test
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

27 28

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side
Kruskal Wallis χ2 test
>2 groups?
= Σ(R2/N) X 12/(N(N+1)) = 14.58
N = total number of samples in all groups
 Kruskal-Wallis test, test for medians,
 > 2 groups, similar number of samples
Compare with χ2 Table
 e.g., mean number of beetles/cup in 4 areas:

A B C D df = Number of groups – 1 = 4 – 1 = 3
27.1 48.5 11.2 44.5
14.4 18.9 0.0 1 72.0 So median number of beetles is
different among areas (Kruskal-Wallis test,
8.2 4.5 32.3 3.3 2 81.8 20 χ2 =14.58, df=3, P<0.01)
18.5 51.0 15.7 55.2
This table: calculated χ2
7.7 3 22.7 8.2 4.5 39.0
should be ≥ critical
N 5 5 5 5 threshold in table
R 36 66.5 21.5 86

29 30

5
2 EM: transformation and Non
parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Today’s topics

1. Data distributions & transformation

2. Non-parametric tests:
 Mann-Whitney U test
 Wilcoxon matched pairs test
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

So: medians significantly differ,

but which ones?

31 32

Introduction Transformation MWU Wilcoxon KW test Graphs Introduction Transformation MWU Wilcoxon KW test Graphs
Normal distribution
95%
Mean in middle
Symmetrical
Means differ significantly Mean=median Means differ significantly
SD: 68%
2SDs: 95%

N beetles
Mean in middle
Symmetrical
Mean=median
SD: 68%
1 2 3 4
2SDs: 95%
Meadow

Dependent variable

33 34

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side
Boxplot
Interpretation
Number of beetles (n/ha)

Medians with
4 quartiles

 Mean, s.d. (or c.l) can not be Means differ significantly * Outlier
used ≈100%
 Data not normal? 75%
 Error bars are not informative 50%
 Use different plot type median
 H0: medians are equal 25%
* ≈0%
Outlier
>1.5*interquartile
range

35 36

6
2 EM: transformation and Non
parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side
Number of beetles (n/ha)
Today’s topics
Number of beetles (n/ha)

Number of observations
1. Data distributions & transformation
2. Non-parametric tests:
 Mann-Whitney U test
 Wilcoxon matched pairs test
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

37 38

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

df = 6

This test: our test variable (U) should be < critical value
…there is a statistically significant difference between the medians (Mann
Whitney U -test, U=8.5, N1= 8 and N2=7, P<0.05)…..
Power of the Mann Whitney U test << t-test

39 40

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

One-tailed / two-tailed test

 One-tailed test yields significant outcome sooner than two-tailed test

 Independent grounds that one sample is drawn from a population
that has for instance a larger mean (you know the direction of the
difference): one-tailed test:
…e.g. pollution (mortality) or diet studies
 Ecology: a lot of noise: often uncertainty about direction effect:
two-tailed
 Later: some tests are one-tailed: e.g., F-test
Anova

 Doubt: always two-tailed.

Posthoc

41 42

7
2 EM: transformation and Non
parametric

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Parametric /Non-parametric tests

Parametric tests: assumes that the data come from a data

distribution characterized by certain parameters (e.g., mean, SD)
Data follows Data does not
normal distribution follow normal
distribution

Anova

Calculate mean, Calculate median,

SD, SE etc. range, max, min

Posthoc

43 44

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

tapply(elespeed$speed, elespeed$daynight, shapiro.test)

tapply(sqrt(elespeed$speed), elespeed$daynight, shapiro.test) Normality test
Example in R
leveneTest(sqrt(speed) ~ daynight,

All data Packages: data = elespeed,

Variance test
center = median)
 readxl
Non-parametric tests  tidyverse t.test(sqrt(elespeed$speed) ~ daynight,
data = elespeed, t-test
Transformation var.equal = TRUE)

Mann-Whitney U
wilcox.test(speed ~ daynight,
Normal distribution data = elespeed_small,
exact = FALSE)
Parametric tests wilcox.test(Pair(massw2, massw3)~1,
data = nestlings)
Power test: +++ Wilcox matched
pairs
kruskal.test(nrbeestles ~ area,
data = beetles) Bar graph + SD
Kruskal-Wallis

45 46

Distributions Transformation MWU Wilcoxon KW test Graphs One/two side

Today’s topics

1. Data distributions & transformation

2. Non-parametric tests:
 Mann-Whitney U test
 Wilcoxon matched pairs test
 Kruskal-Wallis test
3. Graphs for non-normal data
4. One-two sided testing

47