Lecture:
Statistics in Cognitive Science
Hypothesis Testing
Logical Reasoning
• Deductive Reasoning
• Inductive Reasoning
• Abductive Reasoning
2 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
3 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
4 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
5 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
6 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
7 06/11/2023 Statistics in Cognitive Science - Lecture
Deductive Reasoning
8 06/11/2023 Statistics in Cognitive Science - Lecture
Inductive Reasoning
1. I observed that the grass got wet every time when it rained.
2. ?
9 06/11/2023 Statistics in Cognitive Science - Lecture
Inductive Reasoning
1. I observed that the grass got wet every time when it rained.
2. When it rains, the grass gets wet.
10 06/11/2023 Statistics in Cognitive Science - Lecture
Abductive Reasoning
1. I observed that the grass got wet every time when it rained.
2. The grass is wet.
3. ?
11 06/11/2023 Statistics in Cognitive Science - Lecture
Abductive Reasoning
1. I observed that the grass got wet every time when it rained.
2. The grass is wet.
3. It probably rained.
12 06/11/2023 Statistics in Cognitive Science - Lecture
Statistical Inference
• We apply data analysis to infer properties of an underlying distribution of
probability (First sentence on Wikipedia)
• We draw a sample (our observed data set) and assume that it is drawn from a
population
• We infer properties of this population through inferential statistics performed
on our sample
13 06/11/2023 Statistics in Cognitive Science - Lecture
Significance Decisions / Hypothesis Testing
• Research Hypothesis
• Null and Alternative Hypothesis (H0 & H1)
14 06/11/2023 Statistics in Cognitive Science - Lecture
H0 & H1
• H0 = assumption about the population; „there‘s no difference“
The Null-Hypothesis
• H1 = „something is happening“
The Alternative-Hypothesis
15 06/11/2023 Statistics in Cognitive Science - Lecture
But why this complicated setting?
Falsifiability
• H0 = „All swans are white“
• H1 = „Some swans are black“
17 06/11/2023 Statistics in Cognitive Science - Lecture
H0 & H1
If the probability of a result occurring under H 0 is low, then
we reject H0 in favor of H1
18 06/11/2023 Statistics in Cognitive Science - Lecture
Probability
• Probability of events is measured on a scale of 0 to 1
• p represents our calculated probability
• Probability distribution is a histogram with columns measuring the likelihood
of occurrence
19 06/11/2023 Statistics in Cognitive Science - Lecture
Make a one- or two-tailed prediction
20 06/11/2023 Statistics in Cognitive Science - Lecture
Make a one- or two-tailed prediction
21 06/11/2023 Statistics in Cognitive Science - Lecture
Make a one- or two-tailed prediction
22 06/11/2023 Statistics in Cognitive Science - Lecture
Significance Decisions / Hypothesis Testing
• Research Hypothesis
• Null and Alternative Hypothesis (H0 & H1)
• Make statistical assumptions
• Decide which test is appropriate
• Set significance level
• Run your test
• Make a decision
23 06/11/2023 Statistics in Cognitive Science - Lecture
Reject or fail to reject the null hypothesis
• We reject the null hypothesis if our p-value lies under the pre-defined
significance level (α = .05).
• If we fail to reject the null hypothesis, we nonetheless do not conclude that
the null hypothesis is necessarily true
• Solution: Collect more data, or calculate an equivalence test
24 06/11/2023 Statistics in Cognitive Science - Lecture
Reject or fail to reject the null hypothesis
25 06/11/2023 Statistics in Cognitive Science - Lecture
Errors in statistical decision-making
Error Types Null hypothesis is
True False
Correct inference Type II error
Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)
26 06/11/2023 Statistics in Cognitive Science - Lecture
Errors in statistical decision-making
Error Types Null hypothesis is
True False
Correct inference Type II error
Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)
27 06/11/2023 Statistics in Cognitive Science - Lecture
Type I Error
28 06/11/2023 Statistics in Cognitive Science - Lecture
Errors in statistical decision-making
Error Types Null hypothesis is
True False
Correct inference Type II error
Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)
29 06/11/2023 Statistics in Cognitive Science - Lecture
Type II Error
30 06/11/2023 Statistics in Cognitive Science - Lecture
Errors in statistical decision-making
Error Types Null hypothesis is
True False
Correct inference Type II error
Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)
31 06/11/2023 Statistics in Cognitive Science - Lecture
Type II Error
32 06/11/2023 Statistics in Cognitive Science - Lecture
Errors in statistical decision-making
Error Types Null hypothesis is
True False
Correct inference Type II error
Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)
33 06/11/2023 Statistics in Cognitive Science - Lecture
Type I Error
34 06/11/2023 Statistics in Cognitive Science - Lecture
Design an Experiment
1. Define research hypothesis
2. Operationalization
3. Make a one- or two-tailed prediction
4. Set Null (H0) and Alternative (H1) Hypothesis
5. Set significance level
Pick the right statistical test
35 06/11/2023 Statistics in Cognitive Science - Lecture
Design an Experiment
1. Do Cognitive Science students differ in general intelligence from the
normal population?
2. Measure IQ of 9 students
3. CS students do not have average intelligence
4. H0: µ = 100; H1: µ ≠ 100
5. α = .05
36 06/11/2023 Statistics in Cognitive Science - Lecture
What’s out there?
37 06/11/2023 Statistics in Cognitive Science - Lecture
One-Sample t-Test
• Let’s say we know the population mean µ = 100
• If we now collect data and want to know whether our sample is significantly
different from this population, what can we do?
38 06/11/2023 Statistics in Cognitive Science - Lecture
What do we have?
• µ = 100
• our sample size N = 9
• our sample mean
• our sample standard deviation
• our standard error of the mean
39 06/11/2023 Statistics in Cognitive Science - Lecture
What can we do with this?
• We can calculate, how many se our sample mean is away from the population
mean µ (or how far away is our difference from 0)
• How many degrees of freedom does it have?
• N-1
40 06/11/2023 Statistics in Cognitive Science - Lecture
Is t just like z?
• No!
• unlike z, t is not normally distributed (unless N > 120)
• It’s generally broader than a normal distribution
• Student‘s t (William Gossett)
• We have t tables that give us critical values
41 06/11/2023 Statistics in Cognitive Science - Lecture
Density Plot
Density Plot
Example: z-distribution vs t-distribution Densit
df = 8
42 06/11/2023 Statistics in Cognitive Science - Lecture
Density Plot
Example: z-distribution vs t-distribution
df = 8
43 06/11/2023 Statistics in Cognitive Science - Lecture
t-Table
44 06/11/2023 Statistics in Cognitive Science - Lecture
Example: t-distribution
df = 8
45 06/11/2023 Statistics in Cognitive Science - Lecture
How to report?
The average IQ of Cognitive Science students (M = 109) is significantly
different from the average IQ of the general population, t(8) = 3, p = .017.
46 06/11/2023 Statistics in Cognitive Science - Lecture
t-Table
47 06/11/2023 Statistics in Cognitive Science - Lecture
Example: t-distribution
df = 8
48 06/11/2023 Statistics in Cognitive Science - Lecture
How to report?
The average IQ of Cognitive Science students (M = 109) is significantly higher
than the average IQ of the general population, t(8) = 3, p = .009.
49 06/11/2023 Statistics in Cognitive Science - Lecture
Thanks for listening!
