Testing Multiple Linear Restrictions: The F Test

Intuition

• So, far we have seen how to test whether the estimated coefficient in the population
is equal to any given value (usually zero, but not always). For a simple regression
βˆ1 −β1
model e.g. y = β0 + β1 x + u, this is given by t = se(βˆ1 )
.

• In a multiple regression model, the intuition behind testing the individual signifi-
cance is slightly different. For example, in the model y = β0 + β1 x1 + β2 x2 + u, testing
the individual significance of a partial regression coefficient assumes implicitly that
each test of significance is based on a different (i.e., independent) sample. Thus, in
testing the significance of βˆ1 under the hypothesis that HO : β1 = 0, it is assumed
tacitly that the testing is based on a different sample from the one used in testing the
significance of βˆ2 under the null hypothesis that HO : β2 = 0.

• Now, consider the hypothesis HO : β1 = β2 = 0. Can we use a t-test and individually

test the hypothesis HO : β1 = 0 and HO : β2 = 0 i.e. perform two different t-tests? The
answer is no! Why?

• Testing a series of single hypotheses is not equivalent to testing those same hypothe-
ses jointly. The intuitive reason is that in a joint test of several hypotheses any single
hypothesis is "affected" by the information in the other hypotheses.

• This means that conflicting results can be found between separate, individual hy-
pothesis tests, like HO : β1 = 0 and HO : β2 = 0, and a joint hypothesis test of the
form HO : β1 = β2 = 0.

• To test whether two or more parameters are jointly significant, we think of a "full" or
"unrestricted" model which is assumed to be the most appropriate to fit the data.

• The "reduced" model or the "restricted" model is described by the null hypothesis
(often obtained by zeroing on certain parameters apriori).

• The F-test then quantifies how much error is remaining after the two models are fit.

1
Testing Multiple Linear Restrictions: The F Test

Write the unrestricted model with k independent variables as

y = β0 + β1 x1 + β2 x2 + ..... + βk xk + u

the number of parameters in the unrestricted model is k + 1. (Remember to add one for the
intercept.) Suppose that we have q exclusion restrictions to test: that is, the null hypothesis
states that q of the variables in the unrestricted models have zero coefficients. The null
hypothesis is stated as

HO : βk−q+1 = 0, ....βk = 0

which puts q exclusion restrictions on the unrestricted model. The alternative is simply
that it is false; this means that at least one of the parameters listed in the null hypothesis
is different from zero. When we impose the restrictions under HO , we are left with the
restricted model:

y = β0 + β1 x1 + .... + βk−q xk−q + u

The F-statistic is then given by

SSRr −SSRur
q
F = SSRur
n−k−1

where SSRur is the Residual Sum of Squares for the unrestricted model, SSRr is the Resid-
ual Sum of Squares for the restricted model, q is the number of exclusion restrictions on the
unrestricted model, k is the parameters to be estimated (excluding intercept) in the unre-
stricted model and n is the sample size.

For testing exclusion restrictions, it is often more convenient to have a form of the F statistic
that can be computed using the R-squared from the restricted and unrestricted models. One
reason for this is that the R-squared is always between zero and one, whereas the SSRs can
be very large depending on the unit of measurement of y, making the calculation based on

2
the SSRs tedious.

2 −R2
Rur r
q
F = 1−Rur2

n−k−1

We reject the null if the calculated F-statistic is greater than the F-critical value at the α
significance level.

Example - 1

Consider the regression of wage on education controlling for experience across the popula-
tion for women.

lwage = β0 + β1 educ + β2 exper + β3 exper2 + u

Estimation results using a sample of data from US are presented in Table 1. Test for the joint
hypothesis HO : β2 = 0, β3 = 0 in the model.

Table 1: Determinants of Wage

Dependent variable:
lwage
(1) (2)
educ 0.109∗∗∗ 0.107∗∗∗
(0.014) (0.014)

exper 0.042∗∗∗
(0.013)

expersq −0.001∗∗
(0.0004)

Constant −0.185 −0.522∗∗∗

(0.185) (0.199)

Observations 428 428

R2 0.118 0.157
Adjusted R2 0.116 0.151
Residual Std. Error 0.680 (df = 426) 0.666 (df = 424)
Sum Squared Residual 197.001 188.305
∗∗∗ ∗∗∗
F Statistic 56.929 (df = 1; 426) 26.286 (df = 3; 424)
∗
Note: p<0.1; ∗∗ p<0.05; ∗∗∗ p<0.01

3
 SSRr = 197.001

SSRur = 188.305

q=2

n = 428

k=2
197.001−188.305
2
F = 188.305 = 9.79
428−3−1
2 −R2
Rur r 0.157−0.118
q 2
F = 1−Rur2 = 1−0.157 = 9.807
n−k−1 428−3−1

Since Fcalc > Fcrit i.e. 9.79 > 3.00, we can reject the null and conclude that both experience
and experience-square belong in the model.

Example - 2

Consider the following model to predict the semester GPA

termgpa = β0 + β1 attend + β2 priGP A + β3 hwrte + β4 ACT + u

where termgpa is the GPA for the semester, attend is the number of classes attended for the
semester, priGPA is the GPA prior to the semester, hrwte is the percentage of homework as-
signments submitted and ACT is the score on the standardized entrance exam. Estimation
results using a sample data from US are presented in Table 2. Test for the overall signifi-
cance of the model.

The null hypothesis, HO : β1 = 0, β2 = 0, β3 = 0, β4 = 0 is tested against the alternative

hypothesis, HA : β1 ̸= 0, β2 ̸= 0, β3 ̸= 0, β4 ̸= 0.

0.570−0
4
F = 1−0.570 = 221.70
674−4−1

Therefore, we can reject the null in favour of alternative as the Fcal > Fcrit i.e. 221.70 > 2.37.

4
 Table 2: Estimation of Semester GPA
Dependent variable:
termGPA
attend 0.032∗∗∗
(0.005)

hwrte 0.009∗∗∗
(0.001)

priGPA 0.557∗∗∗
(0.042)

ACT 0.036∗∗∗
(0.006)

Constant −1.287∗∗∗
(0.166)

Observations 674
R2 0.570
Adjusted R2 0.567
Residual Std. Error 0.477 (df = 669)
Sum Square Residual 152.216
F Statistic 221.436∗∗∗ (df = 4; 669)
∗
Note: p<0.1; ∗∗ p<0.05; ∗∗∗ p<0.01