Lesson 8: Matrix fundamental spaces
New concepts:
¡ Row space
¡ Basis for a vector space
¡ Dimension of a vector space
¡ Sum, direct sum, and intersection of vector spaces
¡ Orthogonal subspaces, orthogonal complements
¡ Fundamental theorem of linear algebra (FTLA) for A 2 Rmn:
C(A) N (AT ) = Rm ; C(A)?N (AT ); C(A) \ N (AT ) = f0g;
C(AT ) N (A) = Rn ; C(AT )?N (A); C(AT ) \ N (A) = f0g:
Row space
For A 2 Rmn, seen as a linear mapping A: Rn 7! Rm, that given input vector x 2 Rn
returns output vector b 2 Rm ; b = A x, we have de ned the vector space of possible
outputs, the column space of A
C(A) = fb 2 Rm j 9x 2 Rn such that b = Axg Rm
The transpose AT 2 Rnm can also be seen as a linear mapping. Given some input vector
y 2 Rm the mapping returns the output vector c 2 Rn, c = ATy. The set of possible
outputs is the column space of AT . Since columns of AT are rows of A, we can de ne
the row space of A as
R(A) = C(AT ) = fc 2 Rn j 9y 2 Rm such that c = AT y g Rn
Basis for a vector space
De nition. A set of vectors u1; :::; un 2 V is a basis for vector space V if:
1. u1; :::; un are linearly independent;
2. spanfu1; :::; un g = V.
De nition. The number of vectors u1; :::; un 2 V within a basis is the dimension of the vector
space V.
Direct sum, intersection of vector spaces
De nition. Given two vector subspaces (U ; S ; +), (V ; S ; +) of the space (W ; S ; +), the
sum is the set U + V = fu + v j u 2 U ; v 2 V g:
De nition. Given two vector subspaces (U ; S ; +), (V ; S ; +) of the space (W ; S ; +), the
direct sum is the set U V = fu + vj 9!u 2 U ; 9!v 2 V g: (unique decomposition)
De nition. Given two vector subspaces (U ; S ; +), (V ; S ; +) of the space (W ; S ; +), the
intersection is the set
U \ V = fxjx 2 U ; x 2 V g:
De nition. Two vector subspaces (U ; S ; +), (V ; S ; +) of the space (W ; S ; +) are orthogonal
subspaces, denoted U?V if uTv = 0 for any u 2 U ; v 2 V.
De nition. Two vector subspaces (U ; S ; +), (V ; S ; +) of the space (W ; S ; +) are orthogonal
complements, denoted U = V ?, V = U ? if they are orthogonal subspaces and U \ V = f0g,
i.e., the null vector is the only common element of both subspaces.
The four fundamental subspaces for a linear mapping
A matrix A 2 Rmn is a linear mapping from Rn to Rm, A: Rn ! Rm
The transpose AT 2 Rnm is a linear mapping from Rm to Rn, AT : Rm ! Rn
To each matrix A 2 Rmn associate four fundamental subspaces:
1. Column space, C(A) = fb 2 Rm j 9x 2 Rn such that b = Axg Rm, the part of Rm
reachable by linear combination of columns of A
2. Left null space, N (AT ) = fy 2 Rm j AT y = 0g Rm, the part of Rm not reachable
by linear combination of columns of A
3. Row space, R(A) = C(AT ) = fc 2 Rn j 9y 2 Rm such that c = AT yg Rn, the part
of Rn reachable by linear combination of rows of A
4. Null space, N (A) = fx 2 Rn j Ax = 0g Rn, the part of Rn not reachable by linear
combination of rows of A
Fundamental theorem of linear algebra (FTLA)
Theorem. Given the linear mapping associated with matrix A 2 Rmn we have:
1. C(A) N (AT ) = Rm, the direct sum of the column space and left null space is the
codomain of the mapping
2. C(AT ) N (A) = Rn, the direct sum of the row space and null space is the domain of
the mapping
3. C(A)?N (AT ) and C(A) \ N (AT ) = f0g, the column space is orthogonal to the left
null space, and they are orthogonal complements of one another,
C(A) = N (AT )?; N (AT ) = C(A)? :
4. C(AT )?N (A) and C(AT ) \ N (A) = f0g, the row space is orthogonal to the null space,
and they are orthogonal complements of one another,
C(AT ) = N (A)?; N (A) = C(AT )? :
Graphical representation of FTLA
Gil Strang introduced a very useful graphical represenation in "The Fundamental Theorem
of Linear Algebra. Amer. Math. Monthly 100, 848-855, 1993.
r
mn
A2R
r
A: Rn ! Rm b
r = rank(A)
n m
C(AT ) Ax = b A(x + y) = b C(A)
x x+ y
0
Ay = 0
0
n¡r N (A) y
N (AT ) m¡r
Rn = C(AT ) N (A) usually: m > n Rm = N (AT ) C(A)
C(AT )?N (A) N (AT )?C(A)
