Elementary Matrix Theory

Definition
A matrix of A order (or size) m  n is a rectangular array of mn
elements arranged in m rows and n columns. The element in the i th
row and j th column is called the (i, j ) -entry of the matrix and is
denoted by aij .
𝑎11 𝑎12 … 𝑎1𝑛
𝑎21 𝑎22 … 𝑎2𝑛
𝐴 = [𝑎𝑖𝑗 ] = [ ⋮ ⋮ ⋮ ⋮ ]
𝑎𝑚1 𝑎𝑚2 … 𝑎𝑚𝑛

An n  n matrix is called a square matrix of order n . A matrix of order m1 is

called a column vector, and one of order 1 n is called a row vector.

Here, consider matrices whose elements are, in general, complex numbers. Two
matrices A and B are equal when their corresponding elements are equal i.e.,
aij = bij for all i , j (which implies that A and B are of the same order).

Matrix operations:
Addition: Matrices are added (or subtracted) element-wise:
[aij ] + [bij ] = [aij + bij ] , which again assumes that they are of the same order.

A matrix A = [aij ] can also be multiplied by a scalar (complex number) k as

kA = k[aij ] = [kaij ] .

Matrix Multiplication
An m  n matrix A can be multiplied with an p  q matrix B if and only if
n 
n = p . Their product is defined as AB =  aik bkj  .
 k =1 

Example
2 3  2 1 + 3  4 2  (−1) + 3  6  14 16 
5 7   1 −1
=  5 1 + 7  4 5  (−1) + 7  6  = 33 37 
   4 6     
11 13 11  1 + 13  4 11  (−1) + 13  6  63 67 
Matrix multiplication is associative, A( BC ) = ( AB)C ; and distributive,
A( B + C ) = AB + AC , but not commutative (in general), AB  BA . Indeed, the
product BA may not even be defined, as in the case of the matrices in the above
example.

The Identity Matrix

The identity matrix I of order n is defined as the square matrix of order n with
(i, j ) -entry 1 if i = j and 0 if i  j for all i, j = 1,2n .
1 0 … 0
0 1 … 0
𝐼=[ ]
⋮ ⋮ ⋮ ⋮
0 0 … 1
When the order of the identity matrix is not clear from the context, it may be
denoted as I n , where n is the order.

Multiplying any matrix by an identity matrix of appropriate order results in the

same matrix, i.e., if A is any m  n matrix, then I m A = AI n = A . It is easy to
show that the identity matrix I the only matrix with this property. For, suppose
K is another such matrix, then IK = K , because of the property of I , and
IK = I , because of the property of K , and thus, I = K .

Transpose and Trace

The transpose of an m  n matrix A is the n  m matrix, denoted by AT (or A ' )
with the rows of A for columns and columns of A for rows – i.e., if A = [aij ]mn
then AT = [a ji ]nm .

Example
1 0 9
1 2 −1 4   2 −3 −2 
 
If A = 0 −3 1 7  , then A = 
T .
 −1 1 5 
9 −2 5 −1  
 4 7 −1

The conjugate transpose of A (also called the Hermitian transpose or

transjugate) is matrix A * (or AH ) obtained by replacing every element of AT
by its complex conjugate. Recall that the conjugate of a complex number
z = a + ib is z = a − ib . Thus, if A = [aij ]mn , then A* = AT =  a ji  .
nm

Example
2 + i 4 −3 + 2i  2−𝑖 1+𝑖 −6𝑖

If A =  1 − i 1 + 3i 
−5 , then 𝐴* = [ 4
 1 − 3𝑖 4 + 𝑖 ].
 6i 4 − i 1 − 2i  −3 − 2𝑖 −5 1 + 2𝑖

A square matrix A is said to be symmetric if A = AT , skew-symmetric if

A = − AT , and Hermitian of A = A * .

The principal or main diagonal of a square matrix A = [aij ] of order n consists

of the entries aii for every i = 1,2n . The trace of a square matrix is the sum of
all the elements on the main diagonal, and is denoted by trace( A) or
n
tr( A) =  aii .
i =1

Example
10 −1 1 
If A =  3 −2 1 , then tr( A) = a11 + a22 + a33 = 10 + (−2) + 5 = 13 .
 −4 7 5

Properties
1. ( A + B)T = AT + BT and ( A + B)* = A * + B * for all m  n matrices A and B .
2. ( cA) = cAT and ( cA) * = c * A * for all matrices A and scalars c .
T

( )
T
3. AT = ( A *) * = A for all matrices A .

4. ( AB ) = BT AT and ( AB ) * = B * A * (provided the product AB is defined).

T

5. tr( A + B) = tr( A) + tr( B) for all n  n matrices A and B .

6. tr(cA) = c tr( A) for all square matrices A and scalars c .
( )
7. tr AT = tr( A) and tr ( A *) = tr( A) for all square matrices A .
8. tr( AB) = tr( BA) (provided the product AB is defined and is a square matrix).
Example
 3 −1  −2 7 
Let A =   and B =  .
 2 5   1 4 

 −7 17   8 37 
Then AB =   and BA = 11 19  .
 1 34   

We observe that tr( AB) = −7 + 34 = 27 and tr( BA) = 8 + 19 = 27 . Thus, although

the products AB and BA , and indeed, even the main diagonal entries of the
products are different, tr( AB) = tr( BA) .

Determinants
For every square matrix A , we can associate a single (complex) number called
the determinant of A , and denoted by | A | or det( A) , having certain important
properties.

Definition
1. For a 1  1 matrix A = [ a11 ] , the determinant is defined to be | A |= a11 .
2. For an n  n matrix A = [aij ] (where n  1 ), the determinant is defined as
n
| A |=  (−1) j +1a1 j A(1| j )
j =1

where A(i | j ) denotes the (n − 1)  (n − 1) matrix obtained by deleting the i th

row and j th column of A (the determinant A(i | j ) is itself called the minor
and is denoted by M ij .

Example
1 2 
1. Let A =  .
3 4 
1 2
Then | A |=
3 4
= 1 4 − 2  3
= −2 .
 1 3 1
2. Let A =  −1 1 2  .
 
 2 1 −2 

1 3 1
Then | A |= −1 1 2
2 1 −2

1 2 −1 2 −1 1
= 1 − 3 + 1
1 −2 2 −2 2 1
= 1(1 (−2) − 2  1) − 3 ( (−1)  (−2) − 2  2 ) + 1( −1 1 − 1 2 )

= −4 + 6 − 3
= −1 .

Inverse of a Matrix
For a square matrix A of order n , another matrix B of the same order is said to
be an inverse of A if AB = BA = I , the identity matrix of order n . A square
matrix that has no inverse is said to be singular, and one that has an inverse is
said to be non-singular or invertible.

Theorem
Every invertible matrix has a unique inverse.

Proof
Let A be an invertible matrix, and suppose B1 and B2 are inverses of A .

Then B2 = B2 I (where I is the identity matrix of appropriate order)

= B2 ( AB1 ) (as B1 is an inverse of A )

= ( B2 A) B1 (as matrix multiplication is associative)
= IB1 (as B2 is an inverse of A )
= B1 .
 B2 = B1 , i.e., the inverse of A is unique. ∎

The unique inverse of A is denoted by A−1 .

Example
1 3 1  4 −7 −5
If A = −1 1 2 , then A =  −2 4 3  .
  −1
   
 2 1 −2   3 −5 −4 

 1 3 1   4 −7 −5  1 0 0 
For, AA =  −1 1 2   −2 4 3  = 0 1 0  .
−1

 2 1 −2   3 −5 −4  0 0 1 

Properties
( )
−1
1. A−1 = A for all invertible matrices A .

2. ( AB ) = B −1 A−1 for all invertible matrices A and B of the same order.

−1

( ) ( )
−1 T
3. AT = A−1 for all invertible matrices A .

Formula for Inverse

A square matrix is non-singular if and only if its determinant is non-zero. The
inverse of a non-singular matrix A can be calculated in terms of determinants of
A and its submatrices.

Definition
The cofactor Cij of A corresponding to the (i, j ) -entry is defined as
Cij = (−1)i + j M ij , where M ij is the minor.
The cofactor matrix C of A is the matrix with (i, j ) -entry as the
cofactor Cij , for every i , j .
The adjoint (or adjugate) of A , denoted by adj A is the transpose of the
cofactor matrix C .

The inverse of a non-singular matrix A is given by the formula

1
A−1 = ( adj A)
| A|
Example
1 0 2 
Let A =  2 3 1  .
 
1 2 1 

 1 −1 1 
The cofactor matrix is C =  4 −1 −2  and the determinant is | A |= 3 .
 −6 3 3 

 1 4 −6 
Thus, adj A = C =  −1 −1 3  .
T

 1 −2 3 

 1 4 −6 
adj A 1 
A =−1
= −1 −1 3  .
| A| 3 
 1 −2 3 

Special Matrices
Let A = [aij ] be an n  n (real or complex) matrix.
1. A is said to be diagonal if aij = 0 for all i  j .
2. A is said to be tridiagonal if aij = 0 for all i , j with | i − j | 1 , i.e., all the
entries of A are zero except (possibly) for those on the main diagonal
( a11, a22 ann ), the superdiagonal ( a12 , a23 an−1,n ), and the subdiagonal
( a21, a32 an,n−1 ).
3. A is said to be upper-triangular if aij = 0 for i  j , i.e., all entries below the
main diagonal are zero.
4. A is said to be lower-triangular if aij = 0 for i  j , i.e., all entries above the
main diagonal are zero.
5. A is said to be orthogonal if all its entries are real and AT = A−1 , i.e.,
AAT = AT A = I .
6. A is said to be unitary if it has complex entries and A* = A−1 , i.e.,
AA* = A * A = I .
Exercises
1. Classify the following matrices as symmetric, skew-symmetric, Hermitian, or
none of these:
 1 −3
a.  
 −3 4 
 e x e −ix 
b.  
e
ix
e − x 
0 1 1 
c. 1 0 1 
1 1 0 
 2 1+ i 3 
d. 1 − i 0 i 

 3 −i −1
 0 1 1
e.  −1 0 1 
 −1 −1 0 
 1 1 1
f.  −1 1 1
 −1 −1 1
 2 1+ i 3 
g. 1 − i i i 

 3 −i −1
 x 2 − 9 x 2 − 2 x − 1
2. Find the value of x such that A =   is
 4 − 2 x 0 
a. symmetric
b. skew-symmetric
3. Show that every skew-symmetric matrix A = [aij ] of order n has all main
diagonal entries zero, i.e., a11 = a22 = = ann = 0 .
4. Show that every Hermitian matrix has only real numbers along the main
diagonal.
5. Find the determinant of the following:
 1 −1
a. 
3 3 
0 1 1
b. 1 0 1

1 1 0 
 e x e −ix 
c.  
e
ix
e − x 
 cos sin 
d.  
 − sin cos 
e x e2 x 
e.  x 2x

e 2e 
 1 2 −2 
f.  −1 3 0 
 0 −2 1 
6. Determine which of the following are invertible and find the inverse:
 1 −1
a.  
 −3 3 
1 −1
b.  
3 3 
0 1 1 
c. 1 0 1 
1 1 0 
 0 1 1
d.  −1 0 1 
 −1 −1 0 
 cos sin 
e.  
 − sin cos 
1 x x 2 
 
f. 0 1 x 
0 0 1 
 
 1 a b 
g. 0 1 c  , where a , b , c are any three complex numbers.
 
0 0 1 
a 0 0
h.  0 b 0 
 0 0 c 
7. Show that the inverse of every 3  3 upper-triangular matrices with non-zero
diagonal entries is also upper triangular.
Hint: If the i th row of A is multiplied by a non-zero constant p , then the i th
column of A−1 gets divided by p . Use this property to reduce A to a matrix
of the same form as that in Exercise 6g.
8. Find the inverse of a general n  n diagonal matrix with non-zero diagonal
entries.
9. Classify the following matrices as orthogonal, unitary, or neither:
1 0 
a.  
0 1 
0 0 1
b. 0 1 0

1 0 0 
1 −1
c. 
 −1 0 
1 1 
d.  
1 −1
1 1 
 2 2 
e. 
1 − 1 
 2 2 
1 i 
 2 2 
f. 
1 − i 
 2 2 
 cos sin 
g.  
 − sin cos 
 Elementary Row and Column Operations
A rectangular matrix is in echelon form (or row echelon form) if it has the

following three properties:

1. The leading entry of every row is to the left of the leading entry, if any, of
all rows below it.
2. All entries in a column below a leading entry are zeros.

If a matrix in echelon form satisfies the following additional conditions, then it

is in reduced echelon form (or row reduced echelon form)

1. In every row, the first non-zero entry (called the leading entry), if any, is 1.
2. Each leading 1 is the only nonzero entry in its column.

Note that zero rows (i.e., rows with all entries 0) are allowed, but the second
condition implies that all zero rows must occur below all non-zero rows (i.e.,
rows with at least one non-zero entry).

Example
Among the matrices given below, A is in row echelon form, C is in row-
reduced echelon form, while B and D are not (why?).
1 0 2 −1 1 0 2 −1
𝐴 = [0 0 1 4 ] 𝐵 = [0 0 0 0]
0 0 0 0 0 0 1 4
1 0 0 0 −1 −5 1 2 0 0 −1 −5
𝐶 = [0 1 0 0 3 −3] 𝐷 = [0 1 0 4 3 −3]
0 0 0 1 1 2 0 1 0 5 4 −1

The following three operations performed on a matrix are called elementary row
operations:
1. Interchange: Interchange any two rows. Ri  R j denotes interchanging of
the i th and j th rows
2. Scaling: Multiply every entry of a row by the same non-zero scalar. Ri → kRi
denotes multiplying the i th row by the scalar k .
3. Row Addition: Add a multiple of one row of the matrix to another row.
Ri → Ri + kR j denotes adding k times the j th row, to the i th row.
Note: These definitions are motivated by analogous operations that can be
performed on a given system of linear equations to obtain equivalent systems of
linear equations.

We can use elementary row operations to transform a matrix into its row-
reduced echelon form.

Example
4 8 0 0 −4 −20
𝐴 = [−3 −5 0 4 6 12 ]
2 6 0 9 5 −14
 1 2 0 0 −1 −5  R1 → R1 / 4
~  −3 −5 0 4 6 12 
 
 2 6 0 9 5 −14 

1 2 0 0 −1 −5  R2 → R2 + 3R1
~ 0 1 0 4 3 −3 R3 → R3 − 2 R1
 
0 2 0 9 7 −4 

1 2 0 0 −1 −5 R3 → R3 − 2 R2
~ 0 1 0 4 3 −3 (Row Echelon form)
 
0 0 0 1 1 2 

1 2 0 0 −1 −5
~ [0 1 0 0 −1 −11] 𝑅2 → 𝑅2 − 4𝑅3
0 0 0 1 1 2
1 0 0 0 1 17
~ [0 1 0 0 −1 −11] 𝑅1 → 𝑅1 − 2𝑅2
0 0 0 1 1 2

Elementary column operations can also be defined, similar to elementary row

operations. Applying these operations, we can find the analogous column-
reduced echelon form of a matrix.
Example
 4 8 0 0 −4 −20 
A =  −3 −5 0 4 6 12 
 
 2 6 0 9 5 −14 

4 0 0 0 0 0  C2 → C2 − 2C1
~  −3 1 0 4 3 −3 C5 → C5 + C1
 
 2 2 0 9 7 −4  C6 → C6 + 5C1
4 0 0 0 0 0  C4 → C4 − 4C2
~  −3 1 0 0 0 0  C5 → C5 − 3C2
 
 2 2 0 1 1 2  C6 → C6 + 3C2

4 0 0 0 0 0  C5 → C5 − C4
~  −3 1 0 0 0 0  C6 → C6 − 2C4
 
 2 2 0 1 0 0 
4 0 0 0 0 0  C3  C4
~  −3 1 0 0 0 0 (Column echelon form)
 
 2 2 1 0 0 0 

Rank Using Row Operations

Recall that we defined the rank of a matrix as the order of its largest non-
singular square submatrix. It is also possible to define and compute the rank in a
different manner.

Definition
The row rank of a matrix is the number of non-zero rows in the row-
echelon form of that matrix.
The column rank of a matrix is the number of non-zero columns in the
column- echelon form of that matrix.
Example

 2 −1 3 
1 0 1 
A= 
 0 2 −1
 
1 1 4 

1 0 1  R1  R2
2 −1 3
~ 
0 2 −1
 
1 1 4
1 0 1  R2 → R2 + 2 R1
0 −1 1  R4 → R4 − R1
~ 
0 2 −1
 
0 1 3
1 0 1  R3 → R3 + 2 R2
0 −1 1  R4 → R4 + R2
~ 
0 0 1
 
0 0 4
1 0 1
0 1 −1 R2 → − R2
~ 
0 0 1  R4 → R4 − 4 R3
 
0 0 0

As there are three non-zero rows in the row- echelon form of A , the row rank of
A is 3. Now, we find the column rank.
 2 −1 3 
1 0 1 
A= 
 0 2 −1
 
1 1 4 
  −1 2 3  C1  C2
0 1 1
~ 
2 0 −1
 
1 1 4
 −1 0 0  C2 → C2 + 2C1
0 1 1  C3 → C3 + 3C1
~ 
2 4 5
 
1 3 7
 −1 0 0  C3 → C3 − C2
0 1 0
~ 
2 4 1
 
1 3 4

As there are three non-zero columns in the column- echelon form of A , the
column rank of A is 3. We see that the row rank and column rank of A are
equal. Indeed, this is true in general, as stated in the following theorem.

Theorem
The row rank of a matrix is the same as the column rank, and the
common value is the rank of the matrix.

Exercises
Determine the rank of the following matrices by using elementary row
operations:
1 −2 0 
1. 1 1 1  Ans. Rank = 2
 2 −1 1 
1 1 0 
2.  2 −2 1  Ans. Rank = 3
1 −1 0 
1 1 1 1 
3. 1 −1 1 2  Ans. Rank = 2
3 1 3 4 
 1 3 1 1
 −1 1 2 2
4.   Ans. Rank = 4
 2 1 −2 1
 
 1 2 −2 2
1 1 1 
5. 1 0 1  Ans. Rank = 3
1 1 0 

Rank of a matrix and system of equations

Introduction
The rank of a matrix A is the maximum number of linearly independent row
vectors (or column vectors) in A. A simple application of rank comes from the
fact that a linear system of equations AX=B has a solution if and only if the rank
of the coefficient matrix A is the same as the rank of the matrix [A|B] where [A|B]
denotes the coefficient matrix augmented by column vector B.

Objectives
o To find the rank of a given matrix.

o To find whether a linear system of equations AX=B has a solution.

Contents:
Elementary Row Operations:
(i) Multiply a row by a nonzero constant.
(ii) Interchange any two rows.
(iii) Add a nonzero multiple of one row to any other row.

Problem: Find the rank of the following matrix using elementary row operations.

1 2 −1 0 
− 1 3 0 − 4
A=  .
2 1 3 − 2
 
1 1 1 − 1
Solution:

Perform, R2 → R2 + R1, R3 → R3-2R1, R4 → R4-R1

1 2 − 1 0 
0 5 − 1 − 4 
A  
0 − 3 5 − 2 
 
0 − 1 2 − 1 

Perform, R2  R4, R2 → - R2

1 2 − 1 0 
0 1 − 2 1 
A  
0 − 3 5 − 2 
 
0 5 − 1 − 4 

Perform, R3 → R3 + 3R2, R4 → R4 - 5R2, R3 → -R3

1 2 −10
0 1 − 2 1 
A 
0 0 1 − 1
 
0 0 9 − 9

Perform, R4 → R4-9R3

1 2 −10
0 1 − 2 1 
A 
0 0 1 − 1
 
0 0 0 0

The above matrix is an echelon form of A and the number of non- zero rows in the
echelon form of A is 3. Hence we get rank (A) = 3.

9 9 9
1 2 3
Example: Find the rank of the matrix A = 
2 4 6
 
7 7 7
 R1 R R
Solution: Perform, R1 → , R3 → 3 , R4 → 4
9 2 7

1 1 1
1 2 3
A 
1 2 3
 
1 1 1

Perform, R3 → R3-R2, R4 → R4-R1, R2 → R2-R1.

1 1 1
0 1 2
A 
0 0 0
 
0 0 0

We get rank (A) = 2.

Example: Find the rank of the matrix

2 − 2 5 3
4 − 1 1 1 
A=  
3 − 2 3 4
 
1 − 3 7 6 

Solution: Perform, R1  R4

1 − 3 7 6 
4 − 1 1 1 
 
3 − 2 3 4
 
2 − 2 5 3

Perform, R2 → R2-4R1, R3 → R3-3R1, R4 → R4-2R1.

1 − 3 7 6 
0 11 − 27 − 23
 
0 7 − 18 − 14 
 
0 4 −9 −9 
Perform, R4 → (R4+R3) – R2

1 − 3 7 6 
0 11 − 27 − 23
 
0 7 − 18 − 14 
 
0 0 0 0 

R2 R
Perform, R2→ , R3 → 3
11 11

1 − 3 7 6 
0 1 − 27 − 23 
 11 11
0 1 − 18 −2 
 7 
0 0 0 0 

Perform, R3 → R3-R2

1 − 3 7 6 
0 1 − 27 − 23 
 11 11
0 0 9 1 
 77 11 
0 0 0 0 

77
Perform, R3 → R3
9

1 − 3 7 6 
0 1 − 27 − 23 
 11 11
0 0 1 7 
 9 
0 0 0 0 

We get rank (A) = 3.

Linear Equations-Consistency:

Definitions:

A system of m linear equations in n unknowns x1, x2, …, xn is a set of equations of

the form

a11 x1 + a12 x2 + … + a1n xn = b1

a21 x1 + a22 x2 + … + a2n xn = b2

… ... ... _______ (*) say.

... … ...

am1 x1 + am2 x2 + ... + amn xn = bm.

The aij are given numbers, which are called the coefficients of the system. The bi are
also given numbers.

(i) If the bi are all zero, then (*) is called a homogeneous system. If at least one bi is
not zero, then (*) is called a non-homogeneous system.

A solution of (*) is a set of numbers x1, x2, .., xn which satisfy all the m equations. If
the system (*) is homogeneous, it has at least one trivial solution x1 = 0, x2 = 0,…, xn
= 0.

From the definition of matrix multiplication, we can write the above system (*) as

AX = B

a11 a12 .... a1n 

a 
 21 a 22 .... a 2n 
where A = .... .... .... ....  is the coefficient matrix
 
.... .... .... .... 
a m1 a m2 .... a mn 
  mn
  x1   b1 
x  b 
X =  2 and B =  2 
   
  n 1   m 1
xn   b m 

 a11 a12 ..... a1 n b1 

 
 a 21 a 22 ..... a 2 n b2 
The matrix  A B or [A:B] =  .... .... .... .... .... 
 
 .... .... .... .... .... 
 
a m1 a m2 .... a mn bm 

is called the augmented matrix. The matrix  A B determines the system (*)

completely. The matrix equation AX = B may have no solution or unique solution or

infinite number of solutions. A system of equations having one or more solutions is
called a consistent system. Otherwise it is called inconsistent.

Consider a system of non-homogeneous linear equations AX = B.

i) If rank A  rank  A B , then the system is inconsistent.

ii) If rank A = rank  A B = number of unknowns, then the system has unique

solution.
iii) If rank A = rank  A B < number of unknowns, then the system has infinite

number of solutions.

Example: Test for consistency

x+y+z=6

x – y + 2z = 5

3x + y + z = 8
 1 1 1 : 6 
Solution: The augmented matrix  A B = 1 − 1 2 : 5 
 
3 1 1 : 8 

Perform R 2 → - R1 + R 2 and R 3 → -3R1 + R 3

1 1 1 : 6 
 A B = 0 −2 1 : −1 
 
0 −2 −2 : −10

Perform R 3 → -R 2 + R 3

1 1 1 : 6 
 A B = 0 −2 1 : −1
 
0 0 3 : −9 

We get rank (A) = 3 = rank  A B = the number of independent variables. Therefore

the given system of equations is consistent and has unique solution.

Example: Find the condition under which the system

3x + 4y + 5z = 

4x + 5y + 6z = 

5x + 6y + 7z = 

is consistent.

3 4 5 :  
Solution: The Augmented matrix  A B = 4 5 6 :  
5 6 7 :  
Perform, R2 → R2-R1, R3 → R3 – R2

3 4 5 :  
1 1 1 :  −  
 
1 1 1 :  −  

Perform, R1 R2

1 1 1 :  −  
3 4 5 :  

1 1 1 :  −  

Perform, R2 → R2-3R1, R3 → R3 – R1

1 1 1 :  − 
0 1 2 : 4 − 3 
 
0 0 0 :  − 2  +  

This implies that rank (A) = 2 and rank  A B = 2 only if  - 2 +  = 0.

The system is consistent only if  - 2 +  = 0.

Exercises and Answers

1 7 − 1
3 − 5 4 
1. Find the rank of the matrix A =  
4 2 3
 
1 5 2
Answer: rank (A) = 3.

1 2 1
2 1 0
2. Find the rank of the following matrix A = 
3 3 1
 
4 5 2
Answer: rank (A) = 3.
Further Reading
1. Hadley G (2002) “Linear Algebra”, Norosa Publ.
2. Gilbert Strang, “Linear Algebra and its Applicaitons”, Thomson Publ. 2007.
3. J B Fraleigh “A First Course in Abstract Algebra” (3/e).
4. Kreyzig E (2006) “Advanced Engineering”, Wiley Eastern.
5. G. Birkhoff and S. A. Maclane, A Survey of Modern Algebra, Macmillan.
Inverse Using Elementary Operations
It is also possible to find the inverse of a non-singular matrix using elementary
row (or column) operations. Consider a square matrix A of order n , and the
identity matrix I of the same order. We define an augmented matrix [ A | I ] by
appending the columns of I to the columns of A , on the right. Then we reduce
A in [ A | I ] to the identity matrix of order n by performing a series of
elementary row operations on the augmented matrix. When A has been reduced
to the identity matrix, I would have been reduced to A−1 . If it is not possible to
reduce A to the identity matrix, then its row reduced form contains at least one
zero row, which means that A is singular and hence has no inverse.

Example

1 2 
1. A =  
3 4 
1 2 1 0 
[A| I] =  
3 4 0 −1
1 2 1 0 
~ 
0 −2 −3 −1
1 2 1 0 
~ 
0 1 −3 / 2 −1 / 2 
1 0 −2 1 
~ 
0 1 −3 / 2 −1 / 2 
 −2 1 
 A = 3 
 2 −1
2 

0 1 1
2. A = 1 1 1
1 1 3
0 1 1 1 0 0 
[ A | I ] = 1 1 1 0 1 0 
 
1 1 3 0 0 1 
 1 1 1 0 1 0
~ 0 1 1 1 0 0
 
1 1 3 0 0 1 
1 1 1 0 1 0
~ 0 1 1 1 0 0
 
0 0 2 0 −1 1 
1 1 1 0 1 0 
~ 0 1 1 1 0 0 
 
0 0 1 0 −1 / 2 1 / 2 
1 1 1 0 1 0 
~ 0 1 0 1 1 / 2 −1 / 2 
 
0 0 1 0 −1 / 2 1 / 2 
1 0 0 −1 1 0 
~ 0 1 0 1 1 / 2 −1 / 2 
 
0 0 1 0 −1 / 2 1 / 2 

 −1 1 0 
 
−1 
A = 1 1 −1 
 2 2
 0 −1 1 
 2 2

5 8 1 
3. A =  0 2 1 
 4 3 −1
5 8 1 1 0 0
[ A | I ] = 0 2 1 0 1 0
 
 4 3 −1 0 0 1 
1 1.6 0.2 0.2 0 0 
~ 0 2 1 0 1 0
 
 4 3 −1 0 0 1 
 1 1.6 0.2 0.2 0 0
~ 0 2 1 0 1 0
 
0 −3.4 −1.8 −0.8 0 1 
1 1.6 0.2 0.2 0 0

~ 0 1 0.5 0 0.5 0 
 
0 −3.4 −1.8 −0.8 0 1 
1 1.6 0.2 0.2 0 0
~ 0 1 0.5 0 0.5 0 
 
0 0 −0.1 −0.8 1.7 1 
1 1.6 0.2 0.2 0 0 
~ 0 1 0.5 0 0.5 0 
 
0 0 1 8 −17 −10 
1 1.6 0.2 0.2 0 0 
~ 0 1 0 −4 9 5 
 
0 0 1 8 −17 −10 
1 1.6 0 −1.4 3.4 2 
~ 0 1 0 −4 9 5 
 
0 0 1 8 −17 −10 
1 0 0 5 −11
−6 
~ 0 1 0 −4 9 5 
 
0 0 1 8 −17 −10 
5 −11 −6 
 A−1 =  −4 9 5 
 
 8 −17 −10 
1 1 0
4. A =  2 −1 4

1 −2 4 
1 1 0 1 0 0
[ A | I ] = 2 −1 4 0 1 0 
 
1 −2 4 0 0 1 
 1 1 0 10 0
~ 0 −3 4 −2 1 0 
 
0 −3 4 −1 0 1 
1 1 0 1 0 0
~ 0 −3 4 −2 1 0 
 
0 0 0 1 −1 1 
As the last row of A has become zero, it is singular and the inverse cannot
be found.

Exercises
Determine the rank of the following matrices by using elementary row
operations:
1 −2 0 
6. 1 1 1  Ans. Rank = 2
 2 −1 1 
1 1 0
7.  2 −2 1 
 Ans. Rank = 3
1 −1 0 
1 1 1 1
8. 1 −1 1 2 
 Ans. Rank = 2
3 1 3 4 
1 3 1 1
 −1 1 2 2
9.   Ans. Rank = 4
2 1 −2 1 
 
1 2 −2 2 
1 1 1 
10. 1 0 1  Ans. Rank = 3
1 1 0 

Determine the inverse of the following matrices by using elementary row

operations:
 1 0 1 
1 1 0   2 2
1. A =  2 −2 1  Ans. A = 1
−1
0 −1 
 2 2
1 −1 0  0 1 −2 
 
1 0 1 
1 1 0   2 2
2. A = 1 −1 1  Ans. A −1 =  1 0 −1 
 2 2
1 −1 0  0 1 −1 
 
1 3 1 4 −7 −5
3. A =  −1 1 2  Ans. A−1 =  −2 4 3
 
 2 1 −2   3 −5 −4 
1 1 1   −1 1 1
4. A = 1 0 1  Ans. A−1 =  1

−1 0

1 1 0   1 0 −1
1 1 1 
A = 1 0 1  Ans.
 
1 1 0 
Gauss Elimination Method and Gauss Jordan Method

Introduction
In Gauss Elimination Method, we row-reduce the augmented matrix to row-
echelon form and use back substitution to write the solution. In Gauss Jordan
Method, the row-echelon form has an additional property that a column
containing a first entry 1 has zeroes everywhere else. This gives rise to a
direct solution without the need for back substitutions.

Objectives
o To solve a system of linear equations by Gauss Elimination Method.
o To solve a system of linear equations by Gauss Jordan Method.

Gauss elimination method:

This is the elementary elimination method which reduces the system of equations to
an equivalent upper triangular system. The method is well-adapted for computer
operations. We will explain it by considering a system of three equations for sake of
simplicity.

Consider the equations

a1 x + b1y + c1z = d1

a2x + b2y + c2z = d2 …………………… (i)

a3x + b3y + c3z = d3

Step I: To eliminate x from second and third equations.

 a2 
We eliminate x from the second equation by subtracting   times the first equation
 a1 
from the second equation. Similarly we eliminate x from the third equation by
 a3 
eliminating   times the first equation from the third equation. We thus, get the
 a1 
new system
 a1 x + b 1 y + c 1 z = d1

b '2 y + c '2 z = d '2 ……………………(ii)

b '3 y + c '3 z = d '3

Here the first equation is called the pivotal equation and a1 is called the first pivot
element.

Step II: To eliminate y from third equation in (ii).

 b '3 
We eliminate y from the third equation of (ii) by subtracting   times the second
 b ' 
 2
equation from the third equation. We thus, get the new system

a1 x + b1 y + c1z = d1

b '2 y + c '2 z = d '2 ………………..(iii)

c'3 z = d'3

Here the second equation is the pivotal equation and b'2 is the new pivot element.

Step III: To evaluate the unknowns

The values of x, y, z are found from the reduced system (iii) by back substitution.

This also can be derived from the augmented matrix of the system (i),

 a1 b1 c1 d1 
[A|B] = a b2 c2 d 2  …………………(iv)
 2
 a 3 b3 c3 d1 

Using elementary row transformation, reduce the coefficient matrix A to an upper

triangular matrix.

a1 b1 c1 d1 
That is., [A|B] ~  0 b '2 c '2 d'2  ………………..(v)
 0 0 c'3 d ' 3 
From (v) the system of linear equations is equivalent to

a1 x + b1 y + c1z = d1

b '2 y + c '2 z = d'2

c3 ''z = d3 ''

By back substitution we get z, y and x constituting the exact solution of the system (i).

Note: The method will fail if one of the pivot elements a1, b '2 or c3 '' vanishes. In
such cases the method can be modified by rearranging the rows so that the pivots are
non-zero. If this is impossible, then the matrix is singular and the equations have no
solution.

Example: Solve the following system of equations by Gauss elimination method.

5x + y + z + w = 4

x + 7y + z + w = 12

x + y + 6z + w = –5

x + y + z + 4w = –6

We shall write the augmented matrix by interchanging the first equation with the
fourth equation.

1 2 1 4 − 6 R1
1 7 1 1 12  R2

 A B  =  1 1 6 1 −5 R3
 
5 1 1 1 4 R4

Perform: R2 → R2 - R1, R3 → R3 – R1 and R4 → R4-5R1

 1 1 1 4 − 6
0 6 0 −3 18
 A B  = 
0 0 5 −3 1
 
0 −4 −4 −19 34 

2
Perform: R4 → R4 + R2
3

1 1 1 4 −6 
0 6 0 −3 18
 A B  = 
0 0 5 −3 1
 
0 0 −4 −21 46 

4
Perform: R4 → R4 + R3
5

1 1 1 4 −6 
0 6 0 −3 18 

0 0 5 −3 1
 A B  =  
 
 117 234 
0 0 0 − 
 5 5 

Perform R4 → 5R4

1 1 1 4 −6 
0 6 0 −3 18
 A B  = 
0 0 5 −3 1
 
0 0 0 − 117 234 

Now we have

1x + 1y + 1z + 4w = –6

6y + 0z – 3w = 18

5z – 3w = 1

–117w = 234
We get w = –2. By back substitution, we get z = –1, y = 2, x = 1.

Therefore (x, y, z, w) = (1, 2, –1, –2) is the required solution.

ITERATIVE METHODS:-
We shall now describe the iterative or indirect methods, which start from an approximation to
the true solution and if convergent, derive a sequence of closer approximations – the cycle of
computations being repeated till the required accuracy is obtained. This means that in a
direct method the amount of computation is fixed, while in an iterative method the amount of
computation depends on the accuracy required. In the case of matrices with a large number of
zero elements, it will be advantageous to use iterative methods which preserve these
elements.

Jacobi’s Method:
Let the system be given by
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1 

a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2 
a31 x1 + a32 x2 + a33 x3 + ... + a3n xn = b1 

. 
. 

a n1x 2 +a n2 x 2 +a n3 x 3 +...+a nn x n = b n 

In which the diagonal elements aii do not vanish. If this is not the case, then the equations
should be rearranged so that this condition is satisfied. Now, we rewrite the system above as
b1 a12 a a
x1 = − x2 − 13 x3 − ... − 1n xn
a11 a11 a11 a11
b2 a21 a a
x2 = − x1 − 23 x2 − ... − 2 n xn
a22 a22 a22 a22
.
.
bn an1 a a
xn = − x1 − n 2 x2 − ... − n ,n −1 xn −1
ann ann ann ann

Let x1(0) , x2(0) ,....., xn(0) be the initial approximation to the solution.

If x1( k ) , x2( k ) ,....., xn( k ) denotes the approximation to the solution at kth iteration, then the solution

at (k +1)th iteration is given by

 b1 a12 ( k ) a13 ( k ) a
x1( k +1) = − x2 − x3 − ... − 1n xn ( k )
a11 a11 a11 a11
b2 a21 ( k ) a23 ( k ) a
x2( k +1) = − x1 − x2 − ... − 2 n xn ( k )
a22 a22 a22 a22
.
.
bn an1 ( k ) an 2 ( k ) a
xn ( k +1) = − x1 − x2 − ... − n ,n −1 xn −1( k )
ann ann ann ann

This method is due to Jacobi and is called the method of simultaneous displacements.
Gauss Seidel Method:
In Jacobi’s method, to obtain the values at the (k+1)th iteration, only kth iteration values are
used. A simple modification to the above method sometimes yields faster convergence and is
described below.
Here, along with the kth iteration values, all available (k + 1)th iteration values are used. It is
clear, therefore, that this method uses an improved component as soon as it is available and it
is called the method of successive displacements, or the Gauss – Seidel method.

The Jacobi and Gauss – Seidel methods converge, for any choice of the first approximation
xj(0) (j= 1, 2, …, n), if every equation of the system satisfies the condition that sum of the
absolute values of the coefficients aij/aii is almost equal to, or in at least one equation less
than unity, i.e. provided that
n
aij

j = 1, j1 aii
 1, ( i =1, 2,..., n )

where the “ < ” sign should be valid in the case of ‘at least’ one equation. A matrix A
satisfying the above condition is known as a diagonally dominant matrix. It can be shown
that the Gauss – Seidel method converges twice as fast as the Jacobi method. The working of
the methods is illustrated in the following examples.

Example 3:
10x1 – 2x2 – x3 – x4 = 3
- 2x1 + 10x2 – x3 – x4 = 15
- x1 – x2 + 10x3 – 2x4 = 27
- x1 – x2 – 2x3 + 10x4 = - 9
To solve these equations by the iterative methods, we rewrite them as follows:
x1 = 0.3 + 0.2x2 + 0.1x3 + 0.1x4
 x2 = 1.5 + 0.2x1 + 0.1x3 + 0.1x4
x3 = 2.7 + 0.1x1 + 0.1x2 + 0.2x4
x4 = - 0.9 + 0.1x1 + 0.1x2 + 0.2x3

It can be verified that the coefficient matrix is diagonally dominant. The results are given in
the following table:
Gauss – Seidel Method
n x1 x2 x3 x4
1 0.3 1.56 2.886 - 0.1368
2 0.8869 1.9523 2.9566 - 0.0248
3 0.9836 1.9899 2.9924 - 0.0042
4 0.9968 1.9982 2.9987 - 0.0008
5 0.99994 1.9997 2.9998 - 0.0001
6 0.9999 1.9999 3.0 0.0
7 1.0 2.0 3.0 0.0

Jacobi Method
n x1 x2 x3 x4
1 0.3 1.5 2.7 - 0.9
2 0.78 1.74 2.7 - 0.18
3 0.9 1.908 2.916 - 0.108
4 0.9624 1.9608 2.9592 - 0.036
5 0.9845 1.9848 2.9851 - 0.0158
6 0.9939 1.9935 2.9938 - 0.006
7 0.9975 1.9975 2.9976 - 0.0025
8 0.9990 1.9990 2.9990 - 0.0010
9 0.9996 1.9996 2.9996 - 0.0004
10 0.9998 1.9998 2.9998 - 0.0002
11 0.9999 1.9999 2.9999 - 0.0001
12 1.0 2.0 3.0 0.0
From above two tables, it is clear that twelve iterations are required by Jacobi method to
achieve the same accuracy as seven Gauss – Seidel iterations.
Example 4:
Solve the system
6x + y + z = 20
x + 4y – z = 6
x – y + 5z = 7
using Jacobi and Gauss – Seidel Method.
(a) Jacobi method:
We rewrite the given system as
20 1 1
x= - y - z = 3.3333 - 0.167y - 0.1667z
6 6 6
y = 1.5 – 0.25x + 0.25z
z = 1.4 – 0.2x + 0.2y
In matrix form, the above system may be written as
X = C + BX
where
3.3333  0 - 0.1667 - 0.1667  x
C =  1.5  , B = - 0.25
 0 
0.25  and X=  y 
 1.4   − 0.2 0.2 0   z 

Assuming
3.3333
X =  1.5  , We obtain
(0)

 1.4 

3.3333  0 - 0.1667 - 0.1667  3.3333  2.8499

X (1)   
=  1.5  + - 0.25 0 0.25   1.5  = 1.0167 
 1.4   − 0.2 0.2 0   1.4  1.0333 

3.3333  0 - 0.1667 - 0.1667   2.8499  2.9647 

X ( 2)   
=  1.5  + - 0.25 0 0.25  1.0167  = 1.0458 
 1.4   − 0.2 0.2 0  1.0333  1.0656 

Proceeding in this way, we obtain

 2.9991  2.9995
X (8)
= 1.0012  and X = 1.0005 
  (9)

1.0010  1.0004 

We, therefore, conclude that

  3
X = 1  i.e. x = 3, y =1 and z = 1
1

(b) Gauss Seidel Method:

As before, we obtain the first approximation as
 2.8499 
X (1)
= 1.0167 
1.0333 

The subsequent iteration values are given by

x(2)= 2.9916, y(2) = 1.0104, z(2)= 1.0038,
x(3) = 2.9975, y(3)= 1.0016, z(2)= 1.0008,
x(4)= 2.9995, y(4)= 1.0003, z(4) = 1.0002
from which we conclude that
x = 3, y = 1 and z = 1.