Introductory Linear Algebra/Matrices

Motivation

One important application for matrices is solving systems of linear equations. Some of the following definitions may be viewed as 'designed for solving system of linear equations'.

Some terminologies

Definition.

A matrix with m rows and n columns.

(Matrix) A matrix (plural: matrices) is a rectangular array of numbers. A horizontal unit is a row, and a vertical unit is a column. The element in the $\color {blue}i$th row and the $\color {red}j$th column is the $({\color {blue}i},{\color {red}j})$th entry of the matrix.

An $m\times n$ (read 'm by n') is a matrix with $m$ rows and $n$ columns, and $m\times n$ is the size of the matrix. The rows are counted from the top, and the columns are counted from the left. If the size of a matrix is $1\times 1$, we simply refer to this matrix as a number, and no brackets are needed in this case. The set of all $m\times n$ matrices with real entries is denoted by $M_{m\times n}(\mathbb {R} )$. A capital letter is usually used to denote a matrix, while small letters are used to denote its entries. For example, $A=(a_{ij})_{m\times n}$ denotes an $m\times n$ matrix $A$ with entries $a_{ij}$ in which $1\leq i\leq \underbrace {m} _{\text{no. of rows}}$ and $1\leq j\leq \underbrace {n} _{\text{no. of columns}}$. (We may omit the subscript specifying the size of matrix if its size is already mentioned, or its size is not important.)

Definition. (Matrix equality) Two matrices $A=(a_{ij})_{m\times n}$ and $B=(b_{ij})_{k\times \ell }$ are equal if

1. $m=k$,
2. $n=\ell$ and
3. $a_{ij}=b_{ij}$ for each pair $(i,j)$.

We write $A=B$ if $A$ and $B$ are equal.

Remark.

• In other words, if two matrices have the same size and same entries, then they are equal.
• If $A$ and $B$ are not equal, we write $A\neq B$.

Exercise. Consider the following three matrices $A,B$ and $C$.

$$A=(a_{ij})_{m\times n}={\begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}},\quad B=(b_{ij})_{k\times \ell }={\begin{pmatrix}1&4\\2&5\\3&6\end{pmatrix}}\;{\text{and}}\quad C=(c_{ij})_{p\times q}={\begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}}$$

1 Choose correct statement(s) from the following statements.

	$A=B$
	$A=C$
	$B=C$
	$A=B=C$
	$A\neq B\neq C$

2 Choose correct statement(s) from the following statements.

	$a_{12}=2$
	$b_{23}=6$
	$a_{31}=c_{31}=3$
	$a_{ij}=b_{ji}$ for each pair $(i,j)$

3 Choose correct statement(s) from the following statements.

	$m=p=2$
	$n=k=2$
	$m=\ell =3$
	$p=q=2$

Exercise. Let $A=(a_{ij})$ be a $3\times 2$ matrix in which each entry $a_{ij}=i+2j$. Write down $A$ in the form of an array of numbers.

(a11 a12 )

(a21 a22)

(a31 a32)

$A={\begin{pmatrix}3&5\\4&6\\5&7\end{pmatrix}}.$

In particular, if a matrix has the same number of rows and columns, then it has some nice properties. In view of the shape of such a matrix (square-like), we define such matrices as square matrices.

Definition. (Square matrix) A square matrix is a matrix with the same number of rows and columns.

We will also introduce a term, namely main diagonal, which will be useful in some situations.

Definition. (Main diagonal) The main diagonal of an $n\times n$ matrix (which is a square matrix) is the collection of the $(1,1)$th, $(2,2)$th, $\ldots$, $(n,n)$th entries.

Example. The main diagonal of the matrix $I_{3}={\begin{pmatrix}{\color {green}1}&0&0\\0&{\color {green}1}&0\\0&0&{\color {green}1}\end{pmatrix}}$ is the collection of $1,1$ and $1$.

Remark. The matrix $I_{3}$ is the identity matrix (which will be defined later).

Then, we will define some types of matrices for which the definitions are related to the main diagonal.

Definition. (Triangular matrix) A triangular matrix is an upper triangular matrix or a lower triangular matrix (inclusively).

An upper triangular matrix is a square matrix whose entries below its main diagonal are all $0$.

A lower triangular matrix is a square matrix whose entries above its main diagonal are all $0$.

Remark.

• Equivalently and symbolically, a matrix $A=(a_{ij})$ is upper triangular if $a_{ij}=0$ whenever $i>j$,

and is lower triangular if $a_{ij}=0$ if $i<j$.

• Upper triangular and lower triangular matrices are in the form of

$\underbrace {\begin{pmatrix}*&*&*&\cdots &*\\0&*&*&\cdots &*\\0&0&\ddots &\ddots &\vdots \\\vdots &\vdots &\ddots &\ddots &*\\0&0&\cdots &0&*\end{pmatrix}} _{\text{Upper triangular matrix}}\quad {\text{and}}\quad \underbrace {\begin{pmatrix}*&0&\cdots &0&0\\*&*&0&\cdots &0\\*&*&\ddots &\ddots &\vdots \\\vdots &\vdots &\ddots &\ddots &0\\*&*&\cdots &*&*\end{pmatrix}} _{\text{Lower triangular matrix}}$ respectively, in which $*$ is an arbitrary entry (which may or may not be zero).

Definition. (Diagonal matrix) A diagonal matrix is a square matrix whose entries not lying on the main diagonal are all $0$.

Remark.

• A diagonal matrix is both upper triangular and lower triangular.
• A diagonal matrix has the form

${\begin{pmatrix}*&0&0&\cdots &0\\0&*&0&\cdots &0\\0&0&\ddots &\ddots &\vdots \\\vdots &\vdots &\ddots &\ddots &0\\0&0&0&\cdots &*\end{pmatrix}}$ in which $*$ is an arbitrary entry.

Exercise.

1 Choose all correct statement(s) from the following statements.

	A matrix whose every entry is $0$ is diagonal matrix.
	A matrix whose every entry is $0$ is triangular matrix.
	A diagonal matrix is triangular matrix.
	A triangular matrix is diagonal matrix.
	A diagonal matrix is the only type of matrix that is both upper triangular and lower triangular.

2 Choose all upper triangular matrices from the following matrices.

	${\begin{pmatrix}1&2&2\\0&2&3\\0&0&1\end{pmatrix}}$
	${\begin{pmatrix}2&0&0\\1&2&0\\3&5&1\end{pmatrix}}$
	${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}$
	${\begin{pmatrix}1&0&0\\0&2&0\\0&0&3\end{pmatrix}}$
	${\begin{pmatrix}0&1&0&0\\0&0&2&0\\0&0&0&3\end{pmatrix}}$

3 Choose all lower triangular matrices from the following matrices.

	${\begin{pmatrix}1&2&2\\0&2&3\\0&0&1\end{pmatrix}}$
	${\begin{pmatrix}2&0&0\\1&2&0\\3&5&1\end{pmatrix}}$
	${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}$
	${\begin{pmatrix}1&0&0\\0&2&0\\0&0&3\end{pmatrix}}$
	${\begin{pmatrix}0&1&0&0\\0&0&2&0\\0&0&0&3\end{pmatrix}}$

4 Choose all triangular matrices from the following matrices.

	${\begin{pmatrix}1&2&2\\0&2&3\\0&0&1\end{pmatrix}}$
	${\begin{pmatrix}2&0&0\\1&2&0\\3&5&1\end{pmatrix}}$
	${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}$
	${\begin{pmatrix}1&0&0\\0&2&0\\0&0&3\end{pmatrix}}$
	${\begin{pmatrix}0&1&0&0\\0&0&2&0\\0&0&0&3\end{pmatrix}}$

5 Choose all diagonal matrices from the following matrices.

	${\begin{pmatrix}1&2&2\\0&2&3\\0&0&1\end{pmatrix}}$
	${\begin{pmatrix}2&0&0\\1&2&0\\3&5&1\end{pmatrix}}$
	${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}$
	${\begin{pmatrix}1&0&0\\0&2&0\\0&0&3\end{pmatrix}}$
	${\begin{pmatrix}0&1&0&0\\0&0&2&0\\0&0&0&3\end{pmatrix}}$

The last terminology we mention here is submatrix, which will sometimes be used.

Definition. (Submatrix) Let $A$ be a matrix. A submatrix of $A$ is a matrix obtained from $A$ by removing some rows or columns (inclusively).

Remark. By convention, every matrix is a submatrix of itself.

Exercise.

Choose all submatrices of ${\begin{pmatrix}3&5&7\\4&6&8\\5&7&9\end{pmatrix}}$ from the following matrices.

	${\begin{pmatrix}3&5&7\end{pmatrix}}$
	${\begin{pmatrix}3&4&5\end{pmatrix}}$
	${\begin{pmatrix}3\\4\\5\end{pmatrix}}$
	2
	7
	${\begin{pmatrix}3&7\\5&9\end{pmatrix}}$

Matrix operations

In this section, we will cover different matrix operations. Some operations are quite different from that in the number system, in particular, matrix multiplication.

Definition. (Matrix addition and subtraction) Let $A=(a_{ij})_{m\times n}$ and $B=(b_{ij})_{m\times n}$ be two matrices of the same size. We define matrix addition and subtraction by

$$A\pm B=(a_{ij}\pm b_{ij})_{m\times n}.$$

Definition. (Scalar multiplication of matrix) Let $A=(a_{ij})_{m\times n}$ be a matrix. We define the scalar multiplication of the matrix by

$$cA=(ca_{ij})_{m\times n}.$$

Then, we are going to define matrix multiplication, which is quite different from the multiplication in the number system.

Definition.

Illustration of matrix multiplication.

(Matrix multiplication) Let $A=(a_{ij})_{{\color {blue}m}\times {\color {green}n}}$ and $B=(b_{ij})_{{\color {green}n}\times {\color {red}\ell }}$ be two matrices. The matrix product $AB$ of $A$ and $B$ is defined as the ${\color {blue}m}\times {\color {red}\ell }$ matrix whose $({\color {purple}i},{\color {brown}j})$th entry is

$$\sum _{k=1}^{n}(a_{{\color {purple}i}k}b_{k{\color {brown}j}})=a_{{\color {purple}i}1}b_{1{\color {brown}j}}+a_{{\color {purple}i}2}b_{2{\color {brown}j}}+\cdots +a_{{\color {purple}i}n}b_{n{\color {brown}j}}.$$

If the number of columns of $A$ ($\color {blue}m$) is different from the number of rows of $B$ ($\color {red}\ell$), then the product $AB$ is not defined.

On the other hand, a positive power of a square matrix is defined quite similarly to that in number system.

Definition. (Positive power of a square matrix) Let $A$ be a square matrix. The $p$th power of $A$, written $A^{p}$, in which $p$ is a positive number, is the product of $p$ copies of $A$, i.e.,

$$A^{p}=\underbrace {AA\cdots A.} _{{\text{product of}}\;p\;{\text{copies of}}\;A}$$

Exercise.

Choose all correct statement(s) from the following statements.

	${\begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}}{\begin{pmatrix}1&2\\3&4\\5&6\end{pmatrix}}={\begin{pmatrix}22&28\\49&64\end{pmatrix}}$.
	${\begin{pmatrix}1&2\\3&4\\5&6\end{pmatrix}}{\begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}}={\begin{pmatrix}22&28\\49&64\end{pmatrix}}$
	$(a_{ij})_{n\times n}^{2}=(a_{ij}^{2})_{n\times n}$.
	$A+B=B+A$ for each matrix $A$ and $B$.
	$((ij)a_{ij})_{m\times n}=ij(a_{ij})_{m\times n}$.

Then, we will discuss matrix analogs for the numbers zero and one in the number system, namely the zero matrix and the identity matrix, which, in the number system, are analogous to the numbers $0$ and $1$ respectively.

Definition. (Zero matrix) The zero matrix is the $m\times n$ matrix whose entries are all $0$, and is denoted by $O_{m\times n}$ or simply $O$ if there is no ambiguity.

Remark. The zero matrix is analogous to the number $0$ in the number system, because:

1. We have $O+A=A+O=A$ for each matrix $A$ of the same size as the zero matrix.
2. We have $OA=AO=O$ if the products are well-defined, for each matrix $A$.

Definition. (Identity matrix) The $n\times n$ identity matrix, denoted by $I_{n}$ or simply $I$ if there is no ambiguity, is the $n\times n$ diagonal matrix whose diagonal entries are all ${\color {green}1}$.

Remark. The identity matrix is analogous to the number $1$ in the number system, because $AI=IA=A$ if the products are well-defined, for each matrix $A$.

Example.

• the zero matrix $O_{2\times 1}$ is ${\begin{pmatrix}0\\0\end{pmatrix}}$
• the identity matrix $I_{2}$ is ${\begin{pmatrix}1&0\\0&1\end{pmatrix}}$

Proposition. (Properties of matrix operations) Let $A,B$ and $C$ be matrices such that the following operations are well-defined, and let $c$ be a scalar. Then, the following hold.

(i) (associativity of matrix multiplication) $(AB)C=A(BC)$.

(ii) (roles of numbers zero and one) $AO=O,\,OB=O,\,AI=A,\,IB=B$

(iii) (distributivity of matrix multiplication) $\underbrace {A(B+C)=AB+AC} _{\text{left distributivity}},\,\underbrace {(A+B)C=AC+BC} _{\text{right distributivity}}$

(iv) $c(AB)=(cA)B=A(cB)$.

Remark. Matrix multiplication is not commutative in general, i.e., the matrix product $AB$ is different from the matrix product $BA$ in general.

Then, we will introduce an operation that does not exist in the number system, namely transpose.

Definition. (Matrix transpose) Let $A=(a_{{\color {purple}i}{\color {green}j}})_{{\color {blue}m}\times {\color {red}n}}$ be a matrix. The transpose of matrix $A$ is the matrix

$$A^{T}=(a_{{\color {green}j}{\color {purple}i}})_{{\color {red}n}\times {\color {blue}m}}.$$

Remark. We can see from the definition that the transpose of $1\times 1$ matrix is simply itself.

Example.

Illustration of this example.

Let $A$ be the matrix ${\begin{pmatrix}1&2\\3&4\\5&6\end{pmatrix}}$. Then,

$$A^{T}={\begin{pmatrix}1&2\\3&4\\5&6\end{pmatrix}}^{T}={\begin{pmatrix}1&3&5\\2&4&6\end{pmatrix}}\quad {\text{and}}\quad \left(A^{T}\right)^{T}={\begin{pmatrix}1&3&5\\2&4&6\end{pmatrix}}^{T}={\begin{pmatrix}1&2\\3&4\\5&6\end{pmatrix}}=A.$$

Proposition. (Properties of matrix transpose) Let $A$ and $B$ be matrices such that the following operations are well-defined. Then, the following hold.

(i) (self-invertibility) $(A^{T})^{T}=A$

(ii) (linearity) $(aA+bB)^{T}=aA^{T}+bB^{T}.$ for each real number $a$ and $b$

(iii) ('reverse multiplicativity') $\color {green}(AB)^{T}=B^{T}A^{T}$

Definition.

Symmetry pattern of a $5\times 5$ symmetric matrix.

(Symmetric matrix) A matrix $A$ is a symmetric matrix if

$$A^{T}=A.$$

Definition. (Skew-symmetric matrix) A matrix $A$ is a skew-symmetric matrix if

$$A^{T}={\color {green}-}A.$$

Proposition. (Necessary condition for a symmetric and a skew-symmetric matrix) A symmetric matrix or a skew-symmetric matrix must be a square matrix.

Proof. It follows from observing that matrix transpose has the same size as the original matrix if and only if the matrix is square matrix, since the number of rows and number of columns are swapped for the matrix transpose, and the size remains unchanged if and only if the number of rows equals number of columns.

$\Box$

Remark. This does not imply that every square matrix is a symmetric matrix or a skew-symmetric matrix. The condition that a matrix is a square matrix itself is not sufficient (but necessary) for it to be symmetric or skew-symmetric.

Example. ${\begin{pmatrix}1&{\color {blue}2}&{\color {red}3}\\{\color {blue}2}&8&{\color {green}4}\\{\color {red}3}&{\color {green}4}&5\end{pmatrix}}$ is symmetric, while ${\begin{pmatrix}0&{\color {blue}2}&-{\color {red}3}\\-{\color {blue}2}&0&-{\color {green}4}\\{\color {red}3}&{\color {green}4}&0\end{pmatrix}}$ is skew-symmetric, and its transpose is ${\begin{pmatrix}0&-{\color {blue}2}&{\color {red}3}\\{\color {blue}2}&0&{\color {green}4}\\-{\color {red}3}&-{\color {green}4}&0\end{pmatrix}}$. In a skew-symmetric matrix, all entries lying on the main diagonal must be 0.