Linear Algebra/Print version/Part 2

Chapter III - Maps Between Spaces

Section I - Isomorphisms

In the examples following the definition of a vector space we developed the intuition that some spaces are "the same" as others. For instance, the space of two-tall column vectors and the space of two-wide row vectors are not equal because their elements—column vectors and row vectors—are not equal, but we have the idea that these spaces differ only in how their elements appear. We will now make this idea precise.

This section illustrates a common aspect of a mathematical investigation. With the help of some examples, we've gotten an idea. We will next give a formal definition, and then we will produce some results backing our contention that the definition captures the idea. We've seen this happen already, for instance, in the first section of the Vector Space chapter. There, the study of linear systems led us to consider collections closed under linear combinations. We defined such a collection as a vector space, and we followed it with some supporting results.

Of course, that definition wasn't an end point, instead it led to new insights such as the idea of a basis. Here too, after producing a definition, and supporting it, we will get two surprises (pleasant ones). First, we will find that the definition applies to some unforeseen, and interesting, cases. Second, the study of the definition will lead to new ideas. In this way, our investigation will build a momentum.

1 - Definition and Examples

We start with two examples that suggest the right definition.

Example 1.1

Consider the example mentioned above, the space of two-wide row vectors and the space of two-tall column vectors. They are "the same" in that if we associate the vectors that have the same components, e.g.,

${\begin{pmatrix}1&2\end{pmatrix}}\quad \longleftrightarrow \quad {\begin{pmatrix}1\\2\end{pmatrix}}$

then this correspondence preserves the operations, for instance this addition

${\begin{pmatrix}1&2\end{pmatrix}}+{\begin{pmatrix}3&4\end{pmatrix}}={\begin{pmatrix}4&6\end{pmatrix}}\quad \longleftrightarrow \quad {\begin{pmatrix}1\\2\end{pmatrix}}+{\begin{pmatrix}3\\4\end{pmatrix}}={\begin{pmatrix}4\\6\end{pmatrix}}$

and this scalar multiplication.

$5\cdot {\begin{pmatrix}1&2\end{pmatrix}}={\begin{pmatrix}5&10\end{pmatrix}}\quad \longleftrightarrow \quad 5\cdot {\begin{pmatrix}1\\2\end{pmatrix}}={\begin{pmatrix}5\\10\end{pmatrix}}$

More generally stated, under the correspondence

${\begin{pmatrix}a_{0}&a_{1}\end{pmatrix}}\quad \longleftrightarrow \quad {\begin{pmatrix}a_{0}\\a_{1}\end{pmatrix}}$

both operations are preserved:

${\begin{pmatrix}a_{0}&a_{1}\end{pmatrix}}+{\begin{pmatrix}b_{0}&b_{1}\end{pmatrix}}={\begin{pmatrix}a_{0}+b_{0}&a_{1}+b_{1}\end{pmatrix}}\longleftrightarrow {\begin{pmatrix}a_{0}\\a_{1}\end{pmatrix}}+{\begin{pmatrix}b_{0}\\b_{1}\end{pmatrix}}={\begin{pmatrix}a_{0}+b_{0}\\a_{1}+b_{1}\end{pmatrix}}$

and

$r\cdot {\begin{pmatrix}a_{0}&a_{1}\end{pmatrix}}={\begin{pmatrix}ra_{0}&ra_{1}\end{pmatrix}}\quad \longleftrightarrow \quad r\cdot {\begin{pmatrix}a_{0}\\a_{1}\end{pmatrix}}={\begin{pmatrix}ra_{0}\\ra_{1}\end{pmatrix}}$

(all of the variables are real numbers).

Example 1.2

Another two spaces we can think of as "the same" are ${\mathcal {P}}_{2}$, the space of quadratic polynomials, and $\mathbb {R} ^{3}$. A natural correspondence is this.

$a_{0}+a_{1}x+a_{2}x^{2}\quad \longleftrightarrow \quad {\begin{pmatrix}a_{0}\\a_{1}\\a_{2}\end{pmatrix}}\qquad \qquad ({\text{e.g., }}1+2x+3x^{2}\,\longleftrightarrow \,{\begin{pmatrix}1\\2\\3\end{pmatrix}})$

The structure is preserved: corresponding elements add in a corresponding way

${\begin{array}{r}a_{0}+a_{1}x+a_{2}x^{2}\\+\,\,b_{0}+b_{1}x+b_{2}x^{2}\\\hline (a_{0}+b_{0})+(a_{1}+b_{1})x+(a_{2}+b_{2})x^{2}\end{array}}\quad \longleftrightarrow \quad {\begin{pmatrix}a_{0}\\a_{1}\\a_{2}\end{pmatrix}}+{\begin{pmatrix}b_{0}\\b_{1}\\b_{2}\end{pmatrix}}={\begin{pmatrix}a_{0}+b_{0}\\a_{1}+b_{1}\\a_{2}+b_{2}\end{pmatrix}}$

and scalar multiplication corresponds also.

$r\cdot (a_{0}+a_{1}x+a_{2}x^{2})=(ra_{0})+(ra_{1})x+(ra_{2})x^{2}\quad \longleftrightarrow \quad r\cdot {\begin{pmatrix}a_{0}\\a_{1}\\a_{2}\end{pmatrix}}={\begin{pmatrix}ra_{0}\\ra_{1}\\ra_{2}\end{pmatrix}}$

Definition 1.3

An isomorphism between two vector spaces $V$ and $W$ is a map $f:V\to W$ that

1. is a correspondence: $f$ is one-to-one and onto;
2. preserves structure: if ${\vec {v}}_{1},{\vec {v}}_{2}\in V$ then

   $f({\vec {v}}_{1}+{\vec {v}}_{2})=f({\vec {v}}_{1})+f({\vec {v}}_{2})$

   and if ${\vec {v}}\in V$ and $r\in \mathbb {R}$ then

   $f(r{\vec {v}})=r\,f({\vec {v}})$

(we write $V\cong W$, read "$V$ is isomorphic to $W$", when such a map exists).

("Morphism" means map, so "isomorphism" means a map expressing sameness.)

Example 1.4

The vector space $G=\{c_{1}\cos \theta +c_{2}\sin \theta \,{\big |}\,c_{1},c_{2}\in \mathbb {R} \}$ of functions of $\theta$ is isomorphic to the vector space $\mathbb {R} ^{2}$ under this map.

$c_{1}\cos \theta +c_{2}\sin \theta {\stackrel {f}{\longmapsto }}{\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}$

We will check this by going through the conditions in the definition.

We will first verify condition 1, that the map is a correspondence between the sets underlying the spaces.

To establish that $f$ is one-to-one, we must prove that $f({\vec {a}})=f({\vec {b}})$ only when ${\vec {a}}={\vec {b}}$. If

$f(a_{1}\cos \theta +a_{2}\sin \theta )=f(b_{1}\cos \theta +b_{2}\sin \theta )$

then, by the definition of $f$,

${\begin{pmatrix}a_{1}\\a_{2}\end{pmatrix}}={\begin{pmatrix}b_{1}\\b_{2}\end{pmatrix}}$

from which we can conclude that $a_{1}=b_{1}$ and $a_{2}=b_{2}$ because column vectors are equal only when they have equal components. We've proved that $f({\vec {a}})=f({\vec {b}})$ implies that ${\vec {a}}={\vec {b}}$, which shows that $f$ is one-to-one.

To check that $f$ is onto we must check that any member of the codomain $\mathbb {R} ^{2}$ is the image of some member of the domain $G$. But that's clear—any

${\begin{pmatrix}x\\y\end{pmatrix}}\in \mathbb {R} ^{2}$

is the image under $f$ of $x\cos \theta +y\sin \theta \in G$.

Next we will verify condition (2), that $f$ preserves structure.

This computation shows that $f$ preserves addition.

$f{\bigl (}\,(a_{1}\cos \theta +a_{2}\sin \theta )+(b_{1}\cos \theta +b_{2}\sin \theta )\,{\bigr )}$

${\begin{array}{rl}&=f{\bigl (}\,(a_{1}+b_{1})\cos \theta +(a_{2}+b_{2})\sin \theta \,{\bigr )}\\&={\begin{pmatrix}a_{1}+b_{1}\\a_{2}+b_{2}\end{pmatrix}}\\&={\begin{pmatrix}a_{1}\\a_{2}\end{pmatrix}}+{\begin{pmatrix}b_{1}\\b_{2}\end{pmatrix}}\\&=f(a_{1}\cos \theta +a_{2}\sin \theta )+f(b_{1}\cos \theta +b_{2}\sin \theta )\end{array}}$

A similar computation shows that $f$ preserves scalar multiplication.

${\begin{array}{rl}f{\bigl (}\,r\cdot (a_{1}\cos \theta +a_{2}\sin \theta )\,{\bigr )}&=f(\,ra_{1}\cos \theta +ra_{2}\sin \theta \,)\\&={\begin{pmatrix}ra_{1}\\ra_{2}\end{pmatrix}}\\&=r\cdot {\begin{pmatrix}a_{1}\\a_{2}\end{pmatrix}}\\&=r\cdot \,f(a_{1}\cos \theta +a_{2}\sin \theta )\end{array}}$

With that, conditions (1) and (2) are verified, so we know that $f$ is an isomorphism and we can say that the spaces are isomorphic $G\cong \mathbb {R} ^{2}$.

Example 1.5

Let $V$ be the space $\{c_{1}x+c_{2}y+c_{3}z\,{\big |}\,c_{1},c_{2},c_{3}\in \mathbb {R} \}$ of linear combinations of three variables $x$, $y$, and $z$, under the natural addition and scalar multiplication operations. Then $V$ is isomorphic to ${\mathcal {P}}_{2}$, the space of quadratic polynomials.

To show this we will produce an isomorphism map. There is more than one possibility; for instance, here are four.

${\begin{array}{c}c_{1}x+c_{2}y+c_{3}z\end{array}}\quad {\begin{array}{rl}{\stackrel {f_{1}}{\longmapsto }}&c_{1}+c_{2}x+c_{3}x^{2}\\{\stackrel {f_{2}}{\longmapsto }}&c_{2}+c_{3}x+c_{1}x^{2}\\{\stackrel {f_{3}}{\longmapsto }}&-c_{1}-c_{2}x-c_{3}x^{2}\\{\stackrel {f_{4}}{\longmapsto }}&c_{1}+(c_{1}+c_{2})x+(c_{1}+c_{3})x^{2}\end{array}}$

The first map is the more natural correspondence in that it just carries the coefficients over. However, below we shall verify that the second one is an isomorphism, to underline that there are isomorphisms other than just the obvious one (showing that $f_{1}$ is an isomorphism is Problem 3).

To show that $f_{2}$ is one-to-one, we will prove that if $f_{2}(c_{1}x+c_{2}y+c_{3}z)=f_{2}(d_{1}x+d_{2}y+d_{3}z)$ then $c_{1}x+c_{2}y+c_{3}z=d_{1}x+d_{2}y+d_{3}z$. The assumption that $f_{2}(c_{1}x+c_{2}y+c_{3}z)=f_{2}(d_{1}x+d_{2}y+d_{3}z)$ gives, by the definition of $f_{2}$, that $c_{2}+c_{3}x+c_{1}x^{2}=d_{2}+d_{3}x+d_{1}x^{2}$. Equal polynomials have equal coefficients, so $c_{2}=d_{2}$, $c_{3}=d_{3}$, and $c_{1}=d_{1}$. Thus $f_{2}(c_{1}x+c_{2}y+c_{3}z)=f_{2}(d_{1}x+d_{2}y+d_{3}z)$ implies that $c_{1}x+c_{2}y+c_{3}z=d_{1}x+d_{2}y+d_{3}z$ and therefore $f_{2}$ is one-to-one.

The map $f_{2}$ is onto because any member $a+bx+cx^{2}$ of the codomain is the image of some member of the domain, namely it is the image of $cx+ay+bz$. For instance, $2+3x-4x^{2}$ is $f_{2}(-4x+2y+3z)$.

The computations for structure preservation are like those in the prior example. This map preserves addition

$f_{2}{\bigl (}(c_{1}x+c_{2}y+c_{3}z)+(d_{1}x+d_{2}y+d_{3}z){\bigr )}$

${\begin{array}{rl}&=f_{2}{\bigl (}(c_{1}+d_{1})x+(c_{2}+d_{2})y+(c_{3}+d_{3})z{\bigr )}\\&=(c_{2}+d_{2})+(c_{3}+d_{3})x+(c_{1}+d_{1})x^{2}\\&=(c_{2}+c_{3}x+c_{1}x^{2})+(d_{2}+d_{3}x+d_{1}x^{2})\\&=f_{2}(c_{1}x+c_{2}y+c_{3}z)+f_{2}(d_{1}x+d_{2}y+d_{3}z)\end{array}}$

and scalar multiplication.

${\begin{array}{rl}f_{2}{\bigl (}r\cdot (c_{1}x+c_{2}y+c_{3}z){\bigr )}&=f_{2}(rc_{1}x+rc_{2}y+rc_{3}z)\\&=rc_{2}+rc_{3}x+rc_{1}x^{2}\\&=r\cdot (c_{2}+c_{3}x+c_{1}x^{2})\\&=r\cdot \,f_{2}(c_{1}x+c_{2}y+c_{3}z)\end{array}}$

Thus $f_{2}$ is an isomorphism and we write $V\cong {\mathcal {P}}_{2}$.

We are sometimes interested in an isomorphism of a space with itself, called an automorphism. An identity map is an automorphism. The next two examples show that there are others.

Example 1.6

A dilation map $d_{s}:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that multiplies all vectors by a nonzero scalar $s$ is an automorphism of $\mathbb {R} ^{2}$.

A rotation or turning map $t_{\theta }:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that rotates all vectors through an angle $\theta$ is an automorphism.

A third type of automorphism of $\mathbb {R} ^{2}$ is a map $f_{\ell }:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that flips or reflects all vectors over a line $\ell$ through the origin.

See Problem 20.

Example 1.7

Consider the space ${\mathcal {P}}_{5}$ of polynomials of degree 5 or less and the map $f$ that sends a polynomial $p(x)$ to $p(x-1)$. For instance, under this map $x^{2}\mapsto (x-1)^{2}=x^{2}-2x+1$ and $x^{3}+2x\mapsto (x-1)^{3}+2(x-1)=x^{3}-3x^{2}+5x-3$. This map is an automorphism of this space; the check is Problem 12.

This isomorphism of ${\mathcal {P}}_{5}$ with itself does more than just tell us that the space is "the same" as itself. It gives us some insight into the space's structure. For instance, below is shown a family of parabolas, graphs of members of ${\mathcal {P}}_{5}$. Each has a vertex at $y=-1$, and the left-most one has zeroes at $-2.25$ and $-1.75$, the next one has zeroes at $-1.25$ and $-0.75$, etc.

Geometrically, the substitution of $x-1$ for $x$ in any function's argument shifts its graph to the right by one. Thus, $f(p_{0})=p_{1}$ and $f$'s action is to shift all of the parabolas to the right by one. Notice that the picture before $f$ is applied is the same as the picture after $f$ is applied, because while each parabola moves to the right, another one comes in from the left to take its place. This also holds true for cubics, etc. So the automorphism $f$ gives us the insight that $P_{5}$ has a certain horizontal homogeneity; this space looks the same near $x=1$ as near $x=0$.

As described in the preamble to this section, we will next produce some results supporting the contention that the definition of isomorphism above captures our intuition of vector spaces being the same.

Of course the definition itself is persuasive: a vector space consists of two components, a set and some structure, and the definition simply requires that the sets correspond and that the structures correspond also. Also persuasive are the examples above. In particular, Example 1.1, which gives an isomorphism between the space of two-wide row vectors and the space of two-tall column vectors, dramatizes our intuition that isomorphic spaces are the same in all relevant respects. Sometimes people say, where $V\cong W$, that "$W$ is just $V$ painted green"—any differences are merely cosmetic.

Further support for the definition, in case it is needed, is provided by the following results that, taken together, suggest that all the things of interest in a vector space correspond under an isomorphism. Since we studied vector spaces to study linear combinations, "of interest" means "pertaining to linear combinations". Not of interest is the way that the vectors are presented typographically (or their color!).

As an example, although the definition of isomorphism doesn't explicitly say that the zero vectors must correspond, it is a consequence of that definition.

Lemma 1.8

An isomorphism maps a zero vector to a zero vector.

Proof

Where $f:V\to W$ is an isomorphism, fix any ${\vec {v}}\in V$. Then $f({\vec {0}}_{V})=f(0\cdot {\vec {v}})=0\cdot f({\vec {v}})={\vec {0}}_{W}$.

The definition of isomorphism requires that sums of two vectors correspond and that so do scalar multiples. We can extend that to say that all linear combinations correspond.

Lemma 1.9

For any map $f:V\to W$ between vector spaces these statements are equivalent.

1. $f$ preserves structure

   $f({\vec {v}}_{1}+{\vec {v}}_{2})=f({\vec {v}}_{1})+f({\vec {v}}_{2})\quad {\text{and}}\quad f(c{\vec {v}})=c\,f({\vec {v}})$

2. $f$ preserves linear combinations of two vectors

   $f(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})=c_{1}f({\vec {v}}_{1})+c_{2}f({\vec {v}}_{2})$

3. $f$ preserves linear combinations of any finite number of vectors

   $f(c_{1}{\vec {v}}_{1}+\dots +c_{n}{\vec {v}}_{n})=c_{1}f({\vec {v}}_{1})+\dots +c_{n}f({\vec {v}}_{n})$

Proof

Since the implications $3\!\implies \!2$ and $2\!\implies \!1$ are clear, we need only show that $1\!\implies \!3$. Assume statement 1. We will prove statement 3 by induction on the number of summands $n$.

The one-summand base case, that $f(c{\vec {v}}_{1})=c\,f({\vec {v}}_{1})$, is covered by the assumption of statement 1.

For the inductive step assume that statement 3 holds whenever there are $k$ or fewer summands, that is, whenever $n=1$, or $n=2$, ..., or $n=k$. Consider the $k+1$-summand case. The first half of 1 gives

$f(c_{1}{\vec {v}}_{1}+\dots +c_{k}{\vec {v}}_{k}+c_{k+1}{\vec {v}}_{k+1})=f(c_{1}{\vec {v}}_{1}+\dots +c_{k}{\vec {v}}_{k})+f(c_{k+1}{\vec {v}}_{k+1})$

by breaking the sum along the final "$+$". Then the inductive hypothesis lets us break up the $k$-term sum.

$=f(c_{1}{\vec {v}}_{1})+\dots +f(c_{k}{\vec {v}}_{k})+f(c_{k+1}{\vec {v}}_{k+1})$

Finally, the second half of statement 1 gives

$=c_{1}\,f({\vec {v}}_{1})+\dots +c_{k}\,f({\vec {v}}_{k})+c_{k+1}\,f({\vec {v}}_{k+1})$

when applied $k+1$ times.

In addition to adding to the intuition that the definition of isomorphism does indeed preserve the things of interest in a vector space, that lemma's second item is an especially handy way of checking that a map preserves structure.

We close with a summary. The material in this section augments the chapter on Vector Spaces. There, after giving the definition of a vector space, we informally looked at what different things can happen. Here, we defined the relation "$\cong$" between vector spaces and we have argued that it is the right way to split the collection of vector spaces into cases because it preserves the features of interest in a vector space—in particular, it preserves linear combinations. That is, we have now said precisely what we mean by "the same", and by "different", and so we have precisely classified the vector spaces.

Exercises

This exercise is recommended for all readers.

Problem 1

Verify, using Example 1.4 as a model, that the two correspondences given before the definition are isomorphisms.

1. Example 1.1
2. Example 1.2

This exercise is recommended for all readers.

Problem 2

For the map $f:{\mathcal {P}}_{1}\to \mathbb {R} ^{2}$ given by

$a+bx{\stackrel {f}{\longmapsto }}{\begin{pmatrix}a-b\\b\end{pmatrix}}$

Find the image of each of these elements of the domain.

1. $3-2x$
2. $2+2x$
3. $x$

Show that this map is an isomorphism.

Problem 3

Show that the natural map $f_{1}$ from Example 1.5 is an isomorphism.

This exercise is recommended for all readers.

Problem 4

Decide whether each map is an isomorphism (if it is an isomorphism then prove it and if it isn't then state a condition that it fails to satisfy).

1. $f:{\mathcal {M}}_{2\!\times \!2}\to \mathbb {R}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto ad-bc$

2. $f:{\mathcal {M}}_{2\!\times \!2}\to \mathbb {R} ^{4}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto {\begin{pmatrix}a+b+c+d\\a+b+c\\a+b\\a\end{pmatrix}}$

3. $f:{\mathcal {M}}_{2\!\times \!2}\to {\mathcal {P}}_{3}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto c+(d+c)x+(b+a)x^{2}+ax^{3}$

4. $f:{\mathcal {M}}_{2\!\times \!2}\to {\mathcal {P}}_{3}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto c+(d+c)x+(b+a+1)x^{2}+ax^{3}$

Problem 5

Show that the map $f:\mathbb {R} ^{1}\to \mathbb {R} ^{1}$ given by $f(x)=x^{3}$ is one-to-one and onto.Is it an isomorphism?

This exercise is recommended for all readers.

Problem 6

Refer to Example 1.1. Produce two more isomorphisms (of course, that they satisfy the conditions in the definition of isomorphism must be verified).

Problem 7

Refer to Example 1.2. Produce two more isomorphisms (and verify that they satisfy the conditions).

This exercise is recommended for all readers.

Problem 8

Show that, although $\mathbb {R} ^{2}$ is not itself a subspace of $\mathbb {R} ^{3}$, it is isomorphic to the $xy$-plane subspace of $\mathbb {R} ^{3}$.

Problem 9

Find two isomorphisms between $\mathbb {R} ^{16}$ and ${\mathcal {M}}_{4\!\times \!4}$.

This exercise is recommended for all readers.

Problem 10

For what $k$ is ${\mathcal {M}}_{m\!\times \!n}$ isomorphic to $\mathbb {R} ^{k}$?

Problem 11

For what $k$ is ${\mathcal {P}}_{k}$ isomorphic to $\mathbb {R} ^{n}$?

Problem 12

Prove that the map in Example 1.7, from ${\mathcal {P}}_{5}$ to ${\mathcal {P}}_{5}$ given by $p(x)\mapsto p(x-1)$, is a vector space isomorphism.

Problem 13

Why, in Lemma 1.8, must there be a ${\vec {v}}\in V$? That is, why must $V$ be nonempty?

Problem 14

Are any two trivial spaces isomorphic?

Problem 15

In the proof of Lemma 1.9, what about the zero-summands case (that is, if $n$ is zero)?

Problem 16

Show that any isomorphism $f:{\mathcal {P}}_{0}\to \mathbb {R} ^{1}$ has the form $a\mapsto ka$ for some nonzero real number $k$.

This exercise is recommended for all readers.

Problem 17

These prove that isomorphism is an equivalence relation.

1. Show that the identity map ${\mbox{id}}:V\to V$ is an isomorphism. Thus, any vector space is isomorphic to itself.
2. Show that if $f:V\to W$ is an isomorphism then so is its inverse $f^{-1}:W\to V$. Thus, if $V$ is isomorphic to $W$ then also $W$ is isomorphic to $V$.
3. Show that a composition of isomorphisms is an isomorphism: if $f:V\to W$ is an isomorphism and $g:W\to U$ is an isomorphism then so also is $g\circ f:V\to U$. Thus, if $V$ is isomorphic to $W$ and $W$ is isomorphic to $U$, then also $V$ is isomorphic to $U$.

Problem 18

Suppose that $f:V\to W$ preserves structure. Show that $f$ is one-to-one if and only if the unique member of $V$ mapped by $f$ to ${\vec {0}}_{W}$ is ${\vec {0}}_{V}$.

Problem 19

Suppose that $f:V\to W$ is an isomorphism. Prove that the set $\{{\vec {v}}_{1},\dots ,{\vec {v}}_{k}\}\subseteq V$ is linearly dependent if and only if the set of images $\{f({\vec {v}}_{1}),\dots ,f({\vec {v}}_{k})\}\subseteq W$ is linearly dependent.

This exercise is recommended for all readers.

Problem 20

Show that each type of map from Example 1.6 is an automorphism.

1. Dilation $d_{s}$ by a nonzero scalar $s$.
2. Rotation $t_{\theta }$ through an angle $\theta$.
3. Reflection $f_{\ell }$ over a line through the origin.

Hint. For the second and third items, polar coordinates are useful.

Problem 21

Produce an automorphism of ${\mathcal {P}}_{2}$ other than the identity map, and other than a shift map $p(x)\mapsto p(x-k)$.

Problem 22

1. Show that a function $f:\mathbb {R} ^{1}\to \mathbb {R} ^{1}$ is an automorphism if and only if it has the form $x\mapsto kx$ for some $k\neq 0$.
2. Let $f$ be an automorphism of $\mathbb {R} ^{1}$ such that $f(3)=7$. Find $f(-2)$.
3. Show that a function $f:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ is an automorphism if and only if it has the form

   ${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}ax+by\\cx+dy\end{pmatrix}}$

   for some $a,b,c,d\in \mathbb {R}$ with $ad-bc\neq 0$. Hint. Exercises in prior subsections have shown that

   ${\begin{pmatrix}b\\d\end{pmatrix}}{\text{ is not a multiple of }}{\begin{pmatrix}a\\c\end{pmatrix}}$

   if and only if $ad-bc\neq 0$.
4. Let $f$ be an automorphism of $\mathbb {R} ^{2}$ with

   $f({\begin{pmatrix}1\\3\end{pmatrix}})={\begin{pmatrix}2\\-1\end{pmatrix}}\quad {\text{and}}\quad f({\begin{pmatrix}1\\4\end{pmatrix}})={\begin{pmatrix}0\\1\end{pmatrix}}.$

   Find

   $f({\begin{pmatrix}0\\-1\end{pmatrix}}).$

Problem 23

Refer to Lemma 1.8 and Lemma 1.9. Find two more things preserved by isomorphism.

Problem 24

We show that isomorphisms can be tailored to fit in that, sometimes, given vectors in the domain and in the range we can produce an isomorphism associating those vectors.

1. Let $B=\langle {\vec {\beta }}_{1},{\vec {\beta }}_{2},{\vec {\beta }}_{3}\rangle$ be a basis for ${\mathcal {P}}_{2}$ so that any ${\vec {p}}\in {\mathcal {P}}_{2}$ has a unique representation as ${\vec {p}}=c_{1}{\vec {\beta }}_{1}+c_{2}{\vec {\beta }}_{2}+c_{3}{\vec {\beta }}_{3}$, which we denote in this way.

   ${\rm {Rep}}_{B}({\vec {p}})={\begin{pmatrix}c_{1}\\c_{2}\\c_{3}\end{pmatrix}}$

   Show that the ${\rm {Rep}}_{B}(\cdot )$ operation is a function from ${\mathcal {P}}_{2}$ to $\mathbb {R} ^{3}$ (this entails showing that with every domain vector ${\vec {v}}\in {\mathcal {P}}_{2}$ there is an associated image vector in $\mathbb {R} ^{3}$, and further, that with every domain vector ${\vec {v}}\in {\mathcal {P}}_{2}$ there is at most one associated image vector).
2. Show that this ${\rm {Rep}}_{B}(\cdot )$ function is one-to-one and onto.
3. Show that it preserves structure.
4. Produce an isomorphism from ${\mathcal {P}}_{2}$ to $\mathbb {R} ^{3}$ that fits these specifications.

   $x+x^{2}\mapsto {\begin{pmatrix}1\\0\\0\end{pmatrix}}\quad {\text{and}}\quad 1-x\mapsto {\begin{pmatrix}0\\1\\0\end{pmatrix}}$

Problem 25

Prove that a space is $n$-dimensional if and only if it is isomorphic to $\mathbb {R} ^{n}$. Hint. Fix a basis $B$ for the space and consider the map sending a vector over to its representation with respect to $B$.

Problem 26

(Requires the subsection on Combining Subspaces, which is optional.) Let $U$ and $W$ be vector spaces. Define a new vector space, consisting of the set $U\times W=\{({\vec {u}},{\vec {w}})\,{\big |}\,{\vec {u}}\in U{\text{ and }}{\vec {w}}\in W\}$ along with these operations.

$({\vec {u}}_{1},{\vec {w}}_{1})+({\vec {u}}_{2},{\vec {w}}_{2})=({\vec {u}}_{1}+{\vec {u}}_{2},{\vec {w}}_{1}+{\vec {w}}_{2})\quad {\text{and}}\quad r\cdot ({\vec {u}},{\vec {w}})=(r{\vec {u}},r{\vec {w}})$

This is a vector space, the external direct sum of $U$ and $W$.

1. Check that it is a vector space.
2. Find a basis for, and the dimension of, the external direct sum ${\mathcal {P}}_{2}\times \mathbb {R} ^{2}$.
3. What is the relationship among $\dim(U)$, $\dim(W)$, and $\dim(U\times W)$?
4. Suppose that $U$ and $W$ are subspaces of a vector space $V$ such that $V=U\oplus W$ (in this case we say that $V$ is the internal direct sum of $U$ and $W$). Show that the map $f:U\times W\to V$ given by

   $({\vec {u}},{\vec {w}}){\stackrel {f}{\longmapsto }}{\vec {u}}+{\vec {w}}$

   is an isomorphism. Thus if the internal direct sum is defined then the internal and external direct sums are isomorphic.

2 - Dimension Characterizes Isomorphism

In the prior subsection, after stating the definition of an isomorphism, we gave some results supporting the intuition that such a map describes spaces as "the same". Here we will formalize this intuition. While two spaces that are isomorphic are not equal, we think of them as almost equal— as equivalent. In this subsection we shall show that the relationship "is isomorphic to" is an equivalence relation.

Theorem 2.1

Isomorphism is an equivalence relation between vector spaces.

Proof

We must prove that this relation has the three properties of being symmetric, reflexive, and transitive. For each of the three we will use item 2 of Lemma 1.9 and show that the map preserves structure by showing that it preserves linear combinations of two members of the domain.

To check reflexivity, that any space is isomorphic to itself, consider the identity map. It is clearly one-to-one and onto. The calculation showing that it preserves linear combinations is easy.

${\mbox{id}}(c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2})=c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}=c_{1}\cdot {\mbox{id}}({\vec {v}}_{1})+c_{2}\cdot {\mbox{id}}({\vec {v}}_{2})$

To check symmetry, that if $V$ is isomorphic to $W$ via some map $f:V\to W$ then there is an isomorphism going the other way, consider the inverse map $f^{-1}:W\to V$. As stated in the appendix, such an inverse function exists and it is also a correspondence. Thus we have reduced the symmetry issue to checking that, because $f$ preserves linear combinations, so also does $f^{-1}$. Assume that ${\vec {w}}_{1}=f({\vec {v}}_{1})$ and ${\vec {w}}_{2}=f({\vec {v}}_{2})$, i.e., that $f^{-1}({\vec {w}}_{1})={\vec {v}}_{1}$ and $f^{-1}({\vec {w}}_{2})={\vec {v}}_{2}$.

${\begin{array}{rl}f^{-1}(c_{1}\cdot {\vec {w}}_{1}+c_{2}\cdot {\vec {w}}_{2})&=f^{-1}{\bigl (}\,c_{1}\cdot f({\vec {v}}_{1})+c_{2}\cdot f({\vec {v}}_{2})\,{\bigr )}\\&=f^{-1}(\,f{\bigl (}c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})\,{\bigr )}\\&=c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}\\&=c_{1}\cdot f^{-1}({\vec {w}}_{1})+c_{2}\cdot f^{-1}({\vec {w}}_{2})\end{array}}$

Finally, we must check transitivity, that if $V$ is isomorphic to $W$ via some map $f$ and if $W$ is isomorphic to $U$ via some map $g$ then also $V$ is isomorphic to $U$. Consider the composition $g\circ f:V\to U$. The appendix notes that the composition of two correspondences is a correspondence, so we need only check that the composition preserves linear combinations.

${\begin{array}{rl}g\circ f\,{\bigl (}c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2}{\bigr )}&=g{\bigl (}\,f(c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2})\,{\bigr )}\\&=g{\bigl (}\,c_{1}\cdot f({\vec {v}}_{1})+c_{2}\cdot f({\vec {v}}_{2})\,{\bigr )}\\&=c_{1}\cdot g{\bigl (}f({\vec {v}}_{1}))+c_{2}\cdot g(f({\vec {v}}_{2}){\bigr )}\\&=c_{1}\cdot (g\circ f)\,({\vec {v}}_{1})+c_{2}\cdot (g\circ f)\,({\vec {v}}_{2})\end{array}}$

Thus $g\circ f:V\to U$ is an isomorphism.

As a consequence of that result, we know that the universe of vector spaces is partitioned into classes: every space is in one and only one isomorphism class.

All finite dimensional vector spaces:

$V\cong W$

Theorem 2.2

Vector spaces are isomorphic if and only if they have the same dimension.

This follows from the next two lemmas.

Lemma 2.3

If spaces are isomorphic then they have the same dimension.

Proof

We shall show that an isomorphism of two spaces gives a correspondence between their bases. That is, where $f:V\to W$ is an isomorphism and a basis for the domain $V$ is $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$, then the image set $D=\langle f({\vec {\beta }}_{1}),\dots ,f({\vec {\beta }}_{n})\rangle$ is a basis for the codomain $W$. (The other half of the correspondence— that for any basis of $W$ the inverse image is a basis for $V$— follows on recalling that if $f$ is an isomorphism then $f^{-1}$ is also an isomorphism, and applying the prior sentence to $f^{-1}$.)

To see that $D$ spans $W$, fix any ${\vec {w}}\in W$, note that $f$ is onto and so there is a ${\vec {v}}\in V$ with ${\vec {w}}=f({\vec {v}})$, and expand ${\vec {v}}$ as a combination of basis vectors.

${\vec {w}}=f({\vec {v}})=f(v_{1}{\vec {\beta }}_{1}+\dots +v_{n}{\vec {\beta }}_{n})=v_{1}\cdot f({\vec {\beta }}_{1})+\dots +v_{n}\cdot f({\vec {\beta }}_{n})$

For linear independence of $D$, if

${\vec {0}}_{W}=c_{1}f({\vec {\beta }}_{1})+\dots +c_{n}f({\vec {\beta }}_{n})=f(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})$

then, since $f$ is one-to-one and so the only vector sent to ${\vec {0}}_{W}$ is ${\vec {0}}_{V}$, we have that ${\vec {0}}_{V}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}$, implying that all of the $c$'s are zero.

Lemma 2.4

If spaces have the same dimension then they are isomorphic.

Proof

To show that any two spaces of dimension $n$ are isomorphic, we can simply show that any one is isomorphic to $\mathbb {R} ^{n}$. Then we will have shown that they are isomorphic to each other, by the transitivity of isomorphism (which was established in Theorem 2.1).

Let $V$ be $n$-dimensional. Fix a basis $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ for the domain $V$. Consider the representation of the members of that domain with respect to the basis as a function from $V$ to $\mathbb {R} ^{n}$

${\vec {v}}=v_{1}{\vec {\beta }}_{1}+\dots +v_{n}{\vec {\beta }}_{n}\,{\stackrel {{\text{Rep}}_{B}}{\longmapsto }}\,{\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}$

(it is well-defined since every ${\vec {v}}$ has one and only one such representation— see Remark 2.5 below).

This function is one-to-one because if

${\text{Rep}}_{B}(u_{1}{\vec {\beta }}_{1}+\dots +u_{n}{\vec {\beta }}_{n})={\text{Rep}}_{B}(v_{1}{\vec {\beta }}_{1}+\dots +v_{n}{\vec {\beta }}_{n})$

then

${\begin{pmatrix}u_{1}\\\vdots \\u_{n}\end{pmatrix}}={\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}$

and so $u_{1}=v_{1}$, ..., $u_{n}=v_{n}$, and therefore the original arguments $u_{1}{\vec {\beta }}_{1}+\dots +u_{n}{\vec {\beta }}_{n}$ and $v_{1}{\vec {\beta }}_{1}+\dots +v_{n}{\vec {\beta }}_{n}$ are equal.

This function is onto; any $n$-tall vector

${\vec {w}}={\begin{pmatrix}w_{1}\\\vdots \\w_{n}\end{pmatrix}}$

is the image of some ${\vec {v}}\in V$, namely ${\vec {w}}={\rm {Rep}}_{B}(w_{1}{\vec {\beta }}_{1}+\dots +w_{n}{\vec {\beta }}_{n})$.

Finally, this function preserves structure.

${\begin{array}{rl}{\rm {Rep}}_{B}(r\cdot {\vec {u}}+s\cdot {\vec {v}})&={\rm {Rep}}_{B}(\,(ru_{1}+sv_{1}){\vec {\beta }}_{1}+\dots +(ru_{n}+sv_{n}){\vec {\beta }}_{n}\,)\\&={\begin{pmatrix}ru_{1}+sv_{1}\\\vdots \\ru_{n}+sv_{n}\end{pmatrix}}\\&=r\cdot {\begin{pmatrix}u_{1}\\\vdots \\u_{n}\end{pmatrix}}+s\cdot {\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}\\&=r\cdot {\rm {Rep}}_{B}({\vec {u}})+s\cdot {\rm {Rep}}_{B}({\vec {v}})\end{array}}$

Thus the ${\mbox{Rep}}_{B}$ function is an isomorphism and thus any $n$-dimensional space is isomorphic to the $n$-dimensional space $\mathbb {R} ^{n}$. Consequently, any two spaces with the same dimension are isomorphic.

Remark 2.5

The parenthetical comment in that proof about the role played by the "one and only one representation" result requires some explanation. We need to show that (for a fixed $B$) each vector in the domain is associated by ${\mbox{Rep}}_{B}$ with one and only one vector in the codomain.

A contrasting example, where an association doesn't have this property, is illuminating. Consider this subset of ${\mathcal {P}}_{2}$, which is not a basis.

$A=\{1+0x+0x^{2},0+1x+0x^{2},0+0x+1x^{2},1+1x+2x^{2}\}$

Call those four polynomials ${\vec {\alpha }}_{1}$, ..., ${\vec {\alpha }}_{4}$. If, mimicing above proof, we try to write the members of ${\mathcal {P}}_{2}$ as ${\vec {p}}=c_{1}{\vec {\alpha }}_{1}+c_{2}{\vec {\alpha }}_{2}+c_{3}{\vec {\alpha }}_{3}+c_{4}{\vec {\alpha }}_{4}$, and associate ${\vec {p}}$ with the four-tall vector with components $c_{1}$, ..., $c_{4}$ then there is a problem. For, consider ${\vec {p}}(x)=1+x+x^{2}$. The set $A$ spans the space ${\mathcal {P}}_{2}$, so there is at least one four-tall vector associated with ${\vec {p}}$. But $A$ is not linearly independent and so vectors do not have unique decompositions. In this case, both

${\vec {p}}(x)=1{\vec {\alpha }}_{1}+1{\vec {\alpha }}_{2}+1{\vec {\alpha }}_{3}+0{\vec {\alpha }}_{4}\quad {\text{and}}\quad {\vec {p}}(x)=0{\vec {\alpha }}_{1}+0{\vec {\alpha }}_{2}-1{\vec {\alpha }}_{3}+1{\vec {\alpha }}_{4}$

and so there is more than one four-tall vector associated with ${\vec {p}}$.

${\begin{pmatrix}1\\1\\1\\0\end{pmatrix}}\quad {\text{and}}\quad {\begin{pmatrix}0\\0\\-1\\1\end{pmatrix}}$

That is, with input ${\vec {p}}$ this association does not have a well-defined (i.e., single) output value.

Any map whose definition appears possibly ambiguous must be checked to see that it is well-defined. For ${\mbox{Rep}}_{B}$ in the above proof that check is Problem 11.

That ends the proof of Theorem 2.2. We say that the isomorphism classes are characterized by dimension because we can describe each class simply by giving the number that is the dimension of all of the spaces in that class.

This subsection's results give us a collection of representatives of the isomorphism classes.

Corollary 2.6

A finite-dimensional vector space is isomorphic to one and only one of the $\mathbb {R} ^{n}$.

The proofs above pack many ideas into a small space. Through the rest of this chapter we'll consider these ideas again, and fill them out. For a taste of this, we will expand here on the proof of Lemma 2.4.

Example 2.7

The space ${\mathcal {M}}_{2\!\times \!2}$ of $2\!\times \!2$ matrices is isomorphic to $\mathbb {R} ^{4}$. With this basis for the domain

$B=\langle {\begin{pmatrix}1&0\\0&0\end{pmatrix}},{\begin{pmatrix}0&1\\0&0\end{pmatrix}},{\begin{pmatrix}0&0\\1&0\end{pmatrix}},{\begin{pmatrix}0&0\\0&1\end{pmatrix}}\rangle$

the isomorphism given in the lemma, the representation map $f_{1}={\mbox{Rep}}_{B}$, simply carries the entries over.

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}{\stackrel {f_{1}}{\longmapsto }}{\begin{pmatrix}a\\b\\c\\d\end{pmatrix}}$

One way to think of the map $f_{1}$ is: fix the basis $B$ for the domain and the basis ${\mathcal {E}}_{4}$ for the codomain, and associate ${\vec {\beta }}_{1}$ with ${\vec {e}}_{1}$, and ${\vec {\beta }}_{2}$ with ${\vec {e}}_{2}$, etc. Then extend this association to all of the members of two spaces.

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}=a{\vec {\beta }}_{1}+b{\vec {\beta }}_{2}+c{\vec {\beta }}_{3}+d{\vec {\beta }}_{4}\;\;{\stackrel {f_{1}}{\longmapsto }}\;\;a{\vec {e}}_{1}+b{\vec {e}}_{2}+c{\vec {e}}_{3}+d{\vec {e}}_{4}={\begin{pmatrix}a\\b\\c\\d\end{pmatrix}}$

We say that the map has been extended linearly from the bases to the spaces.

We can do the same thing with different bases, for instance, taking this basis for the domain.

$A=\langle {\begin{pmatrix}2&0\\0&0\end{pmatrix}},{\begin{pmatrix}0&2\\0&0\end{pmatrix}},{\begin{pmatrix}0&0\\2&0\end{pmatrix}},{\begin{pmatrix}0&0\\0&2\end{pmatrix}}\rangle$

Associating corresponding members of $A$ and ${\mathcal {E}}_{4}$ and extending linearly

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}=(a/2){\vec {\alpha }}_{1}+(b/2){\vec {\alpha }}_{2}+(c/2){\vec {\alpha }}_{3}+(d/2){\vec {\alpha }}_{4}$

${\stackrel {f_{2}}{\longmapsto }}\;\;(a/2){\vec {e}}_{1}+(b/2){\vec {e}}_{2}+(c/2){\vec {e}}_{3}+(d/2){\vec {e}}_{4}={\begin{pmatrix}a/2\\b/2\\c/2\\d/2\end{pmatrix}}$

gives rise to an isomorphism that is different than $f_{1}$.

The prior map arose by changing the basis for the domain. We can also change the basis for the codomain. Starting with

$B\quad {\text{and}}\quad D=\langle {\begin{pmatrix}1\\0\\0\\0\end{pmatrix}},{\begin{pmatrix}0\\1\\0\\0\end{pmatrix}},{\begin{pmatrix}0\\0\\0\\1\end{pmatrix}},{\begin{pmatrix}0\\0\\1\\0\end{pmatrix}}\rangle$

associating ${\vec {\beta }}_{1}$ with ${\vec {\delta }}_{1}$, etc., and then linearly extending that correspondence to all of the two spaces

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}=a{\vec {\beta }}_{1}+b{\vec {\beta }}_{2}+c{\vec {\beta }}_{3}+d{\vec {\beta }}_{4}\;\;{\stackrel {f_{3}}{\longmapsto }}\;\;a{\vec {\delta }}_{1}+b{\vec {\delta }}_{2}+c{\vec {\delta }}_{3}+d{\vec {\delta }}_{4}={\begin{pmatrix}a\\b\\d\\c\end{pmatrix}}$

gives still another isomorphism.

So there is a connection between the maps between spaces and bases for those spaces. Later sections will explore that connection.

We will close this section with a summary.

Recall that in the first chapter we defined two matrices as row equivalent if they can be derived from each other by elementary row operations (this was the meaning of same-ness that was of interest there). We showed that is an equivalence relation and so the collection of matrices is partitioned into classes, where all the matrices that are row equivalent fall together into a single class. Then, for insight into which matrices are in each class, we gave representatives for the classes, the reduced echelon form matrices.

In this section, except that the appropriate notion of same-ness here is vector space isomorphism, we have followed much the same outline. First we defined isomorphism, saw some examples, and established some properties. Then we showed that it is an equivalence relation, and now we have a set of class representatives, the real vector spaces $\mathbb {R} ^{1}$, $\mathbb {R} ^{2}$, etc.

All finite dimensional vector spaces:

One representative per class

As before, the list of representatives helps us to understand the partition. It is simply a classification of spaces by dimension.

In the second chapter, with the definition of vector spaces, we seemed to have opened up our studies to many examples of new structures besides the familiar $\mathbb {R} ^{n}$'s. We now know that isn't the case. Any finite-dimensional vector space is actually "the same" as a real space. We are thus considering exactly the structures that we need to consider.

The rest of the chapter fills out the work in this section. In particular, in the next section we will consider maps that preserve structure, but are not necessarily correspondences.

Exercises

This exercise is recommended for all readers.

Problem 1

Decide if the spaces are isomorphic.

1. $\mathbb {R} ^{2}$, $\mathbb {R} ^{4}$
2. ${\mathcal {P}}_{5}$, $\mathbb {R} ^{5}$
3. ${\mathcal {M}}_{2\!\times \!3}$, $\mathbb {R} ^{6}$
4. ${\mathcal {P}}_{5}$, ${\mathcal {M}}_{2\!\times \!3}$
5. ${\mathcal {M}}_{2\!\times \!k}$, $\mathbb {C} ^{k}$

Answer

Each pair of spaces is isomorphic if and only if the two have the same dimension. We can, when there is an isomorphism, state a map, but it isn't strictly necessary.

1. No, they have different dimensions.
2. No, they have different dimensions.
3. Yes, they have the same dimension. One isomorphism is this.

   ${\begin{pmatrix}a&b&c\\d&e&f\end{pmatrix}}\mapsto {\begin{pmatrix}a\\\vdots \\f\end{pmatrix}}$

4. Yes, they have the same dimension. This is an isomorphism.

   $a+bx+\cdots +fx^{5}\mapsto {\begin{pmatrix}a&b&c\\d&e&f\end{pmatrix}}$

5. Yes, both have dimension $2k$.

This exercise is recommended for all readers.

Problem 2

Consider the isomorphism ${\rm {Rep}}_{B}(\cdot ):{\mathcal {P}}_{1}\to \mathbb {R} ^{2}$ where $B=\langle 1,1+x\rangle$. Find the image of each of these elements of the domain.

1. $3-2x$;
2. $2+2x$;
3. $x$

Answer

1. ${\rm {Rep}}_{B}(3-2x)={\begin{pmatrix}5\\-2\end{pmatrix}}$
2. ${\begin{pmatrix}0\\2\end{pmatrix}}$
3. ${\begin{pmatrix}-1\\1\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 3

Show that if $m\neq n$ then $\mathbb {R} ^{m}\not \cong \mathbb {R} ^{n}$.

Answer

They have different dimensions.

This exercise is recommended for all readers.

Problem 4

Is ${\mathcal {M}}_{m\!\times \!n}\cong {\mathcal {M}}_{n\!\times \!m}$?

Answer

Yes, both are $mn$-dimensional.

This exercise is recommended for all readers.

Problem 5

Are any two planes through the origin in $\mathbb {R} ^{3}$ isomorphic?

Answer

Yes, any two (nondegenerate) planes are both two-dimensional vector spaces.

Problem 6

Find a set of equivalence class representatives other than the set of $\mathbb {R} ^{n}$'s.

Answer

There are many answers, one is the set of ${\mathcal {P}}_{k}$ (taking ${\mathcal {P}}_{-1}$ to be the trivial vector space).

Problem 7

True or false: between any $n$-dimensional space and $\mathbb {R} ^{n}$ there is exactly one isomorphism.

Answer

False (except when $n=0$). For instance, if $f:V\to \mathbb {R} ^{n}$ is an isomorphism then multiplying by any nonzero scalar, gives another, different, isomorphism. (Between trivial spaces the isomorphisms are unique; the only map possible is ${\vec {0}}_{V}\mapsto 0_{W}$.)

Problem 8

Can a vector space be isomorphic to one of its (proper) subspaces?

Answer

No. A proper subspace has a strictly lower dimension than it's superspace; if $U$ is a proper subspace of $V$ then any linearly independent subset of $U$ must have fewer than $\dim(V)$ members or else that set would be a basis for $V$, and $U$ wouldn't be proper.

This exercise is recommended for all readers.

Problem 9

This subsection shows that for any isomorphism, the inverse map is also an isomorphism. This subsection also shows that for a fixed basis $B$ of an $n$-dimensional vector space $V$, the map ${\text{Rep}}_{B}:V\to \mathbb {R} ^{n}$ is an isomorphism. Find the inverse of this map.

Answer

Where $B=\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$, the inverse is this.

${\begin{pmatrix}c_{1}\\\vdots \\c_{n}\end{pmatrix}}\mapsto c_{1}{\vec {\beta }}_{1}+\cdots +c_{n}{\vec {\beta }}_{n}$

This exercise is recommended for all readers.

Problem 10

Prove these facts about matrices.

1. The row space of a matrix is isomorphic to the column space of its transpose.
2. The row space of a matrix is isomorphic to its column space.

Answer

All three spaces have dimension equal to the rank of the matrix.

Problem 11

Show that the function from Theorem 2.2 is well-defined.

Answer

We must show that if ${\vec {a}}={\vec {b}}$ then $f({\vec {a}})=f({\vec {b}})$. So suppose that $a_{1}{\vec {\beta }}_{1}+\dots +a_{n}{\vec {\beta }}_{n}=b_{1}{\vec {\beta }}_{1}+\dots +b_{n}{\vec {\beta }}_{n}$. Each vector in a vector space (here, the domain space) has a unique representation as a linear combination of basis vectors, so we can conclude that $a_{1}=b_{1}$, ..., $a_{n}=b_{n}$. Thus,

$f({\vec {a}})={\begin{pmatrix}a_{1}\\\vdots \\a_{n}\end{pmatrix}}={\begin{pmatrix}b_{1}\\\vdots \\b_{n}\end{pmatrix}}=f({\vec {b}})$

and so the function is well-defined.

Problem 12

Is the proof of Theorem 2.2 valid when $n=0$?

Answer

Yes, because a zero-dimensional space is a trivial space.

Problem 13

For each, decide if it is a set of isomorphism class representatives.

1. $\{\mathbb {C} ^{k}\,{\big |}\,k\in \mathbb {N} \}$
2. $\{{\mathcal {P}}_{k}\,{\big |}\,k\in \{-1,0,1,\ldots \}\}$
3. $\{{\mathcal {M}}_{m\!\times \!n}\,{\big |}\,m,n\in \mathbb {N} \}$

Answer

1. No, this collection has no spaces of odd dimension.
2. Yes, because ${\mathcal {P}}_{k}\cong \mathbb {R} ^{k+1}$.
3. No, for instance, ${\mathcal {M}}_{2\!\times \!3}\cong {\mathcal {M}}_{3\!\times \!2}$.

Problem 14

Let $f$ be a correspondence between vector spaces $V$ and $W$ (that is, a map that is one-to-one and onto). Show that the spaces $V$ and $W$ are isomorphic via $f$ if and only if there are bases $B\subset V$ and $D\subset W$ such that corresponding vectors have the same coordinates: ${\rm {Rep}}_{B}({\vec {v}})={\rm {Rep}}_{D}(f({\vec {v}}))$.

Answer

One direction is easy: if the two are isomorphic via $f$ then for any basis $B\subseteq V$, the set $D=f(B)$ is also a basis (this is shown in Lemma 2.3). The check that corresponding vectors have the same coordinates: $f(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})=c_{1}f({\vec {\beta }}_{1})+\dots +c_{n}f({\vec {\beta }}_{n})=c_{1}{\vec {\delta }}_{1}+\dots +c_{n}{\vec {\delta }}_{n}$ is routine.

For the other half, assume that there are bases such that corresponding vectors have the same coordinates with respect to those bases. Because $f$ is a correspondence, to show that it is an isomorphism, we need only show that it preserves structure. Because ${\rm {Rep}}_{B}({\vec {v}}\,)={\rm {Rep}}_{D}(f({\vec {v}}\,))$, the map $f$ preserves structure if and only if representations preserve addition: ${\rm {Rep}}_{B}({\vec {v}}_{1}+{\vec {v}}_{2})={\rm {Rep}}_{B}({\vec {v}}_{1})+{\rm {Rep}}_{B}({\vec {v}}_{2})$ and scalar multiplication: ${\rm {Rep}}_{B}(r\cdot {\vec {v}}\,)=r\cdot {\rm {Rep}}_{B}({\vec {v}}\,)$ The addition calculation is this: $(c_{1}+d_{1}){\vec {\beta }}_{1}+\dots +(c_{n}+d_{n}){\vec {\beta }}_{n}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}+d_{1}{\vec {\beta }}_{1}+\dots +d_{n}{\vec {\beta }}_{n}$, and the scalar multiplication calculation is similar.

Problem 15

Consider the isomorphism ${\text{Rep}}_{B}:{\mathcal {P}}_{3}\to \mathbb {R} ^{4}$.

1. Vectors in a real space are orthogonal if and only if their dot product is zero. Give a definition of orthogonality for polynomials.
2. The derivative of a member of ${\mathcal {P}}_{3}$ is in ${\mathcal {P}}_{3}$. Give a definition of the derivative of a vector in $\mathbb {R} ^{4}$.

Answer

1. Pulling the definition back from $\mathbb {R} ^{4}$ to ${\mathcal {P}}_{3}$ gives that $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}$ is orthogonal to $b_{0}+b_{1}x+b_{2}x^{2}+b_{3}x^{3}$ if and only if $a_{0}b_{0}+a_{1}b_{1}+a_{2}b_{2}+a_{3}b_{3}=0$.
2. A natural definition is this.

   $D({\begin{pmatrix}a_{0}\\a_{1}\\a_{2}\\a_{3}\end{pmatrix}})={\begin{pmatrix}a_{1}\\2a_{2}\\3a_{3}\\0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 16

Does every correspondence between bases, when extended to the spaces, give an isomorphism?

Answer

Yes.

Assume that $V$ is a vector space with basis $B=\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$ and that $W$ is another vector space such that the map $f:B\to W$ is a correspondence. Consider the extension ${\hat {f}}:V\to W$ of $f$.

${\hat {f}}(c_{1}{\vec {\beta }}_{1}+\cdots +c_{n}{\vec {\beta }}_{n})=c_{1}f({\vec {\beta }}_{1})+\cdots +c_{n}f({\vec {\beta }}_{n}).$

The map ${\hat {f}}$ is an isomorphism.

First, ${\hat {f}}$ is well-defined because every member of $V$ has one and only one representation as a linear combination of elements of $B$.

Second, ${\hat {f}}$ is one-to-one because every member of $W$ has only one representation as a linear combination of elements of $\langle f({\vec {\beta }}_{1}),\dots ,f({\vec {\beta }}_{n})\rangle$. That map ${\hat {f}}$ is onto because every member of $W$ has at least one representation as a linear combination of members of $\langle f({\vec {\beta }}_{1}),\dots ,f({\vec {\beta }}_{n})\rangle$.

Finally, preservation of structure is routine to check. For instance, here is the preservation of addition calculation.

${\begin{array}{rl}{\hat {f}}(\,(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})+(d_{1}{\vec {\beta }}_{1}+\dots +d_{n}{\vec {\beta }}_{n})\,)&={\hat {f}}(\,(c_{1}+d_{1}){\vec {\beta }}_{1}+\dots +(c_{n}+d_{n}){\vec {\beta }}_{n}\,)\\&=(c_{1}+d_{1})f({\vec {\beta }}_{1})+\dots +(c_{n}+d_{n})f({\vec {\beta }}_{n})\\&=c_{1}f({\vec {\beta }}_{1})+\dots +c_{n}f({\vec {\beta }}_{n})+d_{1}f({\vec {\beta }}_{1})+\dots +d_{n}f({\vec {\beta }}_{n})\\&={\hat {f}}(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})++{\hat {f}}(d_{1}{\vec {\beta }}_{1}+\dots +d_{n}{\vec {\beta }}_{n}).\end{array}}$

Preservation of scalar multiplication is similar.

Problem 17

(Requires the subsection on Combining Subspaces, which is optional.) Suppose that $V=V_{1}\oplus V_{2}$ and that $V$ is isomorphic to the space $U$ under the map $f$. Show that $U=f(V_{1})\oplus f(U_{2})$.

Answer

Because $V_{1}\cap V_{2}=\{{\vec {0}}_{V}\}$ and $f$ is one-to-one we have that $f(V_{1})\cap f(V_{2})=\{{\vec {0}}_{U}\}$. To finish, count the dimensions: $\dim(U)=\dim(V)=\dim(V_{1})+\dim(V_{2})=\dim(f(V_{1}))+\dim(f(V_{2}))$, as required.

Problem 18

Show that this is not a well-defined function from the rational numbers to the integers: with each fraction, associate the value of its numerator.

Answer

Rational numbers have many representations, e.g., $1/2=3/6$, and the numerators can vary among representations.

Section II - Homomorphisms

The definition of isomorphism has two conditions. In this section we will consider the second one, that the map must preserve the algebraic structure of the space. We will focus on this condition by studying maps that are required only to preserve structure; that is, maps that are not required to be correspondences.

Experience shows that this kind of map is tremendously useful in the study of vector spaces. For one thing, as we shall see in the second subsection below, while isomorphisms describe how spaces are the same, these maps describe how spaces can be thought of as alike.

1 - Definition

Definition 1.1

A function between vector spaces $h:V\to W$ that preserves the operations of addition

if ${\vec {v}}_{1},{\vec {v}}_{2}\in V$ then $h({\vec {v}}_{1}+{\vec {v}}_{2})=h({\vec {v}}_{1})+h({\vec {v}}_{2})$

and scalar multiplication

if ${\vec {v}}\in V$ and $r\in \mathbb {R}$ then $h(r\cdot {\vec {v}})=r\cdot h({\vec {v}})$

is a homomorphism or linear map.

Example 1.2

The projection map $\pi :\mathbb {R} ^{3}\to \mathbb {R} ^{2}$

${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}x\\y\end{pmatrix}}$

is a homomorphism.

It preserves addition

$\pi ({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}\!+\!{\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})=\pi ({\begin{pmatrix}x_{1}+x_{2}\\y_{1}+y_{2}\\z_{1}+z_{2}\end{pmatrix}})={\begin{pmatrix}x_{1}+x_{2}\\y_{1}+y_{2}\end{pmatrix}}=\pi ({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})+\pi ({\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})$

and scalar multiplication.

$\pi (r\cdot {\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})=\pi ({\begin{pmatrix}rx_{1}\\ry_{1}\\rz_{1}\end{pmatrix}})={\begin{pmatrix}rx_{1}\\ry_{1}\end{pmatrix}}=r\cdot \pi ({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})$

This map is not an isomorphism since it is not one-to-one. For instance, both ${\vec {0}}$ and ${\vec {e}}_{3}$ in $\mathbb {R} ^{3}$ are mapped to the zero vector in $\mathbb {R} ^{2}$.

Example 1.3

Of course, the domain and codomain might be other than spaces of column vectors. Both of these are homomorphisms; the verifications are straightforward.

1. $f_{1}:{\mathcal {P}}_{2}\to {\mathcal {P}}_{3}$ given by

   $a_{0}+a_{1}x+a_{2}x^{2}\;\mapsto \;a_{0}x+(a_{1}/2)x^{2}+(a_{2}/3)x^{3}$

2. $f_{2}:M_{2\!\times \!2}\to \mathbb {R}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto a+d$

Example 1.4

Between any two spaces there is a zero homomorphism, mapping every vector in the domain to the zero vector in the codomain.

Example 1.5

These two suggest why we use the term "linear map".

1. The map $g:\mathbb {R} ^{3}\to \mathbb {R}$ given by

   ${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {g}{\longmapsto }}3x+2y-4.5z$

   is linear (i.e., is a homomorphism). In contrast, the map ${\hat {g}}:\mathbb {R} ^{3}\to \mathbb {R}$ given by

   ${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {\hat {g}}{\longmapsto }}3x+2y-4.5z+1$

   is not; for instance,

   ${\hat {g}}({\begin{pmatrix}0\\0\\0\end{pmatrix}}+{\begin{pmatrix}1\\0\\0\end{pmatrix}})=4\quad {\text{while}}\quad {\hat {g}}({\begin{pmatrix}0\\0\\0\end{pmatrix}})+{\hat {g}}({\begin{pmatrix}1\\0\\0\end{pmatrix}})=5$

   (to show that a map is not linear we need only produce one example of a linear combination that is not preserved).
2. The first of these two maps $t_{1},t_{2}:\mathbb {R} ^{3}\to \mathbb {R} ^{2}$ is linear while the second is not.

   ${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {t_{1}}{\longmapsto }}{\begin{pmatrix}5x-2y\\x+y\end{pmatrix}}\quad {\text{and}}\quad {\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {t_{2}}{\longmapsto }}{\begin{pmatrix}5x-2y\\xy\end{pmatrix}}$

   Finding an example that the second fails to preserve structure is easy.

What distinguishes the homomorphisms is that the coordinate functions are linear combinations of the arguments. See also Problem 7.

Obviously, any isomorphism is a homomorphism— an isomorphism is a homomorphism that is also a correspondence. So, one way to think of the "homomorphism" idea is that it is a generalization of "isomorphism", motivated by the observation that many of the properties of isomorphisms have only to do with the map's structure preservation property and not to do with it being a correspondence. As examples, these two results from the prior section do not use one-to-one-ness or onto-ness in their proof, and therefore apply to any homomorphism.

Lemma 1.6

A homomorphism sends a zero vector to a zero vector.

Lemma 1.7

Each of these is a necessary and sufficient condition for $f:V\to W$ to be a homomorphism.

1. $f(c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2})=c_{1}\cdot f({\vec {v}}_{1})+c_{2}\cdot f({\vec {v}}_{2})$ for any $c_{1},c_{2}\in \mathbb {R}$ and ${\vec {v}}_{1},{\vec {v}}_{2}\in V$
2. $f(c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n})=c_{1}\cdot f({\vec {v}}_{1})+\dots +c_{n}\cdot f({\vec {v}}_{n})$ for any $c_{1},\dots ,c_{n}\in \mathbb {R}$ and ${\vec {v}}_{1},\ldots ,{\vec {v}}_{n}\in V$

Part 1 is often used to check that a function is linear.

Example 1.8

The map $f:\mathbb {R} ^{2}\to \mathbb {R} ^{4}$ given by

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {f}{\longmapsto }}{\begin{pmatrix}x/2\\0\\x+y\\3y\end{pmatrix}}$

satisfies 1 of the prior result

${\begin{pmatrix}r_{1}(x_{1}/2)+r_{2}(x_{2}/2)\\0\\r_{1}(x_{1}+y_{1})+r_{2}(x_{2}+y_{2})\\r_{1}(3y_{1})+r_{2}(3y_{2})\end{pmatrix}}=r_{1}{\begin{pmatrix}x_{1}/2\\0\\x_{1}+y_{1}\\3y_{1}\end{pmatrix}}+r_{2}{\begin{pmatrix}x_{2}/2\\0\\x_{2}+y_{2}\\3y_{2}\end{pmatrix}}$

and so it is a homomorphism.

However, some of the results that we have seen for isomorphisms fail to hold for homomorphisms in general. Consider the theorem that an isomorphism between spaces gives a correspondence between their bases. Homomorphisms do not give any such correspondence; Example 1.2 shows that there is no such correspondence, and another example is the zero map between any two nontrivial spaces. Instead, for homomorphisms a weaker but still very useful result holds.

Theorem 1.9

A homomorphism is determined by its action on a basis. That is, if $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ is a basis of a vector space $V$ and ${\vec {w}}_{1},\dots ,{\vec {w}}_{n}$ are (perhaps not distinct) elements of a vector space $W$ then there exists a homomorphism from $V$ to $W$ sending ${\vec {\beta }}_{1}$ to ${\vec {w}}_{1}$, ..., and ${\vec {\beta }}_{n}$ to ${\vec {w}}_{n}$, and that homomorphism is unique.

Proof

We will define the map by associating ${\vec {\beta }}_{1}$ with ${\vec {w}}_{1}$, etc., and then extending linearly to all of the domain. That is, where ${\vec {v}}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}$, the map $h:V\to W$ is given by $h({\vec {v}})=c_{1}{\vec {w}}_{1}+\dots +c_{n}{\vec {w}}_{n}$. This is well-defined because, with respect to the basis, the representation of each domain vector ${\vec {v}}$ is unique.

This map is a homomorphism since it preserves linear combinations; where ${\vec {v_{1}}}=c_{1}{\vec {\beta }}_{1}+\cdots +c_{n}{\vec {\beta }}_{n}$ and ${\vec {v_{2}}}=d_{1}{\vec {\beta }}_{1}+\cdots +d_{n}{\vec {\beta }}_{n}$, we have this.

${\begin{array}{rl}h(r_{1}{\vec {v}}_{1}+r_{2}{\vec {v}}_{2})&=h((r_{1}c_{1}+r_{2}d_{1}){\vec {\beta }}_{1}+\dots +(r_{1}c_{n}+r_{2}d_{n}){\vec {\beta }}_{n})\\&=(r_{1}c_{1}+r_{2}d_{1}){\vec {w}}_{1}+\dots +(r_{1}c_{n}+r_{2}d_{n}){\vec {w}}_{n}\\&=r_{1}h({\vec {v}}_{1})+r_{2}h({\vec {v}}_{2})\end{array}}$

And, this map is unique since if ${\hat {h}}:V\to W$ is another homomorphism such that ${\hat {h}}({\vec {\beta }}_{i})={\vec {w}}_{i}$ for each $i$ then $h$ and ${\hat {h}}$ agree on all of the vectors in the domain.

${\begin{array}{rl}{\hat {h}}({\vec {v}})&={\hat {h}}(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})\\&=c_{1}{\hat {h}}({\vec {\beta }}_{1})+\dots +c_{n}{\hat {h}}({\vec {\beta }}_{n})\\&=c_{1}{\vec {w}}_{1}+\dots +c_{n}{\vec {w}}_{n}\\&=h({\vec {v}})\end{array}}$

Thus, $h$ and ${\hat {h}}$ are the same map.

Example 1.10

This result says that we can construct a homomorphism by fixing a basis for the domain and specifying where the map sends those basis vectors. For instance, if we specify a map $h:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that acts on the standard basis ${\mathcal {E}}_{2}$ in this way

$h({\begin{pmatrix}1\\0\end{pmatrix}})={\begin{pmatrix}-1\\1\end{pmatrix}}\quad {\text{and}}\quad h({\begin{pmatrix}0\\1\end{pmatrix}})={\begin{pmatrix}-4\\4\end{pmatrix}}$

then the action of $h$ on any other member of the domain is also specified. For instance, the value of $h$ on this argument

$h({\begin{pmatrix}3\\-2\end{pmatrix}})=h(3\cdot {\begin{pmatrix}1\\0\end{pmatrix}}-2\cdot {\begin{pmatrix}0\\1\end{pmatrix}})=3\cdot h({\begin{pmatrix}1\\0\end{pmatrix}})-2\cdot h({\begin{pmatrix}0\\1\end{pmatrix}})={\begin{pmatrix}5\\-5\end{pmatrix}}$

is a direct consequence of the value of $h$ on the basis vectors.

Later in this chapter we shall develop a scheme, using matrices, that is convienent for computations like this one.

Just as the isomorphisms of a space with itself are useful and interesting, so too are the homomorphisms of a space with itself.

Definition 1.11

A linear map from a space into itself $t:V\to V$ is a linear transformation.

Remark 1.12

In this book we use "linear transformation" only in the case where the codomain equals the domain, but it is widely used in other texts as a general synonym for "homomorphism".

Example 1.13

The map on $\mathbb {R} ^{2}$ that projects all vectors down to the $x$-axis

${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}x\\0\end{pmatrix}}$

is a linear transformation.

Example 1.14

The derivative map $d/dx:{\mathcal {P}}_{n}\to {\mathcal {P}}_{n}$

$a_{0}+a_{1}x+\cdots +a_{n}x^{n}{\stackrel {d/dx}{\longmapsto }}a_{1}+2a_{2}x+3a_{3}x^{2}+\cdots +na_{n}x^{n-1}$

is a linear transformation, as this result from calculus notes: $d(c_{1}f+c_{2}g)/dx=c_{1}\,(df/dx)+c_{2}\,(dg/dx)$.

Example 1.15

The matrix transpose map

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\;\mapsto \;{\begin{pmatrix}a&c\\b&d\end{pmatrix}}$

is a linear transformation of ${\mathcal {M}}_{2\!\times \!2}$. Note that this transformation is one-to-one and onto, and so in fact it is an automorphism.

We finish this subsection about maps by recalling that we can linearly combine maps. For instance, for these maps from $\mathbb {R} ^{2}$ to itself

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {f}{\longmapsto }}{\begin{pmatrix}2x\\3x-2y\end{pmatrix}}\quad {\text{and}}\quad {\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {g}{\longmapsto }}{\begin{pmatrix}0\\5x\end{pmatrix}}$

the linear combination $5f-2g$ is also a map from $R^{2}$ to itself.

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {5f-2g}{\longmapsto }}{\begin{pmatrix}10x\\5x-10y\end{pmatrix}}$

Lemma 1.16

For vector spaces $V$ and $W$, the set of linear functions from $V$ to $W$ is itself a vector space, a subspace of the space of all functions from $V$ to $W$. It is denoted $\mathop {\mathcal {L}} (V,W)$.

Proof

This set is non-empty because it contains the zero homomorphism. So to show that it is a subspace we need only check that it is closed under linear combinations. Let $f,g:V\to W$ be linear. Then their sum is linear

${\begin{array}{rl}(f+g)(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})&=c_{1}f({\vec {v}}_{1})+c_{2}f({\vec {v}}_{2})+c_{1}g({\vec {v}}_{1})+c_{2}g({\vec {v}}_{2})\\&=c_{1}{\bigl (}f+g{\bigr )}({\vec {v}}_{1})+c_{2}{\bigl (}f+g{\bigr )}({\vec {v}}_{2})\end{array}}$

and any scalar multiple is also linear.

${\begin{array}{rl}(r\cdot f)(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})&=r(c_{1}f({\vec {v}}_{1})+c_{2}f({\vec {v}}_{2}))\\&=c_{1}(r\cdot f)({\vec {v}}_{1})+c_{2}(r\cdot f)({\vec {v}}_{2})\end{array}}$

Hence $\mathop {\mathcal {L}} (V,W)$ is a subspace.

We started this section by isolating the structure preservation property of isomorphisms. That is, we defined homomorphisms as a generalization of isomorphisms. Some of the properties that we studied for isomorphisms carried over unchanged, while others were adapted to this more general setting.

It would be a mistake, though, to view this new notion of homomorphism as derived from, or somehow secondary to, that of isomorphism. In the rest of this chapter we shall work mostly with homomorphisms, partly because any statement made about homomorphisms is automatically true about isomorphisms, but more because, while the isomorphism concept is perhaps more natural, experience shows that the homomorphism concept is actually more fruitful and more central to further progress.

Exercises

This exercise is recommended for all readers.

Problem 1

Decide if each $h:\mathbb {R} ^{3}\to \mathbb {R} ^{2}$ is linear.

1. $h({\begin{pmatrix}x\\y\\z\end{pmatrix}})={\begin{pmatrix}x\\x+y+z\end{pmatrix}}$
2. $h({\begin{pmatrix}x\\y\\z\end{pmatrix}})={\begin{pmatrix}0\\0\end{pmatrix}}$
3. $h({\begin{pmatrix}x\\y\\z\end{pmatrix}})={\begin{pmatrix}1\\1\end{pmatrix}}$
4. $h({\begin{pmatrix}x\\y\\z\end{pmatrix}})={\begin{pmatrix}2x+y\\3y-4z\end{pmatrix}}$

Answer

1. Yes. The verification is straightforward.

   ${\begin{array}{rl}h(c_{1}\cdot {\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})&=h({\begin{pmatrix}c_{1}x_{1}+c_{2}x_{2}\\c_{1}y_{1}+c_{2}y_{2}\\c_{1}z_{1}+c_{2}z_{2}\end{pmatrix}})\\&={\begin{pmatrix}c_{1}x_{1}+c_{2}x_{2}\\c_{1}x_{1}+c_{2}x_{2}+c_{1}y_{1}+c_{2}y_{2}+c_{1}z_{1}+c_{2}z_{2}\end{pmatrix}}\\&=c_{1}\cdot {\begin{pmatrix}x_{1}\\x_{1}+y_{1}+z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}x_{2}\\c_{2}+y_{2}+z_{2}\end{pmatrix}}\\&=c_{1}\cdot h({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})+c_{2}\cdot h({\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})\end{array}}$

2. Yes. The verification is easy.

   ${\begin{array}{rl}h(c_{1}\cdot {\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})&=h({\begin{pmatrix}c_{1}x_{1}+c_{2}x_{2}\\c_{1}y_{1}+c_{2}y_{2}\\c_{1}z_{1}+c_{2}z_{2}\end{pmatrix}})\\&={\begin{pmatrix}0\\0\end{pmatrix}}\\&=c_{1}\cdot h({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})+c_{2}\cdot h({\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})\end{array}}$

3. No. An example of an addition that is not respected is this.

   $h({\begin{pmatrix}0\\0\\0\end{pmatrix}}+{\begin{pmatrix}0\\0\\0\end{pmatrix}})={\begin{pmatrix}1\\1\end{pmatrix}}\neq h({\begin{pmatrix}0\\0\\0\end{pmatrix}})+h({\begin{pmatrix}0\\0\\0\end{pmatrix}})$

4. Yes. The verification is straightforward.

   ${\begin{array}{rl}h(c_{1}\cdot {\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})&=h({\begin{pmatrix}c_{1}x_{1}+c_{2}x_{2}\\c_{1}y_{1}+c_{2}y_{2}\\c_{1}z_{1}+c_{2}z_{2}\end{pmatrix}})\\&={\begin{pmatrix}2(c_{1}x_{1}+c_{2}x_{2})+(c_{1}y_{1}+c_{2}y_{2})\\3(c_{1}y_{1}+c_{2}y_{2})-4(c_{1}z_{1}+c_{2}z_{2})\end{pmatrix}}\\&=c_{1}\cdot {\begin{pmatrix}2x_{1}+y_{1}\\3y_{1}-4z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}2x_{2}+y_{2}\\3y_{2}-4z_{2}\end{pmatrix}}\\&=c_{1}\cdot h({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})+c_{2}\cdot h({\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})\end{array}}$

This exercise is recommended for all readers.

Problem 2

Decide if each map $h:{\mathcal {M}}_{2\!\times \!2}\to \mathbb {R}$ is linear.

1. $h({\begin{pmatrix}a&b\\c&d\end{pmatrix}})=a+d$
2. $h({\begin{pmatrix}a&b\\c&d\end{pmatrix}})=ad-bc$
3. $h({\begin{pmatrix}a&b\\c&d\end{pmatrix}})=2a+3b+c-d$
4. $h({\begin{pmatrix}a&b\\c&d\end{pmatrix}})=a^{2}+b^{2}$

Answer

For each, we must either check that linear combinations are preserved, or give an example of a linear combination that is not.

1. Yes. The check that it preserves combinations is routine.

   ${\begin{array}{rl}h(r_{1}\cdot {\begin{pmatrix}a_{1}&b_{1}\\c_{1}&d_{1}\end{pmatrix}}+r_{2}\cdot {\begin{pmatrix}a_{2}&b_{2}\\c_{2}&d_{2}\end{pmatrix}})&=h({\begin{pmatrix}r_{1}a_{1}+r_{2}a_{2}&r_{1}b_{1}+r_{2}b_{2}\\r_{1}c_{1}+r_{2}c_{2}&r_{1}d_{1}+r_{2}d_{2}\end{pmatrix}})\\&=(r_{1}a_{1}+r_{2}a_{2})+(r_{1}d_{1}+r_{2}d_{2})\\&=r_{1}(a_{1}+d_{1})+r_{2}(a_{2}+d_{2})\\&=r_{1}\cdot h({\begin{pmatrix}a_{1}&b_{1}\\c_{1}&d_{1}\end{pmatrix}})+r_{2}\cdot h({\begin{pmatrix}a_{2}&b_{2}\\c_{2}&d_{2}\end{pmatrix}})\end{array}}$

2. No. For instance, not preserved is multiplication by the scalar $2$.

   $h(2\cdot {\begin{pmatrix}1&0\\0&1\end{pmatrix}})=h({\begin{pmatrix}2&0\\0&2\end{pmatrix}})=4\quad {\text{while}}\quad 2\cdot h({\begin{pmatrix}1&0\\0&1\end{pmatrix}})=2\cdot 1=2$

3. Yes. This is the check that it preserves combinations of two members of the domain.

   ${\begin{array}{rl}h(r_{1}\cdot {\begin{pmatrix}a_{1}&b_{1}\\c_{1}&d_{1}\end{pmatrix}}+r_{2}\cdot {\begin{pmatrix}a_{2}&b_{2}\\c_{2}&d_{2}\end{pmatrix}})&=h({\begin{pmatrix}r_{1}a_{1}+r_{2}a_{2}&r_{1}b_{1}+r_{2}b_{2}\\r_{1}c_{1}+r_{2}c_{2}&r_{1}d_{1}+r_{2}d_{2}\end{pmatrix}})\\&=2(r_{1}a_{1}+r_{2}a_{2})+3(r_{1}b_{1}+r_{2}b_{2})+(r_{1}c_{1}+r_{2}c_{2})-(r_{1}d_{1}+r_{2}d_{2})\\&=r_{1}(2a_{1}+3b_{1}+c_{1}-d_{1})+r_{2}(2a_{2}+3b_{2}+c_{2}-d_{2})\\&=r_{1}\cdot h({\begin{pmatrix}a_{1}&b_{1}\\c_{1}&d_{1}\end{pmatrix}}+r_{2}\cdot h({\begin{pmatrix}a_{2}&b_{2}\\c_{2}&d_{2}\end{pmatrix}})\end{array}}$

4. No. An example of a combination that is not preserved is this.

   $h({\begin{pmatrix}1&0\\0&0\end{pmatrix}}+{\begin{pmatrix}1&0\\0&0\end{pmatrix}})=h({\begin{pmatrix}2&0\\0&0\end{pmatrix}})=4\quad {\text{while}}\quad h({\begin{pmatrix}1&0\\0&0\end{pmatrix}})+h({\begin{pmatrix}1&0\\0&0\end{pmatrix}})=1+1=2$

This exercise is recommended for all readers.

Problem 3

Show that these two maps are homomorphisms.

1. $d/dx:{\mathcal {P}}_{3}\to {\mathcal {P}}_{2}$ given by $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}$ maps to $a_{1}+2a_{2}x+3a_{3}x^{2}$
2. $\int :{\mathcal {P}}_{2}\to {\mathcal {P}}_{3}$ given by $b_{0}+b_{1}x+b_{2}x^{2}$ maps to $b_{0}x+(b_{1}/2)x^{2}+(b_{2}/3)x^{3}$

Are these maps inverse to each other?

Answer

The check that each is a homomorphisms is routine. Here is the check for the differentiation map.

${\frac {d}{dx}}(r\cdot (a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3})+s\cdot (b_{0}+b_{1}x+b_{2}x^{2}+b_{3}x^{3}))$

${\begin{array}{rl}&={\frac {d}{dx}}((ra_{0}+sb_{0})+(ra_{1}+sb_{1})x+(ra_{2}+sb_{2})x^{2}+(ra_{3}+sb_{3})x^{3})\\&=(ra_{1}+sb_{1})+2(ra_{2}+sb_{2})x+3(ra_{3}+sb_{3})x^{2}\\&=r\cdot (a_{1}+2a_{2}x+3a_{3}x^{2})+s\cdot (b_{1}+2b_{2}x+3b_{3}x^{2})\\&=r\cdot {\frac {d}{dx}}(a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3})+s\cdot {\frac {d}{dx}}(b_{0}+b_{1}x+b_{2}x^{2}+b_{3}x^{3})\end{array}}$

(An alternate proof is to simply note that this is a property of differentiation that is familar from calculus.)

These two maps are not inverses as this composition does not act as the identity map on this element of the domain.

$1\in {\mathcal {P}}_{3}\;{\stackrel {d/dx}{\longmapsto }}\;0\in {\mathcal {P}}_{2}\;{\stackrel {\int }{\longmapsto }}\;0\in {\mathcal {P}}_{3}$

Problem 4

Is (perpendicular) projection from $\mathbb {R} ^{3}$ to the $xz$-plane a homomorphism? Projection to the $yz$-plane? To the $x$-axis? The $y$-axis? The $z$-axis? Projection to the origin?

Answer

Each of these projections is a homomorphism. Projection to the $xz$-plane and to the $yz$-plane are these maps.

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}x\\0\\z\end{pmatrix}}\qquad {\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}0\\y\\z\end{pmatrix}}$

Projection to the $x$-axis, to the $y$-axis, and to the $z$-axis are these maps.

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}x\\0\\0\end{pmatrix}}\qquad {\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}0\\y\\0\end{pmatrix}}\qquad {\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}0\\0\\z\end{pmatrix}}$

And projection to the origin is this map.

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}0\\0\\0\end{pmatrix}}$

Verification that each is a homomorphism is straightforward. (The last one, of course, is the zero transformation on $\mathbb {R} ^{3}$.)

Problem 5

Show that, while the maps from Example 1.3 preserve linear operations, they are not isomorphisms.

Answer

The first is not onto; for instance, there is no polynomial that is sent the constant polynomial $p(x)=1$. The second is not one-to-one; both of these members of the domain

${\begin{pmatrix}1&0\\0&0\end{pmatrix}}\quad {\text{and}}\quad {\begin{pmatrix}0&0\\0&1\end{pmatrix}}$

are mapped to the same member of the codomain, $1\in \mathbb {R}$.

Problem 6

Is an identity map a linear transformation?

Answer

Yes; in any space ${\text{id}}(c\cdot {\vec {v}}+d\cdot {\vec {w}})=c\cdot {\vec {v}}+d\cdot {\vec {w}}=c\cdot {\text{id}}({\vec {v}})+d\cdot {\text{id}}({\vec {w}})$.

This exercise is recommended for all readers.

Problem 7

Stating that a function is "linear" is different than stating that its graph is a line.

1. The function $f_{1}:\mathbb {R} \to \mathbb {R}$ given by $f_{1}(x)=2x-1$ has a graph that is a line. Show that it is not a linear function.
2. The function $f_{2}:\mathbb {R} ^{2}\to \mathbb {R}$ given by

   ${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto x+2y$

   does not have a graph that is a line. Show that it is a linear function.

Answer

1. This map does not preserve structure since $f(1+1)=3$, while $f(1)+f(1)=2$.
2. The check is routine.

   ${\begin{array}{rl}f(r_{1}\cdot {\begin{pmatrix}x_{1}\\y_{1}\end{pmatrix}}+r_{2}\cdot {\begin{pmatrix}x_{2}\\y_{2}\end{pmatrix}})&=f({\begin{pmatrix}r_{1}x_{1}+r_{2}x_{2}\\r_{1}y_{1}+r_{2}y_{2}\end{pmatrix}})\\&=(r_{1}x_{1}+r_{2}x_{2})+2(r_{1}y_{1}+r_{2}y_{2})\\&=r_{1}\cdot (x_{1}+2y_{1})+r_{2}\cdot (x_{2}+2y_{2})\\&=r_{1}\cdot f({\begin{pmatrix}x_{1}\\y_{1}\end{pmatrix}})+r_{2}\cdot f({\begin{pmatrix}x_{2}\\y_{2}\end{pmatrix}})\end{array}}$

This exercise is recommended for all readers.

Problem 8

Part of the definition of a linear function is that it respects addition. Does a linear function respect subtraction?

Answer

Yes. Where $h:V\to W$ is linear, $h({\vec {u}}-{\vec {v}})=h({\vec {u}}+(-1)\cdot {\vec {v}})=h({\vec {u}})+(-1)\cdot h({\vec {v}})=h({\vec {u}})-h({\vec {v}})$.

Problem 9

Assume that $h$ is a linear transformation of $V$ and that $\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$ is a basis of $V$. Prove each statement.

1. If $h({\vec {\beta }}_{i})={\vec {0}}$ for each basis vector then $h$ is the zero map.
2. If $h({\vec {\beta }}_{i})={\vec {\beta }}_{i}$ for each basis vector then $h$ is the identity map.
3. If there is a scalar $r$ such that $h({\vec {\beta }}_{i})=r\cdot {\vec {\beta }}_{i}$ for each basis vector then $h({\vec {v}})=r\cdot {\vec {v}}$ for all vectors in $V$.

Answer

1. Let ${\vec {v}}\in V$ be represented with respect to the basis as ${\vec {v}}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}$. Then $h({\vec {v}})=h(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})=c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})=c_{1}\cdot {\vec {0}}+\dots +c_{n}\cdot {\vec {0}}={\vec {0}}$.
2. This argument is similar to the prior one. Let ${\vec {v}}\in V$ be represented with respect to the basis as ${\vec {v}}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}$. Then $h(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})=c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}={\vec {v}}$.
3. As above, only $c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})=c_{1}r{\vec {\beta }}_{1}+\dots +c_{n}r{\vec {\beta }}_{n}=r(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})=r{\vec {v}}$.

This exercise is recommended for all readers.

Problem 10

Consider the vector space $\mathbb {R} ^{+}$ where vector addition and scalar multiplication are not the ones inherited from $\mathbb {R}$ but rather are these: $a+b$ is the product of $a$ and $b$, and $r\cdot a$ is the $r$-th power of $a$. (This was shown to be a vector space in an earlier exercise.) Verify that the natural logarithm map $\ln :\mathbb {R} ^{+}\to \mathbb {R}$ is a homomorphism between these two spaces. Is it an isomorphism?

Answer

That it is a homomorphism follows from the familiar rules that the logarithm of a product is the sum of the logarithms $\ln(ab)=\ln(a)+\ln(b)$ and that the logarithm of a power is the multiple of the logarithm $\ln(a^{r})=r\ln(a)$. This map is an isomorphism because it has an inverse, namely, the exponential map, so it is a correspondence, and therefore it is an isomorphism.

This exercise is recommended for all readers.

Problem 11

Consider this transformation of $\mathbb {R} ^{2}$.

${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}x/2\\y/3\end{pmatrix}}$

Find the image under this map of this ellipse.

$\{{\begin{pmatrix}x\\y\end{pmatrix}}\,{\big |}\,(x^{2}/4)+(y^{2}/9)=1\}$

Answer

Where ${\hat {x}}=x/2$ and ${\hat {y}}=y/3$, the image set is

$\{{\begin{pmatrix}{\hat {x}}\\{\hat {y}}\end{pmatrix}}\,{\big |}\,{\frac {\displaystyle (2{\hat {x}})^{2}}{\displaystyle 4}}+{\frac {\displaystyle (3{\hat {y}})^{2}}{\displaystyle 9}}=1\}=\{{\begin{pmatrix}{\hat {x}}\\{\hat {y}}\end{pmatrix}}\,{\big |}\,{\hat {x}}^{2}+{\hat {y}}^{2}=1\}$

the unit circle in the ${\hat {x}}{\hat {y}}$-plane.

This exercise is recommended for all readers.

Problem 12

Imagine a rope wound around the earth's equator so that it fits snugly (suppose that the earth is a sphere). How much extra rope must be added to raise the circle to a constant six feet off the ground?

Answer

The circumference function $r\mapsto 2\pi r$ is linear. Thus we have $2\pi \cdot (r_{\text{earth}}+6)-2\pi \cdot (r_{\text{earth}})=12\pi$. Observe that it takes the same amount of extra rope to raise the circle from tightly wound around a basketball to six feet above that basketball as it does to raise it from tightly wound around the earth to six feet above the earth.

This exercise is recommended for all readers.

Problem 13

Verify that this map $h:\mathbb {R} ^{3}\to \mathbb {R}$

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\;\mapsto \;{\begin{pmatrix}x\\y\\z\end{pmatrix}}\cdot {\begin{pmatrix}3\\-1\\-1\end{pmatrix}}=3x-y-z$

is linear. Generalize.

Answer

Verifying that it is linear is routine.

${\begin{array}{rl}h(c_{1}\cdot {\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})&=h({\begin{pmatrix}c_{1}x_{1}+c_{2}x_{2}\\c_{1}y_{1}+c_{2}y_{2}\\c_{1}z_{1}+c_{2}z_{2}\end{pmatrix}})\\&=3(c_{1}x_{1}+c_{2}x_{2})-(c_{1}y_{1}+c_{2}y_{2})-(c_{1}z_{1}+c_{2}z_{2})\\&=c_{1}\cdot (3x_{1}-y_{1}-z_{1})+c_{2}\cdot (3x_{2}-y_{2}-z_{2})\\&=c_{1}\cdot h({\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}})+c_{2}\cdot h({\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}})\end{array}}$

The natural guess at a generalization is that for any fixed ${\vec {k}}\in \mathbb {R} ^{3}$ the map ${\vec {v}}\mapsto {\vec {v}}\cdot {\vec {k}}$ is linear. This statement is true. It follows from properties of the dot product we have seen earlier: $({\vec {v}}+{\vec {u}})\cdot {\vec {k}}={\vec {v}}\cdot {\vec {k}}+{\vec {u}}\cdot {\vec {k}}$ and $(r{\vec {v}})\cdot {\vec {k}}=r({\vec {v}}\cdot {\vec {k}})$. (The natural guess at a generalization of this generalization, that the map from $\mathbb {R} ^{n}$ to $\mathbb {R}$ whose action consists of taking the dot product of its argument with a fixed vector ${\vec {k}}\in \mathbb {R} ^{n}$ is linear, is also true.)

Problem 14

Show that every homomorphism from $\mathbb {R} ^{1}$ to $\mathbb {R} ^{1}$ acts via multiplication by a scalar. Conclude that every nontrivial linear transformation of $\mathbb {R} ^{1}$ is an isomorphism. Is that true for transformations of $\mathbb {R} ^{2}$? $\mathbb {R} ^{n}$?

Answer

Let $h:\mathbb {R} ^{1}\to \mathbb {R} ^{1}$ be linear. A linear map is determined by its action on a basis, so fix the basis $\langle 1\rangle$ for $\mathbb {R} ^{1}$. For any $r\in \mathbb {R} ^{1}$ we have that $h(r)=h(r\cdot 1)=r\cdot h(1)$ and so $h$ acts on any argument $r$ by multiplying it by the constant $h(1)$. If $h(1)$ is not zero then the map is a correspondence— its inverse is division by $h(1)$— so any nontrivial transformation of $\mathbb {R} ^{1}$ is an isomorphism.

This projection map is an example that shows that not every transformation of $\mathbb {R} ^{n}$ acts via multiplication by a constant when $n>1$, including when $n=2$.

${\begin{pmatrix}x_{1}\\x_{2}\\\vdots \\x_{n}\end{pmatrix}}\mapsto {\begin{pmatrix}x_{1}\\0\\\vdots \\0\end{pmatrix}}$

Problem 15

1. Show that for any scalars $a_{1,1},\dots ,a_{m,n}$ this map $h:\mathbb {R} ^{n}\to \mathbb {R} ^{m}$ is a homomorphism.

   ${\begin{pmatrix}x_{1}\\\vdots \\x_{n}\end{pmatrix}}\mapsto {\begin{pmatrix}a_{1,1}x_{1}+\dots +a_{1,n}x_{n}\\\vdots \\a_{m,1}x_{1}+\cdots +a_{m,n}x_{n}\end{pmatrix}}$

2. Show that for each $i$, the $i$-th derivative operator $d^{i}/dx^{i}$ is a linear transformation of ${\mathcal {P}}_{n}$. Conclude that for any scalars $c_{k},\ldots ,c_{0}$ this map is a linear transformation of that space.

   $f\mapsto {\frac {d^{k}}{dx^{k}}}f+c_{k-1}{\frac {d^{k-1}}{dx^{k-1}}}f+\dots +c_{1}{\frac {d}{dx}}f+c_{0}f$

Answer

1. Where $c$ and $d$ are scalars, we have this.

   ${\begin{array}{rl}h(c\cdot {\begin{pmatrix}x_{1}\\\vdots \\x_{n}\end{pmatrix}}+d\cdot {\begin{pmatrix}y_{1}\\\vdots \\y_{n}\end{pmatrix}})&=h({\begin{pmatrix}cx_{1}+dy_{1}\\\vdots \\cx_{n}+dy_{n}\end{pmatrix}})\\&={\begin{pmatrix}a_{1,1}(cx_{1}+dy_{1})+\dots +a_{1,n}(cx_{n}+dy_{n})\\\vdots \\a_{m,1}(cx_{1}+dy_{1})+\dots +a_{m,n}(cx_{n}+dy_{n})\end{pmatrix}}\\&=c\cdot {\begin{pmatrix}a_{1,1}x_{1}+\dots +a_{1,n}x_{n}\\\vdots \\a_{m,1}x_{1}+\dots +a_{m,n}x_{n}\end{pmatrix}}+d\cdot {\begin{pmatrix}a_{1,1}y_{1}+\dots +a_{1,n}y_{n}\\\vdots \\a_{m,1}y_{1}+\dots +a_{m,n}y_{n}\end{pmatrix}}\\&=c\cdot h({\begin{pmatrix}x_{1}\\\vdots \\x_{n}\end{pmatrix}})+d\cdot h({\begin{pmatrix}y_{1}\\\vdots \\y_{n}\end{pmatrix}})\end{array}}$

2. Each power $i$ of the derivative operator is linear because of these rules familiar from calculus.

   ${\frac {d^{i}}{dx^{i}}}(\,f(x)+g(x)\,)={\frac {d^{i}}{dx^{i}}}f(x)+{\frac {d^{i}}{dx^{i}}}g(x)\quad {\text{and}}\quad {\frac {d^{i}}{dx^{i}}}\,r\cdot f(x)=r\cdot {\frac {d^{i}}{dx^{i}}}f(x)$

   Thus the given map is a linear transformation of ${\mathcal {P}}_{n}$ because any linear combination of linear maps is also a linear map.

Problem 16

Lemma 1.16 shows that a sum of linear functions is linear and that a scalar multiple of a linear function is linear. Show also that a composition of linear functions is linear.

Answer

(This argument has already appeared, as part of the proof that isomorphism is an equivalence.) Let $f:U\to V$ and $g:V\to W$ be linear. For any ${\vec {u}}_{1},{\vec {u}}_{2}\in U$ and scalars $c_{1},c_{2}$ combinations are preserved.

$g\circ f(c_{1}{\vec {u}}_{1}+c_{2}{\vec {u}}_{2})=g(\,f(c_{1}{\vec {u}}_{1}+c_{2}{\vec {u}}_{2})\,)=g(\,c_{1}f({\vec {u}}_{1})+c_{2}f({\vec {u}}_{2})\,)$

$=c_{1}\cdot g(f({\vec {u}}_{1}))+c_{2}\cdot g(f({\vec {u}}_{2}))=c_{1}\cdot g\circ f({\vec {u}}_{1})+c_{2}\cdot g\circ f({\vec {u}}_{2})$

This exercise is recommended for all readers.

Problem 17

Where $f:V\to W$ is linear, suppose that $f({\vec {v}}_{1})={\vec {w}}_{1}$, ..., $f({\vec {v}}_{n})={\vec {w}}_{n}$ for some vectors ${\vec {w}}_{1}$, ..., ${\vec {w}}_{n}$ from $W$.

1. If the set of ${\vec {w}}\,$'s is independent, must the set of ${\vec {v}}\,$'s also be independent?
2. If the set of ${\vec {v}}\,$'s is independent, must the set of ${\vec {w}}\,$'s also be independent?
3. If the set of ${\vec {w}}\,$'s spans $W$, must the set of ${\vec {v}}\,$'s span $V$?
4. If the set of ${\vec {v}}\,$'s spans $V$, must the set of ${\vec {w}}\,$'s span $W$?

Answer

1. Yes. The set of ${\vec {w}}\,$'s cannot be linearly independent if the set of ${\vec {v}}\,$'s is linearly dependent because any nontrivial relationship in the domain ${\vec {0}}_{V}=c_{1}{\vec {v}}_{1}+\dots +c_{n}{\vec {v}}_{n}$ would give a nontrivial relationship in the range $f({\vec {0}}_{V})={\vec {0}}_{W}=f(c_{1}{\vec {v}}_{1}+\dots +c_{n}{\vec {v}}_{n})=c_{1}f({\vec {v}}_{1})+\dots +c_{n}f({\vec {v}}_{n})=c_{1}{\vec {w}}+\dots +c_{n}{\vec {w}}_{n}$.
2. Not necessarily. For instance, the transformation of $\mathbb {R} ^{2}$ given by

   ${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}x+y\\x+y\end{pmatrix}}$

   sends this linearly independent set in the domain to a linearly dependent image.

   $\{{\vec {v}}_{1},{\vec {v}}_{2}\}=\{{\begin{pmatrix}1\\0\end{pmatrix}},{\begin{pmatrix}1\\1\end{pmatrix}}\}\;\mapsto \;\{{\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}2\\2\end{pmatrix}}\}=\{{\vec {w}}_{1},{\vec {w}}_{2}\}$

3. Not necessarily. An example is the projection map $\pi :\mathbb {R} ^{3}\to \mathbb {R} ^{2}$

   ${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}x\\y\end{pmatrix}}$

   and this set that does not span the domain but maps to a set that does span the codomain.

   $\{{\begin{pmatrix}1\\0\\0\end{pmatrix}},{\begin{pmatrix}0\\1\\0\end{pmatrix}}\}{\stackrel {\pi }{\longmapsto }}\{{\begin{pmatrix}1\\0\end{pmatrix}},{\begin{pmatrix}0\\1\end{pmatrix}}\}$

4. Not necessarily. For instance, the injection map $\iota :\mathbb {R} ^{2}\to \mathbb {R} ^{3}$ sends the standard basis ${\mathcal {E}}_{2}$ for the domain to a set that does not span the codomain. (Remark. However, the set of ${\vec {w}}$'s does span the range. A proof is easy.)

Problem 18

Generalize Example 1.15 by proving that the matrix transpose map is linear. What is the domain and codomain?

Answer

Recall that the entry in row $i$ and column $j$ of the transpose of $M$ is the entry $m_{j,i}$ from row $j$ and column $i$ of $M$. Now, the check is routine.

${\begin{array}{rl}{{[r\cdot {\begin{pmatrix}&\vdots \\\cdots &a_{i,j}&\cdots \\&\vdots \end{pmatrix}}+s\cdot {\begin{pmatrix}&\vdots \\\cdots &b_{i,j}&\cdots \\&\vdots \end{pmatrix}}]}^{\rm {trans}}}&={{\begin{pmatrix}&\vdots \\\cdots &ra_{i,j}+sb_{i,j}&\cdots \\&\vdots \end{pmatrix}}^{\rm {trans}}}\\&={\begin{pmatrix}&\vdots \\\cdots &ra_{j,i}+sb_{j,i}&\cdots \\&\vdots \end{pmatrix}}\\&=r\cdot {\begin{pmatrix}&\vdots \\\cdots &a_{j,i}&\cdots \\&\vdots \end{pmatrix}}+s\cdot {\begin{pmatrix}&\vdots \\\cdots &b_{j,i}&\cdots \\&\vdots \end{pmatrix}}\\&=r\cdot {{\begin{pmatrix}&\vdots \\\cdots &a_{j,i}&\cdots \\&\vdots \end{pmatrix}}^{\rm {trans}}}+s\cdot {{\begin{pmatrix}&\vdots \\\cdots &b_{j,i}&\cdots \\&\vdots \end{pmatrix}}^{\rm {trans}}}\end{array}}$

The domain is ${\mathcal {M}}_{m\!\times \!n}$ while the codomain is ${\mathcal {M}}_{n\!\times \!m}$.

Problem 19

1. Where ${\vec {u}},{\vec {v}}\in \mathbb {R} ^{n}$, the line segment connecting them is defined to be the set $\ell =\{t\cdot {\vec {u}}+(1-t)\cdot {\vec {v}}\,{\big |}\,t\in [0..1]\}$. Show that the image, under a homomorphism $h$, of the segment between ${\vec {u}}$ and ${\vec {v}}$ is the segment between $h({\vec {u}})$ and $h({\vec {v}})$.
2. A subset of $\mathbb {R} ^{n}$ is convex if, for any two points in that set, the line segment joining them lies entirely in that set. (The inside of a sphere is convex while the skin of a sphere is not.) Prove that linear maps from $\mathbb {R} ^{n}$ to $\mathbb {R} ^{m}$ preserve the property of set convexity.

Answer

1. For any homomorphism $h:\mathbb {R} ^{n}\to \mathbb {R} ^{m}$ we have

   $h(\ell )=\{h(t\cdot {\vec {u}}+(1-t)\cdot {\vec {v}})\,{\big |}\,t\in [0..1]\}=\{t\cdot h({\vec {u}})+(1-t)\cdot h({\vec {v}})\,{\big |}\,t\in [0..1]\}$

   which is the line segment from $h({\vec {u}})$ to $h({\vec {v}})$.
2. We must show that if a subset of the domain is convex then its image, as a subset of the range, is also convex. Suppose that $C\subseteq \mathbb {R} ^{n}$ is convex and consider its image $h(C)$. To show $h(C)$ is convex we must show that for any two of its members, ${\vec {d}}_{1}$ and ${\vec {d}}_{2}$, the line segment connecting them

   $\ell =\{t\cdot {\vec {d}}_{1}+(1-t)\cdot {\vec {d}}_{2}\,{\big |}\,t\in [0..1]\}$

   is a subset of $h(C)$. Fix any member ${\hat {t}}\cdot {\vec {d}}_{1}+(1-{\hat {t}})\cdot {\vec {d}}_{2}$ of that line segment. Because the endpoints of $\ell$ are in the image of $C$, there are members of $C$ that map to them, say $h({\vec {c}}_{1})={\vec {d}}_{1}$ and $h({\vec {c}}_{2})={\vec {d}}_{2}$. Now, where ${\hat {t}}$ is the scalar that is fixed in the first sentence of this paragraph, observe that $h({\hat {t}}\cdot {\vec {c}}_{1}+(1-{\hat {t}})\cdot {\vec {c}}_{2})={\hat {t}}\cdot h({\vec {c}}_{1})+(1-{\hat {t}})\cdot h({\vec {c}}_{2})={\hat {t}}\cdot {\vec {d}}_{1}+(1-{\hat {t}})\cdot {\vec {d}}_{2}$ Thus, any member of $\ell$ is a member of $h(C)$, and so $h(C)$ is convex.

This exercise is recommended for all readers.

Problem 20

Let $h:\mathbb {R} ^{n}\to \mathbb {R} ^{m}$ be a homomorphism.

1. Show that the image under $h$ of a line in $\mathbb {R} ^{n}$ is a (possibly degenerate) line in $\mathbb {R} ^{n}$.
2. What happens to a $k$-dimensional linear surface?

Answer

1. For ${\vec {v}}_{0},{\vec {v}}_{1}\in \mathbb {R} ^{n}$, the line through ${\vec {v}}_{0}$ with direction ${\vec {v}}_{1}$ is the set $\{{\vec {v}}_{0}+t\cdot {\vec {v}}_{1}\,{\big |}\,t\in \mathbb {R} \}$. The image under $h$ of that line $\{h({\vec {v}}_{0}+t\cdot {\vec {v}}_{1})\,{\big |}\,t\in \mathbb {R} \}=\{h({\vec {v}}_{0})+t\cdot h({\vec {v}}_{1})\,{\big |}\,t\in \mathbb {R} \}$ is the line through $h({\vec {v}}_{0})$ with direction $h({\vec {v}}_{1})$. If $h({\vec {v}}_{1})$ is the zero vector then this line is degenerate.
2. A $k$-dimensional linear surface in $\mathbb {R} ^{n}$ maps to a (possibly degenerate) $k$-dimensional linear surface in $\mathbb {R} ^{m}$. The proof is just like that the one for the line.

Problem 21

Prove that the restriction of a homomorphism to a subspace of its domain is another homomorphism.

Answer

Suppose that $h:V\to W$ is a homomorphism and suppose that $S$ is a subspace of $V$. Consider the map ${\hat {h}}:S\to W$ defined by ${\hat {h}}({\vec {s}})=h({\vec {s}})$. (The only difference between ${\hat {h}}$ and $h$ is the difference in domain.) Then this new map is linear: ${\hat {h}}(c_{1}\cdot {\vec {s}}_{1}+c_{2}\cdot {\vec {s}}_{2})=h(c_{1}{\vec {s}}_{1}+c_{2}{\vec {s}}_{2})=c_{1}h({\vec {s}}_{1})+c_{2}h({\vec {s}}_{2})=c_{1}\cdot {\hat {h}}({\vec {s}}_{1})+c_{2}\cdot {\hat {h}}({\vec {s}}_{2})$.

Problem 22

Assume that $h:V\to W$ is linear.

1. Show that the rangespace of this map $\{h({\vec {v}})\,{\big |}\,{\vec {v}}\in V\}$ is a subspace of the codomain $W$.
2. Show that the nullspace of this map $\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})={\vec {0}}_{W}\}$ is a subspace of the domain $V$.
3. Show that if $U$ is a subspace of the domain $V$ then its image $\{h({\vec {u}})\,{\big |}\,{\vec {u}}\in U\}$ is a subspace of the codomain $W$. This generalizes the first item.
4. Generalize the second item.

Answer

This will appear as a lemma in the next subsection.

1. The range is nonempty because $V$ is nonempty. To finish we need to show that it is closed under combinations. A combination of range vectors has the form, where ${\vec {v}}_{1},\dots ,{\vec {v}}_{n}\in V$,

   $c_{1}\cdot h({\vec {v}}_{1})+\dots +c_{n}\cdot h({\vec {v}}_{n})=h(c_{1}{\vec {v}}_{1})+\dots +h(c_{n}{\vec {v}}_{n})=h(c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n}),$

   which is itself in the range as $c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n}$ is a member of domain $V$. Therefore the range is a subspace.
2. The nullspace is nonempty since it contains ${\vec {0}}_{V}$, as ${\vec {0}}_{V}$ maps to ${\vec {0}}_{W}$. It is closed under linear combinations because, where ${\vec {v}}_{1},\dots ,{\vec {v}}_{n}\in V$ are elements of the inverse image set $\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})={\vec {0}}_{W}\}$, for $c_{1},\ldots ,c_{n}\in \mathbb {R}$

   ${\vec {0}}_{W}=c_{1}\cdot h({\vec {v}}_{1})+\dots +c_{n}\cdot h({\vec {v}}_{n})=h(c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n})$

   and so $c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n}$ is also in the inverse image of ${\vec {0}}_{W}$.
3. This image of $U$ nonempty because $U$ is nonempty. For closure under combinations, where ${\vec {u}}_{1},\ldots ,{\vec {u}}_{n}\in U$,

   $c_{1}\cdot h({\vec {u}}_{1})+\dots +c_{n}\cdot h({\vec {u}}_{n})=h(c_{1}\cdot {\vec {u}}_{1})+\dots +h(c_{n}\cdot {\vec {u}}_{n})=h(c_{1}\cdot {\vec {u}}_{1}+\dots +c_{n}\cdot {\vec {u}}_{n})$

   which is itself in $h(U)$ as $c_{1}\cdot {\vec {u}}_{1}+\dots +c_{n}\cdot {\vec {u}}_{n}$ is in $U$. Thus this set is a subspace.
4. The natural generalization is that the inverse image of a subspace of is a subspace. Suppose that $X$ is a subspace of $W$. Note that ${\vec {0}}_{W}\in X$ so the set $\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})\in X\}$ is not empty. To show that this set is closed under combinations, let ${\vec {v}}_{1},\dots ,{\vec {v}}_{n}$ be elements of $V$ such that $h({\vec {v}}_{1})={\vec {x}}_{1}$, ..., $h({\vec {v}}_{n})={\vec {x}}_{n}$ and note that

   $h(c_{1}\cdot {\vec {v}}_{1}+\dots +c_{n}\cdot {\vec {v}}_{n})=c_{1}\cdot h({\vec {v}}_{1})+\dots +c_{n}\cdot h({\vec {v}}_{n})=c_{1}\cdot {\vec {x}}_{1}+\dots +c_{n}\cdot {\vec {x}}_{n}$

   so a linear combination of elements of $h^{-1}(X)$ is also in $h^{-1}(X)$.

Problem 23

Consider the set of isomorphisms from a vector space to itself. Is this a subspace of the space $\mathop {\mathcal {L}} (V,V)$ of homomorphisms from the space to itself?

Answer

No; the set of isomorphisms does not contain the zero map (unless the space is trivial).

Problem 24

Does Theorem 1.9 need that $\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$ is a basis? That is, can we still get a well-defined and unique homomorphism if we drop either the condition that the set of ${\vec {\beta }}$'s be linearly independent, or the condition that it span the domain?

Answer

If $\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$ doesn't span the space then the map needn't be unique. For instance, if we try to define a map from $\mathbb {R} ^{2}$ to itself by specifying only that ${\vec {e}}_{1}$ is sent to itself, then there is more than one homomorphism possible; both the identity map and the projection map onto the first component fit this condition.

If we drop the condition that $\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$ is linearly independent then we risk an inconsistent specification (i.e, there could be no such map). An example is if we consider $\langle {\vec {e}}_{2},{\vec {e}}_{1},2{\vec {e}}_{1}\rangle$, and try to define a map from $\mathbb {R} ^{2}$ to itself that sends ${\vec {e}}_{2}$ to itself, and sends both ${\vec {e}}_{1}$ and $2{\vec {e}}_{1}$ to ${\vec {e}}_{1}$. No homomorphism can satisfy these three conditions.

Problem 25

Let $V$ be a vector space and assume that the maps $f_{1},f_{2}:V\to \mathbb {R} ^{1}$ are linear.

1. Define a map $F:V\to \mathbb {R} ^{2}$ whose component functions are the given linear ones.

   ${\vec {v}}\mapsto {\begin{pmatrix}f_{1}({\vec {v}})\\f_{2}({\vec {v}})\end{pmatrix}}$

   Show that $F$ is linear.
2. Does the converse hold— is any linear map from $V$ to $\mathbb {R} ^{2}$ made up of two linear component maps to $\mathbb {R} ^{1}$?
3. Generalize.

Answer

1. Briefly, the check of linearity is this.

   $F(r_{1}\cdot {\vec {v}}_{1}+r_{2}\cdot {\vec {v}}_{2})={\begin{pmatrix}f_{1}(r_{1}{\vec {v}}_{1}+r_{2}{\vec {v}}_{2})\\f_{2}(r_{1}{\vec {v}}_{1}+r_{2}{\vec {v}}_{2})\end{pmatrix}}=r_{1}{\begin{pmatrix}f_{1}({\vec {v}}_{1})\\f_{2}({\vec {v}}_{1})\end{pmatrix}}+r_{2}{\begin{pmatrix}f_{1}({\vec {v}}_{2})\\f_{2}({\vec {v}}_{2})\end{pmatrix}}=r_{1}\cdot F({\vec {v}}_{1})+r_{2}\cdot F({\vec {v}}_{2})$

2. Yes. Let $\pi _{1}:\mathbb {R} ^{2}\to \mathbb {R} ^{1}$ and $\pi _{2}:\mathbb {R} ^{2}\to \mathbb {R} ^{1}$ be the projections

   ${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {\pi _{1}}{\longmapsto }}x\quad {\text{and}}\quad {\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {\pi _{2}}{\longmapsto }}y$

   onto the two axes. Now, where $f_{1}({\vec {v}})=\pi _{1}(F({\vec {v}}))$ and $f_{2}({\vec {v}})=\pi _{2}(F({\vec {v}}))$ we have the desired component functions.

   $F({\vec {v}})={\begin{pmatrix}f_{1}({\vec {v}})\\f_{2}({\vec {v}})\end{pmatrix}}$

   They are linear because they are the composition of linear functions, and the fact that the composition of linear functions is linear was shown as part of the proof that isomorphism is an equivalence relation (alternatively, the check that they are linear is straightforward).
3. In general, a map from a vector space $V$ to an $\mathbb {R} ^{n}$ is linear if and only if each of the component functions is linear. The verification is as in the prior item.

2 - Rangespace and Nullspace

Isomorphisms and homomorphisms both preserve structure. The difference is that homomorphisms needn't be onto and needn't be one-to-one. This means that homomorphisms are a more general kind of map, subject to fewer restrictions than isomorphisms. We will examine what can happen with homomorphisms that is prevented by the extra restrictions satisfied by isomorphisms.

We first consider the effect of dropping the onto requirement, of not requiring as part of the definition that a homomorphism be onto its codomain. For instance, the injection map $\iota :\mathbb {R} ^{2}\to \mathbb {R} ^{3}$

${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}x\\y\\0\end{pmatrix}}$

is not an isomorphism because it is not onto. Of course, being a function, a homomorphism is onto some set, namely its range; the map $\iota$ is onto the $xy$-plane subset of $\mathbb {R} ^{3}$.

Lemma 2.1

Under a homomorphism, the image of any subspace of the domain is a subspace of the codomain. In particular, the image of the entire space, the range of the homomorphism, is a subspace of the codomain.

Proof

Let $h:V\to W$ be linear and let $S$ be a subspace of the domain $V$. The image $h(S)$ is a subset of the codomain $W$. It is nonempty because $S$ is nonempty and thus to show that $h(S)$ is a subspace of $W$ we need only show that it is closed under linear combinations of two vectors. If $h({\vec {s}}_{1})$ and $h({\vec {s}}_{2})$ are members of $h(S)$ then $c_{1}\cdot h({\vec {s}}_{1})+c_{2}\cdot h({\vec {s}}_{2})=h(c_{1}\cdot {\vec {s}}_{1})+h(c_{2}\cdot {\vec {s}}_{2})=h(c_{1}\cdot {\vec {s}}_{1}+c_{2}\cdot {\vec {s}}_{2})$ is also a member of $h(S)$ because it is the image of $c_{1}\cdot {\vec {s}}_{1}+c_{2}\cdot {\vec {s}}_{2}$ from $S$.

Definition 2.2

The rangespace of a homomorphism $h:V\to W$ is

${\mathcal {R}}(h)=\{h({\vec {v}})\,{\big |}\,{\vec {v}}\in V\}$

sometimes denoted $h(V)$. The dimension of the rangespace is the map's rank.

(We shall soon see the connection between the rank of a map and the rank of a matrix.)

Example 2.3

Recall that the derivative map $d/dx:{\mathcal {P}}_{3}\to {\mathcal {P}}_{3}$ given by $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\mapsto a_{1}+2a_{2}x+3a_{3}x^{2}$ is linear. The rangespace ${\mathcal {R}}(d/dx)$ is the set of quadratic polynomials $\{r+sx+tx^{2}\,{\big |}\,r,s,t\in \mathbb {R} \}$. Thus, the rank of this map is three.

Example 2.4

With this homomorphism $h:M_{2\!\times \!2}\to {\mathcal {P}}_{3}$

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto (a+b+2d)+0x+cx^{2}+cx^{3}$

an image vector in the range can have any constant term, must have an $x$ coefficient of zero, and must have the same coefficient of $x^{2}$ as of $x^{3}$. That is, the rangespace is ${\mathcal {R}}(h)=\{r+0x+sx^{2}+sx^{3}\,{\big |}\,r,s\in \mathbb {R} \}$ and so the rank is two.

The prior result shows that, in passing from the definition of isomorphism to the more general definition of homomorphism, omitting the "onto" requirement doesn't make an essential difference. Any homomorphism is onto its rangespace.

However, omitting the "one-to-one" condition does make a difference. A homomorphism may have many elements of the domain that map to one element of the codomain. Below is a "bean " sketch of a many-to-one map between sets. It shows three elements of the codomain that are each the image of many members of the domain.

Recall that for any function $h:V\to W$, the set of elements of $V$ that are mapped to ${\vec {w}}\in W$ is the inverse image $h^{-1}({\vec {w}})=\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})={\vec {w}}\}$. Above, the three sets of many elements on the left are inverse images.

Example 2.5

Consider the projection $\pi :\mathbb {R} ^{3}\to \mathbb {R} ^{2}$

${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}x\\y\end{pmatrix}}$

which is a homomorphism that is many-to-one. In this instance, an inverse image set is a vertical line of vectors in the domain.

Example 2.6

This homomorphism $h:\mathbb {R} ^{2}\to \mathbb {R} ^{1}$

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {h}{\longmapsto }}x+y$

is also many-to-one; for a fixed $w\in \mathbb {R} ^{1}$, the inverse image $h^{-1}(w)$

is the set of plane vectors whose components add to $w$.

The above examples have only to do with the fact that we are considering functions, specifically, many-to-one functions. They show the inverse images as sets of vectors that are related to the image vector ${\vec {w}}$. But these are more than just arbitrary functions, they are homomorphisms; what do the two preservation conditions say about the relationships?

In generalizing from isomorphisms to homomorphisms by dropping the one-to-one condition, we lose the property that we've stated intuitively as: the domain is "the same as" the range. That is, we lose that the domain corresponds perfectly to the range in a one-vector-by-one-vector way.

What we shall keep, as the examples below illustrate, is that a homomorphism describes a way in which the domain is "like", or "analogous to", the range.

Example 2.7

We think of $\mathbb {R} ^{3}$ as being like $\mathbb {R} ^{2}$, except that vectors have an extra component. That is, we think of the vector with components $x$, $y$, and $z$ as like the vector with components $x$ and $y$. In defining the projection map $\pi$, we make precise which members of the domain we are thinking of as related to which members of the codomain.

Understanding in what way the preservation conditions in the definition of homomorphism show that the domain elements are like the codomain elements is easiest if we draw $\mathbb {R} ^{2}$ as the $xy$-plane inside of $\mathbb {R} ^{3}$. (Of course, $\mathbb {R} ^{2}$ is a set of two-tall vectors while the $xy$-plane is a set of three-tall vectors with a third component of zero, but there is an obvious correspondence.) Then, $\pi ({\vec {v}})$ is the "shadow" of ${\vec {v}}$ in the plane and the preservation of addition property says that

${\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}$ above ${\begin{pmatrix}x_{1}\\y_{1}\end{pmatrix}}$	plus	${\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}}$ above ${\begin{pmatrix}x_{2}\\y_{2}\end{pmatrix}}$	equals	${\begin{pmatrix}x_{1}+y_{1}\\y_{1}+y_{2}\\z_{1}+z_{2}\end{pmatrix}}$ above ${\begin{pmatrix}x_{1}+x_{2}\\y_{1}+y_{2}\end{pmatrix}}$

Briefly, the shadow of a sum $\pi ({\vec {v}}_{1}+{\vec {v}}_{2})$ equals the sum of the shadows $\pi ({\vec {v}}_{1})+\pi ({\vec {v}}_{2})$. (Preservation of scalar multiplication has a similar interpretation.)

Redrawing by separating the two spaces, moving the codomain $\mathbb {R} ^{2}$ to the right, gives an uglier picture but one that is more faithful to the "bean" sketch.

Again in this drawing, the vectors that map to ${\vec {w}}_{1}$ lie in the domain in a vertical line (only one such vector is shown, in gray). Call any such member of this inverse image a "${\vec {w}}_{1}$ vector". Similarly, there is a vertical line of "${\vec {w}}_{2}$ vectors" and a vertical line of "${\vec {w}}_{1}+{\vec {w}}_{2}$ vectors". Now, $\pi$ has the property that if $\pi ({\vec {v}}_{1})={\vec {w}}_{1}$ and $\pi ({\vec {v}}_{2})={\vec {w}}_{2}$ then $\pi ({\vec {v}}_{1}+{\vec {v}}_{2})=\pi ({\vec {v}}_{1})+\pi ({\vec {v}}_{2})={\vec {w}}_{1}+{\vec {w}}_{2}$. This says that the vector classes add, in the sense that any ${\vec {w}}_{1}$ vector plus any ${\vec {w}}_{2}$ vector equals a ${\vec {w}}_{1}+{\vec {w}}_{2}$ vector, (A similar statement holds about the classes under scalar multiplication.)

Thus, although the two spaces $\mathbb {R} ^{3}$ and $\mathbb {R} ^{2}$ are not isomorphic, $\pi$ describes a way in which they are alike: vectors in $\mathbb {R} ^{3}$ add as do the associated vectors in $\mathbb {R} ^{2}$— vectors add as their shadows add.

Example 2.8

A homomorphism can be used to express an analogy between spaces that is more subtle than the prior one. For the map

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {h}{\longmapsto }}x+y$

from Example 2.6 fix two numbers $w_{1},w_{2}$ in the range $\mathbb {R}$. A ${\vec {v}}_{1}$ that maps to $w_{1}$ has components that add to $w_{1}$, that is, the inverse image $h^{-1}(w_{1})$ is the set of vectors with endpoint on the diagonal line $x+y=w_{1}$. Call these the "$w_{1}$ vectors". Similarly, we have the "$w_{2}$ vectors" and the "$w_{1}+w_{2}$ vectors". Then the addition preservation property says that

a "$w_{1}$ vector"	plus	a "$w_{2}$ vector"	equals	a "$w_{1}+w_{2}$ vector".

Restated, if a $w_{1}$ vector is added to a $w_{2}$ vector then the result is mapped by $h$ to a $w_{1}+w_{2}$ vector. Briefly, the image of a sum is the sum of the images. Even more briefly, $h({\vec {v}}_{1}+{\vec {v}}_{2})=h({\vec {v}}_{1})+h({\vec {v}}_{2})$. (The preservation of scalar multiplication condition has a similar restatement.)

Example 2.9

The inverse images can be structures other than lines. For the linear map $h:\mathbb {R} ^{3}\to \mathbb {R} ^{2}$

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}x\\x\end{pmatrix}}$

the inverse image sets are planes $x=0$, $x=1$, etc., perpendicular to the $x$-axis.

We won't describe how every homomorphism that we will use is an analogy because the formal sense that we make of "alike in that ..." is "a homomorphism exists such that ...". Nonetheless, the idea that a homomorphism between two spaces expresses how the domain's vectors fall into classes that act like the range's vectors is a good way to view homomorphisms.

Another reason that we won't treat all of the homomorphisms that we see as above is that many vector spaces are hard to draw (e.g., a space of polynomials). However, there is nothing bad about gaining insights from those spaces that we are able to draw, especially when those insights extend to all vector spaces. We derive two such insights from the three examples 2.7 , 2.8, and 2.9.

First, in all three examples, the inverse images are lines or planes, that is, linear surfaces. In particular, the inverse image of the range's zero vector is a line or plane through the origin— a subspace of the domain.

Lemma 2.10

For any homomorphism, the inverse image of a subspace of the range is a subspace of the domain. In particular, the inverse image of the trivial subspace of the range is a subspace of the domain.

Proof

Let $h:V\to W$ be a homomorphism and let $S$ be a subspace of the rangespace $h$. Consider $h^{-1}(S)=\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})\in S\}$, the inverse image of the set $S$. It is nonempty because it contains ${\vec {0}}_{V}$, since $h({\vec {0}}_{V})={\vec {0}}_{W}$, which is an element $S$, as $S$ is a subspace. To show that $h^{-1}(S)$ is closed under linear combinations, let ${\vec {v}}_{1}$ and ${\vec {v}}_{2}$ be elements, so that $h({\vec {v}}_{1})$ and $h({\vec {v}}_{2})$ are elements of $S$, and then $c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}$ is also in the inverse image because $h(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})=c_{1}h({\vec {v}}_{1})+c_{2}h({\vec {v}}_{2})$ is a member of the subspace $S$.

Definition 2.11

The nullspace or kernel of a linear map $h:V\to W$ is the inverse image of $0_{W}$

${\mathcal {N}}(h)=h^{-1}({\vec {0}}_{W})=\{{\vec {v}}\in V\,{\big |}\,h({\vec {v}})={\vec {0}}_{W}\}.$

The dimension of the nullspace is the map's nullity.

Example 2.12

The map from Example 2.3 has this nullspace ${\mathcal {N}}(d/dx)=\{a_{0}+0x+0x^{2}+0x^{3}\,{\big |}\,a_{0}\in \mathbb {R} \}$.

Example 2.13

The map from Example 2.4 has this nullspace.

${\mathcal {N}}(h)=\{{\begin{pmatrix}a&b\\0&-(a+b)/2\end{pmatrix}}\,{\big |}\,a,b\in \mathbb {R} \}$

Now for the second insight from the above pictures. In Example 2.7, each of the vertical lines is squashed down to a single point— $\pi$, in passing from the domain to the range, takes all of these one-dimensional vertical lines and "zeroes them out", leaving the range one dimension smaller than the domain. Similarly, in Example 2.8, the two-dimensional domain is mapped to a one-dimensional range by breaking the domain into lines (here, they are diagonal lines), and compressing each of those lines to a single member of the range. Finally, in Example 2.9, the domain breaks into planes which get "zeroed out", and so the map starts with a three-dimensional domain but ends with a one-dimensional range— this map "subtracts" two from the dimension. (Notice that, in this third example, the codomain is two-dimensional but the range of the map is only one-dimensional, and it is the dimension of the range that is of interest.)

Theorem 2.14

A linear map's rank plus its nullity equals the dimension of its domain.

Proof

Let $h:V\to W$ be linear and let $B_{N}=\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{k}\rangle$ be a basis for the nullspace. Extend that to a basis $B_{V}=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{k},{\vec {\beta }}_{k+1},\dots ,{\vec {\beta }}_{n}\rangle$ for the entire domain. We shall show that $B_{R}=\langle h({\vec {\beta }}_{k+1}),\dots ,h({\vec {\beta }}_{n})\rangle$ is a basis for the rangespace. Then counting the size of these bases gives the result.

To see that $B_{R}$ is linearly independent, consider the equation $c_{k+1}h({\vec {\beta }}_{k+1})+\dots +c_{n}h({\vec {\beta }}_{n})={\vec {0}}_{W}$. This gives that $h(c_{k+1}{\vec {\beta }}_{k+1}+\dots +c_{n}{\vec {\beta }}_{n})={\vec {0}}_{W}$ and so $c_{k+1}{\vec {\beta }}_{k+1}+\dots +c_{n}{\vec {\beta }}_{n}$ is in the nullspace of $h$. As $B_{N}$ is a basis for this nullspace, there are scalars $c_{1},\dots ,c_{k}\in \mathbb {R}$ satisfying this relationship.

$c_{1}{\vec {\beta }}_{1}+\dots +c_{k}{\vec {\beta }}_{k}=c_{k+1}{\vec {\beta }}_{k+1}+\dots +c_{n}{\vec {\beta }}_{n}$

But $B_{V}$ is a basis for $V$ so each scalar equals zero. Therefore $B_{R}$ is linearly independent.

To show that $B_{R}$ spans the rangespace, consider $h({\vec {v}})\in {\mathcal {R}}(h)$ and write ${\vec {v}}$ as a linear combination ${\vec {v}}=c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n}$ of members of $B_{V}$. This gives $h({\vec {v}})=h(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})=c_{1}h({\vec {\beta }}_{1})+\dots +c_{k}h({\vec {\beta }}_{k})+c_{k+1}h({\vec {\beta }}_{k+1})+\dots +c_{n}h({\vec {\beta }}_{n})$ and since ${\vec {\beta }}_{1}$, ..., ${\vec {\beta }}_{k}$ are in the nullspace, we have that $h({\vec {v}})={\vec {0}}+\dots +{\vec {0}}+c_{k+1}h({\vec {\beta }}_{k+1})+\dots +c_{n}h({\vec {\beta }}_{n})$. Thus, $h({\vec {v}})$ is a linear combination of members of $B_{R}$, and so $B_{R}$ spans the space.

Example 2.15

Where $h:\mathbb {R} ^{3}\to \mathbb {R} ^{4}$ is

${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {h}{\longmapsto }}{\begin{pmatrix}x\\0\\y\\0\end{pmatrix}}$

the rangespace and nullspace are

${\mathcal {R}}(h)=\{{\begin{pmatrix}a\\0\\b\\0\end{pmatrix}}\,{\big |}\,a,b\in \mathbb {R} \}\quad {\text{and}}\quad {\mathcal {N}}(h)=\{{\begin{pmatrix}0\\0\\z\end{pmatrix}}\,{\big |}\,z\in \mathbb {R} \}$

and so the rank of $h$ is two while the nullity is one.

Example 2.16

If $t:\mathbb {R} \to \mathbb {R}$ is the linear transformation $x\mapsto -4x,$ then the range is ${\mathcal {R}}(t)=\mathbb {R} ^{1}$, and so the rank of $t$ is one and the nullity is zero.

Corollary 2.17

The rank of a linear map is less than or equal to the dimension of the domain. Equality holds if and only if the nullity of the map is zero.

We know that an isomorphism exists between two spaces if and only if their dimensions are equal. Here we see that for a homomorphism to exist, the dimension of the range must be less than or equal to the dimension of the domain. For instance, there is no homomorphism from $\mathbb {R} ^{2}$ onto $\mathbb {R} ^{3}$. There are many homomorphisms from $\mathbb {R} ^{2}$ into $\mathbb {R} ^{3}$, but none is onto all of three-space.

The rangespace of a linear map can be of dimension strictly less than the dimension of the domain (Example 2.3's derivative transformation on ${\mathcal {P}}_{3}$ has a domain of dimension four but a range of dimension three). Thus, under a homomorphism, linearly independent sets in the domain may map to linearly dependent sets in the range (for instance, the derivative sends $\{1,x,x^{2},x^{3}\}$ to $\{0,1,2x,3x^{2}\}$). That is, under a homomorphism, independence may be lost. In contrast, dependence stays.

Lemma 2.18

Under a linear map, the image of a linearly dependent set is linearly dependent.

Proof

Suppose that $c_{1}{\vec {v}}_{1}+\dots +c_{n}{\vec {v}}_{n}={\vec {0}}_{V}$, with some $c_{i}$ nonzero. Then, because $h(c_{1}{\vec {v}}_{1}+\dots +c_{n}{\vec {v}}_{n})=c_{1}h({\vec {v}}_{1})+\dots +c_{n}h({\vec {v}}_{n})$ and because $h({\vec {0}}_{V})={\vec {0}}_{W}$, we have that $c_{1}h({\vec {v}}_{1})+\dots +c_{n}h({\vec {v}}_{n})={\vec {0}}_{W}$ with some nonzero $c_{i}$.

When is independence not lost? One obvious sufficient condition is when the homomorphism is an isomorphism. This condition is also necessary; see Problem 14. We will finish this subsection comparing homomorphisms with isomorphisms by observing that a one-to-one homomorphism is an isomorphism from its domain onto its range.

Definition 2.19

A linear map that is one-to-one is nonsingular.

(In the next section we will see the connection between this use of "nonsingular" for maps and its familiar use for matrices.)

Example 2.20

This nonsingular homomorphism $\iota :\mathbb {R} ^{2}\to \mathbb {R} ^{3}$

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {\iota }{\longmapsto }}{\begin{pmatrix}x\\y\\0\end{pmatrix}}$

gives the obvious correspondence between $\mathbb {R} ^{2}$ and the $xy$-plane inside of $\mathbb {R} ^{3}$.

The prior observation allows us to adapt some results about isomorphisms to this setting.

Theorem 2.21

In an $n$-dimensional vector space $V$, these:

1. $h$ is nonsingular, that is, one-to-one
2. $h$ has a linear inverse
3. ${\mathcal {N}}(h)=\{{\vec {0}}\,\}$, that is, ${\text{nullity}}\,(h)=0$
4. $\mathop {\mbox{rank}} (h)=n$
5. if $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ is a basis for $V$ then $\langle h({\vec {\beta }}_{1}),\dots ,h({\vec {\beta }}_{n})\rangle$is a basis for ${\mathcal {R}}(h)$

are equivalent statements about a linear map $h:V\to W$.

Proof

We will first show that $1\Longleftrightarrow 2$. We will then show that $1\implies 3\implies 4\implies 5\implies 2$.

For $1\Longrightarrow 2$, suppose that the linear map $h$ is one-to-one, and so has an inverse. The domain of that inverse is the range of $h$ and so a linear combination of two members of that domain has the form $c_{1}h({\vec {v}}_{1})+c_{2}h({\vec {v}}_{2})$. On that combination, the inverse $h^{-1}$ gives this.

${\begin{array}{rl}h^{-1}(c_{1}h({\vec {v}}_{1})+c_{2}h({\vec {v}}_{2}))&=h^{-1}(h(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}))\\&=h^{-1}\circ h\,(c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2})\\&=c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}\\&=c_{1}h^{-1}\circ h\,({\vec {v}}_{1})+c_{2}h^{-1}\circ h\,({\vec {v}}_{2})\\&=c_{1}\cdot h^{-1}(h({\vec {v}}_{1}))+c_{2}\cdot h^{-1}(h({\vec {v}}_{2}))\end{array}}$

Thus the inverse of a one-to-one linear map is automatically linear. But this also gives the $1\Longrightarrow 2$ implication, because the inverse itself must be one-to-one.

Of the remaining implications, $1\implies 3$ holds because any homomorphism maps ${\vec {0}}_{V}$ to ${\vec {0}}_{W}$, but a one-to-one map sends at most one member of $V$ to ${\vec {0}}_{W}$.

Next, $3\implies 4$ is true since rank plus nullity equals the dimension of the domain.

For $4\implies 5$, to show that $\langle h({\vec {\beta }}_{1}),\dots ,h({\vec {\beta }}_{n})\rangle$ is a basis for the rangespace we need only show that it is a spanning set, because by assumption the range has dimension $n$. Consider $h({\vec {v}})\in {\mathcal {R}}(h)$. Expressing ${\vec {v}}$ as a linear combination of basis elements produces $h({\vec {v}})=h(c_{1}{\vec {\beta }}_{1}+c_{2}{\vec {\beta }}_{2}+\cdots +c_{n}{\vec {\beta }}_{n})$, which gives that $h({\vec {v}})=c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})$, as desired.

Finally, for the $5\implies 2$ implication, assume that $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ is a basis for $V$ so that $\langle h({\vec {\beta }}_{1}),\dots ,h({\vec {\beta }}_{n})\rangle$ is a basis for ${\mathcal {R}}(h)$. Then every ${\vec {w}}\in {\mathcal {R}}(h)$ a the unique representation ${\vec {w}}=c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})$. Define a map from ${\mathcal {R}}(h)$ to $V$ by

${\vec {w}}\;\mapsto \;c_{1}{\vec {\beta }}_{1}+c_{2}{\vec {\beta }}_{2}+\cdots +c_{n}{\vec {\beta }}_{n}$

(uniqueness of the representation makes this well-defined). Checking that it is linear and that it is the inverse of $h$ are easy.

We've now seen that a linear map shows how the structure of the domain is like that of the range. Such a map can be thought to organize the domain space into inverse images of points in the range. In the special case that the map is one-to-one, each inverse image is a single point and the map is an isomorphism between the domain and the range.

Exercises

This exercise is recommended for all readers.

Problem 1

Let $h:{\mathcal {P}}_{3}\to {\mathcal {P}}_{4}$ be given by $p(x)\mapsto x\cdot p(x)$. Which of these are in the nullspace? Which are in the rangespace?

1. $x^{3}$
2. $0$
3. $7$
4. $12x-0.5x^{3}$
5. $1+3x^{2}-x^{3}$

This exercise is recommended for all readers.

Problem 2

Find the nullspace, nullity, rangespace, and rank of each map.

1. $h:\mathbb {R} ^{2}\to {\mathcal {P}}_{3}$ given by

   ${\begin{pmatrix}a\\b\end{pmatrix}}\mapsto a+ax+ax^{2}$

2. $h:{\mathcal {M}}_{2\!\times \!2}\to \mathbb {R}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto a+d$

3. $h:{\mathcal {M}}_{2\!\times \!2}\to {\mathcal {P}}_{2}$ given by

   ${\begin{pmatrix}a&b\\c&d\end{pmatrix}}\mapsto a+b+c+dx^{2}$

4. the zero map $Z:\mathbb {R} ^{3}\to \mathbb {R} ^{4}$

This exercise is recommended for all readers.

Problem 3

Find the nullity of each map.

1. $h:\mathbb {R} ^{5}\to \mathbb {R} ^{8}$ of rank five
2. $h:{\mathcal {P}}_{3}\to {\mathcal {P}}_{3}$ of rank one
3. $h:\mathbb {R} ^{6}\to \mathbb {R} ^{3}$, an onto map
4. $h:{\mathcal {M}}_{3\!\times \!3}\to {\mathcal {M}}_{3\!\times \!3}$, onto

This exercise is recommended for all readers.

Problem 4

What is the nullspace of the differentiation transformation $d/dx:{\mathcal {P}}_{n}\to {\mathcal {P}}_{n}$? What is the nullspace of the second derivative, as a transformation of ${\mathcal {P}}_{n}$? The $k$-th derivative?

Problem 5

Example 2.7 restates the first condition in the definition of homomorphism as "the shadow of a sum is the sum of the shadows". Restate the second condition in the same style.

Problem 6

For the homomorphism $h:{\mathcal {P}}_{3}\to {\mathcal {P}}_{3}$ given by $h(a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3})=a_{0}+(a_{0}+a_{1})x+(a_{2}+a_{3})x^{3}$ find these.

1. ${\mathcal {N}}(h)$
2. $h^{-1}(2-x^{3})$
3. $h^{-1}(1+x^{2})$

This exercise is recommended for all readers.

Problem 7

For the map $f:\mathbb {R} ^{2}\to \mathbb {R}$ given by

$f({\begin{pmatrix}x\\y\end{pmatrix}})=2x+y$

sketch these inverse image sets: $f^{-1}(-3)$, $f^{-1}(0)$, and $f^{-1}(1)$.

This exercise is recommended for all readers.

Problem 8

Each of these transformations of ${\mathcal {P}}_{3}$ is nonsingular. Find the inverse function of each.

1. $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\mapsto a_{0}+a_{1}x+2a_{2}x^{2}+3a_{3}x^{3}$
2. $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\mapsto a_{0}+a_{2}x+a_{1}x^{2}+a_{3}x^{3}$
3. $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\mapsto a_{1}+a_{2}x+a_{3}x^{2}+a_{0}x^{3}$
4. $a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\mapsto a_{0}+(a_{0}+a_{1})x+(a_{0}+a_{1}+a_{2})x^{2}+(a_{0}+a_{1}+a_{2}+a_{3})x^{3}$

Problem 9

Describe the nullspace and rangespace of a transformation given by ${\vec {v}}\mapsto 2{\vec {v}}$.

Problem 10

List all pairs $({\text{rank}}(h),{\text{nullity}}(h))$ that are possible for linear maps from $\mathbb {R} ^{5}$ to $\mathbb {R} ^{3}$.

Problem 11

Does the differentiation map $d/dx:{\mathcal {P}}_{n}\to {\mathcal {P}}_{n}$ have an inverse?

This exercise is recommended for all readers.

Problem 12

Find the nullity of the map $h:{\mathcal {P}}_{n}\to \mathbb {R}$ given by

$a_{0}+a_{1}x+\dots +a_{n}x^{n}\mapsto \int _{x=0}^{x=1}a_{0}+a_{1}x+\dots +a_{n}x^{n}\,dx.$

Problem 13

1. Prove that a homomorphism is onto if and only if its rank equals the dimension of its codomain.
2. Conclude that a homomorphism between vector spaces with the same dimension is one-to-one if and only if it is onto.

Problem 14

Show that a linear map is nonsingular if and only if it preserves linear independence.

Problem 15

Corollary 2.17 says that for there to be an onto homomorphism from a vector space $V$ to a vector space $W$, it is necessary that the dimension of $W$ be less than or equal to the dimension of $V$. Prove that this condition is also sufficient; use Theorem 1.9 to show that if the dimension of $W$ is less than or equal to the dimension of $V$, then there is a homomorphism from $V$ to $W$ that is onto.

Problem 16

Let $h:V\to \mathbb {R}$ be a homomorphism, but not the zero homomorphism. Prove that if $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ is a basis for the nullspace and if ${\vec {v}}\in V$ is not in the nullspace then $\langle {\vec {v}},{\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ is a basis for the entire domain $V$.

This exercise is recommended for all readers.

Problem 17

Recall that the nullspace is a subset of the domain and the rangespace is a subset of the codomain. Are they necessarily distinct? Is there a homomorphism that has a nontrivial intersection of its nullspace and its rangespace?

Problem 18

Prove that the image of a span equals the span of the images. That is, where $h:V\to W$ is linear, prove that if $S$ is a subset of $V$ then $h([S])$ equals $[h(S)]$. This generalizes Lemma 2.1 since it shows that if $U$ is any subspace of $V$ then its image $\{h({\vec {u}})\,{\big |}\,{\vec {u}}\in U\}$ is a subspace of $W$, because the span of the set $U$ is $U$.

This exercise is recommended for all readers.

Problem 19

1. Prove that for any linear map $h:V\to W$ and any ${\vec {w}}\in W$, the set $h^{-1}({\vec {w}})$ has the form

   $\{{\vec {v}}+{\vec {n}}\,{\big |}\,{\vec {n}}\in {\mathcal {N}}(h)\}$

   for ${\vec {v}}\in V$ with $h({\vec {v}})={\vec {w}}$ (if $h$ is not onto then this set may be empty). Such a set is a coset of ${\mathcal {N}}(h)$ and is denoted ${\vec {v}}+{\mathcal {N}}(h)$.
2. Consider the map $t:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ given by

   ${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {t}{\longmapsto }}{\begin{pmatrix}ax+by\\cx+dy\end{pmatrix}}$

   for some scalars $a$, $b$, $c$, and $d$. Prove that $t$ is linear.
3. Conclude from the prior two items that for any linear system of the form

   ${\begin{array}{*{2}{rc}r}ax&+&by&=&e\\cx&+&dy&=&f\end{array}}$

   the solution set can be written (the vectors are members of $\mathbb {R} ^{2}$)

   $\{{\vec {p}}+{\vec {h}}\,{\big |}\,{\vec {h}}{\text{ satisfies the associated homogeneous system}}\}$

   where ${\vec {p}}$ is a particular solution of that linear system (if there is no particular solution then the above set is empty).
4. Show that this map $h:\mathbb {R} ^{n}\to \mathbb {R} ^{m}$ is linear

   ${\begin{pmatrix}x_{1}\\\vdots \\x_{n}\end{pmatrix}}\mapsto {\begin{pmatrix}a_{1,1}x_{1}+\dots +a_{1,n}x_{n}\\\vdots \\a_{m,1}x_{1}+\dots +a_{m,n}x_{n}\end{pmatrix}}$

   for any scalars $a_{1,1}$, ..., $a_{m,n}$. Extend the conclusion made in the prior item.
5. Show that the $k$-th derivative map is a linear transformation of ${\mathcal {P}}_{n}$ for each $k$. Prove that this map is a linear transformation of that space

   $f\mapsto {\frac {d^{k}}{dx^{k}}}f+c_{k-1}{\frac {d^{k-1}}{dx^{k-1}}}f+\dots +c_{1}{\frac {d}{dx}}f+c_{0}f$

   for any scalars $c_{k}$, ..., $c_{0}$. Draw a conclusion as above.

Problem 20

Prove that for any transformation $t:V\to V$ that is rank one, the map given by composing the operator with itself $t\circ t:V\to V$ satisfies $t\circ t=r\cdot t$ for some real number $r$.

Problem 21

Show that for any space $V$ of dimension $n$, the dual space

$\mathop {\mathcal {L}} (V,\mathbb {R} )=\{h:V\to \mathbb {R} \,{\big |}\,h{\text{ is linear}}\}$

is isomorphic to $\mathbb {R} ^{n}$. It is often denoted $V^{\ast }$. Conclude that $V^{\ast }\cong V$.

Problem 22

Show that any linear map is the sum of maps of rank one.

Problem 23

Is "is homomorphic to" an equivalence relation? (Hint: the difficulty is to decide on an appropriate meaning for the quoted phrase.)

Problem 24

Show that the rangespaces and nullspaces of powers of linear maps $t:V\to V$ form descending

$V\supseteq {\mathcal {R}}(t)\supseteq {\mathcal {R}}(t^{2})\supseteq \ldots$

and ascending

$\{{\vec {0}}\}\subseteq {\mathcal {N}}(t)\subseteq {\mathcal {N}}(t^{2})\subseteq \ldots$

chains. Also show that if $k$ is such that ${\mathcal {R}}(t^{k})={\mathcal {R}}(t^{k+1})$ then all following rangespaces are equal: ${\mathcal {R}}(t^{k})={\mathcal {R}}(t^{k+1})={\mathcal {R}}(t^{k+2})\ldots \,$. Similarly, if ${\mathcal {N}}(t^{k})={\mathcal {N}}(t^{k+1})$ then ${\mathcal {N}}(t^{k})={\mathcal {N}}(t^{k+1})={\mathcal {N}}(t^{k+2})=\ldots \,$.

Section III - Computing Linear Maps

The prior section shows that a linear map is determined by its action on a basis. In fact, the equation

$h({\vec {v}})=h(c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{n}\cdot {\vec {\beta }}_{n})=c_{1}\cdot h({\vec {\beta }}_{1})+\dots +c_{n}\cdot h({\vec {\beta }}_{n})$

shows that, if we know the value of the map on the vectors in a basis, then we can compute the value of the map on any vector ${\vec {v}}$ at all. We just need to find the $c$'s to express ${\vec {v}}$ with respect to the basis.

This section gives the scheme that computes, from the representation of a vector in the domain ${\rm {Rep}}_{B}({\vec {v}})$, the representation of that vector's image in the codomain ${\rm {Rep}}_{D}(h({\vec {v}}))$, using the representations of $h({\vec {\beta }}_{1})$, ..., $h({\vec {\beta }}_{n})$.

1 - Representing Linear Maps with Matrices

Example 1.1

Consider a map $h$ with domain $\mathbb {R} ^{2}$ and codomain $\mathbb {R} ^{3}$ (fixing

$B=\langle {\begin{pmatrix}2\\0\end{pmatrix}},{\begin{pmatrix}1\\4\end{pmatrix}}\rangle \quad {\text{and}}\quad D=\langle {\begin{pmatrix}1\\0\\0\end{pmatrix}},{\begin{pmatrix}0\\-2\\0\end{pmatrix}},{\begin{pmatrix}1\\0\\1\end{pmatrix}}\rangle$

as the bases for these spaces) that is determined by this action on the vectors in the domain's basis.

${\begin{pmatrix}2\\0\end{pmatrix}}{\stackrel {h}{\longmapsto }}{\begin{pmatrix}1\\1\\1\end{pmatrix}}\qquad {\begin{pmatrix}1\\4\end{pmatrix}}{\stackrel {h}{\longmapsto }}{\begin{pmatrix}1\\2\\0\end{pmatrix}}$

To compute the action of this map on any vector at all from the domain, we first express $h({\vec {\beta }}_{1})$ and $h({\vec {\beta }}_{2})$ with respect to the codomain's basis:

${\begin{pmatrix}1\\1\\1\end{pmatrix}}=0{\begin{pmatrix}1\\0\\0\end{pmatrix}}-{\frac {1}{2}}{\begin{pmatrix}0\\-2\\0\end{pmatrix}}+1{\begin{pmatrix}1\\0\\1\end{pmatrix}}\quad {\text{so}}\quad {\rm {Rep}}_{D}(h({\vec {\beta }}_{1}))={\begin{pmatrix}0\\-1/2\\1\end{pmatrix}}_{D}$

and

${\begin{pmatrix}1\\2\\0\end{pmatrix}}=1{\begin{pmatrix}1\\0\\0\end{pmatrix}}-1{\begin{pmatrix}0\\-2\\0\end{pmatrix}}+0{\begin{pmatrix}1\\0\\1\end{pmatrix}}\quad {\text{so}}\quad {\rm {Rep}}_{D}(h({\vec {\beta }}_{2}))={\begin{pmatrix}1\\-1\\0\end{pmatrix}}_{D}$

(these are easy to check). Then, as described in the preamble, for any member ${\vec {v}}$ of the domain, we can express the image $h({\vec {v}})$ in terms of the $h({\vec {\beta }})$'s.

${\begin{array}{rl}h({\vec {v}})&=h(c_{1}\cdot {\begin{pmatrix}2\\0\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}1\\4\end{pmatrix}})\\&=c_{1}\cdot h({\begin{pmatrix}2\\0\end{pmatrix}})+c_{2}\cdot h({\begin{pmatrix}1\\4\end{pmatrix}})\\&=c_{1}\cdot (0{\begin{pmatrix}1\\0\\0\end{pmatrix}}\!-{\frac {1}{2}}{\begin{pmatrix}0\\-2\\0\end{pmatrix}}\!+1{\begin{pmatrix}1\\0\\1\end{pmatrix}}\!)+c_{2}\cdot (1{\begin{pmatrix}1\\0\\0\end{pmatrix}}\!-1{\begin{pmatrix}0\\-2\\0\end{pmatrix}}\!+0{\begin{pmatrix}1\\0\\1\end{pmatrix}}\!)\\&=(0c_{1}+1c_{2})\cdot {\begin{pmatrix}1\\0\\0\end{pmatrix}}+(-{\frac {1}{2}}c_{1}-1c_{2})\cdot {\begin{pmatrix}0\\-2\\0\end{pmatrix}}+(1c_{1}+0c_{2})\cdot {\begin{pmatrix}1\\0\\1\end{pmatrix}}\end{array}}$

Thus,

with ${\rm {Rep}}_{B}({\vec {v}})={\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}$ then ${\rm {Rep}}_{D}(\,h({\vec {v}})\,)={\begin{pmatrix}0c_{1}+1c_{2}\\-(1/2)c_{1}-1c_{2}\\1c_{1}+0c_{2}\end{pmatrix}}$.

For instance,

with ${\rm {Rep}}_{B}({\begin{pmatrix}4\\8\end{pmatrix}})={\begin{pmatrix}1\\2\end{pmatrix}}_{B}$ then ${\rm {Rep}}_{D}(\,h({\begin{pmatrix}4\\8\end{pmatrix}})\,)={\begin{pmatrix}2\\-5/2\\1\end{pmatrix}}$.

We will express computations like the one above with a matrix notation.

${\begin{pmatrix}0&1\\-1/2&-1\\1&0\end{pmatrix}}_{B,D}{\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}_{B}={\begin{pmatrix}0c_{1}+1c_{2}\\(-1/2)c_{1}-1c_{2}\\1c_{1}+0c_{2}\end{pmatrix}}_{D}$

In the middle is the argument ${\vec {v}}$ to the map, represented with respect to the domain's basis $B$ by a column vector with components $c_{1}$ and $c_{2}$. On the right is the value $h({\vec {v}})$ of the map on that argument, represented with respect to the codomain's basis $D$ by a column vector with components $0c_{1}+1c_{2}$, etc. The matrix on the left is the new thing. It consists of the coefficients from the vector on the right, $0$ and $1$ from the first row, $-1/2$ and $-1$ from the second row, and $1$ and $0$ from the third row.

This notation simply breaks the parts from the right, the coefficients and the $c$'s, out separately on the left, into a vector that represents the map's argument and a matrix that we will take to represent the map itself.

Definition 1.2

Suppose that $V$ and $W$ are vector spaces of dimensions $n$ and $m$ with bases $B$ and $D$, and that $h:V\to W$ is a linear map. If

${\rm {Rep}}_{D}(h({\vec {\beta }}_{1}))={\begin{pmatrix}h_{1,1}\\h_{2,1}\\\vdots \\h_{m,1}\end{pmatrix}}_{D}\;\ldots \;{\rm {Rep}}_{D}(h({\vec {\beta }}_{n}))={\begin{pmatrix}h_{1,n}\\h_{2,n}\\\vdots \\h_{m,n}\end{pmatrix}}_{D}$

then

${\rm {Rep}}_{B,D}(h)=\left({\begin{array}{cccc}h_{1,1}&h_{1,2}&\ldots &h_{1,n}\\h_{2,1}&h_{2,2}&\ldots &h_{2,n}\\&\vdots \\h_{m,1}&h_{m,2}&\ldots &h_{m,n}\end{array}}\right)_{B,D}$

is the matrix representation of $h$ with respect to $B,D$.

Briefly, the vectors representing the $h({\vec {\beta }})$'s are adjoined to make the matrix representing the map.

${\rm {Rep}}_{B,D}(h)=\left({\begin{array}{c|c|c}\vdots &&\vdots \\{\rm {Rep}}_{D}(\,h({\vec {\beta }}_{1})\,)&\cdots &{\rm {Rep}}_{D}(\,h({\vec {\beta }}_{n})\,)\\\vdots &&\vdots \end{array}}\right)$

Observe that the number of columns $n$ of the matrix is the dimension of the domain of the map, and the number of rows $m$ is the dimension of the codomain.

Example 1.3

If $h:\mathbb {R} ^{3}\to {\mathcal {P}}_{1}$ is given by

${\begin{pmatrix}a_{1}\\a_{2}\\a_{3}\end{pmatrix}}{\stackrel {h}{\longmapsto }}(2a_{1}+a_{2})+(-a_{3})x$

then where

$B=\langle {\begin{pmatrix}0\\0\\1\end{pmatrix}},{\begin{pmatrix}0\\2\\0\end{pmatrix}},{\begin{pmatrix}2\\0\\0\end{pmatrix}}\rangle \quad {\text{and}}\quad D=\langle 1+x,-1+x\rangle$

the action of $h$ on $B$ is given by

${\begin{pmatrix}0\\0\\1\end{pmatrix}}{\stackrel {h}{\longmapsto }}-x\qquad {\begin{pmatrix}0\\2\\0\end{pmatrix}}{\stackrel {h}{\longmapsto }}2\qquad {\begin{pmatrix}2\\0\\0\end{pmatrix}}{\stackrel {h}{\longmapsto }}4$

and a simple calculation gives

${\rm {Rep}}_{D}(-x)={\begin{pmatrix}-1/2\\-1/2\end{pmatrix}}_{D}\quad {\rm {Rep}}_{D}(2)={\begin{pmatrix}1\\-1\end{pmatrix}}_{D}\quad {\rm {Rep}}_{D}(4)={\begin{pmatrix}2\\-2\end{pmatrix}}_{D}$

showing that this is the matrix representing $h$ with respect to the bases.

${\rm {Rep}}_{B,D}(h)={\begin{pmatrix}-1/2&1&2\\-1/2&-1&-2\end{pmatrix}}_{B,D}$

We will use lower case letters for a map, upper case for the matrix, and lower case again for the entries of the matrix. Thus for the map $h$, the matrix representing it is $H$, with entries $h_{i,j}$.

Theorem 1.4

Assume that $V$ and $W$ are vector spaces of dimensions $m$ and $n$ with bases $B$ and $D$, and that $h:V\to W$ is a linear map. If $h$ is represented by

${\rm {Rep}}_{B,D}(h)=\left({\begin{array}{cccc}h_{1,1}&h_{1,2}&\ldots &h_{1,n}\\h_{2,1}&h_{2,2}&\ldots &h_{2,n}\\&\vdots \\h_{m,1}&h_{m,2}&\ldots &h_{m,n}\end{array}}\right)_{B,D}$

and ${\vec {v}}\in V$ is represented by

${\rm {Rep}}_{B}({\vec {v}})={\begin{pmatrix}c_{1}\\c_{2}\\\vdots \\c_{n}\end{pmatrix}}_{B}$

then the representation of the image of ${\vec {v}}$ is this.

${\rm {Rep}}_{D}(\,h({\vec {v}})\,)={\begin{pmatrix}h_{1,1}c_{1}+h_{1,2}c_{2}+\dots +h_{1,n}c_{n}\\h_{2,1}c_{1}+h_{2,2}c_{2}+\dots +h_{2,n}c_{n}\\\vdots \\h_{m,1}c_{1}+h_{m,2}c_{2}+\dots +h_{m,n}c_{n}\end{pmatrix}}_{D}$

Proof

Problem 18.

We will think of the matrix ${\rm {Rep}}_{B,D}(h)$ and the vector ${\rm {Rep}}_{B}({\vec {v}})$ as combining to make the vector ${\rm {Rep}}_{D}(h({\vec {v}}))$.

Definition 1.5

The matrix-vector product of a $m\!\times \!n$ matrix and a $n\!\times \!1$ vector is this.

$\left({\begin{array}{cccc}a_{1,1}&a_{1,2}&\ldots &a_{1,n}\\a_{2,1}&a_{2,2}&\ldots &a_{2,n}\\&\vdots \\a_{m,1}&a_{m,2}&\ldots &a_{m,n}\end{array}}\right){\begin{pmatrix}c_{1}\\\vdots \\c_{n}\end{pmatrix}}={\begin{pmatrix}a_{1,1}c_{1}+a_{1,2}c_{2}+\dots +a_{1,n}c_{n}\\a_{2,1}c_{1}+a_{2,2}c_{2}+\dots +a_{2,n}c_{n}\\\vdots \\a_{m,1}c_{1}+a_{m,2}c_{2}+\dots +a_{m,n}c_{n}\end{pmatrix}}$

The point of Definition 1.2 is to generalize Example 1.1, that is, the point of the definition is Theorem 1.4, that the matrix describes how to get from the representation of a domain vector with respect to the domain's basis to the representation of its image in the codomain with respect to the codomain's basis. With Definition 1.5, we can restate this as: application of a linear map is represented by the matrix-vector product of the map's representative and the vector's representative.

Example 1.6

With the matrix from Example 1.3 we can calculate where that map sends this vector.

${\vec {v}}={\begin{pmatrix}4\\1\\0\end{pmatrix}}$

This vector is represented, with respect to the domain basis $B$, by

${\rm {Rep}}_{B}({\vec {v}})={\begin{pmatrix}0\\1/2\\2\end{pmatrix}}_{B}$

and so this is the representation of the value $h({\vec {v}})$ with respect to the codomain basis $D$.

${\begin{array}{rl}{\rm {Rep}}_{D}(h({\vec {v}}))&={\begin{pmatrix}-1/2&1&2\\-1/2&-1&-2\end{pmatrix}}_{B,D}{\begin{pmatrix}0\\1/2\\2\end{pmatrix}}_{B}\\&={\begin{pmatrix}(-1/2)\cdot 0+1\cdot (1/2)+2\cdot 2\\(-1/2)\cdot 0-1\cdot (1/2)-2\cdot 2\end{pmatrix}}_{D}={\begin{pmatrix}9/2\\-9/2\end{pmatrix}}_{D}\end{array}}$

To find $h({\vec {v}})$ itself, not its representation, take $(9/2)(1+x)-(9/2)(-1+x)=9$.

Example 1.7

Let $\pi :\mathbb {R} ^{3}\to \mathbb {R} ^{2}$ be projection onto the $xy$-plane. To give a matrix representing this map, we first fix bases.

$B=\langle {\begin{pmatrix}1\\0\\0\end{pmatrix}},{\begin{pmatrix}1\\1\\0\end{pmatrix}},{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\rangle \qquad D=\langle {\begin{pmatrix}2\\1\end{pmatrix}},{\begin{pmatrix}1\\1\end{pmatrix}}\rangle$

For each vector in the domain's basis, we find its image under the map.

${\begin{pmatrix}1\\0\\0\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}1\\0\end{pmatrix}}\quad {\begin{pmatrix}1\\1\\0\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}1\\1\end{pmatrix}}\quad {\begin{pmatrix}-1\\0\\1\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}-1\\0\end{pmatrix}}$

Then we find the representation of each image with respect to the codomain's basis

${\rm {Rep}}_{D}({\begin{pmatrix}1\\0\end{pmatrix}})={\begin{pmatrix}1\\-1\end{pmatrix}}\quad {\rm {Rep}}_{D}({\begin{pmatrix}1\\1\end{pmatrix}})={\begin{pmatrix}0\\1\end{pmatrix}}\quad {\rm {Rep}}_{D}({\begin{pmatrix}-1\\0\end{pmatrix}})={\begin{pmatrix}-1\\1\end{pmatrix}}$

(these are easily checked). Finally, adjoining these representations gives the matrix representing $\pi$ with respect to $B,D$.

${\rm {Rep}}_{B,D}(\pi )={\begin{pmatrix}1&0&-1\\-1&1&1\end{pmatrix}}_{B,D}$

We can illustrate Theorem 1.4 by computing the matrix-vector product representing the following statement about the projection map.

$\pi ({\begin{pmatrix}2\\2\\1\end{pmatrix}})={\begin{pmatrix}2\\2\end{pmatrix}}$

Representing this vector from the domain with respect to the domain's basis

${\rm {Rep}}_{B}({\begin{pmatrix}2\\2\\1\end{pmatrix}})={\begin{pmatrix}1\\2\\1\end{pmatrix}}_{B}$

gives this matrix-vector product.

${\rm {Rep}}_{D}(\,\pi ({\begin{pmatrix}2\\2\\1\end{pmatrix}})\,)={\begin{pmatrix}1&0&-1\\-1&1&1\end{pmatrix}}_{B,D}{\begin{pmatrix}1\\2\\1\end{pmatrix}}_{B}={\begin{pmatrix}0\\2\end{pmatrix}}_{D}$

Expanding this representation into a linear combination of vectors from $D$

$0\cdot {\begin{pmatrix}2\\1\end{pmatrix}}+2\cdot {\begin{pmatrix}1\\1\end{pmatrix}}={\begin{pmatrix}2\\2\end{pmatrix}}$

checks that the map's action is indeed reflected in the operation of the matrix. (We will sometimes compress these three displayed equations into one

${\begin{pmatrix}2\\2\\1\end{pmatrix}}={\begin{pmatrix}1\\2\\1\end{pmatrix}}_{B}\;{\overset {h}{\underset {H}{\longmapsto }}}\;{\begin{pmatrix}0\\2\end{pmatrix}}_{D}={\begin{pmatrix}2\\2\end{pmatrix}}$

in the course of a calculation.)

We now have two ways to compute the effect of projection, the straightforward formula that drops each three-tall vector's third component to make a two-tall vector, and the above formula that uses representations and matrix-vector multiplication. Compared to the first way, the second way might seem complicated. However, it has advantages. The next example shows that giving a formula for some maps is simplified by this new scheme.

Example 1.8

To represent a rotation map $t_{\theta }:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that turns all vectors in the plane counterclockwise through an angle $\theta$

we start by fixing bases. Using ${\mathcal {E}}_{2}$ both as a domain basis and as a codomain basis is natural, Now, we find the image under the map of each vector in the domain's basis.

${\begin{pmatrix}1\\0\end{pmatrix}}{\stackrel {t_{\theta }}{\longmapsto }}{\begin{pmatrix}\cos \theta \\\sin \theta \end{pmatrix}}\qquad {\begin{pmatrix}0\\1\end{pmatrix}}{\stackrel {t_{\theta }}{\longmapsto }}{\begin{pmatrix}-\sin \theta \\\cos \theta \end{pmatrix}}$

Then we represent these images with respect to the codomain's basis. Because this basis is ${\mathcal {E}}_{2}$, vectors are represented by themselves. Finally, adjoining the representations gives the matrix representing the map.

${\rm {Rep}}_{{\mathcal {E}}_{2},{\mathcal {E}}_{2}}(t_{\theta })={\begin{pmatrix}\cos \theta &-\sin \theta \\\sin \theta &\cos \theta \end{pmatrix}}$

The advantage of this scheme is that just by knowing how to represent the image of the two basis vectors, we get a formula that tells us the image of any vector at all; here a vector rotated by $\theta =\pi /6$.

${\begin{pmatrix}3\\-2\end{pmatrix}}\;{\stackrel {t_{\pi /6}}{\longmapsto }}\;{\begin{pmatrix}{\sqrt {3}}/2&-1/2\\1/2&{\sqrt {3}}/2\end{pmatrix}}{\begin{pmatrix}3\\-2\end{pmatrix}}\approx {\begin{pmatrix}3.598\\-0.232\end{pmatrix}}$

(Again, we are using the fact that, with respect to ${\mathcal {E}}_{2}$, vectors represent themselves.)

We have already seen the addition and scalar multiplication operations of matrices and the dot product operation of vectors. Matrix-vector multiplication is a new operation in the arithmetic of vectors and matrices. Nothing in Definition 1.5 requires us to view it in terms of representations. We can get some insight into this operation by turning away from what is being represented, and instead focusing on how the entries combine.

Example 1.9

In the definition the width of the matrix equals the height of the vector. Hence, the first product below is defined while the second is not.

${\begin{pmatrix}1&0&0\\4&3&1\end{pmatrix}}{\begin{pmatrix}1\\0\\2\end{pmatrix}}={\begin{pmatrix}1\\6\end{pmatrix}}\qquad {\begin{pmatrix}1&0&0\\4&3&1\end{pmatrix}}{\begin{pmatrix}1\\0\end{pmatrix}}$

One reason that this product is not defined is purely formal: the definition requires that the sizes match, and these sizes don't match. Behind the formality, though, is a reason why we will leave it undefined— the matrix represents a map with a three-dimensional domain while the vector represents a member of a two-dimensional space.

A good way to view a matrix-vector product is as the dot products of the rows of the matrix with the column vector.

${\begin{pmatrix}&\vdots \\a_{i,1}&a_{i,2}&\ldots &a_{i,n}\\&\vdots \end{pmatrix}}{\begin{pmatrix}c_{1}\\c_{2}\\\vdots \\c_{n}\end{pmatrix}}={\begin{pmatrix}\vdots \\a_{i,1}c_{1}+a_{i,2}c_{2}+\ldots +a_{i,n}c_{n}\\\vdots \end{pmatrix}}$

Looked at in this row-by-row way, this new operation generalizes dot product.

Matrix-vector product can also be viewed column-by-column.

${\begin{array}{rl}\left({\begin{array}{cccc}h_{1,1}&h_{1,2}&\ldots &h_{1,n}\\h_{2,1}&h_{2,2}&\ldots &h_{2,n}\\&\vdots \\h_{m,1}&h_{m,2}&\ldots &h_{m,n}\end{array}}\right){\begin{pmatrix}c_{1}\\c_{2}\\\vdots \\c_{n}\end{pmatrix}}&={\begin{pmatrix}h_{1,1}c_{1}+h_{1,2}c_{2}+\dots +h_{1,n}c_{n}\\h_{2,1}c_{1}+h_{2,2}c_{2}+\dots +h_{2,n}c_{n}\\\vdots \\h_{m,1}c_{1}+h_{m,2}c_{2}+\dots +h_{m,n}c_{n}\end{pmatrix}}\\&=c_{1}{\begin{pmatrix}h_{1,1}\\h_{2,1}\\\vdots \\h_{m,1}\end{pmatrix}}+\dots +c_{n}{\begin{pmatrix}h_{1,n}\\h_{2,n}\\\vdots \\h_{m,n}\end{pmatrix}}\end{array}}$

Example 1.10

${\begin{pmatrix}1&0&-1\\2&0&3\end{pmatrix}}{\begin{pmatrix}2\\-1\\1\end{pmatrix}}=2{\begin{pmatrix}1\\2\end{pmatrix}}-1{\begin{pmatrix}0\\0\end{pmatrix}}+1{\begin{pmatrix}-1\\3\end{pmatrix}}={\begin{pmatrix}1\\7\end{pmatrix}}$

The result has the columns of the matrix weighted by the entries of the vector. This way of looking at it brings us back to the objective stated at the start of this section, to compute $h(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})$ as $c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})$.

We began this section by noting that the equality of these two enables us to compute the action of $h$ on any argument knowing only $h({\vec {\beta }}_{1})$, ..., $h({\vec {\beta }}_{n})$. We have developed this into a scheme to compute the action of the map by taking the matrix-vector product of the matrix representing the map and the vector representing the argument. In this way, any linear map is represented with respect to some bases by a matrix. In the next subsection, we will show the converse, that any matrix represents a linear map.

Exercises

This exercise is recommended for all readers.

Problem 1

Multiply the matrix

${\begin{pmatrix}1&3&1\\0&-1&2\\1&1&0\end{pmatrix}}$

by each vector (or state "not defined").

1. ${\begin{pmatrix}2\\1\\0\end{pmatrix}}$
2. ${\begin{pmatrix}-2\\-2\end{pmatrix}}$
3. ${\begin{pmatrix}0\\0\\0\end{pmatrix}}$

Problem 2

Perform, if possible, each matrix-vector multiplication.

1. ${\begin{pmatrix}2&1\\3&-1/2\end{pmatrix}}{\begin{pmatrix}4\\2\end{pmatrix}}$
2. ${\begin{pmatrix}1&1&0\\-2&1&0\end{pmatrix}}{\begin{pmatrix}1\\3\\1\end{pmatrix}}$
3. ${\begin{pmatrix}1&1\\-2&1\end{pmatrix}}{\begin{pmatrix}1\\3\\1\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 3

Solve this matrix equation.

${\begin{pmatrix}2&1&1\\0&1&3\\1&-1&2\end{pmatrix}}{\begin{pmatrix}x\\y\\z\end{pmatrix}}={\begin{pmatrix}8\\4\\4\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 4

For a homomorphism from ${\mathcal {P}}_{2}$ to ${\mathcal {P}}_{3}$ that sends

$1\mapsto 1+x,\quad x\mapsto 1+2x,\quad {\text{and}}\quad x^{2}\mapsto x-x^{3}$

where does $1-3x+2x^{2}$ go?

This exercise is recommended for all readers.

Problem 5

Assume that $h:\mathbb {R} ^{2}\to \mathbb {R} ^{3}$ is determined by this action.

${\begin{pmatrix}1\\0\end{pmatrix}}\mapsto {\begin{pmatrix}2\\2\\0\end{pmatrix}}\qquad {\begin{pmatrix}0\\1\end{pmatrix}}\mapsto {\begin{pmatrix}0\\1\\-1\end{pmatrix}}$

Using the standard bases, find

1. the matrix representing this map;
2. a general formula for $h({\vec {v}})$.

This exercise is recommended for all readers.

Problem 6

Let $d/dx:{\mathcal {P}}_{3}\to {\mathcal {P}}_{3}$ be the derivative transformation.

1. Represent $d/dx$ with respect to $B,B$ where $B=\langle 1,x,x^{2},x^{3}\rangle$.
2. Represent $d/dx$ with respect to $B,D$ where $D=\langle 1,2x,3x^{2},4x^{3}\rangle$.

This exercise is recommended for all readers.

Problem 7

Represent each linear map with respect to each pair of bases.

1. $d/dx:{\mathcal {P}}_{n}\to {\mathcal {P}}_{n}$ with respect to $B,B$ where $B=\langle 1,x,\dots ,x^{n}\rangle$, given by

   $a_{0}+a_{1}x+a_{2}x^{2}+\dots +a_{n}x^{n}\mapsto a_{1}+2a_{2}x+\dots +na_{n}x^{n-1}$

2. $\int :{\mathcal {P}}_{n}\to {\mathcal {P}}_{n+1}$ with respect to $B_{n},B_{n+1}$ where $B_{i}=\langle 1,x,\dots ,x^{i}\rangle$, given by

   $a_{0}+a_{1}x+a_{2}x^{2}+\dots +a_{n}x^{n}\mapsto a_{0}x+{\frac {a_{1}}{2}}x^{2}+\dots +{\frac {a_{n}}{n+1}}x^{n+1}$

3. $\int _{0}^{1}:{\mathcal {P}}_{n}\to \mathbb {R}$ with respect to $B,{\mathcal {E}}_{1}$ where $B=\langle 1,x,\dots ,x^{n}\rangle$ and ${\mathcal {E}}_{1}=\langle 1\rangle$, given by

   $a_{0}+a_{1}x+a_{2}x^{2}+\dots +a_{n}x^{n}\mapsto a_{0}+{\frac {a_{1}}{2}}+\dots +{\frac {a_{n}}{n+1}}$

4. ${\text{eval}}_{3}:{\mathcal {P}}_{n}\to \mathbb {R}$ with respect to $B,{\mathcal {E}}_{1}$ where $B=\langle 1,x,\dots ,x^{n}\rangle$ and ${\mathcal {E}}_{1}=\langle 1\rangle$, given by

   $a_{0}+a_{1}x+a_{2}x^{2}+\dots +a_{n}x^{n}\mapsto a_{0}+a_{1}\cdot 3+a_{2}\cdot 3^{2}+\dots +a_{n}\cdot 3^{n}$

5. ${\text{slide}}_{-1}:{\mathcal {P}}_{n}\to {\mathcal {P}}_{n}$ with respect to $B,B$ where $B=\langle 1,x,\ldots ,x^{n}\rangle$, given by

   $a_{0}+a_{1}x+a_{2}x^{2}+\dots +a_{n}x^{n}\mapsto a_{0}+a_{1}\cdot (x+1)+\dots +a_{n}\cdot (x+1)^{n}$

Problem 8

Represent the identity map on any nontrivial space with respect to $B,B$, where $B$ is any basis.

Problem 9

Represent, with respect to the natural basis, the transpose transformation on the space ${\mathcal {M}}_{2\!\times \!2}$ of $2\!\times \!2$ matrices.

Problem 10

Assume that $B=\langle {\vec {\beta }}_{1},{\vec {\beta }}_{2},{\vec {\beta }}_{3},{\vec {\beta }}_{4}\rangle$ is a basis for a vector space. Represent with respect to $B,B$ the transformation that is determined by each.

1. ${\vec {\beta }}_{1}\mapsto {\vec {\beta }}_{2}$, ${\vec {\beta }}_{2}\mapsto {\vec {\beta }}_{3}$, ${\vec {\beta }}_{3}\mapsto {\vec {\beta }}_{4}$, ${\vec {\beta }}_{4}\mapsto {\vec {0}}$
2. ${\vec {\beta }}_{1}\mapsto {\vec {\beta }}_{2}$, ${\vec {\beta }}_{2}\mapsto {\vec {0}}$, ${\vec {\beta }}_{3}\mapsto {\vec {\beta }}_{4}$, ${\vec {\beta }}_{4}\mapsto {\vec {0}}$
3. ${\vec {\beta }}_{1}\mapsto {\vec {\beta }}_{2}$, ${\vec {\beta }}_{2}\mapsto {\vec {\beta }}_{3}$, ${\vec {\beta }}_{3}\mapsto {\vec {0}}$, ${\vec {\beta }}_{4}\mapsto {\vec {0}}$

Problem 11

Example 1.8 shows how to represent the rotation transformation of the plane with respect to the standard basis. Express these other transformations also with respect to the standard basis.

1. the dilation map $d_{s}$, which multiplies all vectors by the same scalar $s$
2. the reflection map $f_{\ell }$, which reflects all all vectors across a line $\ell$ through the origin

This exercise is recommended for all readers.

Problem 12

Consider a linear transformation of $\mathbb {R} ^{2}$ determined by these two.

${\begin{pmatrix}1\\1\end{pmatrix}}\mapsto {\begin{pmatrix}2\\0\end{pmatrix}}\qquad {\begin{pmatrix}1\\0\end{pmatrix}}\mapsto {\begin{pmatrix}-1\\0\end{pmatrix}}$

1. Represent this transformation with respect to the standard bases.
2. Where does the transformation send this vector?

   ${\begin{pmatrix}0\\5\end{pmatrix}}$

3. Represent this transformation with respect to these bases.

   $B=\langle {\begin{pmatrix}1\\-1\end{pmatrix}},{\begin{pmatrix}1\\1\end{pmatrix}}\rangle \qquad D=\langle {\begin{pmatrix}2\\2\end{pmatrix}},{\begin{pmatrix}-1\\1\end{pmatrix}}\rangle$

4. Using $B$ from the prior item, represent the transformation with respect to $B,B$.

Problem 13

Suppose that $h:V\to W$ is nonsingular so that by Theorem II.2.21, for any basis $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle \subset V$ the image $h(B)=\langle h({\vec {\beta }}_{1}),\dots ,h({\vec {\beta }}_{n})\rangle$ is a basis for $W$.

1. Represent the map $h$ with respect to $B,h(B)$.
2. For a member ${\vec {v}}$ of the domain, where the representation of ${\vec {v}}$ has components $c_{1}$, ..., $c_{n}$, represent the image vector $h({\vec {v}})$ with respect to the image basis $h(B)$.

Problem 14

Give a formula for the product of a matrix and ${\vec {e}}_{i}$, the column vector that is all zeroes except for a single one in the $i$-th position.

This exercise is recommended for all readers.

Problem 15

For each vector space of functions of one real variable, represent the derivative transformation with respect to $B,B$.

1. $\{a\cos x+b\sin x\,{\big |}\,a,b\in \mathbb {R} \}$, $B=\langle \cos x,\sin x\rangle$
2. $\{ae^{x}+be^{2x}\,{\big |}\,a,b\in \mathbb {R} \}$, $B=\langle e^{x},e^{2x}\rangle$
3. $\{a+bx+ce^{x}+dxe^{x}\,{\big |}\,a,b,c,d\in \mathbb {R} \}$, $B=\langle 1,x,e^{x},xe^{x}\rangle$

Problem 16

Find the range of the linear transformation of $\mathbb {R} ^{2}$ represented with respect to the standard bases by each matrix.

1. ${\begin{pmatrix}1&0\\0&0\end{pmatrix}}$
2. ${\begin{pmatrix}0&0\\3&2\end{pmatrix}}$
3. a matrix of the form ${\begin{pmatrix}a&b\\2a&2b\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 17

Can one matrix represent two different linear maps? That is, can ${\rm {Rep}}_{B,D}(h)={\rm {Rep}}_{{\hat {B}},{\hat {D}}}({\hat {h}})$?

Problem 18

Prove Theorem 1.4.

This exercise is recommended for all readers.

Problem 19

Example 1.8 shows how to represent rotation of all vectors in the plane through an angle $\theta$ about the origin, with respect to the standard bases.

1. Rotation of all vectors in three-space through an angle $\theta$ about the $x$-axis is a transformation of $\mathbb {R} ^{3}$. Represent it with respect to the standard bases. Arrange the rotation so that to someone whose feet are at the origin and whose head is at $(1,0,0)$, the movement appears clockwise.
2. Repeat the prior item, only rotate about the $y$-axis instead. (Put the person's head at ${\vec {e}}_{2}$.)
3. Repeat, about the $z$-axis.
4. Extend the prior item to $\mathbb {R} ^{4}$. (Hint: "rotate about the $z$-axis" can be restated as "rotate parallel to the $xy$-plane".)

Problem 20 (Schur's Triangularization Lemma)

1. Let $U$ be a subspace of $V$ and fix bases $B_{U}\subseteq B_{V}$. What is the relationship between the representation of a vector from $U$ with respect to $B_{U}$ and the representation of that vector (viewed as a member of $V$) with respect to $B_{V}$?
2. What about maps?
3. Fix a basis $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ for $V$ and observe that the spans

   $[\{{\vec {0}}\}]=\{{\vec {0}}\}\subset [\{{\vec {\beta }}_{1}\}]\subset [\{{\vec {\beta }}_{1},{\vec {\beta }}_{2}\}]\subset \quad \cdots \quad \subset [B]=V$

   form a strictly increasing chain of subspaces. Show that for any linear map $h:V\to W$ there is a chain $W_{0}=\{{\vec {0}}\}\subseteq W_{1}\subseteq \dots \subseteq W_{m}=W$ of subspaces of $W$ such that

   $h([\{{\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i}\}])\subset W_{i}$

   for each $i$.
4. Conclude that for every linear map $h:V\to W$ there are bases $B,D$ so the matrix representing $h$ with respect to $B,D$ is upper-triangular (that is, each entry $h_{i,j}$ with $i>j$ is zero).
5. Is an upper-triangular representation unique?

2 - Any Matrix Represents a Linear Map

The prior subsection shows that the action of a linear map $h$ is described by a matrix $H$, with respect to appropriate bases, in this way.

${\vec {v}}={\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}_{B}\;{\overset {h}{\underset {H}{\longmapsto }}}\;{\begin{pmatrix}h_{1,1}v_{1}+\dots +h_{1,n}v_{n}\\\vdots \\h_{m,1}v_{1}+\dots +h_{m,n}v_{n}\end{pmatrix}}_{D}=h({\vec {v}})$

In this subsection, we will show the converse, that each matrix represents a linear map.

Recall that, in the definition of the matrix representation of a linear map, the number of columns of the matrix is the dimension of the map's domain and the number of rows of the matrix is the dimension of the map's codomain. Thus, for instance, a $2\!\times \!3$ matrix cannot represent a map from $\mathbb {R} ^{5}$ to $\mathbb {R} ^{4}$. The next result says that, beyond this restriction on the dimensions, there are no other limitations: the $2\!\times \!3$ matrix represents a map from any three-dimensional space to any two-dimensional space.

Theorem 2.1

Any matrix represents a homomorphism between vector spaces of appropriate dimensions, with respect to any pair of bases.

Proof

For the matrix

$H=\left({\begin{array}{cccc}h_{1,1}&h_{1,2}&\ldots &h_{1,n}\\h_{2,1}&h_{2,2}&\ldots &h_{2,n}\\&\vdots \\h_{m,1}&h_{m,2}&\ldots &h_{m,n}\end{array}}\right)$

fix any $n$-dimensional domain space $V$ and any $m$-dimensional codomain space $W$. Also fix bases $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ and $D=\langle {\vec {\delta }}_{1},\dots ,{\vec {\delta }}_{m}\rangle$ for those spaces. Define a function $h:V\to W$ by: where ${\vec {v}}$ in the domain is represented as

${\rm {Rep}}_{B}({\vec {v}})={\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}_{B}$

then its image $h({\vec {v}})$ is the member the codomain represented by

${\rm {Rep}}_{D}(\,h({\vec {v}})\,)={\begin{pmatrix}h_{1,1}v_{1}+\dots +h_{1,n}v_{n}\\\vdots \\h_{m,1}v_{1}+\dots +h_{m,n}v_{n}\end{pmatrix}}_{D}$

that is, $h({\vec {v}})=h(v_{1}{\vec {\beta }}_{1}+\dots +v_{n}{\vec {\beta }}_{n})$ is defined to be $(h_{1,1}v_{1}+\dots +h_{1,n}v_{n})\cdot {\vec {\delta }}_{1}+\dots +(h_{m,1}v_{1}+\dots +h_{m,n}v_{n})\cdot {\vec {\delta }}_{m}$. (This is well-defined by the uniqueness of the representation ${\rm {Rep}}_{B}({\vec {v}})$.)

Observe that $h$ has simply been defined to make it the map that is represented with respect to $B,D$ by the matrix $H$. So to finish, we need only check that $h$ is linear. If ${\vec {v}},{\vec {u}}\in V$ are such that

${\rm {Rep}}_{B}({\vec {v}})={\begin{pmatrix}v_{1}\\\vdots \\v_{n}\end{pmatrix}}\quad {\text{and}}\quad {\rm {Rep}}_{B}({\vec {u}})={\begin{pmatrix}u_{1}\\\vdots \\u_{n}\end{pmatrix}}$

and $c,d\in \mathbb {R}$ then the calculation

${\begin{array}{rl}h(c{\vec {v}}+d{\vec {u}})&={\bigl (}h_{1,1}(cv_{1}+du_{1})+\dots +h_{1,n}(cv_{n}+du_{n}){\bigr )}\cdot {\vec {\delta }}_{1}+\\&\quad \cdots +{\bigl (}h_{m,1}(cv_{1}+du_{1})+\dots +h_{m,n}(cv_{n}+du_{n}){\bigr )}\cdot {\vec {\delta }}_{m}\\&=c\cdot h({\vec {v}})+d\cdot h({\vec {u}})\end{array}}$

provides this verification.

Example 2.2

Which map the matrix represents depends on which bases are used. If

$H={\begin{pmatrix}1&0\\0&0\end{pmatrix}},\quad B_{1}=D_{1}=\langle {\begin{pmatrix}1\\0\end{pmatrix}},{\begin{pmatrix}0\\1\end{pmatrix}}\rangle ,\quad {\text{and}}\quad B_{2}=D_{2}=\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\rangle ,$

then $h_{1}:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ represented by $H$ with respect to $B_{1},D_{1}$ maps

${\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}={\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}_{B_{1}}\quad \mapsto \quad {\begin{pmatrix}c_{1}\\0\end{pmatrix}}_{D_{1}}={\begin{pmatrix}c_{1}\\0\end{pmatrix}}$

while $h_{2}:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ represented by $H$ with respect to $B_{2},D_{2}$ is this map.

${\begin{pmatrix}c_{1}\\c_{2}\end{pmatrix}}={\begin{pmatrix}c_{2}\\c_{1}\end{pmatrix}}_{B_{2}}\quad \mapsto \quad {\begin{pmatrix}c_{2}\\0\end{pmatrix}}_{D_{2}}={\begin{pmatrix}0\\c_{2}\end{pmatrix}}$

These two are different. The first is projection onto the $x$ axis, while the second is projection onto the $y$ axis.

So not only is any linear map described by a matrix but any matrix describes a linear map. This means that we can, when convenient, handle linear maps entirely as matrices, simply doing the computations, without have to worry that a matrix of interest does not represent a linear map on some pair of spaces of interest. (In practice, when we are working with a matrix but no spaces or bases have been specified, we will often take the domain and codomain to be $\mathbb {R} ^{n}$ and $\mathbb {R} ^{m}$ and use the standard bases. In this case, because the representation is transparent— the representation with respect to the standard basis of ${\vec {v}}$ is ${\vec {v}}$— the column space of the matrix equals the range of the map. Consequently, the column space of $H$ is often denoted by ${\mathcal {R}}(H)$.)

With the theorem, we have characterized linear maps as those maps that act in this matrix way. Each linear map is described by a matrix and each matrix describes a linear map. We finish this section by illustrating how a matrix can be used to tell things about its maps.

Theorem 2.3

The rank of a matrix equals the rank of any map that it represents.

Proof

Suppose that the matrix $H$ is $m\!\times \!n$. Fix domain and codomain spaces $V$ and $W$ of dimension $n$ and $m$, with bases $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ and $D$. Then $H$ represents some linear map $h$ between those spaces with respect to these bases whose rangespace

${\begin{array}{rl}\{h({\vec {v}})\,{\big |}\,{\vec {v}}\in V\}&=\{h(c_{1}{\vec {\beta }}_{1}+\dots +c_{n}{\vec {\beta }}_{n})\,{\big |}\,c_{1},\dots ,c_{n}\in \mathbb {R} \}\\&=\{c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})\,{\big |}\,c_{1},\dots ,c_{n}\in \mathbb {R} \}\end{array}}$

is the span $[\{h({\vec {\beta }}_{1}),\dots ,h({\vec {\beta }}_{n})\}]$. The rank of $h$ is the dimension of this rangespace.

The rank of the matrix is its column rank (or its row rank; the two are equal). This is the dimension of the column space of the matrix, which is the span of the set of column vectors $[\{{\rm {Rep}}_{D}(h({\vec {\beta }}_{1})),\dots ,{\rm {Rep}}_{D}(h({\vec {\beta }}_{n}))\}]$.

To see that the two spans have the same dimension, recall that a representation with respect to a basis gives an isomorphism ${\mbox{Rep}}_{D}:W\to \mathbb {R} ^{m}$. Under this isomorphism, there is a linear relationship among members of the rangespace if and only if the same relationship holds in the column space, e.g, ${\vec {0}}=c_{1}h({\vec {\beta }}_{1})+\dots +c_{n}h({\vec {\beta }}_{n})$ if and only if ${\vec {0}}=c_{1}{\rm {Rep}}_{D}(h({\vec {\beta }}_{1}))+\dots +c_{n}{\rm {Rep}}_{D}(h({\vec {\beta }}_{n}))$. Hence, a subset of the rangespace is linearly independent if and only if the corresponding subset of the column space is linearly independent. This means that the size of the largest linearly independent subset of the rangespace equals the size of the largest linearly independent subset of the column space, and so the two spaces have the same dimension.

Example 2.4

Any map represented by

${\begin{pmatrix}1&2&2\\1&2&1\\0&0&3\\0&0&2\end{pmatrix}}$

must, by definition, be from a three-dimensional domain to a four-dimensional codomain. In addition, because the rank of this matrix is two (we can spot this by eye or get it with Gauss' method), any map represented by this matrix has a two-dimensional rangespace.

Corollary 2.5

Let $h$ be a linear map represented by a matrix $H$. Then $h$ is onto if and only if the rank of $H$ equals the number of its rows, and $h$ is one-to-one if and only if the rank of $H$ equals the number of its columns.

Proof

For the first half, the dimension of the rangespace of $h$ is the rank of $h$, which equals the rank of $H$ by the theorem. Since the dimension of the codomain of $h$ is the number of rows in $H$, if the rank of $H$ equals the number of rows, then the dimension of the rangespace equals the dimension of the codomain. But a subspace with the same dimension as its superspace must equal that superspace (a basis for the rangespace is a linearly independent subset of the codomain, whose size is equal to the dimension of the codomain, and so this set is a basis for the codomain).

For the second half, a linear map is one-to-one if and only if it is an isomorphism between its domain and its range, that is, if and only if its domain has the same dimension as its range. But the number of columns in $H$ is the dimension of $h$'s domain, and by the theorem the rank of $H$ equals the dimension of $h$'s range.

The above results end any confusion caused by our use of the word "rank" to mean apparently different things when applied to matrices and when applied to maps. We can also justify the dual use of "nonsingular". We've defined a matrix to be nonsingular if it is square and is the matrix of coefficients of a linear system with a unique solution, and we've defined a linear map to be nonsingular if it is one-to-one.

Corollary 2.6

A square matrix represents nonsingular maps if and only if it is a nonsingular matrix. Thus, a matrix represents an isomorphism if and only if it is square and nonsingular.

Proof

Immediate from the prior result.

Example 2.7

Any map from $\mathbb {R} ^{2}$ to ${\mathcal {P}}_{1}$ represented with respect to any pair of bases by

${\begin{pmatrix}1&2\\0&3\end{pmatrix}}$

is nonsingular because this matrix has rank two.

Example 2.8

Any map $g:V\to W$ represented by

${\begin{pmatrix}1&2\\3&6\end{pmatrix}}$

is not nonsingular because this matrix is not nonsingular.

We've now seen that the relationship between maps and matrices goes both ways: fixing bases, any linear map is represented by a matrix and any matrix describes a linear map. That is, by fixing spaces and bases we get a correspondence between maps and matrices. In the rest of this chapter we will explore this correspondence. For instance, we've defined for linear maps the operations of addition and scalar multiplication and we shall see what the corresponding matrix operations are. We shall also see the matrix operation that represent the map operation of composition. And, we shall see how to find the matrix that represents a map's inverse.

Exercises

This exercise is recommended for all readers.

Problem 1

Decide if the vector is in the column space of the matrix.

1. ${\begin{pmatrix}2&1\\2&5\end{pmatrix}}$, ${\begin{pmatrix}1\\-3\end{pmatrix}}$
2. ${\begin{pmatrix}4&-8\\2&-4\end{pmatrix}}$, ${\begin{pmatrix}0\\1\end{pmatrix}}$
3. ${\begin{pmatrix}1&-1&1\\1&1&-1\\-1&-1&1\end{pmatrix}}$, ${\begin{pmatrix}2\\0\\0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

Decide if each vector lies in the range of the map from $\mathbb {R} ^{3}$ to $\mathbb {R} ^{2}$ represented with respect to the standard bases by the matrix.

1. ${\begin{pmatrix}1&1&3\\0&1&4\end{pmatrix}}$, ${\begin{pmatrix}1\\3\end{pmatrix}}$
2. ${\begin{pmatrix}2&0&3\\4&0&6\end{pmatrix}}$, ${\begin{pmatrix}1\\1\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 3

Consider this matrix, representing a transformation of $\mathbb {R} ^{2}$, and these bases for that space.

${\frac {1}{2}}\cdot {\begin{pmatrix}1&1\\-1&1\end{pmatrix}}\qquad B=\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\rangle \quad D=\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\rangle$

1. To what vector in the codomain is the first member of $B$ mapped?
2. The second member?
3. Where is a general vector from the domain (a vector with components $x$ and $y$) mapped? That is, what transformation of $\mathbb {R} ^{2}$ is represented with respect to $B,D$ by this matrix?

Problem 4

What transformation of $F=\{a\cos \theta +b\sin \theta \,{\big |}\,a,b\in \mathbb {R} \}$ is represented with respect to $B=\langle \cos \theta -\sin \theta ,\sin \theta \rangle$ and $D=\langle \cos \theta +\sin \theta ,\cos \theta \rangle$ by this matrix?

${\begin{pmatrix}0&0\\1&0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 5

Decide if $1+2x$ is in the range of the map from $\mathbb {R} ^{3}$ to ${\mathcal {P}}_{2}$ represented with respect to ${\mathcal {E}}_{3}$ and $\langle 1,1+x^{2},x\rangle$ by this matrix.

${\begin{pmatrix}1&3&0\\0&1&0\\1&0&1\end{pmatrix}}$

Problem 6

Example 2.8 gives a matrix that is nonsingular, and is therefore associated with maps that are nonsingular.

1. Find the set of column vectors representing the members of the nullspace of any map represented by this matrix.
2. Find the nullity of any such map.
3. Find the set of column vectors representing the members of the rangespace of any map represented by this matrix.
4. Find the rank of any such map.
5. Check that rank plus nullity equals the dimension of the domain.

This exercise is recommended for all readers.

Problem 7

Because the rank of a matrix equals the rank of any map it represents, if one matrix represents two different maps $H={\rm {Rep}}_{B,D}(h)={\rm {Rep}}_{{\hat {B}},{\hat {D}}}({\hat {h}})$ (where $h,{\hat {h}}:V\to W$) then the dimension of the rangespace of $h$ equals the dimension of the rangespace of ${\hat {h}}$. Must these equal-dimensioned rangespaces actually be the same?

This exercise is recommended for all readers.

Problem 8

Let $V$ be an $n$-dimensional space with bases $B$ and $D$. Consider a map that sends, for ${\vec {v}}\in V$, the column vector representing ${\vec {v}}$ with respect to $B$ to the column vector representing ${\vec {v}}$ with respect to $D$. Show that is a linear transformation of $\mathbb {R} ^{n}$.

Problem 9

Example 2.2 shows that changing the pair of bases can change the map that a matrix represents, even though the domain and codomain remain the same. Could the map ever not change? Is there a matrix $H$, vector spaces $V$ and $W$, and associated pairs of bases $B_{1},D_{1}$ and $B_{2},D_{2}$ (with $B_{1}\neq B_{2}$ or $D_{1}\neq D_{2}$ or both) such that the map represented by $H$ with respect to $B_{1},D_{1}$ equals the map represented by $H$ with respect to $B_{2},D_{2}$?

This exercise is recommended for all readers.

Problem 10

A square matrix is a diagonal matrix if it is all zeroes except possibly for the entries on its upper-left to lower-right diagonal— its $1,1$ entry, its $2,2$ entry, etc. Show that a linear map is an isomorphism if there are bases such that, with respect to those bases, the map is represented by a diagonal matrix with no zeroes on the diagonal.

Problem 11

Describe geometrically the action on $\mathbb {R} ^{2}$ of the map represented with respect to the standard bases ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$ by this matrix.

${\begin{pmatrix}3&0\\0&2\end{pmatrix}}$

Do the same for these.

${\begin{pmatrix}1&0\\0&0\end{pmatrix}}\quad {\begin{pmatrix}0&1\\1&0\end{pmatrix}}\quad {\begin{pmatrix}1&3\\0&1\end{pmatrix}}$

Problem 12

The fact that for any linear map the rank plus the nullity equals the dimension of the domain shows that a necessary condition for the existence of a homomorphism between two spaces, onto the second space, is that there be no gain in dimension. That is, where $h:V\to W$ is onto, the dimension of $W$ must be less than or equal to the dimension of $V$.

1. Show that this (strong) converse holds: no gain in dimension implies that there is a homomorphism and, further, any matrix with the correct size and correct rank represents such a map.
2. Are there bases for $\mathbb {R} ^{3}$ such that this matrix

   $H={\begin{pmatrix}1&0&0\\2&0&0\\0&1&0\end{pmatrix}}$

   represents a map from $\mathbb {R} ^{3}$ to $\mathbb {R} ^{3}$ whose range is the $xy$ plane subspace of $\mathbb {R} ^{3}$?

Problem 13

Let $V$ be an $n$-dimensional space and suppose that ${\vec {x}}\in \mathbb {R} ^{n}$. Fix a basis $B$ for $V$ and consider the map $h_{\vec {x}}:V\to \mathbb {R}$ given ${\vec {v}}\mapsto {\vec {x}}\cdot {\rm {Rep}}_{B}({\vec {v}})$ by the dot product.

1. Show that this map is linear.
2. Show that for any linear map $g:V\to \mathbb {R}$ there is an ${\vec {x}}\in \mathbb {R} ^{n}$ such that $g=h_{\vec {x}}$.
3. In the prior item we fixed the basis and varied the ${\vec {x}}$ to get all possible linear maps. Can we get all possible linear maps by fixing an ${\vec {x}}$ and varying the basis?

Problem 14

Let $V,W,X$ be vector spaces with bases $B,C,D$.

1. Suppose that $h:V\to W$ is represented with respect to $B,C$ by the matrix $H$. Give the matrix representing the scalar multiple $rh$ (where $r\in \mathbb {R}$) with respect to $B,C$ by expressing it in terms of $H$.
2. Suppose that $h,g:V\to W$ are represented with respect to $B,C$ by $H$ and $G$. Give the matrix representing $h+g$ with respect to $B,C$ by expressing it in terms of $H$ and $G$.
3. Suppose that $h:V\to W$ is represented with respect to $B,C$ by $H$ and $g:W\to X$ is represented with respect to $C,D$ by $G$. Give the matrix representing $g\circ h$ with respect to $B,D$ by expressing it in terms of $H$ and $G$.

Section IV - Matrix Operations

The prior section shows how matrices represent linear maps. A good strategy, on seeing a new idea, is to explore how it interacts with some already-established ideas. In the first subsection we will ask how the representation of the sum of two maps $f+g$ is related to the representations of the two maps, and how the representation of a scalar product $r\cdot h$ of a map is related to the representation of that map. In later subsections we will see how to represent map composition and map inverse.

1 - Sums and Scalar Products

Recall that for two maps $f$ and $g$ with the same domain and codomain, the map sum $f+g$ has this definition.

${\vec {v}}\;{\stackrel {f+g}{\longmapsto }}\;f({\vec {v}})+g({\vec {v}})$

The easiest way to see how the representations of the maps combine to represent the map sum is with an example.

Example 1.1

Suppose that $f,g:\mathbb {R} ^{2}\to \mathbb {R} ^{3}$ are represented with respect to the bases $B$ and $D$ by these matrices.

$F={\rm {Rep}}_{B,D}(f)={\begin{pmatrix}1&3\\2&0\\1&0\end{pmatrix}}_{B,D}\qquad G={\rm {Rep}}_{B,D}(g)={\begin{pmatrix}0&0\\-1&-2\\2&4\end{pmatrix}}_{B,D}$

Then, for any ${\vec {v}}\in V$ represented with respect to $B$, computation of the representation of $f({\vec {v}})+g({\vec {v}})$

${\begin{pmatrix}1&3\\2&0\\1&0\end{pmatrix}}{\begin{pmatrix}v_{1}\\v_{2}\end{pmatrix}}+{\begin{pmatrix}0&0\\-1&-2\\2&4\end{pmatrix}}{\begin{pmatrix}v_{1}\\v_{2}\end{pmatrix}}={\begin{pmatrix}1v_{1}+3v_{2}\\2v_{1}+0v_{2}\\1v_{1}+0v_{2}\end{pmatrix}}+{\begin{pmatrix}0v_{1}+0v_{2}\\-1v_{1}-2v_{2}\\2v_{1}+4v_{2}\end{pmatrix}}$

gives this representation of $f+g\,({\vec {v}})$.

${\begin{pmatrix}(1+0)v_{1}+(3+0)v_{2}\\(2-1)v_{1}+(0-2)v_{2}\\(1+2)v_{1}+(0+4)v_{2}\end{pmatrix}}={\begin{pmatrix}1v_{1}+3v_{2}\\1v_{1}-2v_{2}\\3v_{1}+4v_{2}\end{pmatrix}}$

Thus, the action of $f+g$ is described by this matrix-vector product.

${\begin{pmatrix}1&3\\1&-2\\3&4\end{pmatrix}}_{B,D}{\begin{pmatrix}v_{1}\\v_{2}\end{pmatrix}}_{B}={\begin{pmatrix}1v_{1}+3v_{2}\\1v_{1}-2v_{2}\\3v_{1}+4v_{2}\end{pmatrix}}_{D}$

This matrix is the entry-by-entry sum of original matrices, e.g., the $1,1$ entry of ${\rm {Rep}}_{B,D}(f+g)$ is the sum of the $1,1$ entry of $F$ and the $1,1$ entry of $G$.

Representing a scalar multiple of a map works the same way.

Example 1.2

If $t$ is a transformation represented by

${\rm {Rep}}_{B,D}(t)={\begin{pmatrix}1&0\\1&1\end{pmatrix}}_{B,D}\quad {\text{so that}}\quad {\vec {v}}={\begin{pmatrix}v_{1}\\v_{2}\end{pmatrix}}_{B}\mapsto {\begin{pmatrix}v_{1}\\v_{1}+v_{2}\end{pmatrix}}_{D}=t({\vec {v}})$

then the scalar multiple map $5t$ acts in this way.

${\vec {v}}={\begin{pmatrix}v_{1}\\v_{2}\end{pmatrix}}_{B}\;\longmapsto \;{\begin{pmatrix}5v_{1}\\5v_{1}+5v_{2}\end{pmatrix}}_{D}=5\cdot t({\vec {v}})$

Therefore, this is the matrix representing $5t$.

${\rm {Rep}}_{B,D}(5t)={\begin{pmatrix}5&0\\5&5\end{pmatrix}}_{B,D}$

Definition 1.3

The sum of two same-sized matrices is their entry-by-entry sum. The scalar multiple of a matrix is the result of entry-by-entry scalar multiplication.

Remark 1.4

These extend the vector addition and scalar multiplication operations that we defined in the first chapter.

Theorem 1.5

Let $h,g:V\to W$ be linear maps represented with respect to bases $B,D$ by the matrices $H$ and $G$, and let $r$ be a scalar. Then the map $h+g:V\to W$ is represented with respect to $B,D$ by $H+G$, and the map $r\cdot h:V\to W$ is represented with respect to $B,D$ by $rH$.

Proof

Problem 2; generalize the examples above.

A notable special case of scalar multiplication is multiplication by zero. For any map $0\cdot h$ is the zero homomorphism and for any matrix $0\cdot H$ is the zero matrix.

Example 1.6

The zero map from any three-dimensional space to any two-dimensional space is represented by the $2\!\times \!3$ zero matrix

$Z={\begin{pmatrix}0&0&0\\0&0&0\end{pmatrix}}$

no matter which domain and codomain bases are used.

Exercises

This exercise is recommended for all readers.

Problem 1

Perform the indicated operations, if defined.

1. ${\begin{pmatrix}5&-1&2\\6&1&1\end{pmatrix}}+{\begin{pmatrix}2&1&4\\3&0&5\end{pmatrix}}$
2. $6\cdot {\begin{pmatrix}2&-1&-1\\1&2&3\end{pmatrix}}$
3. ${\begin{pmatrix}2&1\\0&3\end{pmatrix}}+{\begin{pmatrix}2&1\\0&3\end{pmatrix}}$
4. $4{\begin{pmatrix}1&2\\3&-1\end{pmatrix}}+5{\begin{pmatrix}-1&4\\-2&1\end{pmatrix}}$
5. $3{\begin{pmatrix}2&1\\3&0\end{pmatrix}}+2{\begin{pmatrix}1&1&4\\3&0&5\end{pmatrix}}$

Problem 2

Prove Theorem 1.5.

1. Prove that matrix addition represents addition of linear maps.
2. Prove that matrix scalar multiplication represents scalar multiplication of linear maps.

This exercise is recommended for all readers.

Problem 3

Prove each, where the operations are defined, where $G$, $H$, and $J$ are matrices, where $Z$ is the zero matrix, and where $r$ and $s$ are scalars.

1. Matrix addition is commutative $G+H=H+G$.
2. Matrix addition is associative $G+(H+J)=(G+H)+J$.
3. The zero matrix is an additive identity $G+Z=G$.
4. $0\cdot G=Z$
5. $(r+s)G=rG+sG$
6. Matrices have an additive inverse $G+(-1)\cdot G=Z$.
7. $r(G+H)=rG+rH$
8. $(rs)G=r(sG)$

Problem 4

Fix domain and codomain spaces. In general, one matrix can represent many different maps with respect to different bases. However, prove that a zero matrix represents only a zero map. Are there other such matrices?

This exercise is recommended for all readers.

Problem 5

Let $V$ and $W$ be vector spaces of dimensions $n$ and $m$. Show that the space $\mathop {\mathcal {L}} (V,W)$ of linear maps from $V$ to $W$ is isomorphic to ${\mathcal {M}}_{m\!\times \!n}$.

This exercise is recommended for all readers.

Problem 6

Show that it follows from the prior questions that for any six transformations $t_{1},\dots ,t_{6}:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ there are scalars $c_{1},\dots ,c_{6}\in \mathbb {R}$ such that $c_{1}t_{1}+\dots +c_{6}t_{6}$ is the zero map. (Hint: this is a bit of a misleading question.)

Problem 7

The trace of a square matrix is the sum of the entries on the main diagonal (the $1,1$ entry plus the $2,2$ entry, etc.; we will see the significance of the trace in Chapter Five). Show that ${\mbox{trace}}(H+G)={\mbox{trace}}(H)+{\mbox{trace}}(G)$. Is there a similar result for scalar multiplication?

Problem 8

Recall that the transpose of a matrix $M$ is another matrix, whose $i,j$ entry is the $j,i$ entry of $M$. Verifiy these identities.

1. ${{(G+H)}^{\rm {trans}}}={{G}^{\rm {trans}}}+{{H}^{\rm {trans}}}$
2. ${{(r\cdot H)}^{\rm {trans}}}=r\cdot {{H}^{\rm {trans}}}$

This exercise is recommended for all readers.

Problem 9

A square matrix is symmetric if each $i,j$ entry equals the $j,i$ entry, that is, if the matrix equals its transpose.

1. Prove that for any $H$, the matrix $H+{{H}^{\rm {trans}}}$ is symmetric. Does every symmetric matrix have this form?
2. Prove that the set of $n\!\times \!n$ symmetric matrices is a subspace of ${\mathcal {M}}_{n\!\times \!n}$.

This exercise is recommended for all readers.

Problem 10

1. How does matrix rank interact with scalar multiplication— can a scalar product of a rank $n$ matrix have rank less than $n$? Greater?
2. How does matrix rank interact with matrix addition— can a sum of rank $n$ matrices have rank less than $n$? Greater?

2 - Matrix Multiplication

After representing addition and scalar multiplication of linear maps in the prior subsection, the natural next map operation to consider is composition.

Lemma 2.1

A composition of linear maps is linear.

Proof

(This argument has appeared earlier, as part of the proof that isomorphism is an equivalence relation between spaces.) Let $h:V\to W$ and $g:W\to U$ be linear. The calculation

$g\circ h\,{\bigl (}c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2}{\bigr )}=g{\bigl (}\,h(c_{1}\cdot {\vec {v}}_{1}+c_{2}\cdot {\vec {v}}_{2})\,{\bigr )}=g{\bigl (}\,c_{1}\cdot h({\vec {v}}_{1})+c_{2}\cdot h({\vec {v}}_{2})\,{\bigr )}$

$=c_{1}\cdot g{\bigl (}h({\vec {v}}_{1}))+c_{2}\cdot g(h({\vec {v}}_{2}){\bigr )}=c_{1}\cdot (g\circ h)({\vec {v}}_{1})+c_{2}\cdot (g\circ h)({\vec {v}}_{2})$

shows that $g\circ h:V\to U$ preserves linear combinations.

To see how the representation of the composite arises out of the representations of the two compositors, consider an example.

Example 2.2

Let $h:\mathbb {R} ^{4}\to \mathbb {R} ^{2}$ and $g:\mathbb {R} ^{2}\to \mathbb {R} ^{3}$, fix bases $B\subset \mathbb {R} ^{4}$, $C\subset \mathbb {R} ^{2}$, $D\subset \mathbb {R} ^{3}$, and let these be the representations.

$H={\rm {Rep}}_{B,C}(h)={\begin{pmatrix}4&6&8&2\\5&7&9&3\end{pmatrix}}_{B,C}\qquad G={\rm {Rep}}_{C,D}(g)={\begin{pmatrix}1&1\\0&1\\1&0\end{pmatrix}}_{C,D}$

To represent the composition $g\circ h:\mathbb {R} ^{4}\to \mathbb {R} ^{3}$ we fix a ${\vec {v}}$, represent $h$ of ${\vec {v}}$, and then represent $g$ of that. The representation of $h({\vec {v}})$ is the product of $h$'s matrix and ${\vec {v}}$'s vector.

${\rm {Rep}}_{C}(\,h({\vec {v}})\,)={\begin{pmatrix}4&6&8&2\\5&7&9&3\end{pmatrix}}_{B,C}{\begin{pmatrix}v_{1}\\v_{2}\\v_{3}\\v_{4}\end{pmatrix}}_{B}={\begin{pmatrix}4v_{1}+6v_{2}+8v_{3}+2v_{4}\\5v_{1}+7v_{2}+9v_{3}+3v_{4}\end{pmatrix}}_{C}$

The representation of $g(\,h({\vec {v}})\,)$ is the product of $g$'s matrix and $h({\vec {v}})$'s vector.

${\begin{array}{rl}{\rm {Rep}}_{D}(\,g(h({\vec {v}}))\,)\!&=\!{\begin{pmatrix}1&1\\0&1\\1&0\end{pmatrix}}_{C,D}{\begin{pmatrix}4v_{1}+6v_{2}+8v_{3}+2v_{4}\\5v_{1}+7v_{2}+9v_{3}+3v_{4}\end{pmatrix}}_{C}\\&=\!{\begin{pmatrix}1\cdot (4v_{1}+6v_{2}+8v_{3}+2v_{4})+1\cdot (5v_{1}+7v_{2}+9v_{3}+3v_{4})\\0\cdot (4v_{1}+6v_{2}+8v_{3}+2v_{4})+1\cdot (5v_{1}+7v_{2}+9v_{3}+3v_{4})\\1\cdot (4v_{1}+6v_{2}+8v_{3}+2v_{4})+0\cdot (5v_{1}+7v_{2}+9v_{3}+3v_{4})\end{pmatrix}}_{D}\end{array}}$

Distributing and regrouping on the $v$'s gives

$={\begin{pmatrix}(1\cdot 4+1\cdot 5)v_{1}+(1\cdot 6+1\cdot 7)v_{2}+(1\cdot 8+1\cdot 9)v_{3}+(1\cdot 2+1\cdot 3)v_{4}\\(0\cdot 4+1\cdot 5)v_{1}+(0\cdot 6+1\cdot 7)v_{2}+(0\cdot 8+1\cdot 9)v_{3}+(0\cdot 2+1\cdot 3)v_{4}\\(1\cdot 4+0\cdot 5)v_{1}+(1\cdot 6+0\cdot 7)v_{2}+(1\cdot 8+0\cdot 9)v_{3}+(1\cdot 2+0\cdot 3)v_{4}\end{pmatrix}}_{D}$

which we recognizing as the result of this matrix-vector product.

$={\begin{pmatrix}1\cdot 4+1\cdot 5&1\cdot 6+1\cdot 7&1\cdot 8+1\cdot 9&1\cdot 2+1\cdot 3\\0\cdot 4+1\cdot 5&0\cdot 6+1\cdot 7&0\cdot 8+1\cdot 9&0\cdot 2+1\cdot 3\\1\cdot 4+0\cdot 5&1\cdot 6+0\cdot 7&1\cdot 8+0\cdot 9&1\cdot 2+0\cdot 3\end{pmatrix}}_{B,D}{\begin{pmatrix}v_{1}\\v_{2}\\v_{3}\\v_{4}\end{pmatrix}}_{D}$

Thus, the matrix representing $g\circ h$ has the rows of $G$ combined with the columns of $H$.

Definition 2.3

The matrix-multiplicative product of the $m\!\times \!r$ matrix $G$ and the $r\!\times \!n$ matrix $H$ is the $m\!\times \!n$ matrix $P$, where

$p_{i,j}=g_{i,1}h_{1,j}+g_{i,2}h_{2,j}+\dots +g_{i,r}h_{r,j}$

that is, the $i,j$-th entry of the product is the dot product of the $i$-th row and the $j$-th column.

$GH={\begin{pmatrix}&\vdots \\g_{i,1}&g_{i,2}&\ldots &g_{i,r}\\&\vdots \end{pmatrix}}{\begin{pmatrix}&h_{1,j}\\\ldots &h_{2,j}&\ldots \\&\vdots \\&h_{r,j}\end{pmatrix}}={\begin{pmatrix}&\vdots \\\ldots &p_{i,j}&\ldots \\&\vdots \end{pmatrix}}$

Example 2.4

The matrices from Example 2.2 combine in this way.

${\begin{pmatrix}1\cdot 4+1\cdot 5&1\cdot 6+1\cdot 7&1\cdot 8+1\cdot 9&1\cdot 2+1\cdot 3\\0\cdot 4+1\cdot 5&0\cdot 6+1\cdot 7&0\cdot 8+1\cdot 9&0\cdot 2+1\cdot 3\\1\cdot 4+0\cdot 5&1\cdot 6+0\cdot 7&1\cdot 8+0\cdot 9&1\cdot 2+0\cdot 3\end{pmatrix}}={\begin{pmatrix}9&13&17&5\\5&7&9&3\\4&6&8&2\end{pmatrix}}$

Example 2.5

${\begin{pmatrix}2&0\\4&6\\8&2\end{pmatrix}}{\begin{pmatrix}1&3\\5&7\end{pmatrix}}={\begin{pmatrix}2\cdot 1+0\cdot 5&2\cdot 3+0\cdot 7\\4\cdot 1+6\cdot 5&4\cdot 3+6\cdot 7\\8\cdot 1+2\cdot 5&8\cdot 3+2\cdot 7\end{pmatrix}}={\begin{pmatrix}2&6\\34&54\\18&38\end{pmatrix}}$

Theorem 2.6

A composition of linear maps is represented by the matrix product of the representatives.

Proof

(This argument parallels Example 2.2.) Let $h:V\to W$ and $g:W\to X$ be represented by $H$ and $G$ with respect to bases $B\subset V$, $C\subset W$, and $D\subset X$, of sizes $n$, $r$, and $m$. For any ${\vec {v}}\in V$, the $k$-th component of ${\rm {Rep}}_{C}(\,h({\vec {v}})\,)$ is

$h_{k,1}v_{1}+\cdots +h_{k,n}v_{n}$

and so the $i$-th component of ${\rm {Rep}}_{D}(\,g\circ h\,({\vec {v}})\,)$ is this.

$g_{i,1}\cdot (h_{1,1}v_{1}+\dots +h_{1,n}v_{n})+g_{i,2}\cdot (h_{2,1}v_{1}+\dots +h_{2,n}v_{n})$

$+\dots +g_{i,r}\cdot (h_{r,1}v_{1}+\dots +h_{r,n}v_{n})$

Distribute and regroup on the $v$'s.

$=(g_{i,1}h_{1,1}+g_{i,2}h_{2,1}+\dots +g_{i,r}h_{r,1})\cdot v_{1}$

$+\dots +(g_{i,1}h_{1,n}+g_{i,2}h_{2,n}+\dots +g_{i,r}h_{r,n})\cdot v_{n}$

Finish by recognizing that the coefficient of each $v_{j}$

$g_{i,1}h_{1,j}+g_{i,2}h_{2,j}+\dots +g_{i,r}h_{r,j}$

matches the definition of the $i,j$ entry of the product $GH$.

The theorem is an example of a result that supports a definition. We can picture what the definition and theorem together say with this arrow diagram ("wrt" abbreviates "with respect to").

Above the arrows, the maps show that the two ways of going from $V$ to $X$, straight over via the composition or else by way of $W$, have the same effect

${\vec {v}}{\stackrel {g\circ h}{\longmapsto }}g(h({\vec {v}}))\qquad {\vec {v}}{\stackrel {h}{\longmapsto }}h({\vec {v}}){\stackrel {g}{\longmapsto }}g(h({\vec {v}}))$

(this is just the definition of composition). Below the arrows, the matrices indicate that the product does the same thing— multiplying $GH$ into the column vector ${\rm {Rep}}_{B}({\vec {v}})$ has the same effect as multiplying the column first by $H$ and then multiplying the result by $G$.

${\rm {Rep}}_{B,D}(g\circ h)=GH={\rm {Rep}}_{C,D}(g)\,{\rm {Rep}}_{B,C}(h)$

The definition of the matrix-matrix product operation does not restrict us to view it as a representation of a linear map composition. We can get insight into this operation by studying it as a mechanical procedure. The striking thing is the way that rows and columns combine.

One aspect of that combination is that the sizes of the matrices involved is significant. Briefly, $m\!\times \!r{\text{ times }}r\!\times \!n{\text{ equals }}m\!\times \!n$.

Example 2.7

This product is not defined

${\begin{pmatrix}-1&2&0\\0&10&1.1\end{pmatrix}}{\begin{pmatrix}0&0\\0&2\end{pmatrix}}$

because the number of columns on the left does not equal the number of rows on the right.

In terms of the underlying maps, the fact that the sizes must match up reflects the fact that matrix multiplication is defined only when a corresponding function composition

${\text{dimension }}n{\text{ space}}\;{\stackrel {h}{\longrightarrow }}\;{\text{dimension }}r{\text{ space}}\;{\stackrel {g}{\longrightarrow }}\;{\text{dimension }}m{\text{ space}}$

is possible.

Remark 2.8

The order in which these things are written can be confusing. In the "$m\!\times \!r{\text{ times }}r\!\times \!n{\text{ equals }}m\!\times \!n$" equation, the number written first $m$ is the dimension of $g$'s codomain and is thus the number that appears last in the map dimension description above. The explanation is that while $f$ is done first and then $g$ is applied, that composition is written $g\circ f$, from the notation "$g(f({\vec {v}}))$". (Some people try to lessen confusion by reading "$g\circ f$" aloud as "$g$ following $f$".) That order then carries over to matrices: $g\circ f$ is represented by $GF$.

Another aspect of the way that rows and columns combine in the matrix product operation is that in the definition of the $i,j$ entry

$p_{i,j}=g_{i,{\color {red}1}}h_{{\color {red}1},j}+g_{i,{\color {red}2}}h_{{\color {red}2},j}+\dots +g_{i,{\color {red}r}}h_{{\color {red}r},j}$

the red subscripts on the $g$'s are column indicators while those on the $h$'s indicate rows. That is, summation takes place over the columns of $G$ but over the rows of $H$; left is treated differently than right, so $GH$ may be unequal to $HG$. Matrix multiplication is not commutative.

Example 2.9

Matrix multiplication hardly ever commutes. Test that by multiplying randomly chosen matrices both ways.

${\begin{pmatrix}1&2\\3&4\end{pmatrix}}{\begin{pmatrix}5&6\\7&8\end{pmatrix}}={\begin{pmatrix}19&22\\43&50\end{pmatrix}}\qquad {\begin{pmatrix}5&6\\7&8\end{pmatrix}}{\begin{pmatrix}1&2\\3&4\end{pmatrix}}={\begin{pmatrix}23&34\\31&46\end{pmatrix}}$

Example 2.10

Commutativity can fail more dramatically:

${\begin{pmatrix}5&6\\7&8\end{pmatrix}}{\begin{pmatrix}1&2&0\\3&4&0\end{pmatrix}}={\begin{pmatrix}23&34&0\\31&46&0\end{pmatrix}}$

while

${\begin{pmatrix}1&2&0\\3&4&0\end{pmatrix}}{\begin{pmatrix}5&6\\7&8\end{pmatrix}}$

isn't even defined.

Remark 2.11

The fact that matrix multiplication is not commutative may be puzzling at first sight, perhaps just because most algebraic operations in elementary mathematics are commutative. But on further reflection, it isn't so surprising. After all, matrix multiplication represents function composition, which is not commutative— if $f(x)=2x$ and $g(x)=x+1$ then $g\circ f(x)=2x+1$ while $f\circ g(x)=2(x+1)=2x+2$. True, this $g$ is not linear and we might have hoped that linear functions commute, but this perspective shows that the failure of commutativity for matrix multiplication fits into a larger context.

Except for the lack of commutativity, matrix multiplication is algebraically well-behaved. Below are some nice properties and more are in Problem 10 and Problem 11.

Theorem 2.12

If $F$, $G$, and $H$ are matrices, and the matrix products are defined, then the product is associative $(FG)H=F(GH)$ and distributes over matrix addition $F(G+H)=FG+FH$ and $(G+H)F=GF+HF$.

Proof

Associativity holds because matrix multiplication represents function composition, which is associative: the maps $(f\circ g)\circ h$ and $f\circ (g\circ h)$ are equal as both send ${\vec {v}}$ to $f(g(h({\vec {v}})))$.

Distributivity is similar. For instance, the first one goes $f\circ (g+h)\,({\vec {v}})=f{\bigl (}\,(g+h)({\vec {v}})\,{\bigr )}=f{\bigl (}\,g({\vec {v}})+h({\vec {v}})\,{\bigr )}=f(g({\vec {v}}))+f(h({\vec {v}}))=f\circ g({\vec {v}})+f\circ h({\vec {v}})$ (the third equality uses the linearity of $f$).

Remark 2.13

We could alternatively prove that result by slogging through the indices. For example, associativity goes: the $i,j$-th entry of $(FG)H$ is

${\begin{array}{rl}&(f_{i,1}g_{1,1}+f_{i,2}g_{2,1}+\dots +f_{i,r}g_{r,1})h_{1,j}\\&\quad +(f_{i,1}g_{1,2}+f_{i,2}g_{2,2}+\dots +f_{i,r}g_{r,2})h_{2,j}\\&\quad \;\;\vdots \\&\quad +(f_{i,1}g_{1,s}+f_{i,2}g_{2,s}+\dots +f_{i,r}g_{r,s})h_{s,j}\end{array}}$

(where $F$, $G$, and $H$ are $m\!\times \!r$, $r\!\times \!s$, and $s\!\times \!n$ matrices), distribute

${\begin{array}{rl}&f_{i,1}g_{1,1}h_{1,j}+f_{i,2}g_{2,1}h_{1,j}+\dots +f_{i,r}g_{r,1}h_{1,j}\\&\quad +f_{i,1}g_{1,2}h_{2,j}+f_{i,2}g_{2,2}h_{2,j}+\dots +f_{i,r}g_{r,2}h_{2,j}\\&\quad \;\;\vdots \\&\quad +f_{i,1}g_{1,s}h_{s,j}+f_{i,2}g_{2,s}h_{s,j}+\dots +f_{i,r}g_{r,s}h_{s,j}\end{array}}$

and regroup around the $f$'s

${\begin{array}{rl}&f_{i,1}(g_{1,1}h_{1,j}+g_{1,2}h_{2,j}+\dots +g_{1,s}h_{s,j})\\&\quad +f_{i,2}(g_{2,1}h_{1,j}+g_{2,2}h_{2,j}+\dots +g_{2,s}h_{s,j})\\&\quad \;\;\vdots \\&\quad +f_{i,r}(g_{r,1}h_{1,j}+g_{r,2}h_{2,j}+\dots +g_{r,s}h_{s,j})\end{array}}$

to get the $i,j$ entry of $F(GH)$.

Contrast these two ways of verifying associativity, the one in the proof and the one just above. The argument just above is hard to understand in the sense that, while the calculations are easy to check, the arithmetic seems unconnected to any idea (it also essentially repeats the proof of Theorem 2.6 and so is inefficient). The argument in the proof is shorter, clearer, and says why this property "really" holds. This illustrates the comments made in the preamble to the chapter on vector spaces— at least some of the time an argument from higher-level constructs is clearer.

We have now seen how the representation of the composition of two linear maps is derived from the representations of the two maps. We have called the combination the product of the two matrices. This operation is extremely important. Before we go on to study how to represent the inverse of a linear map, we will explore it some more in the next subsection.

Exercises

This exercise is recommended for all readers.

Problem 1

Compute, or state "not defined".

1. ${\begin{pmatrix}3&1\\-4&2\end{pmatrix}}{\begin{pmatrix}0&5\\0&0.5\end{pmatrix}}$
2. ${\begin{pmatrix}1&1&-1\\4&0&3\end{pmatrix}}{\begin{pmatrix}2&-1&-1\\3&1&1\\3&1&1\end{pmatrix}}$
3. ${\begin{pmatrix}2&-7\\7&4\end{pmatrix}}{\begin{pmatrix}1&0&5\\-1&1&1\\3&8&4\end{pmatrix}}$
4. ${\begin{pmatrix}5&2\\3&1\end{pmatrix}}{\begin{pmatrix}-1&2\\3&-5\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

Where

$A={\begin{pmatrix}1&-1\\2&0\\\end{pmatrix}}\quad B={\begin{pmatrix}5&2\\4&4\\\end{pmatrix}}\quad C={\begin{pmatrix}-2&3\\-4&1\\\end{pmatrix}}$

compute or state "not defined".

1. $AB$
2. $(AB)C$
3. $BC$
4. $A(BC)$

Problem 3

Which products are defined?

1. $3\!\times \!2$ times $2\!\times \!3$
2. $2\!\times \!3$ times $3\!\times \!2$
3. $2\!\times \!2$ times $3\!\times \!3$
4. $3\!\times \!3$ times $2\!\times \!2$

This exercise is recommended for all readers.

Problem 4

Give the size of the product or state "not defined".

1. a $2\!\times \!3$ matrix times a $3\!\times \!1$ matrix
2. a $1\!\times \!12$ matrix times a $12\!\times \!1$ matrix
3. a $2\!\times \!3$ matrix times a $2\!\times \!1$ matrix
4. a $2\!\times \!2$ matrix times a $2\!\times \!2$ matrix

This exercise is recommended for all readers.

Problem 5

Find the system of equations resulting from starting with

${\begin{array}{*{3}{rc}r}h_{1,1}x_{1}&+&h_{1,2}x_{2}&+&h_{1,3}x_{3}&=&d_{1}\\h_{2,1}x_{1}&+&h_{2,2}x_{2}&+&h_{2,3}x_{3}&=&d_{2}\end{array}}$

and making this change of variable (i.e., substitution).

${\begin{array}{*{2}{rc}r}x_{1}&=&g_{1,1}y_{1}&+&g_{1,2}y_{2}\\x_{2}&=&g_{2,1}y_{1}&+&g_{2,2}y_{2}\\x_{3}&=&g_{3,1}y_{1}&+&g_{3,2}y_{2}\end{array}}$

Problem 6

As Definition 2.3 points out, the matrix product operation generalizes the dot product. Is the dot product of a $1\!\times \!n$ row vector and a $n\!\times \!1$ column vector the same as their matrix-multiplicative product?

This exercise is recommended for all readers.

Problem 7

Represent the derivative map on ${\mathcal {P}}_{n}$ with respect to $B,B$ where $B$ is the natural basis $\langle 1,x,\ldots ,x^{n}\rangle$. Show that the product of this matrix with itself is defined; what the map does it represent?

Problem 8

Show that composition of linear transformations on $\mathbb {R} ^{1}$ is commutative. Is this true for any one-dimensional space?

Problem 9

Why is matrix multiplication not defined as entry-wise multiplication? That would be easier, and commutative too.

This exercise is recommended for all readers.

Problem 10

1. Prove that $H^{p}H^{q}=H^{p+q}$ and $(H^{p})^{q}=H^{pq}$ for positive integers $p,q$.
2. Prove that $(rH)^{p}=r^{p}\cdot H^{p}$ for any positive integer $p$ and scalar $r\in \mathbb {R}$.

This exercise is recommended for all readers.

Problem 11

1. How does matrix multiplication interact with scalar multiplication: is $r(GH)=(rG)H$? Is $G(rH)=r(GH)$?
2. How does matrix multiplication interact with linear combinations: is $F(rG+sH)=r(FG)+s(FH)$? Is $(rF+sG)H=rFH+sGH$?

Problem 12

We can ask how the matrix product operation interacts with the transpose operation.

1. Show that ${{(GH)}^{\rm {trans}}}={{H}^{\rm {trans}}}{{G}^{\rm {trans}}}$.
2. A square matrix is symmetric if each $i,j$ entry equals the $j,i$ entry, that is, if the matrix equals its own transpose. Show that the matrices $H{{H}^{\rm {trans}}}$ and ${{H}^{\rm {trans}}}H$ are symmetric.

This exercise is recommended for all readers.

Problem 13

Rotation of vectors in $\mathbb {R} ^{3}$ about an axis is a linear map. Show that linear maps do not commute by showing geometrically that rotations do not commute.

Problem 14

In the proof of Theorem 2.12 some maps are used. What are the domains and codomains?

Problem 15

How does matrix rank interact with matrix multiplication?

1. Can the product of rank $n$ matrices have rank less than $n$? Greater?
2. Show that the rank of the product of two matrices is less than or equal to the minimum of the rank of each factor.

Problem 16

Is "commutes with" an equivalence relation among $n\!\times \!n$ matrices?

This exercise is recommended for all readers.

Problem 17

(This will be used in the Matrix Inverses exercises.) Here is another property of matrix multiplication that might be puzzling at first sight.

1. Prove that the composition of the projections $\pi _{x},\pi _{y}:\mathbb {R} ^{3}\to \mathbb {R} ^{3}$ onto the $x$ and $y$ axes is the zero map despite that neither one is itself the zero map.
2. Prove that the composition of the derivatives $d^{2}/dx^{2},\,d^{3}/dx^{3}:{\mathcal {P}}_{4}\to {\mathcal {P}}_{4}$ is the zero map despite that neither is the zero map.
3. Give a matrix equation representing the first fact.
4. Give a matrix equation representing the second.

When two things multiply to give zero despite that neither is zero, each is said to be a zero divisor.

Problem 18

Show that, for square matrices, $(S+T)(S-T)$ need not equal $S^{2}-T^{2}$.

This exercise is recommended for all readers.

Problem 19

Represent the identity transformation ${\mbox{id}}:V\to V$ with respect to $B,B$ for any basis $B$. This is the identity matrix $I$. Show that this matrix plays the role in matrix multiplication that the number $1$ plays in real number multiplication: $HI=IH=H$ (for all matrices $H$ for which the product is defined).

Problem 20

In real number algebra, quadratic equations have at most two solutions. That is not so with matrix algebra. Show that the $2\!\times \!2$ matrix equation $T^{2}=I$ has more than two solutions, where $I$ is the identity matrix (this matrix has ones in its $1,1$ and $2,2$ entries and zeroes elsewhere; see Problem 19).

Problem 21

1. Prove that for any $2\!\times \!2$ matrix $T$ there are scalars $c_{0},\dots ,c_{4}$ that are not all $0$ such that the combination $c_{4}T^{4}+c_{3}T^{3}+c_{2}T^{2}+c_{1}T+c_{0}I$ is the zero matrix (where $I$ is the $2\!\times \!2$ identity matrix, with $1$'s in its $1,1$ and $2,2$ entries and zeroes elsewhere; see Problem 19).
2. Let $p(x)$ be a polynomial $p(x)=c_{n}x^{n}+\dots +c_{1}x+c_{0}$. If $T$ is a square matrix we define $p(T)$ to be the matrix $c_{n}T^{n}+\dots +c_{1}T+I$ (where $I$ is the appropriately-sized identity matrix). Prove that for any square matrix there is a polynomial such that $p(T)$ is the zero matrix.
3. The minimal polynomial $m(x)$ of a square matrix is the polynomial of least degree, and with leading coefficient $1$, such that $m(T)$ is the zero matrix. Find the minimal polynomial of this matrix.

   ${\begin{pmatrix}{\sqrt {3}}/2&-1/2\\1/2&{\sqrt {3}}/2\end{pmatrix}}$

   (This is the representation with respect to ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$, the standard basis, of a rotation through $\pi /6$ radians counterclockwise.)

Problem 22

The infinite-dimensional space ${\mathcal {P}}$ of all finite-degree polynomials gives a memorable example of the non-commutativity of linear maps. Let $d/dx:{\mathcal {P}}\to {\mathcal {P}}$ be the usual derivative and let $s:{\mathcal {P}}\to {\mathcal {P}}$ be the shift map.

$a_{0}+a_{1}x+\dots +a_{n}x^{n}\;{\stackrel {s}{\longmapsto }}\;0+a_{0}x+a_{1}x^{2}+\dots +a_{n}x^{n+1}$

Show that the two maps don't commute $d/dx\circ s\neq s\circ d/dx$; in fact, not only is $(d/dx\circ s)-(s\circ d/dx)$ not the zero map, it is the identity map.

Problem 23

Recall the notation for the sum of the sequence of numbers $a_{1},a_{2},\dots ,a_{n}$.

$\sum _{i=1}^{n}a_{i}=a_{1}+a_{2}+\dots +a_{n}$

In this notation, the $i,j$ entry of the product of $G$ and $H$ is this.

$p_{i,j}=\sum _{k=1}^{r}g_{i,k}h_{k,j}$

Using this notation,

1. reprove that matrix multiplication is associative;
2. reprove Theorem 2.6.

3 - Mechanics of Matrix Multiplication

In this subsection we consider matrix multiplication as a mechanical process, putting aside for the moment any implications about the underlying maps. As described earlier, the striking thing about matrix multiplication is the way rows and columns combine. The $i,j$ entry of the matrix product is the dot product of row $i$ of the left matrix with column $j$ of the right one. For instance, here a second row and a third column combine to make a $2,3$ entry.

${\begin{pmatrix}1&1\\{\color {red}0}&{\color {red}1}\\1&0\end{pmatrix}}{\begin{pmatrix}4&6&{\color {red}8}&2\\5&7&{\color {red}9}&3\end{pmatrix}}={\begin{pmatrix}9&13&17&5\\5&7&{\color {red}9}&3\\4&6&8&2\end{pmatrix}}$

We can view this as the left matrix acting by multiplying its rows, one at a time, into the columns of the right matrix. Of course, another perspective is that the right matrix uses its columns to act on the left matrix's rows. Below, we will examine actions from the left and from the right for some simple matrices.

The first case, the action of a zero matrix, is very easy.

Example 3.1

Multiplying by an appropriately-sized zero matrix from the left or from the right

${\begin{pmatrix}0&0\\0&0\end{pmatrix}}{\begin{pmatrix}1&3&2\\-1&1&-1\end{pmatrix}}={\begin{pmatrix}0&0&0\\0&0&0\end{pmatrix}}\qquad {\begin{pmatrix}2&3\\1&4\end{pmatrix}}{\begin{pmatrix}0&0\\0&0\end{pmatrix}}={\begin{pmatrix}0&0\\0&0\end{pmatrix}}$

results in a zero matrix.

After zero matrices, the matrices whose actions are easiest to understand are the ones with a single nonzero entry.

Definition 3.2

A matrix with all zeroes except for a one in the $i,j$ entry is an $i,j$ unit matrix.

Example 3.3

This is the $1,2\,$ unit matrix with three rows and two columns, multiplying from the left.

${\begin{pmatrix}0&1\\0&0\\0&0\end{pmatrix}}{\begin{pmatrix}5&6\\7&8\end{pmatrix}}={\begin{pmatrix}7&8\\0&0\\0&0\end{pmatrix}}$

Acting from the left, an $i,j$ unit matrix copies row $j$ of the multiplicand into row $i$ of the result. From the right an $i,j$ unit matrix copies column $i$ of the multiplicand into column $j$ of the result.

${\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}{\begin{pmatrix}0&1\\0&0\\0&0\end{pmatrix}}={\begin{pmatrix}0&1\\0&4\\0&7\end{pmatrix}}$

Example 3.4

Rescaling these matrices simply rescales the result. This is the action from the left of the matrix that is twice the one in the prior example.

${\begin{pmatrix}0&2\\0&0\\0&0\end{pmatrix}}{\begin{pmatrix}5&6\\7&8\end{pmatrix}}={\begin{pmatrix}14&16\\0&0\\0&0\end{pmatrix}}$

And this is the action of the matrix that is minus three times the one from the prior example.

${\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}{\begin{pmatrix}0&-3\\0&0\\0&0\end{pmatrix}}={\begin{pmatrix}0&-3\\0&-12\\0&-21\end{pmatrix}}$

Next in complication are matrices with two nonzero entries. There are two cases. If a left-multiplier has entries in different rows then their actions don't interact.

Example 3.5

${\begin{array}{rl}{\begin{pmatrix}1&0&0\\0&0&2\\0&0&0\end{pmatrix}}{\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}&=({\begin{pmatrix}1&0&0\\0&0&0\\0&0&0\end{pmatrix}}+{\begin{pmatrix}0&0&0\\0&0&2\\0&0&0\end{pmatrix}}){\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}\\&={\begin{pmatrix}1&2&3\\0&0&0\\0&0&0\end{pmatrix}}+{\begin{pmatrix}0&0&0\\14&16&18\\0&0&0\end{pmatrix}}\\&={\begin{pmatrix}1&2&3\\14&16&18\\0&0&0\end{pmatrix}}\end{array}}$

But if the left-multiplier's nonzero entries are in the same row then that row of the result is a combination.

Example 3.6

${\begin{array}{rl}{\begin{pmatrix}1&0&2\\0&0&0\\0&0&0\end{pmatrix}}{\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}&=({\begin{pmatrix}1&0&0\\0&0&0\\0&0&0\end{pmatrix}}+{\begin{pmatrix}0&0&2\\0&0&0\\0&0&0\end{pmatrix}}){\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}\\&={\begin{pmatrix}1&2&3\\0&0&0\\0&0&0\end{pmatrix}}+{\begin{pmatrix}14&16&18\\0&0&0\\0&0&0\end{pmatrix}}\\&={\begin{pmatrix}15&18&21\\0&0&0\\0&0&0\end{pmatrix}}\end{array}}$

Right-multiplication acts in the same way, with columns.

These observations about matrices that are mostly zeroes extend to arbitrary matrices.

Lemma 3.7

In a product of two matrices $G$ and $H$, the columns of $GH$ are formed by taking $G$ times the columns of $H$

$G\cdot \left({\begin{array}{c|c|c}\vdots &&\vdots \\{\vec {h}}_{1}&\cdots &{\vec {h}}_{n}\\\vdots &&\vdots \end{array}}\right)=\left({\begin{array}{c|c|c}\vdots &&\vdots \\G\cdot {\vec {h}}_{1}&\cdots &G\cdot {\vec {h}}_{n}\\\vdots &&\vdots \end{array}}\right)$

and the rows of $GH$ are formed by taking the rows of $G$ times $H$

$\left({\begin{array}{c}\cdots \;{\vec {g}}_{1}\;\cdots \\\hline \vdots \\\hline \cdots \;{\vec {g}}_{r}\;\cdots \end{array}}\right)\cdot H=\left({\begin{array}{c}\cdots \;{\vec {g}}_{1}\cdot H\;\cdots \\\hline \vdots \\\hline \cdots \;{\vec {g}}_{r}\cdot H\;\cdots \end{array}}\right)$

(ignoring the extra parentheses).

Proof

We will show the $2\!\times \!2$ case and leave the general case as an exercise.

$GH={\begin{pmatrix}g_{1,1}&g_{1,2}\\g_{2,1}&g_{2,2}\end{pmatrix}}{\begin{pmatrix}h_{1,1}&h_{1,2}\\h_{2,1}&h_{2,2}\end{pmatrix}}={\begin{pmatrix}g_{1,1}h_{1,1}+g_{1,2}h_{2,1}&g_{1,1}h_{1,2}+g_{1,2}h_{2,2}\\g_{2,1}h_{1,1}+g_{2,2}h_{2,1}&g_{2,1}h_{1,2}+g_{2,2}h_{2,2}\end{pmatrix}}$

The right side of the first equation in the result

$\left({\begin{array}{c|c}G{\begin{pmatrix}h_{1,1}\\h_{2,1}\end{pmatrix}}&G{\begin{pmatrix}h_{1,2}\\h_{2,2}\end{pmatrix}}\end{array}}\right)=\left({\begin{array}{c|c}{\begin{pmatrix}g_{1,1}h_{1,1}+g_{1,2}h_{2,1}\\g_{2,1}h_{1,1}+g_{2,2}h_{2,1}\end{pmatrix}}&{\begin{pmatrix}g_{1,1}h_{1,2}+g_{1,2}h_{2,2}\\g_{2,1}h_{1,2}+g_{2,2}h_{2,2}\end{pmatrix}}\end{array}}\right)$

is indeed the same as the right side of GH, except for the extra parentheses (the ones marking the columns as column vectors). The other equation is similarly easy to recognize.

An application of those observations is that there is a matrix that just copies out the rows and columns.

Definition 3.8

The main diagonal (or principal diagonal or diagonal) of a square matrix goes from the upper left to the lower right.

Definition 3.9

An identity matrix is square and has with all entries zero except for ones in the main diagonal.

$I_{n\!\times \!n}={\begin{pmatrix}1&0&\ldots &0\\0&1&\ldots &0\\&\vdots \\0&0&\ldots &1\end{pmatrix}}$

Example 3.10

The $3\!\times \!3$ identity leaves its multiplicand unchanged both from the left

${\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}{\begin{pmatrix}2&3&6\\1&3&8\\-7&1&0\end{pmatrix}}={\begin{pmatrix}2&3&6\\1&3&8\\-7&1&0\end{pmatrix}}$

and from the right.

${\begin{pmatrix}2&3&6\\1&3&8\\-7&1&0\end{pmatrix}}{\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}={\begin{pmatrix}2&3&6\\1&3&8\\-7&1&0\end{pmatrix}}$

Example 3.11

So does the $2\!\times \!2$ identity matrix.

${\begin{pmatrix}1&-2\\0&-2\\1&-1\\4&3\end{pmatrix}}{\begin{pmatrix}1&0\\0&1\\\end{pmatrix}}={\begin{pmatrix}1&-2\\0&-2\\1&-1\\4&3\end{pmatrix}}$

In short, an identity matrix is the identity element of the set of $n\!\times \!n$ matrices with respect to the operation of matrix multiplication.

We next see two ways to generalize the identity matrix.

The first is that if the ones are relaxed to arbitrary reals, the resulting matrix will rescale whole rows or columns.

Definition 3.12

A diagonal matrix is square and has zeros off the main diagonal.

${\begin{pmatrix}a_{1,1}&0&\ldots &0\\0&a_{2,2}&\ldots &0\\&\vdots \\0&0&\ldots &a_{n,n}\end{pmatrix}}$

Example 3.13

From the left, the action of multiplication by a diagonal matrix is to rescales the rows.

${\begin{pmatrix}2&0\\0&-1\end{pmatrix}}{\begin{pmatrix}2&1&4&-1\\-1&3&4&4\end{pmatrix}}={\begin{pmatrix}4&2&8&-2\\1&-3&-4&-4\end{pmatrix}}$

From the right such a matrix rescales the columns.

${\begin{pmatrix}1&2&1\\2&2&2\end{pmatrix}}{\begin{pmatrix}3&0&0\\0&2&0\\0&0&-2\end{pmatrix}}={\begin{pmatrix}3&4&-2\\6&4&-4\end{pmatrix}}$

The second generalization of identity matrices is that we can put a single one in each row and column in ways other than putting them down the diagonal.

Definition 3.14

A permutation matrix is square and is all zeros except for a single one in each row and column.

Example 3.15

From the left these matrices permute rows.

${\begin{pmatrix}0&0&1\\1&0&0\\0&1&0\end{pmatrix}}{\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}={\begin{pmatrix}7&8&9\\1&2&3\\4&5&6\end{pmatrix}}$

From the right they permute columns.

${\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}}{\begin{pmatrix}0&0&1\\1&0&0\\0&1&0\end{pmatrix}}={\begin{pmatrix}2&3&1\\5&6&4\\8&9&7\end{pmatrix}}$

We finish this subsection by applying these observations to get matrices that perform Gauss' method and Gauss-Jordan reduction.

Example 3.16

We have seen how to produce a matrix that will rescale rows. Multiplying by this diagonal matrix rescales the second row of the other by a factor of three.

${\begin{pmatrix}1&0&0\\0&3&0\\0&0&1\end{pmatrix}}{\begin{pmatrix}0&2&1&1\\0&1/3&1&-1\\1&0&2&0\end{pmatrix}}={\begin{pmatrix}0&2&1&1\\0&1&3&-3\\1&0&2&0\end{pmatrix}}$

We have seen how to produce a matrix that will swap rows. Multiplying by this permutation matrix swaps the first and third rows.

${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}{\begin{pmatrix}0&2&1&1\\0&1&3&-3\\1&0&2&0\end{pmatrix}}={\begin{pmatrix}1&0&2&0\\0&1&3&-3\\0&2&1&1\end{pmatrix}}$

To see how to perform a pivot, we observe something about those two examples. The matrix that rescales the second row by a factor of three arises in this way from the identity.

${\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}{\xrightarrow[{}]{3\rho _{2}}}{\begin{pmatrix}1&0&0\\0&3&0\\0&0&1\end{pmatrix}}$

Similarly, the matrix that swaps first and third rows arises in this way.

${\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}{\xrightarrow[{}]{\rho _{1}\leftrightarrow \rho _{3}}}{\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}$

Example 3.17

The $3\!\times \!3$ matrix that arises as

${\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}{\xrightarrow[{}]{-2\rho _{2}+\rho _{3}}}{\begin{pmatrix}1&0&0\\0&1&0\\0&-2&1\end{pmatrix}}$

will, when it acts from the left, perform the pivot operation $-2\rho _{2}+\rho _{3}$.

${\begin{pmatrix}1&0&0\\0&1&0\\0&-2&1\end{pmatrix}}{\begin{pmatrix}1&0&2&0\\0&1&3&-3\\0&2&1&1\end{pmatrix}}={\begin{pmatrix}1&0&2&0\\0&1&3&-3\\0&0&-5&7\end{pmatrix}}$

Definition 3.18

The elementary reduction matrices are obtained from identity matrices with one Gaussian operation. We denote them:

1. $I{\xrightarrow[{}]{k\rho _{i}}}M_{i}(k)$ for $k\neq 0$;
2. $I{\xrightarrow[{}]{\rho _{i}\leftrightarrow \rho _{j}}}P_{i,j}$ for $i\neq j$;
3. $I{\xrightarrow[{}]{k\rho _{i}+\rho _{j}}}C_{i,j}(k)$ for $i\neq j$.

Lemma 3.19

Gaussian reduction can be done through matrix multiplication.

1. If $H{\xrightarrow[{}]{k\rho _{i}}}G$ then $M_{i}(k)H=G$.
2. If $H{\xrightarrow[{}]{\rho _{i}\leftrightarrow \rho _{j}}}G$ then $P_{i,j}H=G$.
3. If $H{\xrightarrow[{}]{k\rho _{i}+\rho _{j}}}G$ then $C_{i,j}(k)H=G$.

Proof

Clear.

Example 3.20

This is the first system, from the first chapter, on which we performed Gauss' method.

${\begin{array}{*{3}{rc}r}&&&&3x_{3}&=&9\\x_{1}&+&5x_{2}&-&2x_{3}&=&2\\(1/3)x_{1}&+&2x_{2}&&&=&3\end{array}}$

It can be reduced with matrix multiplication. Swap the first and third rows,

${\begin{pmatrix}0&0&1\\0&1&0\\1&0&0\end{pmatrix}}\left({\begin{array}{*{3}{c}|c}0&0&3&9\\1&5&-2&2\\1/3&2&0&3\end{array}}\right)=\left({\begin{array}{*{3}{c}|c}1/3&2&0&3\\1&5&-2&2\\0&0&3&9\end{array}}\right)$

triple the first row,

${\begin{pmatrix}3&0&0\\0&1&0\\0&0&1\end{pmatrix}}\left({\begin{array}{*{3}{c}|c}1/3&2&0&3\\1&5&-2&2\\0&0&3&9\end{array}}\right)=\left({\begin{array}{*{3}{c}|c}1&6&0&9\\1&5&-2&2\\0&0&3&9\end{array}}\right)$

and then add $-1$ times the first row to the second.

${\begin{pmatrix}1&0&0\\-1&1&0\\0&0&1\end{pmatrix}}\left({\begin{array}{*{3}{c}|c}1&6&0&9\\1&5&-2&2\\0&0&3&9\end{array}}\right)=\left({\begin{array}{*{3}{c}|c}1&6&0&9\\0&-1&-2&-7\\0&0&3&9\end{array}}\right)$

Now back substitution will give the solution.

Example 3.21

Gauss-Jordan reduction works the same way. For the matrix ending the prior example, first adjust the leading entries

${\begin{pmatrix}1&0&0\\0&-1&0\\0&0&1/3\end{pmatrix}}\left({\begin{array}{*{3}{c}|c}1&6&0&9\\0&-1&-2&-7\\0&0&3&9\end{array}}\right)=\left({\begin{array}{*{3}{c}|c}1&6&0&9\\0&1&2&7\\0&0&1&3\end{array}}\right)$

and to finish, clear the third column and then the second column.

${\begin{pmatrix}1&-6&0\\0&1&0\\0&0&1\end{pmatrix}}{\begin{pmatrix}1&0&0\\0&1&-2\\0&0&1\end{pmatrix}}\left({\begin{array}{*{3}{c}|c}1&6&0&9\\0&1&2&7\\0&0&1&3\end{array}}\right)=\left({\begin{array}{*{3}{c}|c}1&0&0&3\\0&1&0&1\\0&0&1&3\end{array}}\right)$

We have observed the following result, which we shall use in the next subsection.

Corollary 3.22

For any matrix $H$ there are elementary reduction matrices $R_{1}$, ..., $R_{r}$ such that $R_{r}\cdot R_{r-1}\cdots R_{1}\cdot H$ is in reduced echelon form.

Until now we have taken the point of view that our primary objects of study are vector spaces and the maps between them, and have adopted matrices only for computational convenience. This subsection show that this point of view isn't the whole story. Matrix theory is a fascinating and fruitful area.

In the rest of this book we shall continue to focus on maps as the primary objects, but we will be pragmatic— if the matrix point of view gives some clearer idea then we shall use it.

Exercises

This exercise is recommended for all readers.

Problem 1

Predict the result of each multiplication by an elementary reduction matrix, and then check by multiplying it out.

1. ${\begin{pmatrix}3&0\\0&0\end{pmatrix}}{\begin{pmatrix}1&2\\3&4\end{pmatrix}}$
2. ${\begin{pmatrix}4&0\\0&2\end{pmatrix}}{\begin{pmatrix}1&2\\3&4\end{pmatrix}}$
3. ${\begin{pmatrix}1&0\\-2&1\end{pmatrix}}{\begin{pmatrix}1&2\\3&4\end{pmatrix}}$
4. ${\begin{pmatrix}1&2\\3&4\end{pmatrix}}{\begin{pmatrix}1&-1\\0&1\end{pmatrix}}$
5. ${\begin{pmatrix}1&2\\3&4\end{pmatrix}}{\begin{pmatrix}0&1\\1&0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

The need to take linear combinations of rows and columns in tables of numbers arises often in practice. For instance, this is a map of part of Vermont and New York.

In part because of Lake Champlain, there are no roads directly connecting some pairs of towns. For instance, there is no way to go from Winooski to Grand Isle without going through Colchester. (Of course, many other roads and towns have been left off to simplify the graph. From top to bottom of this map is about forty miles.)

1. The incidence matrix of a map is the square matrix whose $i,j$ entry is the number of roads from city $i$ to city $j$. Produce the incidence matrix of this map (take the cities in alphabetical order).
2. A matrix is symmetric if it equals its transpose. Show that an incidence matrix is symmetric. (These are all two-way streets. Vermont doesn't have many one-way streets.)
3. What is the significance of the square of the incidence matrix? The cube?

This exercise is recommended for all readers.

Problem 3

This table gives the number of hours of each type done by each worker, and the associated pay rates. Use matrices to compute the wages due.

regular   overtime Alan     40   12 Betty     35   6 Catherine     40   18 Donald     28   0

wage regular     $25.00 overtime     $45.00

(Remark. This illustrates, as did the prior problem, that in practice we often want to compute linear combinations of rows and columns in a context where we really aren't interested in any associated linear maps.)

Problem 4

Find the product of this matrix with its transpose.

${\begin{pmatrix}\cos \theta &-\sin \theta \\\sin \theta &\cos \theta \end{pmatrix}}$

This exercise is recommended for all readers.

Problem 5

Prove that the diagonal matrices form a subspace of ${\mathcal {M}}_{n\!\times \!n}$. What is its dimension?

Problem 6

Does the identity matrix represent the identity map if the bases are unequal?

Problem 7

Show that every multiple of the identity commutes with every square matrix. Are there other matrices that commute with all square matrices?

Problem 8

Prove or disprove: nonsingular matrices commute.

This exercise is recommended for all readers.

Problem 9

Show that the product of a permutation matrix and its transpose is an identity matrix.

Problem 10

Show that if the first and second rows of $G$ are equal then so are the first and second rows of $GH$. Generalize.

Problem 11

Describe the product of two diagonal matrices.

Problem 12

Write

${\begin{pmatrix}1&0\\-3&3\end{pmatrix}}$

as the product of two elementary reduction matrices.

This exercise is recommended for all readers.

Problem 13

Show that if $G$ has a row of zeros then $GH$ (if defined) has a row of zeros. Does that work for columns?

Problem 14

Show that the set of unit matrices forms a basis for ${\mathcal {M}}_{n\!\times \!m}$.

Problem 15

Find the formula for the $n$-th power of this matrix.

${\begin{pmatrix}1&1\\1&0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 16

The trace of a square matrix is the sum of the entries on its diagonal (its significance appears in Chapter Five). Show that ${\text{trace}}\,(GH)={\text{trace}}\,(HG)$.

This exercise is recommended for all readers.

Problem 17

A square matrix is upper triangular if its only nonzero entries lie above, or on, the diagonal. Show that the product of two upper triangular matrices is upper triangular. Does this hold for lower triangular also?

Problem 18

A square matrix is a Markov matrix if each entry is between zero and one and the sum along each row is one. Prove that a product of Markov matrices is Markov.

This exercise is recommended for all readers.

Problem 19

Give an example of two matrices of the same rank with squares of differing rank.

Problem 20

Combine the two generalizations of the identity matrix, the one allowing entires to be other than ones, and the one allowing the single one in each row and column to be off the diagonal. What is the action of this type of matrix?

Problem 21

On a computer multiplications are more costly than additions, so people are interested in reducing the number of multiplications used to compute a matrix product.

1. How many real number multiplications are needed in formula we gave for the product of a $m\!\times \!r$ matrix and a $r\!\times \!n$ matrix?
2. Matrix multiplication is associative, so all associations yield the same result. The cost in number of multiplications, however, varies. Find the association requiring the fewest real number multiplications to compute the matrix product of a $5\!\times \!10$ matrix, a $10\!\times \!20$ matrix, a $20\!\times \!5$ matrix, and a $5\!\times \!1$ matrix.
3. (Very hard.) Find a way to multiply two $2\!\times \!2$ matrices using only seven multiplications instead of the eight suggested by the naive approach.

? Problem 22

If $A$ and $B$ are square matrices of the same size such that $ABAB=0$, does it follow that $BABA=0$? (Putnam Exam 1990)

Problem 23

Demonstrate these four assertions to get an alternate proof that column rank equals row rank. (Liebeck 1966)

1. ${\vec {y}}\cdot {\vec {y}}={\vec {0}}$ iff ${\vec {y}}={\vec {0}}$.
2. $A{\vec {x}}={\vec {0}}$ iff ${{A}^{\rm {trans}}}A{\vec {x}}={\vec {0}}$.
3. $\dim({\mathcal {R}}(A))=\dim({\mathcal {R}}({{A}^{\rm {trans}}}A))$.
4. ${\text{col rank}}(A)={\text{col rank}}({{A}^{\rm {trans}}})={\text{row rank}}(A)$.

Problem 24

Prove (where $A$ is an $n\!\times \!n$ matrix and so defines a transformation of any $n$-dimensional space $V$ with respect to $B,B$ where $B$ is a basis) that $\dim({\mathcal {R}}(A)\cap {\mathcal {N}}(A))=\dim({\mathcal {R}}(A))-\dim({\mathcal {R}}(A^{2}))$. Conclude

1. ${\mathcal {N}}(A)\subset {\mathcal {R}}(A)$ iff $\dim({\mathcal {N}}(A))=\dim({\mathcal {R}}(A))-\dim({\mathcal {R}}(A^{2}))$;
2. ${\mathcal {R}}(A)\subseteq {\mathcal {N}}(A)$ iff $A^{2}=0$;
3. ${\mathcal {R}}(A)={\mathcal {N}}(A)$ iff $A^{2}=0$ and $\dim({\mathcal {N}}(A))=\dim({\mathcal {R}}(A))$ ;
4. $\dim({\mathcal {R}}(A)\cap {\mathcal {N}}(A))=0$ iff $\dim({\mathcal {R}}(A))=\dim({\mathcal {R}}(A^{2}))$ ;
5. (Requires the Direct Sum subsection, which is optional.) $V={\mathcal {R}}(A)\oplus {\mathcal {N}}(A)$ iff $\dim({\mathcal {R}}(A))=\dim({\mathcal {R}}(A^{2}))$.

(Ackerson 1955)

4 - Inverses

We now consider how to represent the inverse of a linear map.

We start by recalling some facts about function inverses. Some functions have no inverse, or have an inverse on the left side or right side only.

Example 4.1

Where $\pi :\mathbb {R} ^{3}\to \mathbb {R} ^{2}$ is the projection map

${\begin{pmatrix}x\\y\\z\end{pmatrix}}\mapsto {\begin{pmatrix}x\\y\end{pmatrix}}$

and $\eta :\mathbb {R} ^{2}\to \mathbb {R} ^{3}$ is the embedding

${\begin{pmatrix}x\\y\end{pmatrix}}\mapsto {\begin{pmatrix}x\\y\\0\end{pmatrix}}$

the composition $\pi \circ \eta$ is the identity map on $\mathbb {R} ^{2}$.

${\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {\eta }{\longmapsto }}{\begin{pmatrix}x\\y\\0\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}x\\y\end{pmatrix}}$

We say $\pi$ is a left inverse map of $\eta$ or, what is the same thing, that $\eta$ is a right inverse map of $\pi$. However, composition in the other order $\eta \circ \pi$ doesn't give the identity map— here is a vector that is not sent to itself under $\eta \circ \pi$.

${\begin{pmatrix}0\\0\\1\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}0\\0\end{pmatrix}}{\stackrel {\eta }{\longmapsto }}{\begin{pmatrix}0\\0\\0\end{pmatrix}}$

In fact, the projection $\pi$ has no left inverse at all. For, if $f$ were to be a left inverse of $\pi$ then we would have

${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {\pi }{\longmapsto }}{\begin{pmatrix}x\\y\end{pmatrix}}{\stackrel {f}{\longmapsto }}{\begin{pmatrix}x\\y\\z\end{pmatrix}}$

for all of the infinitely many $z$'s. But no function $f$ can send a single argument to more than one value.

(An example of a function with no inverse on either side is the zero transformation on $\mathbb {R} ^{2}$.) Some functions have a two-sided inverse map, another function that is the inverse of the first, both from the left and from the right. For instance, the map given by ${\vec {v}}\mapsto 2\cdot {\vec {v}}$ has the two-sided inverse ${\vec {v}}\mapsto (1/2)\cdot {\vec {v}}$. In this subsection we will focus on two-sided inverses. The appendix shows that a function has a two-sided inverse if and only if it is both one-to-one and onto. The appendix also shows that if a function $f$ has a two-sided inverse then it is unique, and so it is called "the" inverse, and is denoted $f^{-1}$. So our purpose in this subsection is, where a linear map $h$ has an inverse, to find the relationship between ${\rm {Rep}}_{B,D}(h)$ and ${\rm {Rep}}_{D,B}(h^{-1})$ (recall that we have shown, in Theorem II.2.21 of Section II of this chapter, that if a linear map has an inverse then the inverse is a linear map also).

Definition 4.2

A matrix $G$ is a left inverse matrix of the matrix $H$ if $GH$ is the identity matrix. It is a right inverse matrix if $HG$ is the identity. A matrix $H$ with a two-sided inverse is an invertible matrix. That two-sided inverse is called the inverse matrix and is denoted $H^{-1}$.

Because of the correspondence between linear maps and matrices, statements about map inverses translate into statements about matrix inverses.

Lemma 4.3

If a matrix has both a left inverse and a right inverse then the two are equal.

Theorem 4.4

A matrix is invertible if and only if it is nonsingular.

Proof

(For both results.) Given a matrix $H$, fix spaces of appropriate dimension for the domain and codomain. Fix bases for these spaces. With respect to these bases, $H$ represents a map $h$. The statements are true about the map and therefore they are true about the matrix.

Lemma 4.5

A product of invertible matrices is invertible— if $G$ and $H$ are invertible and if $GH$ is defined then $GH$ is invertible and $(GH)^{-1}=H^{-1}G^{-1}$.

Proof

(This is just like the prior proof except that it requires two maps.) Fix appropriate spaces and bases and consider the represented maps $h$ and $g$. Note that $h^{-1}g^{-1}$ is a two-sided map inverse of $gh$ since $(h^{-1}g^{-1})(gh)=h^{-1}({\mbox{id}})h=h^{-1}h={\mbox{id}}$ and $(gh)(h^{-1}g^{-1})=g({\mbox{id}})g^{-1}=gg^{-1}={\mbox{id}}$. This equality is reflected in the matrices representing the maps, as required.

Here is the arrow diagram giving the relationship between map inverses and matrix inverses. It is a special case of the diagram for function composition and matrix multiplication.

Beyond its place in our general program of seeing how to represent map operations, another reason for our interest in inverses comes from solving linear systems. A linear system is equivalent to a matrix equation, as here.

${\begin{array}{*{2}{rc}r}x_{1}&+&x_{2}&=&3\\2x_{1}&-&x_{2}&=&2\end{array}}\quad \Longleftrightarrow \quad {\begin{pmatrix}1&1\\2&-1\end{pmatrix}}{\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix}}={\begin{pmatrix}3\\2\end{pmatrix}}\qquad \qquad (*)$

By fixing spaces and bases (e.g., $\mathbb {R} ^{2},\mathbb {R} ^{2}$ and ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$), we take the matrix $H$ to represent some map $h$. Then solving the system is the same as asking: what domain vector ${\vec {x}}$ is mapped by $h$ to the result ${\vec {d}}\,$? If we could invert $h$ then we could solve the system by multiplying ${\rm {Rep}}_{D,B}(h^{-1})\cdot {\rm {Rep}}_{D}({\vec {d}})$ to get ${\rm {Rep}}_{B}({\vec {x}})$.

Example 4.6

We can find a left inverse for the matrix just given

${\begin{pmatrix}m&n\\p&q\end{pmatrix}}{\begin{pmatrix}1&1\\2&-1\end{pmatrix}}={\begin{pmatrix}1&0\\0&1\end{pmatrix}}$

by using Gauss' method to solve the resulting linear system.

${\begin{array}{*{4}{rc}r}m&+&2n&&&&&=&1\\m&-&n&&&&&=&0\\&&&&p&+&2q&=&0\\&&&&p&-&q&=&1\end{array}}$

Answer: $m=1/3$, $n=1/3$, $p=2/3$, and $q=-1/3$. This matrix is actually the two-sided inverse of $H$, as can easily be checked. With it we can solve the system ($*$) above by applying the inverse.

${\begin{pmatrix}x\\y\end{pmatrix}}={\begin{pmatrix}1/3&1/3\\2/3&-1/3\end{pmatrix}}{\begin{pmatrix}3\\2\end{pmatrix}}={\begin{pmatrix}5/3\\4/3\end{pmatrix}}$

Remark 4.7

Why solve systems this way, when Gauss' method takes less arithmetic (this assertion can be made precise by counting the number of arithmetic operations, as computer algorithm designers do)? Beyond its conceptual appeal of fitting into our program of discovering how to represent the various map operations, solving linear systems by using the matrix inverse has at least two advantages.

First, once the work of finding an inverse has been done, solving a system with the same coefficients but different constants is easy and fast: if we change the entries on the right of the system ($*$) then we get a related problem

${\begin{pmatrix}1&1\\2&-1\end{pmatrix}}{\begin{pmatrix}x\\y\end{pmatrix}}={\begin{pmatrix}5\\1\end{pmatrix}}$

with a related solution method.

${\begin{pmatrix}x\\y\end{pmatrix}}={\begin{pmatrix}1/3&1/3\\2/3&-1/3\end{pmatrix}}{\begin{pmatrix}5\\1\end{pmatrix}}={\begin{pmatrix}2\\3\end{pmatrix}}$

In applications, solving many systems having the same matrix of coefficients is common.

Another advantage of inverses is that we can explore a system's sensitivity to changes in the constants. For example, tweaking the $3$ on the right of the system ($*$) to

${\begin{pmatrix}1&1\\2&-1\end{pmatrix}}{\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix}}={\begin{pmatrix}3.01\\2\end{pmatrix}}$

can be solved with the inverse.

${\begin{pmatrix}1/3&1/3\\2/3&-1/3\end{pmatrix}}{\begin{pmatrix}3.01\\2\end{pmatrix}}={\begin{pmatrix}(1/3)(3.01)+(1/3)(2)\\(2/3)(3.01)-(1/3)(2)\end{pmatrix}}$

to show that $x_{1}$ changes by $1/3$ of the tweak while $x_{2}$ moves by $2/3$ of that tweak. This sort of analysis is used, for example, to decide how accurately data must be specified in a linear model to ensure that the solution has a desired accuracy.

We finish by describing the computational procedure usually used to find the inverse matrix.

Lemma 4.8

A matrix is invertible if and only if it can be written as the product of elementary reduction matrices. The inverse can be computed by applying to the identity matrix the same row steps, in the same order, as are used to Gauss-Jordan reduce the invertible matrix.

Proof

A matrix $H$ is invertible if and only if it is nonsingular and thus Gauss-Jordan reduces to the identity. By Corollary 3.22 this reduction can be done with elementary matrices $R_{r}\cdot R_{r-1}\dots R_{1}\cdot H=I$. This equation gives the two halves of the result.

First, elementary matrices are invertible and their inverses are also elementary. Applying $R_{r}^{-1}$ to the left of both sides of that equation, then $R_{r-1}^{-1}$, etc., gives $H$ as the product of elementary matrices $H=R_{1}^{-1}\cdots R_{r}^{-1}\cdot I$ (the $I$ is here to cover the trivial $r=0$ case).

Second, matrix inverses are unique and so comparison of the above equation with $H^{-1}H=I$ shows that $H^{-1}=R_{r}\cdot R_{r-1}\dots R_{1}\cdot I$. Therefore, applying $R_{1}$ to the identity, followed by $R_{2}$, etc., yields the inverse of $H$.

Example 4.9

To find the inverse of

${\begin{pmatrix}1&1\\2&-1\end{pmatrix}}$

we do Gauss-Jordan reduction, meanwhile performing the same operations on the identity. For clerical convenience we write the matrix and the identity side-by-side, and do the reduction steps together.

${\begin{array}{rcl}\left({\begin{array}{cc|cc}1&1&1&0\\2&-1&0&1\end{array}}\right)&{\xrightarrow[{}]{-2\rho _{1}+\rho _{2}}}&\left({\begin{array}{cc|cc}1&1&1&0\\0&-3&-2&1\end{array}}\right)\\&{\xrightarrow[{}]{-1/3\rho _{2}}}&\left({\begin{array}{cc|cc}1&1&1&0\\0&1&2/3&-1/3\end{array}}\right)\\&{\xrightarrow[{}]{-\rho _{2}+\rho _{1}}}&\left({\begin{array}{cc|cc}1&0&1/3&1/3\\0&1&2/3&-1/3\end{array}}\right)\end{array}}$

This calculation has found the inverse.

${\begin{pmatrix}1&1\\2&-1\end{pmatrix}}^{-1}={\begin{pmatrix}1/3&1/3\\2/3&-1/3\end{pmatrix}}$

Example 4.10

This one happens to start with a row swap.

${\begin{array}{rcl}\left({\begin{array}{ccc|ccc}0&3&-1&1&0&0\\1&0&1&0&1&0\\1&-1&0&0&0&1\end{array}}\right)&{\xrightarrow[{}]{\rho _{1}\leftrightarrow \rho _{2}}}&\left({\begin{array}{ccc|ccc}1&0&1&0&1&0\\0&3&-1&1&0&0\\1&-1&0&0&0&1\end{array}}\right)\\&{\xrightarrow[{}]{-\rho _{1}+\rho _{3}}}&\left({\begin{array}{ccc|ccc}1&0&1&0&1&0\\0&3&-1&1&0&0\\0&-1&-1&0&-1&1\end{array}}\right)\\&\vdots \\&{\xrightarrow[{}]{}}&\left({\begin{array}{ccc|ccc}1&0&0&1/4&1/4&3/4\\0&1&0&1/4&1/4&-1/4\\0&0&1&-1/4&3/4&-3/4\end{array}}\right)\end{array}}$

Example 4.11

A non-invertible matrix is detected by the fact that the left half won't reduce to the identity.

$\left({\begin{array}{cc|cc}1&1&1&0\\2&2&0&1\end{array}}\right){\xrightarrow[{}]{-2\rho _{1}+\rho _{2}}}\left({\begin{array}{cc|cc}1&1&1&0\\0&0&-2&1\end{array}}\right)$

This procedure will find the inverse of a general $n\!\times \!n$ matrix. The $2\!\times \!2$ case is handy.

Corollary 4.12

The inverse for a $2\!\times \!2$ matrix exists and equals

${\begin{pmatrix}a&b\\c&d\end{pmatrix}}^{-1}={\frac {1}{ad-bc}}{\begin{pmatrix}d&-b\\-c&a\end{pmatrix}}$

if and only if $ad-bc\neq 0$.

Proof

This computation is Problem 10.

We have seen here, as in the Mechanics of Matrix Multiplication subsection, that we can exploit the correspondence between linear maps and matrices. So we can fruitfully study both maps and matrices, translating back and forth to whichever helps us the most.

Over the entire four subsections of this section we have developed an algebra system for matrices. We can compare it with the familiar algebra system for the real numbers. Here we are working not with numbers but with matrices. We have matrix addition and subtraction operations, and they work in much the same way as the real number operations, except that they only combine same-sized matrices. We also have a matrix multiplication operation and an operation inverse to multiplication. These are somewhat like the familiar real number operations (associativity, and distributivity over addition, for example), but there are differences (failure of commutativity, for example). And, we have scalar multiplication, which is in some ways another extension of real number multiplication. This matrix system provides an example that algebra systems other than the elementary one can be interesting and useful.

Exercises

Problem 1

Supply the intermediate steps in Example 4.10.

This exercise is recommended for all readers.

Problem 2

Use Corollary 4.12 to decide if each matrix has an inverse.

1. ${\begin{pmatrix}2&1\\-1&1\end{pmatrix}}$
2. ${\begin{pmatrix}0&4\\1&-3\end{pmatrix}}$
3. ${\begin{pmatrix}2&-3\\-4&6\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 3

For each invertible matrix in the prior problem, use Corollary 4.12 to find its inverse.

This exercise is recommended for all readers.

Problem 4

Find the inverse, if it exists, by using the Gauss-Jordan method. Check the answers for the $2\!\times \!2$ matrices with Corollary 4.12.

1. ${\begin{pmatrix}3&1\\0&2\end{pmatrix}}$
2. ${\begin{pmatrix}2&1/2\\3&1\end{pmatrix}}$
3. ${\begin{pmatrix}2&-4\\-1&2\end{pmatrix}}$
4. ${\begin{pmatrix}1&1&3\\0&2&4\\-1&1&0\end{pmatrix}}$
5. ${\begin{pmatrix}0&1&5\\0&-2&4\\2&3&-2\end{pmatrix}}$
6. ${\begin{pmatrix}2&2&3\\1&-2&-3\\4&-2&-3\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 5

What matrix has this one for its inverse?

${\begin{pmatrix}1&3\\2&5\end{pmatrix}}$

Problem 6

How does the inverse operation interact with scalar multiplication and addition of matrices?

1. What is the inverse of $rH$?
2. Is $(H+G)^{-1}=H^{-1}+G^{-1}$?

This exercise is recommended for all readers.

Problem 7

Is $(T^{k})^{-1}=(T^{-1})^{k}$?

Problem 8

Is $H^{-1}$ invertible?

Problem 9

For each real number $\theta$ let $t_{\theta }:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ be represented with respect to the standard bases by this matrix.

${\begin{pmatrix}\cos \theta &-\sin \theta \\\sin \theta &\cos \theta \end{pmatrix}}$

Show that $t_{\theta _{1}+\theta _{2}}=t_{\theta _{1}}\cdot t_{\theta _{2}}$. Show also that ${t_{\theta }}^{-1}=t_{-\theta }$.

Problem 10

Do the calculations for the proof of Corollary 4.12.

Problem 11

Show that this matrix

$H={\begin{pmatrix}1&0&1\\0&1&0\end{pmatrix}}$

has infinitely many right inverses. Show also that it has no left inverse.

Problem 12

In Example 4.1, how many left inverses has $\eta$?

Problem 13

If a matrix has infinitely many right-inverses, can it have infinitely many left-inverses? Must it have?

This exercise is recommended for all readers.

Problem 14

Assume that $H$ is invertible and that $HG$ is the zero matrix. Show that $G$ is a zero matrix.

Problem 15

Prove that if $H$ is invertible then the inverse commutes with a matrix $GH^{-1}=H^{-1}G$ if and only if $H$ itself commutes with that matrix $GH=HG$.

This exercise is recommended for all readers.

Problem 16

Show that if $T$ is square and if $T^{4}$ is the zero matrix then $(I-T)^{-1}=I+T+T^{2}+T^{3}$. Generalize.

This exercise is recommended for all readers.

Problem 17

Let $D$ be diagonal. Describe $D^{2}$, $D^{3}$, ... , etc. Describe $D^{-1}$, $D^{-2}$, ... , etc. Define $D^{0}$ appropriately.

Problem 18

Prove that any matrix row-equivalent to an invertible matrix is also invertible.

Problem 19

The first question below appeared as Problem 15 in the Matrix Multiplication subsection.

1. Show that the rank of the product of two matrices is less than or equal to the minimum of the rank of each.
2. Show that if $T$ and $S$ are square then $TS=I$ if and only if $ST=I$.

Problem 20

Show that the inverse of a permutation matrix is its transpose.

Problem 21

The first two parts of this question appeared as Problem 12. of the Matrix Multiplication subsection

1. Show that ${{(GH)}^{\rm {trans}}}={{H}^{\rm {trans}}}{{G}^{\rm {trans}}}$.
2. A square matrix is symmetric if each $i,j$ entry equals the $j,i$ entry (that is, if the matrix equals its transpose). Show that the matrices $H{{H}^{\rm {trans}}}$ and ${{H}^{\rm {trans}}}H$ are symmetric.
3. Show that the inverse of the transpose is the transpose of the inverse.
4. Show that the inverse of a symmetric matrix is symmetric.

This exercise is recommended for all readers.

Problem 22

The items starting this question appeared as Problem 17 of the Matrix Multiplication subsection.

1. Prove that the composition of the projections $\pi _{x},\pi _{y}:\mathbb {R} ^{3}\to \mathbb {R} ^{3}$ is the zero map despite that neither is the zero map.
2. Prove that the composition of the derivatives $d^{2}/dx^{2},\,d^{3}/dx^{3}:{\mathcal {P}}_{4}\to {\mathcal {P}}_{4}$ is the zero map despite that neither map is the zero map.
3. Give matrix equations representing each of the prior two items.

When two things multiply to give zero despite that neither is zero, each is said to be a zero divisor. Prove that no zero divisor is invertible.

Problem 23

In real number algebra, there are exactly two numbers, $1$ and $-1$, that are their own multiplicative inverse. Does $H^{2}=I$ have exactly two solutions for $2\!\times \!2$ matrices?

Problem 24

Is the relation "is a two-sided inverse of" transitive? Reflexive? Symmetric?

Problem 25

Prove: if the sum of the elements in each row of a square matrix is $k$, then the sum of the elements in each row of the inverse matrix is $1/k$. (Wilansky 1951)

Section V - Change of Basis

Representations, whether of vectors or of maps, vary with the bases. For instance, with respect to the two bases ${\mathcal {E}}_{2}$ and

$B=\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\rangle$

for $\mathbb {R} ^{2}$, the vector ${\vec {e}}_{1}$ has two different representations.

${\rm {Rep}}_{{\mathcal {E}}_{2}}({\vec {e}}_{1})={\begin{pmatrix}1\\0\end{pmatrix}}\qquad {\rm {Rep}}_{B}({\vec {e}}_{1})={\begin{pmatrix}1/2\\1/2\end{pmatrix}}$

Similarly, with respect to ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$ and ${\mathcal {E}}_{2},B$, the identity map has two different representations.

${\rm {Rep}}_{{\mathcal {E}}_{2},{\mathcal {E}}_{2}}({\text{id}})={\begin{pmatrix}1&0\\0&1\end{pmatrix}}\qquad {\rm {Rep}}_{{\mathcal {E}}_{2},B}({\text{id}})={\begin{pmatrix}1/2&1/2\\1/2&-1/2\end{pmatrix}}$

With our point of view that the objects of our studies are vectors and maps, in fixing bases we are adopting a scheme of tags or names for these objects, that are convienent for computation. We will now see how to translate among these names— we will see exactly how representations vary as the bases vary.

1 - Changing Representations of Vectors

In converting ${\rm {Rep}}_{B}({\vec {v}})$ to ${\rm {Rep}}_{D}({\vec {v}})$ the underlying vector ${\vec {v}}$ doesn't change. Thus, this translation is accomplished by the identity map on the space, described so that the domain space vectors are represented with respect to $B$ and the codomain space vectors are represented with respect to $D$.

(The diagram is vertical to fit with the ones in the next subsection.)

Definition 1.1

The change of basis matrixfor bases $B,D\subset V$ is the representation of the identity map ${\mbox{id}}:V\to V$ with respect to those bases.

${\rm {Rep}}_{B,D}({\mbox{id}})=\left({\begin{array}{c|c|c}\vdots &&\vdots \\{\rm {Rep}}_{D}({\vec {\beta }}_{1})&\;\cdots \;&{\rm {Rep}}_{D}({\vec {\beta }}_{n})\\\vdots &&\vdots \end{array}}\right)$

Lemma 1.2

Left-multiplication by the change of basis matrix for $B,D$ converts a representation with respect to $B$ to one with respect to $D$. Conversly, if left-multiplication by a matrix changes bases $M\cdot {\rm {Rep}}_{B}({\vec {v}})={\rm {Rep}}_{D}({\vec {v}})$ then $M$ is a change of basis matrix.

Proof

For the first sentence, for each ${\vec {v}}$, as matrix-vector multiplication represents a map application, ${\rm {Rep}}_{B,D}({\mbox{id}})\cdot {\rm {Rep}}_{B}({\vec {v}})={\rm {Rep}}_{D}(\,{\mbox{id}}({\vec {v}})\,)={\rm {Rep}}_{D}({\vec {v}})$. For the second sentence, with respect to $B,D$ the matrix $M$ represents some linear map, whose action is ${\vec {v}}\mapsto {\vec {v}}$, and is therefore the identity map.

Example 1.3

With these bases for $\mathbb {R} ^{2}$,

$B=\langle {\begin{pmatrix}2\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\rangle \qquad D=\langle {\begin{pmatrix}-1\\1\end{pmatrix}},{\begin{pmatrix}1\\1\end{pmatrix}}\rangle$

because

${\rm {Rep}}_{D}(\,{\mbox{id}}({\begin{pmatrix}2\\1\end{pmatrix}}))={\begin{pmatrix}-1/2\\3/2\end{pmatrix}}_{D}\qquad {\rm {Rep}}_{D}(\,{\mbox{id}}({\begin{pmatrix}1\\0\end{pmatrix}}))={\begin{pmatrix}-1/2\\1/2\end{pmatrix}}_{D}$

the change of basis matrix is this.

${\rm {Rep}}_{B,D}({\rm {id)={\begin{pmatrix}-1/2&-1/2\\3/2&1/2\end{pmatrix}}}}$

We can see this matrix at work by finding the two representations of ${\vec {e}}_{2}$

${\rm {Rep}}_{B}({\begin{pmatrix}0\\1\end{pmatrix}})={\begin{pmatrix}1\\-2\end{pmatrix}}\qquad {\rm {Rep}}_{D}({\begin{pmatrix}0\\1\end{pmatrix}})={\begin{pmatrix}1/2\\1/2\end{pmatrix}}$

and checking that the conversion goes as expected.

${\begin{pmatrix}-1/2&-1/2\\3/2&1/2\end{pmatrix}}{\begin{pmatrix}1\\-2\end{pmatrix}}={\begin{pmatrix}1/2\\1/2\end{pmatrix}}$

We finish this subsection by recognizing that the change of basis matrices are familiar.

Lemma 1.4

A matrix changes bases if and only if it is nonsingular.

Proof

For one direction, if left-multiplication by a matrix changes bases then the matrix represents an invertible function, simply because the function is inverted by changing the bases back. Such a matrix is itself invertible, and so nonsingular.

To finish, we will show that any nonsingular matrix $M$ performs a change of basis operation from any given starting basis $B$ to some ending basis. Because the matrix is nonsingular, it will Gauss-Jordan reduce to the identity, so there are elementatry reduction matrices such that $R_{r}\cdots R_{1}\cdot M=I$. Elementary matrices are invertible and their inverses are also elementary, so multiplying from the left first by ${R_{r}}^{-1}$, then by ${R_{r-1}}^{-1}$, etc., gives $M$ as a product of elementary matrices $M={R_{1}}^{-1}\cdots {R_{r}}^{-1}$. Thus, we will be done if we show that elementary matrices change a given basis to another basis, for then ${R_{r}}^{-1}$ changes $B$ to some other basis $B_{r}$, and ${R_{r-1}}^{-1}$ changes $B_{r}$ to some $B_{r-1}$, ..., and the net effect is that $M$ changes $B$ to $B_{1}$. We will prove this about elementary matrices by covering the three types as separate cases.

Applying a row-multiplication matrix

$M_{i}(k){\begin{pmatrix}c_{1}\\\vdots \\c_{i}\\\vdots \\c_{n}\end{pmatrix}}={\begin{pmatrix}c_{1}\\\vdots \\kc_{i}\\\vdots \\c_{n}\end{pmatrix}}$

changes a representation with respect to $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i},\dots ,{\vec {\beta }}_{n}\rangle$ to one with respect to $\langle {\vec {\beta }}_{1},\dots ,(1/k){\vec {\beta }}_{i},\dots ,{\vec {\beta }}_{n}\rangle$ in this way.

${\vec {v}}=c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{i}\cdot {\vec {\beta }}_{i}+\dots +c_{n}\cdot {\vec {\beta }}_{n}$

$\mapsto \;c_{1}\cdot {\vec {\beta }}_{1}+\dots +kc_{i}\cdot (1/k){\vec {\beta }}_{i}+\dots +c_{n}\cdot {\vec {\beta }}_{n}={\vec {v}}$

Similarly, left-multiplication by a row-swap matrix $P_{i,j}$ changes a representation with respect to the basis $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i},\dots ,{\vec {\beta }}_{j},\dots ,{\vec {\beta }}_{n}\rangle$ into one with respect to the basis $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{j},\dots ,{\vec {\beta }}_{i},\dots ,{\vec {\beta }}_{n}\rangle$ in this way.

${\vec {v}}=c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{i}\cdot {\vec {\beta }}_{i}+\dots +c_{j}{\vec {\beta }}_{j}+\dots +c_{n}\cdot {\vec {\beta }}_{n}$

$\mapsto \;c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{j}\cdot {\vec {\beta }}_{j}+\dots +c_{i}\cdot {\vec {\beta }}_{i}+\dots +c_{n}\cdot {\vec {\beta }}_{n}={\vec {v}}$

And, a representation with respect to $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i},\dots ,{\vec {\beta }}_{j},\dots ,{\vec {\beta }}_{n}\rangle$ changes via left-multiplication by a row-combination matrix $C_{i,j}(k)$ into a representation with respect to $\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i}-k{\vec {\beta }}_{j},\dots ,{\vec {\beta }}_{j},\dots ,{\vec {\beta }}_{n}\rangle$

${\vec {v}}=c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{i}\cdot {\vec {\beta }}_{i}+c_{j}{\vec {\beta }}_{j}+\dots +c_{n}\cdot {\vec {\beta }}_{n}$

$\mapsto \;c_{1}\cdot {\vec {\beta }}_{1}+\dots +c_{i}\cdot ({\vec {\beta }}_{i}-k{\vec {\beta }}_{j})+\dots +(kc_{i}+c_{j})\cdot {\vec {\beta }}_{j}+\dots +c_{n}\cdot {\vec {\beta }}_{n}={\vec {v}}$

(the definition of reduction matrices specifies that $i\neq k$ and $k\neq 0$ and so this last one is a basis).

Corollary 1.5

A matrix is nonsingular if and only if it represents the identity map with respect to some pair of bases.

In the next subsection we will see how to translate among representations of maps, that is, how to change ${\rm {Rep}}_{B,D}(h)$ to ${\rm {Rep}}_{{\hat {B}},{\hat {D}}}(h)$. The above corollary is a special case of this, where the domain and range are the same space, and where the map is the identity map.

Exercises

This exercise is recommended for all readers.

Problem 1

In $\mathbb {R} ^{2}$, where

$D=\langle {\begin{pmatrix}2\\1\end{pmatrix}},{\begin{pmatrix}-2\\4\end{pmatrix}}\rangle$

find the change of basis matrices from $D$ to ${\mathcal {E}}_{2}$ and from ${\mathcal {E}}_{2}$ to $D$. Multiply the two.

This exercise is recommended for all readers.

Problem 2

Find the change of basis matrix for $B,D\subseteq \mathbb {R} ^{2}$.

1. $B={\mathcal {E}}_{2}$, $D=\langle {\vec {e}}_{2},{\vec {e}}_{1}\rangle$
2. $B={\mathcal {E}}_{2}$, $D=\langle {\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}1\\4\end{pmatrix}}\rangle$
3. $B=\langle {\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}1\\4\end{pmatrix}}\rangle$, $D={\mathcal {E}}_{2}$
4. $B=\langle {\begin{pmatrix}-1\\1\end{pmatrix}},{\begin{pmatrix}2\\2\end{pmatrix}}\rangle$, $D=\langle {\begin{pmatrix}0\\4\end{pmatrix}},{\begin{pmatrix}1\\3\end{pmatrix}}\rangle$

Problem 3

For the bases in Problem 2, find the change of basis matrix in the other direction, from $D$ to $B$.

This exercise is recommended for all readers.

Problem 4

Find the change of basis matrix for each $B,D\subseteq {\mathcal {P}}_{2}$.

1. $B=\langle 1,x,x^{2}\rangle ,D=\langle x^{2},1,x\rangle$
2. $B=\langle 1,x,x^{2}\rangle ,D=\langle 1,1+x,1+x+x^{2}\rangle$
3. $B=\langle 2,2x,x^{2}\rangle ,D=\langle 1+x^{2},1-x^{2},x+x^{2}\rangle$

This exercise is recommended for all readers.

Problem 5

Decide if each changes bases on $\mathbb {R} ^{2}$. To what basis is ${\mathcal {E}}_{2}$ changed?

1. ${\begin{pmatrix}5&0\\0&4\end{pmatrix}}$
2. ${\begin{pmatrix}2&1\\3&1\end{pmatrix}}$
3. ${\begin{pmatrix}-1&4\\2&-8\end{pmatrix}}$
4. ${\begin{pmatrix}1&-1\\1&1\end{pmatrix}}$

Problem 6

Find bases such that this matrix represents the identity map with respect to those bases.

${\begin{pmatrix}3&1&4\\2&-1&1\\0&0&4\end{pmatrix}}$

Problem 7

Conside the vector space of real-valued functions with basis $\langle \sin(x),\cos(x)\rangle$. Show that $\langle 2\sin(x)+\cos(x),3\cos(x)\rangle$ is also a basis for this space. Find the change of basis matrix in each direction.

Problem 8

Where does this matrix

${\begin{pmatrix}\cos(2\theta )&\sin(2\theta )\\\sin(2\theta )&-\cos(2\theta )\end{pmatrix}}$

send the standard basis for $\mathbb {R} ^{2}$? Any other bases? Hint. Consider the inverse.

This exercise is recommended for all readers.

Problem 9

What is the change of basis matrix with respect to $B,B$?

Problem 10

Prove that a matrix changes bases if and only if it is invertible.

Problem 11

Finish the proof of Lemma 1.4.

This exercise is recommended for all readers.

Problem 12

Let $H$ be a $n\!\times \!n$ nonsingular matrix. What basis of $\mathbb {R} ^{n}$ does $H$ change to the standard basis?

This exercise is recommended for all readers.

Problem 13

1. In ${\mathcal {P}}_{3}$ with basis $B=\langle 1+x,1-x,x^{2}+x^{3},x^{2}-x^{3}\rangle$ we have this represenatation.

   ${\rm {Rep}}_{B}(1-x+3x^{2}-x^{3})={\begin{pmatrix}0\\1\\1\\2\end{pmatrix}}_{B}$

   Find a basis $D$ giving this different representation for the same polynomial.

   ${\rm {Rep}}_{D}(1-x+3x^{2}-x^{3})={\begin{pmatrix}1\\0\\2\\0\end{pmatrix}}_{D}$

2. State and prove that any nonzero vector representation can be changed to any other.

Hint. The proof of Lemma 1.4 is constructive— it not only says the bases change, it shows how they change.

Problem 14

Let $V,W$ be vector spaces, and let $B,{\hat {B}}$ be bases for $V$ and $D,{\hat {D}}$ be bases for $W$. Where $h:V\to W$ is linear, find a formula relating ${\rm {Rep}}_{B,D}(h)$ to ${\rm {Rep}}_{{\hat {B}},{\hat {D}}}(h)$.

This exercise is recommended for all readers.

Problem 15

Show that the columns of an $n\!\times \!n$ change of basis matrix form a basis for $\mathbb {R} ^{n}$. Do all bases appear in that way: can the vectors from any $\mathbb {R} ^{n}$ basis make the columns of a change of basis matrix?

This exercise is recommended for all readers.

Problem 16

Find a matrix having this effect.

${\begin{pmatrix}1\\3\end{pmatrix}}\;\mapsto \;{\begin{pmatrix}4\\-1\end{pmatrix}}$

That is, find a $M$ that left-multiplies the starting vector to yield the ending vector. Is there a matrix having these two effects?

1. ${\begin{pmatrix}1\\3\end{pmatrix}}\mapsto {\begin{pmatrix}1\\1\end{pmatrix}}\quad {\begin{pmatrix}2\\-1\end{pmatrix}}\mapsto {\begin{pmatrix}-1\\-1\end{pmatrix}}$
2. ${\begin{pmatrix}1\\3\end{pmatrix}}\mapsto {\begin{pmatrix}1\\1\end{pmatrix}}\quad {\begin{pmatrix}2\\6\end{pmatrix}}\mapsto {\begin{pmatrix}-1\\-1\end{pmatrix}}$

Give a necessary and sufficient condition for there to be a matrix such that ${\vec {v}}_{1}\mapsto {\vec {w}}_{1}$ and ${\vec {v}}_{2}\mapsto {\vec {w}}_{2}$.

2 - Changing Map Representations

The first subsection shows how to convert the representation of a vector with respect to one basis to the representation of that same vector with respect to another basis. Here we will see how to convert the representation of a map with respect to one pair of bases to the representation of that map with respect to a different pair. That is, we want the relationship between the matrices in this arrow diagram.

To move from the lower-left of this diagram to the lower-right we can either go straight over, or else up to $V_{B}$ then over to $W_{D}$ and then down. Restated in terms of the matrices, we can calculate ${\hat {H}}={\rm {Rep}}_{{\hat {B}},{\hat {D}}}(h)$ either by simply using ${\hat {B}}$ and ${\hat {D}}$, or else by first changing bases with ${\rm {Rep}}_{{\hat {B}},B}({\mbox{id}})$ then multiplying by $H={\rm {Rep}}_{B,D}(h)$ and then changing bases with ${\rm {Rep}}_{D,{\hat {D}}}({\mbox{id}})$. This equation summarizes.

${\hat {H}}={\rm {Rep}}_{D,{\hat {D}}}({\mbox{id}})\cdot H\cdot {\rm {Rep}}_{{\hat {B}},B}({\mbox{id}})\qquad \qquad (*)$

(To compare this equation with the sentence before it, remember that the equation is read from right to left because function composition is read right to left and matrix multiplication represent the composition.)

Example 2.1

The matrix

$T={\begin{pmatrix}\cos(\pi /6)&-\sin(\pi /6)\\\sin(\pi /6)&\cos(\pi /6)\end{pmatrix}}={\begin{pmatrix}{\sqrt {3}}/2&-1/2\\1/2&{\sqrt {3}}/2\end{pmatrix}}$

represents, with respect to ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$, the transformation $t:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ that rotates vectors $\pi /6$ radians counterclockwise.

We can translate that representation with respect to ${\mathcal {E}}_{2},{\mathcal {E}}_{2}$ to one with respect to

${\hat {B}}=\langle {\begin{pmatrix}1\\1\end{pmatrix}}{\begin{pmatrix}0\\2\end{pmatrix}}\rangle \qquad {\hat {D}}=\langle {\begin{pmatrix}-1\\0\end{pmatrix}}{\begin{pmatrix}2\\3\end{pmatrix}}\rangle$

by using the arrow diagram and formula ($*$) above.

From this, we can use the formula:

${\hat {T}}={\rm {Rep}}_{{\mathcal {E}}_{2},{\hat {D}}}({\mbox{id}})\cdot T\cdot {\rm {Rep}}_{{\hat {B}},{\mathcal {E}}_{2}}({\mbox{id}})$

Note that ${\rm {Rep}}_{{\mathcal {E}}_{2},{\hat {D}}}({\mbox{id}})$ can be calculated as the matrix inverse of ${\rm {Rep}}_{{\hat {D}},{\mathcal {E}}_{2}}({\mbox{id}})$.

${\begin{array}{rl}{\rm {Rep}}_{{\hat {B}},{\hat {D}}}(t)&={\begin{pmatrix}-1&2\\0&3\end{pmatrix}}^{-1}{\begin{pmatrix}{\sqrt {3}}/2&-1/2\\1/2&{\sqrt {3}}/2\end{pmatrix}}{\begin{pmatrix}1&0\\1&2\end{pmatrix}}\\&={\begin{pmatrix}(5-{\sqrt {3}})/6&(3+2{\sqrt {3}})/3\\(1+{\sqrt {3}})/6&{\sqrt {3}}/3\end{pmatrix}}\end{array}}$

Although the new matrix is messier-appearing, the map that it represents is the same. For instance, to replicate the effect of $t$ in the picture, start with ${\hat {B}}$,

${\rm {Rep}}_{\hat {B}}({\begin{pmatrix}1\\3\end{pmatrix}})={\begin{pmatrix}1\\1\end{pmatrix}}_{\hat {B}}$

apply ${\hat {T}}$,

${\begin{pmatrix}(5-{\sqrt {3}})/6&(3+2{\sqrt {3}})/3\\(1+{\sqrt {3}})/6&{\sqrt {3}}/3\end{pmatrix}}_{{\hat {B}},{\hat {D}}}{\begin{pmatrix}1\\1\end{pmatrix}}_{\hat {B}}={\begin{pmatrix}(11+3{\sqrt {3}})/6\\(1+3{\sqrt {3}})/6\end{pmatrix}}_{\hat {D}}$

and check it against ${\hat {D}}$

${\frac {11+3{\sqrt {3}}}{6}}\cdot {\begin{pmatrix}-1\\0\end{pmatrix}}+{\frac {1+3{\sqrt {3}}}{6}}\cdot {\begin{pmatrix}2\\3\end{pmatrix}}={\begin{pmatrix}(-3+{\sqrt {3}})/2\\(1+3{\sqrt {3}})/2\end{pmatrix}}$

to see that it is the same result as above.

Example 2.2

On $\mathbb {R} ^{3}$ the map

${\begin{pmatrix}x\\y\\z\end{pmatrix}}{\stackrel {t}{\longmapsto }}{\begin{pmatrix}y+z\\x+z\\x+y\end{pmatrix}}$

that is represented with respect to the standard basis in this way

${\rm {Rep}}_{{\mathcal {E}}_{3},{\mathcal {E}}_{3}}(t)={\begin{pmatrix}0&1&1\\1&0&1\\1&1&0\end{pmatrix}}$

can also be represented with respect to another basis

if $B=\langle {\begin{pmatrix}1\\-1\\0\end{pmatrix}},{\begin{pmatrix}1\\1\\-2\end{pmatrix}},{\begin{pmatrix}1\\1\\1\end{pmatrix}}\rangle$      then ${\rm {Rep}}_{B,B}(t)={\begin{pmatrix}-1&0&0\\0&-1&0\\0&0&2\end{pmatrix}}$

in a way that is simpler, in that the action of a diagonal matrix is easy to understand.

Naturally, we usually prefer basis changes that make the representation easier to understand. When the representation with respect to equal starting and ending bases is a diagonal matrix we say the map or matrix has been diagonalized. In Chaper Five we shall see which maps and matrices are diagonalizable, and where one is not, we shall see how to get a representation that is nearly diagonal.

We finish this subsection by considering the easier case where representations are with respect to possibly different starting and ending bases. Recall that the prior subsection shows that a matrix changes bases if and only if it is nonsingular. That gives us another version of the above arrow diagram and equation ($*$).

Definition 2.3

Same-sized matrices $H$ and ${\hat {H}}$ are matrix equivalent if there are nonsingular matrices $P$ and $Q$ such that ${\hat {H}}=PHQ$.

Corollary 2.4

Matrix equivalent matrices represent the same map, with respect to appropriate pairs of bases.

Problem 10 checks that matrix equivalence is an equivalence relation. Thus it partitions the set of matrices into matrix equivalence classes.

All matrices:

$H$ matrix equivalent to ${\hat {H}}$

We can get some insight into the classes by comparing matrix equivalence with row equivalence (recall that matrices are row equivalent when they can be reduced to each other by row operations). In ${\hat {H}}=PHQ$, the matrices $P$ and $Q$ are nonsingular and thus each can be written as a product of elementary reduction matrices (see Lemma 4.8 in the previous subsection). Left-multiplication by the reduction matrices making up $P$ has the effect of performing row operations. Right-multiplication by the reduction matrices making up $Q$ performs column operations. Therefore, matrix equivalence is a generalization of row equivalence— two matrices are row equivalent if one can be converted to the other by a sequence of row reduction steps, while two matrices are matrix equivalent if one can be converted to the other by a sequence of row reduction steps followed by a sequence of column reduction steps.

Thus, if matrices are row equivalent then they are also matrix equivalent (since we can take $Q$ to be the identity matrix and so perform no column operations). The converse, however, does not hold.

Example 2.5

These two

${\begin{pmatrix}1&0\\0&0\end{pmatrix}}\qquad {\begin{pmatrix}1&1\\0&0\end{pmatrix}}$

are matrix equivalent because the second can be reduced to the first by the column operation of taking $-1$ times the first column and adding to the second. They are not row equivalent because they have different reduced echelon forms (in fact, both are already in reduced form).

We will close this section by finding a set of representatives for the matrix equivalence classes.

Theorem 2.6

Any $m\!\times \!n$ matrix of rank $k$ is matrix equivalent to the $m\!\times \!n$ matrix that is all zeros except that the first $k$ diagonal entries are ones.

${\begin{pmatrix}1&0&\ldots &0&0&\ldots &0\\0&1&\ldots &0&0&\ldots &0\\&\vdots \\0&0&\ldots &1&0&\ldots &0\\0&0&\ldots &0&0&\ldots &0\\&\vdots \\0&0&\ldots &0&0&\ldots &0\end{pmatrix}}$

Sometimes this is described as a block partial-identity form.

$\left({\begin{array}{c|c}I&Z\\\hline Z&Z\end{array}}\right)$

Proof

As discussed above, Gauss-Jordan reduce the given matrix and combine all the reduction matrices used there to make $P$. Then use the leading entries to do column reduction and finish by swapping columns to put the leading ones on the diagonal. Combine the reduction matrices used for those column operations into $Q$.

Example 2.7

We illustrate the proof by finding the $P$ and $Q$ for this matrix.

${\begin{pmatrix}1&2&1&-1\\0&0&1&-1\\2&4&2&-2\end{pmatrix}}$

First Gauss-Jordan row-reduce.

${\begin{pmatrix}1&-1&0\\0&1&0\\0&0&1\end{pmatrix}}{\begin{pmatrix}1&0&0\\0&1&0\\-2&0&1\end{pmatrix}}{\begin{pmatrix}1&2&1&-1\\0&0&1&-1\\2&4&2&-2\end{pmatrix}}={\begin{pmatrix}1&2&0&0\\0&0&1&-1\\0&0&0&0\end{pmatrix}}$

Then column-reduce, which involves right-multiplication.

${\begin{pmatrix}1&2&0&0\\0&0&1&-1\\0&0&0&0\end{pmatrix}}{\begin{pmatrix}1&-2&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1\end{pmatrix}}{\begin{pmatrix}1&0&0&0\\0&1&0&0\\0&0&1&1\\0&0&0&1\end{pmatrix}}={\begin{pmatrix}1&0&0&0\\0&0&1&0\\0&0&0&0\end{pmatrix}}$

Finish by swapping columns.

${\begin{pmatrix}1&0&0&0\\0&0&1&0\\0&0&0&0\end{pmatrix}}{\begin{pmatrix}1&0&0&0\\0&0&1&0\\0&1&0&0\\0&0&0&1\end{pmatrix}}={\begin{pmatrix}1&0&0&0\\0&1&0&0\\0&0&0&0\end{pmatrix}}$

Finally, combine the left-multipliers together as $P$ and the right-multipliers together as $Q$ to get the $PHQ$ equation.

${\begin{pmatrix}1&-1&0\\0&1&0\\-2&0&1\end{pmatrix}}{\begin{pmatrix}1&2&1&-1\\0&0&1&-1\\2&4&2&-2\end{pmatrix}}{\begin{pmatrix}1&0&-2&0\\0&0&1&0\\0&1&0&1\\0&0&0&1\end{pmatrix}}={\begin{pmatrix}1&0&0&0\\0&1&0&0\\0&0&0&0\end{pmatrix}}$

Corollary 2.8

Two same-sized matrices are matrix equivalent if and only if they have the same rank. That is, the matrix equivalence classes are characterized by rank.

Proof

Two same-sized matrices with the same rank are equivalent to the same block partial-identity matrix.

Example 2.9

The $2\!\times \!2$ matrices have only three possible ranks: zero, one, or two. Thus there are three matrix-equivalence classes.

All $2\!\times \!2$ matrices: Three equivalence
classes

Each class consists of all of the $2\!\times \!2$ matrices with the same rank. There is only one rank zero matrix, so that class has only one member, but the other two classes each have infinitely many members.

In this subsection we have seen how to change the representation of a map with respect to a first pair of bases to one with respect to a second pair. That led to a definition describing when matrices are equivalent in this way. Finally we noted that, with the proper choice of (possibly different) starting and ending bases, any map can be represented in block partial-identity form.

One of the nice things about this representation is that, in some sense, we can completely understand the map when it is expressed in this way: if the bases are $B=\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{n}\rangle$ and $D=\langle {\vec {\delta }}_{1},\dots ,{\vec {\delta }}_{m}\rangle$ then the map sends

$c_{1}{\vec {\beta }}_{1}+\dots +c_{k}{\vec {\beta }}_{k}+c_{k+1}{\vec {\beta }}_{k+1}+\dots +c_{n}{\vec {\beta }}_{n}\;\longmapsto \;c_{1}{\vec {\delta }}_{1}+\dots +c_{k}{\vec {\delta }}_{k}+{\vec {0}}+\dots +{\vec {0}}$

where $k$ is the map's rank. Thus, we can understand any linear map as a kind of projection.

${\begin{pmatrix}c_{1}\\\vdots \\c_{k}\\c_{k+1}\\\vdots \\c_{n}\end{pmatrix}}_{B}\;\mapsto \;{\begin{pmatrix}c_{1}\\\vdots \\c_{k}\\0\\\vdots \\0\end{pmatrix}}_{D}$

Of course, "understanding" a map expressed in this way requires that we understand the relationship between $B$ and $D$. However, despite that difficulty, this is a good classification of linear maps. }}

Exercises

This exercise is recommended for all readers.

Problem 1

Decide if these matrices are matrix equivalent.

1. ${\begin{pmatrix}1&3&0\\2&3&0\end{pmatrix}}$, ${\begin{pmatrix}2&2&1\\0&5&-1\end{pmatrix}}$
2. ${\begin{pmatrix}0&3\\1&1\end{pmatrix}}$, ${\begin{pmatrix}4&0\\0&5\end{pmatrix}}$
3. ${\begin{pmatrix}1&3\\2&6\end{pmatrix}}$, ${\begin{pmatrix}1&3\\2&-6\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

Find the canonical representative of the matrix-equivalence class of each matrix.

1. ${\begin{pmatrix}2&1&0\\4&2&0\end{pmatrix}}$
2. ${\begin{pmatrix}0&1&0&2\\1&1&0&4\\3&3&3&-1\end{pmatrix}}$

Problem 3

Suppose that, with respect to

$B={\mathcal {E}}_{2}\qquad D=\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\rangle$

the transformation $t:\mathbb {R} ^{2}\to \mathbb {R} ^{2}$ is represented by this matrix.

${\begin{pmatrix}1&2\\3&4\end{pmatrix}}$

Use change of basis matrices to represent $t$ with respect to each pair.

1. ${\hat {B}}=\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}1\\1\end{pmatrix}}\rangle$, ${\hat {D}}=\langle {\begin{pmatrix}-1\\0\end{pmatrix}},{\begin{pmatrix}2\\1\end{pmatrix}}\rangle$
2. ${\hat {B}}=\langle {\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\rangle$, ${\hat {D}}=\langle {\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}2\\1\end{pmatrix}}\rangle$

This exercise is recommended for all readers.

Problem 4

What sizes are $P$ and $Q$ in the equation ${\hat {H}}=PHQ$?

This exercise is recommended for all readers.

Problem 5

Use Theorem 2.6 to show that a square matrix is nonsingular if and only if it is equivalent to an identity matrix.

This exercise is recommended for all readers.

Problem 6

Show that, where $A$ is a nonsingular square matrix, if $P$ and $Q$ are nonsingular square matrices such that $PAQ=I$ then $QP=A^{-1}$.

This exercise is recommended for all readers.

Problem 7

Why does Theorem 2.6 not show that every matrix is diagonalizable (see Example 2.2)?

Problem 8

Must matrix equivalent matrices have matrix equivalent transposes?

Problem 9

What happens in Theorem 2.6 if $k=0$?

This exercise is recommended for all readers.

Problem 10

Show that matrix-equivalence is an equivalence relation.

This exercise is recommended for all readers.

Problem 11

Show that a zero matrix is alone in its matrix equivalence class. Are there other matrices like that?

Problem 12

What are the matrix equivalence classes of matrices of transformations on $\mathbb {R} ^{1}$? $\mathbb {R} ^{3}$?

Problem 13

How many matrix equivalence classes are there?

Problem 14

Are matrix equivalence classes closed under scalar multiplication? Addition?

Problem 15

Let $t:\mathbb {R} ^{n}\to \mathbb {R} ^{n}$ represented by $T$ with respect to ${\mathcal {E}}_{n},{\mathcal {E}}_{n}$.

1. Find ${\rm {Rep}}_{B,B}(t)$ in this specific case.

   $T={\begin{pmatrix}1&1\\3&-1\end{pmatrix}}\qquad B=\langle {\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}-1\\-1\end{pmatrix}}\rangle$

2. Describe ${\rm {Rep}}_{B,B}(t)$ in the general case where $B=\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{n}\rangle$.

Problem 16

1. Let $V$ have bases $B_{1}$ and $B_{2}$ and suppose that $W$ has the basis $D$. Where $h:V\to W$, find the formula that computes ${\rm {Rep}}_{B_{2},D}(h)$ from ${\rm {Rep}}_{B_{1},D}(h)$.
2. Repeat the prior question with one basis for $V$ and two bases for $W$.

Problem 17

1. If two matrices are matrix-equivalent and invertible, must their inverses be matrix-equivalent?
2. If two matrices have matrix-equivalent inverses, must the two be matrix-equivalent?
3. If two matrices are square and matrix-equivalent, must their squares be matrix-equivalent?
4. If two matrices are square and have matrix-equivalent squares, must they be matrix-equivalent?

This exercise is recommended for all readers.

Problem 18

Square matrices are similar if they represent the same transformation, but each with respect to the same ending as starting basis. That is, ${\rm {Rep}}_{B_{1},B_{1}}(t)$ is similar to ${\rm {Rep}}_{B_{2},B_{2}}(t)$.

1. Give a definition of matrix similarity like that of Definition 2.3.
2. Prove that similar matrices are matrix equivalent.
3. Show that similarity is an equivalence relation.
4. Show that if $T$ is similar to ${\hat {T}}$ then $T^{2}$ is similar to ${\hat {T}}^{2}$, the cubes are similar, etc. Contrast with the prior exercise.
5. Prove that there are matrix equivalent matrices that are not similar.

Section VI - Projection

This section is optional; only the last two sections of Chapter Five require this material.

We have described the projection $\pi$ from $\mathbb {R} ^{3}$ into its $xy$ plane subspace as a "shadow map". This shows why, but it also shows that some shadows fall upward.

So perhaps a better description is: the projection of ${\vec {v}}$ is the ${\vec {p}}$ in the plane with the property that someone standing on ${\vec {p}}$ and looking straight up or down sees ${\vec {v}}$. In this section we will generalize this to other projections, both orthogonal (i.e., "straight up and down") and nonorthogonal.

1 - Orthogonal Projection Onto a Line

We first consider orthogonal projection onto a line. To orthogonally project a vector ${\vec {v}}$ onto a line $\ell$, mark the point on the line at which someone standing on that point could see ${\vec {v}}$ by looking straight up or down (from that person's point of view).

The picture shows someone who has walked out on the line until the tip of ${\vec {v}}$ is straight overhead. That is, where the line is described as the span of some nonzero vector $\ell =\{c\cdot {\vec {s}}\,{\big |}\,c\in \mathbb {R} \}$, the person has walked out to find the coefficient $c_{\vec {p}}\,$ with the property that ${\vec {v}}-c_{\vec {p}}\cdot {\vec {s}}\,$ is orthogonal to $c_{\vec {p}}\cdot {\vec {s}}$.

We can solve for this coefficient by noting that because ${\vec {v}}-c_{\vec {p}}{\vec {s}}\,$ is orthogonal to a scalar multiple of ${\vec {s}}\,$ it must be orthogonal to ${\vec {s}}\,$ itself, and then the consequent fact that the dot product $({\vec {v}}-c_{\vec {p}}{\vec {s}})\cdot {\vec {s}}\,$ is zero gives that $c_{\vec {p}}={\vec {v}}\cdot {\vec {s}}/{\vec {s}}\cdot {\vec {s}}$.

Definition 1.1

The orthogonal projection of ${\vec {v}}$ onto the line spanned by a nonzero ${\vec {s}}\,$ is this vector.

${\mbox{proj}}_{[{\vec {s}}\,]}({\vec {v}})={\frac {{\vec {v}}\cdot {\vec {s}}}{{\vec {s}}\cdot {\vec {s}}}}{\vec {s}}$

Problem 13 checks that the outcome of the calculation depends only on the line and not on which vector ${\vec {s}}\,$ happens to be used to describe that line.

Remark 1.2

The wording of that definition says "spanned by ${\vec {s}}\,$" instead the more formal "the span of the set $\{{\vec {s}}\,\}$". This casual first phrase is common.

Example 1.3

To orthogonally project the vector ${\binom {2}{3}}$ onto the line $y=2x$, we first pick a direction vector for the line. For instance,

${\vec {s}}={\begin{pmatrix}1\\2\end{pmatrix}}$

will do. Then the calculation is routine.

${\frac {{\begin{pmatrix}2\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\2\end{pmatrix}}}{{\begin{pmatrix}1\\2\end{pmatrix}}\cdot {\begin{pmatrix}1\\2\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\2\end{pmatrix}}={\displaystyle {\frac {8}{5}}}\cdot {\begin{pmatrix}1\\2\end{pmatrix}}={\begin{pmatrix}8/5\\16/5\end{pmatrix}}$

Example 1.4

In $\mathbb {R} ^{3}$, the orthogonal projection of a general vector

${\begin{pmatrix}x\\y\\z\end{pmatrix}}$

onto the $y$-axis is

${\frac {{\begin{pmatrix}x\\y\\z\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\\0\end{pmatrix}}}{{\begin{pmatrix}0\\1\\0\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}0\\1\\0\end{pmatrix}}={\begin{pmatrix}0\\y\\0\end{pmatrix}}$

which matches our intuitive expectation.

The picture above with the stick figure walking out on the line until ${\vec {v}}$'s tip is overhead is one way to think of the orthogonal projection of a vector onto a line. We finish this subsection with two other ways.

Example 1.5

A railroad car left on an east-west track without its brake is pushed by a wind blowing toward the northeast at fifteen miles per hour; what speed will the car reach?

For the wind we use a vector of length $15$ that points toward the northeast.

${\vec {v}}={\begin{pmatrix}15{\sqrt {1/2}}\\15{\sqrt {1/2}}\end{pmatrix}}$

The car can only be affected by the part of the wind blowing in the east-west direction— the part of ${\vec {v}}$ in the direction of the $x$-axis is this (the picture has the same perspective as the railroad car picture above).

$\displaystyle {\vec {p}}={\begin{pmatrix}15{\sqrt {1/2}}\\0\end{pmatrix}}$

So the car will reach a velocity of $15{\sqrt {1/2}}$ miles per hour toward the east.

Thus, another way to think of the picture that precedes the definition is that it shows ${\vec {v}}$ as decomposed into two parts, the part with the line (here, the part with the tracks, ${\vec {p}}$), and the part that is orthogonal to the line (shown here lying on the north-south axis). These two are "not interacting" or "independent", in the sense that the east-west car is not at all affected by the north-south part of the wind (see Problem 5). So the orthogonal projection of ${\vec {v}}$ onto the line spanned by ${\vec {s}}\,$ can be thought of as the part of ${\vec {v}}\,$ that lies in the direction of ${\vec {s}}$.

Finally, another useful way to think of the orthogonal projection is to have the person stand not on the line, but on the vector that is to be projected to the line. This person has a rope over the line and pulls it tight, naturally making the rope orthogonal to the line.

That is, we can think of the projection ${\vec {p}}\,$ as being the vector in the line that is closest to ${\vec {v}}$ (see Problem 11).

Example 1.6

A submarine is tracking a ship moving along the line $y=3x+2$. Torpedo range is one-half mile. Can the sub stay where it is, at the origin on the chart below, or must it move to reach a place where the ship will pass within range?

The formula for projection onto a line does not immediately apply because the line doesn't pass through the origin, and so isn't the span of any ${\vec {s}}$. To adjust for this, we start by shifting the entire map down two units. Now the line is $y=3x$, which is a subspace, and we can project to get the point ${\vec {p}}$ of closest approach, the point on the line through the origin closest to

${\vec {v}}={\begin{pmatrix}0\\-2\end{pmatrix}}$

the sub's shifted position.

${\vec {p}}={\frac {{\begin{pmatrix}0\\-2\end{pmatrix}}\cdot {\begin{pmatrix}1\\3\end{pmatrix}}}{{\begin{pmatrix}1\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\3\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\3\end{pmatrix}}={\begin{pmatrix}-3/5\\-9/5\end{pmatrix}}$

The distance between ${\vec {v}}$ and ${\vec {p}}$ is approximately $0.63$ miles and so the sub must move to get in range.

This subsection has developed a natural projection map: orthogonal projection onto a line. As suggested by the examples, it is often called for in applications. The next subsection shows how the definition of orthogonal projection onto a line gives us a way to calculate especially convienent bases for vector spaces, again something that is common in applications. The final subsection completely generalizes projection, orthogonal or not, onto any subspace at all.

Exercises

This exercise is recommended for all readers.

Problem 1

Project the first vector orthogonally onto the line spanned by the second vector.

1. ${\begin{pmatrix}2\\1\end{pmatrix}}$, ${\begin{pmatrix}3\\-2\end{pmatrix}}$
2. ${\begin{pmatrix}2\\1\end{pmatrix}}$, ${\begin{pmatrix}3\\0\end{pmatrix}}$
3. ${\begin{pmatrix}1\\1\\4\end{pmatrix}}$, ${\begin{pmatrix}1\\2\\-1\end{pmatrix}}$
4. ${\begin{pmatrix}1\\1\\4\end{pmatrix}}$, ${\begin{pmatrix}3\\3\\12\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

Project the vector orthogonally onto the line.

1. ${\begin{pmatrix}2\\-1\\4\end{pmatrix}},\{c{\begin{pmatrix}-3\\1\\-3\end{pmatrix}}\,{\big |}\,c\in \mathbb {R} \}$
2. ${\begin{pmatrix}-1\\-1\end{pmatrix}}$, the line $y=3x$

Problem 3

Although the development of Definition 1.1 is guided by the pictures, we are not restricted to spaces that we can draw. In $\mathbb {R} ^{4}$ project this vector onto this line.

${\vec {v}}={\begin{pmatrix}1\\2\\1\\3\end{pmatrix}}\qquad \ell =\{c\cdot {\begin{pmatrix}-1\\1\\-1\\1\end{pmatrix}}\,{\big |}\,c\in \mathbb {R} \}$

This exercise is recommended for all readers.

Problem 4

Definition 1.1 uses two vectors ${\vec {s}}$ and ${\vec {v}}$. Consider the transformation of $\mathbb {R} ^{2}$ resulting from fixing

${\vec {s}}={\begin{pmatrix}3\\1\end{pmatrix}}$

and projecting ${\vec {v}}$ onto the line that is the span of ${\vec {s}}$. Apply it to these vectors.

1. ${\begin{pmatrix}1\\2\end{pmatrix}}$
2. ${\begin{pmatrix}0\\4\end{pmatrix}}$

Show that in general the projection tranformation is this.

${\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix}}\mapsto {\begin{pmatrix}(9x_{1}+3x_{2})/10\\(3x_{1}+x_{2})/10\end{pmatrix}}$

Express the action of this transformation with a matrix.

Problem 5

Example 1.5 suggests that projection breaks ${\vec {v}}$ into two parts, ${\mbox{proj}}_{[{\vec {s}}\,]}({{\vec {v}}\,})$ and ${\vec {v}}-{\mbox{proj}}_{[{\vec {s}}\,]}({{\vec {v}}\,})$, that are "not interacting". Recall that the two are orthogonal. Show that any two nonzero orthogonal vectors make up a linearly independent set.

Problem 6

1. What is the orthogonal projection of ${\vec {v}}$ onto a line if ${\vec {v}}$ is a member of that line?
2. Show that if ${\vec {v}}$ is not a member of the line then the set $\{{\vec {v}},{\vec {v}}-{\mbox{proj}}_{[{\vec {s}}\,]}({{\vec {v}}\,})\}$ is linearly independent.

Problem 7

Definition 1.1 requires that ${\vec {s}}\,$ be nonzero. Why? What is the right definition of the orthogonal projection of a vector onto the (degenerate) line spanned by the zero vector?

Problem 8

Are all vectors the projection of some other vector onto some line?

This exercise is recommended for all readers.

Problem 9

Show that the projection of ${\vec {v}}$ onto the line spanned by ${\vec {s}}$ has length equal to the absolute value of the number ${\vec {v}}\cdot {\vec {s}}$ divided by the length of the vector ${\vec {s}}$.

Problem 10

Find the formula for the distance from a point to a line.

Problem 11

Find the scalar $c$ such that $(cs_{1},cs_{2})$ is a minimum distance from the point $(v_{1},v_{2})$ by using calculus (i.e., consider the distance function, set the first derivative equal to zero, and solve). Generalize to $\mathbb {R} ^{n}$.

This exercise is recommended for all readers.

Problem 12

Prove that the orthogonal projection of a vector onto a line is shorter than the vector.

This exercise is recommended for all readers.

Problem 13

Show that the definition of orthogonal projection onto a line does not depend on the spanning vector: if ${\vec {s}}$ is a nonzero multiple of ${\vec {q}}$ then $({\vec {v}}\cdot {\vec {s}}/{\vec {s}}\cdot {\vec {s}}\,)\cdot {\vec {s}}$ equals $({\vec {v}}\cdot {\vec {q}}/{\vec {q}}\cdot {\vec {q}}\,)\cdot {\vec {q}}$.

This exercise is recommended for all readers.

Problem 14

Consider the function mapping to plane to itself that takes a vector to its projection onto the line $y=x$. These two each show that the map is linear, the first one in a way that is bound to the coordinates (that is, it fixes a basis and then computes) and the second in a way that is more conceptual.

1. Produce a matrix that describes the function's action.
2. Show also that this map can be obtained by first rotating everything in the plane $\pi /4$ radians clockwise, then projecting onto the $x$-axis, and then rotating $\pi /4$ radians counterclockwise.

Problem 15

For ${\vec {a}},{\vec {b}}\in \mathbb {R} ^{n}$ let ${\vec {v}}_{1}$ be the projection of ${\vec {a}}$ onto the line spanned by ${\vec {b}}$, let ${\vec {v}}_{2}$ be the projection of ${\vec {v}}_{1}$ onto the line spanned by ${\vec {a}}$, let ${\vec {v}}_{3}$ be the projection of ${\vec {v}}_{2}$ onto the line spanned by ${\vec {b}}$, etc., back and forth between the spans of ${\vec {a}}$ and ${\vec {b}}$. That is, ${\vec {v}}_{i+1}$ is the projection of ${\vec {v}}_{i}$ onto the span of ${\vec {a}}$ if $i+1$ is even, and onto the span of ${\vec {b}}$ if $i+1$ is odd. Must that sequence of vectors eventually settle down— must there be a sufficiently large $i$ such that ${\vec {v}}_{i+2}$ equals ${\vec {v}}_{i}$ and ${\vec {v}}_{i+3}$ equals ${\vec {v}}_{i+1}$? If so, what is the earliest such $i$?

2 - Gram-Schmidt Orthogonalization

This subsection is optional. It requires material from the prior, also optional, subsection. The work done here will only be needed in the final two sections of Chapter Five.

The prior subsection suggests that projecting onto the line spanned by ${\vec {s}}$ decomposes a vector ${\vec {v}}$ into two parts

$\displaystyle {\vec {v}}={\mbox{proj}}_{[{\vec {s}}\,]}({\vec {v}})\,+\,\left({\vec {v}}-{\mbox{proj}}_{[{\vec {s}}\,]}({\vec {v}})\right)$

that are orthogonal and so are "not interacting". We will now develop that suggestion.

Definition 2.1

Vectors ${\vec {v}}_{1},\dots ,{\vec {v}}_{k}\in \mathbb {R} ^{n}$ are mutually orthogonal when any two are orthogonal: if $i\neq j$ then the dot product ${\vec {v}}_{i}\cdot {\vec {v}}_{j}$ is zero.

Theorem 2.2

If the vectors in a set $\{{\vec {v}}_{1},\dots ,{\vec {v}}_{k}\}\subset \mathbb {R} ^{n}$ are mutually orthogonal and nonzero then that set is linearly independent.

Proof

Consider a linear relationship $c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}+\dots +c_{k}{\vec {v}}_{k}={\vec {0}}$. If $i\in [1..k]$ then taking the dot product of ${\vec {v}}_{i}$ with both sides of the equation

${\begin{array}{rl}{\vec {v}}_{i}\cdot (c_{1}{\vec {v}}_{1}+c_{2}{\vec {v}}_{2}+\dots +c_{k}{\vec {v}}_{k})&={\vec {v}}_{i}\cdot {\vec {0}}\\c_{i}\cdot ({\vec {v}}_{i}\cdot {\vec {v}}_{i})&=0\end{array}}$

shows, since ${\vec {v}}_{i}$ is nonzero, that $c_{i}$ is zero.

Corollary 2.3

If the vectors in a size $k$ subset of a $k$ dimensional space are mutually orthogonal and nonzero then that set is a basis for the space.

Proof

Any linearly independent size $k$ subset of a $k$ dimensional space is a basis.

Of course, the converse of Corollary 2.3 does not hold— not every basis of every subspace of $\mathbb {R} ^{n}$ is made of mutually orthogonal vectors. However, we can get the partial converse that for every subspace of $\mathbb {R} ^{n}$ there is at least one basis consisting of mutually orthogonal vectors.

Example 2.4

The members ${\vec {\beta }}_{1}$ and ${\vec {\beta }}_{2}$ of this basis for $\mathbb {R} ^{2}$ are not orthogonal.

$\displaystyle B=\left\langle {\begin{pmatrix}4\\2\end{pmatrix}},{\begin{pmatrix}1\\3\end{pmatrix}}\right\rangle$

However, we can derive from $B$ a new basis for the same space that does have mutually orthogonal members. For the first member of the new basis we simply use ${\vec {\beta }}_{1}$.

${\vec {\kappa }}_{1}={\begin{pmatrix}4\\2\end{pmatrix}}$

For the second member of the new basis, we take away from ${\vec {\beta }}_{2}$ its part in the direction of ${\vec {\kappa }}_{1}$,

$\displaystyle {\vec {\kappa }}_{2}={\begin{pmatrix}1\\3\end{pmatrix}}-{\mbox{proj}}_{\scriptstyle [{\vec {\kappa }}_{1}]}({\begin{pmatrix}1\\3\end{pmatrix}})={\begin{pmatrix}1\\3\end{pmatrix}}-{\begin{pmatrix}2\\1\end{pmatrix}}={\begin{pmatrix}-1\\2\end{pmatrix}}$

which leaves the part, ${\vec {\kappa }}_{2}$ pictured above, of ${\vec {\beta }}_{2}$ that is orthogonal to ${\vec {\kappa }}_{1}$ (it is orthogonal by the definition of the projection onto the span of ${\vec {\kappa }}_{1}$). Note that, by the corollary, $\{{\vec {\kappa }}_{1},{\vec {\kappa }}_{2}\}$ is a basis for $\mathbb {R} ^{2}$.

Definition 2.5

An orthogonal basis for a vector space is a basis of mutually orthogonal vectors.

Example 2.6

To turn this basis for $\mathbb {R} ^{3}$

$\left\langle {\begin{pmatrix}1\\1\\1\end{pmatrix}},{\begin{pmatrix}0\\2\\0\end{pmatrix}},{\begin{pmatrix}1\\0\\3\end{pmatrix}}\right\rangle$

into an orthogonal basis, we take the first vector as it is given.

${\vec {\kappa }}_{1}={\begin{pmatrix}1\\1\\1\end{pmatrix}}$

We get ${\vec {\kappa }}_{2}$ by starting with the given second vector ${\vec {\beta }}_{2}$ and subtracting away the part of it in the direction of ${\vec {\kappa }}_{1}$.

${\vec {\kappa }}_{2}={\begin{pmatrix}0\\2\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}0\\2\\0\end{pmatrix}})={\begin{pmatrix}0\\2\\0\end{pmatrix}}-{\begin{pmatrix}2/3\\2/3\\2/3\end{pmatrix}}={\begin{pmatrix}-2/3\\4/3\\-2/3\end{pmatrix}}$

Finally, we get ${\vec {\kappa }}_{3}$ by taking the third given vector and subtracting the part of it in the direction of ${\vec {\kappa }}_{1}$, and also the part of it in the direction of ${\vec {\kappa }}_{2}$.

${\vec {\kappa }}_{3}={\begin{pmatrix}1\\0\\3\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}1\\0\\3\end{pmatrix}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\begin{pmatrix}1\\0\\3\end{pmatrix}})={\begin{pmatrix}-1\\0\\1\end{pmatrix}}$

Again the corollary gives that

$\left\langle {\begin{pmatrix}1\\1\\1\end{pmatrix}},{\begin{pmatrix}-2/3\\4/3\\-2/3\end{pmatrix}},{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\right\rangle$

is a basis for the space.

The next result verifies that the process used in those examples works with any basis for any subspace of an $\mathbb {R} ^{n}$ (we are restricted to $\mathbb {R} ^{n}$ only because we have not given a definition of orthogonality for other vector spaces).

Theorem 2.7 (Gram-Schmidt orthogonalization)

If $\left\langle {\vec {\beta }}_{1},\ldots {\vec {\beta }}_{k}\right\rangle$ is a basis for a subspace of $\mathbb {R} ^{n}$ then, where

${\begin{array}{rl}{\vec {\kappa }}_{1}&={\vec {\beta }}_{1}\\{\vec {\kappa }}_{2}&={\vec {\beta }}_{2}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{2}})\\{\vec {\kappa }}_{3}&={\vec {\beta }}_{3}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{3}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({{\vec {\beta }}_{3}})\\&\vdots \\{\vec {\kappa }}_{k}&={\vec {\beta }}_{k}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{k}})-\cdots -{\mbox{proj}}_{[{\vec {\kappa }}_{k-1}]}({{\vec {\beta }}_{k}})\end{array}}$

the ${\vec {\kappa }}\,$'s form an orthogonal basis for the same subspace.

Proof

We will use induction to check that each ${\vec {\kappa }}_{i}$ is nonzero, is in the span of $\left\langle {\vec {\beta }}_{1},\ldots {\vec {\beta }}_{i}\right\rangle$ and is orthogonal to all preceding vectors: ${\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{i}=\cdots ={\vec {\kappa }}_{i-1}\cdot {\vec {\kappa }}_{i}=0$. With those, and with Corollary 2.3, we will have that $\left\langle {\vec {\kappa }}_{1},\ldots {\vec {\kappa }}_{k}\right\rangle$ is a basis for the same space as $\left\langle {\vec {\beta }}_{1},\ldots {\vec {\beta }}_{k}\right\rangle$.

We shall cover the cases up to $i=3$, which give the sense of the argument. Completing the details is Problem 15.

The $i=1$ case is trivial— setting ${\vec {\kappa }}_{1}$ equal to ${\vec {\beta }}_{1}$ makes it a nonzero vector since ${\vec {\beta }}_{1}$ is a member of a basis, it is obviously in the desired span, and the "orthogonal to all preceding vectors" condition is vacuously met.

For the $i=2$ case, expand the definition of ${\vec {\kappa }}_{2}$.

${\vec {\kappa }}_{2}={\vec {\beta }}_{2}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{2}})={\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}={\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}$

This expansion shows that ${\vec {\kappa }}_{2}$ is nonzero or else this would be a non-trivial linear dependence among the ${\vec {\beta }}$'s (it is nontrivial because the coefficient of ${\vec {\beta }}_{2}$ is $1$) and also shows that ${\vec {\kappa }}_{2}$ is in the desired span. Finally, ${\vec {\kappa }}_{2}$ is orthogonal to the only preceding vector

${\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{2}={\vec {\kappa }}_{1}\cdot ({\vec {\beta }}_{2}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{2}}))=0$

because this projection is orthogonal.

The $i=3$ case is the same as the $i=2$ case except for one detail. As in the $i=2$ case, expanding the definition

${\begin{array}{rl}{\vec {\kappa }}_{3}&={\vec {\beta }}_{3}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}\\&={\vec {\beta }}_{3}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\bigl (}{\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}{\bigr )}\end{array}}$

shows that ${\vec {\kappa }}_{3}$ is nonzero and is in the span. A calculation shows that ${\vec {\kappa }}_{3}$ is orthogonal to the preceding vector ${\vec {\kappa }}_{1}$.

${\begin{array}{rl}{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{3}&={\vec {\kappa }}_{1}\cdot {\bigl (}{\vec {\beta }}_{3}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{3}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({{\vec {\beta }}_{3}}){\bigr )}\\&={\vec {\kappa }}_{1}\cdot {\bigl (}{\vec {\beta }}_{3}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{3}}){\bigr )}-{\vec {\kappa }}_{1}\cdot {\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({{\vec {\beta }}_{3}})\\&=0\end{array}}$

(Here's the difference from the $i=2$ case— the second line has two kinds of terms. The first term is zero because this projection is orthogonal, as in the $i=2$ case. The second term is zero because ${\vec {\kappa }}_{1}$ is orthogonal to ${\vec {\kappa }}_{2}$ and so is orthogonal to any vector in the line spanned by ${\vec {\kappa }}_{2}$.) The check that ${\vec {\kappa }}_{3}$ is also orthogonal to the other preceding vector ${\vec {\kappa }}_{2}$ is similar.

Beyond having the vectors in the basis be orthogonal, we can do more; we can arrange for each vector to have length one by dividing each by its own length (we can normalize the lengths).

Example 2.8

Normalizing the length of each vector in the orthogonal basis of Example 2.6 produces this orthonormal basis.

$\left\langle {\begin{pmatrix}1/{\sqrt {3}}\\1/{\sqrt {3}}\\1/{\sqrt {3}}\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {6}}\\2/{\sqrt {6}}\\-1/{\sqrt {6}}\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {2}}\\0\\1/{\sqrt {2}}\end{pmatrix}}\right\rangle$

Besides its intuitive appeal, and its analogy with the standard basis ${\mathcal {E}}_{n}$ for $\mathbb {R} ^{n}$, an orthonormal basis also simplifies some computations. See Exercise 9, for example.

Exercises

Problem 1

Perform the Gram-Schmidt process on each of these bases for $\mathbb {R} ^{2}$.

1. $\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}2\\1\end{pmatrix}}\right\rangle$
2. $\left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\3\end{pmatrix}}\right\rangle$
3. $\left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\0\end{pmatrix}}\right\rangle$

Then turn those orthogonal bases into orthonormal bases.

This exercise is recommended for all readers.

Problem 2

Perform the Gram-Schmidt process on each of these bases for $\mathbb {R} ^{3}$.

1. $\left\langle {\begin{pmatrix}2\\2\\2\end{pmatrix}},{\begin{pmatrix}1\\0\\-1\end{pmatrix}},{\begin{pmatrix}0\\3\\1\end{pmatrix}}\right\rangle$
2. $\left\langle {\begin{pmatrix}1\\-1\\0\end{pmatrix}},{\begin{pmatrix}0\\1\\0\end{pmatrix}},{\begin{pmatrix}2\\3\\1\end{pmatrix}}\right\rangle$

Then turn those orthogonal bases into orthonormal bases.

This exercise is recommended for all readers.

Problem 3

Find an orthonormal basis for this subspace of $\mathbb {R} ^{3}$: the plane $x-y+z=0$.

Problem 4

Find an orthonormal basis for this subspace of $\mathbb {R} ^{4}$.

$\{{\begin{pmatrix}x\\y\\z\\w\end{pmatrix}}\,{\big |}\,x-y-z+w=0{\text{ and }}x+z=0\}$

Problem 5

Show that any linearly independent subset of $\mathbb {R} ^{n}$ can be orthogonalized without changing its span.

This exercise is recommended for all readers.

Problem 6

What happens if we apply the Gram-Schmidt process to a basis that is already orthogonal?

Problem 7

Let $\left\langle {\vec {\kappa }}_{1},\dots ,{\vec {\kappa }}_{k}\right\rangle$ be a set of mutually orthogonal vectors in $\mathbb {R} ^{n}$.

1. Prove that for any ${\vec {v}}$ in the space, the vector ${\vec {v}}-({\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,}))$ is orthogonal to each of ${\vec {\kappa }}_{1}$, ..., ${\vec {\kappa }}_{k}$.
2. Illustrate the prior item in $\mathbb {R} ^{3}$ by using ${\vec {e}}_{1}$ as ${\vec {\kappa }}_{1}$, using ${\vec {e}}_{2}$ as ${\vec {\kappa }}_{2}$, and taking ${\vec {v}}$ to have components $1$, $2$, and $3$.
3. Show that ${\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,})$ is the vector in the span of the set of ${\vec {\kappa }}$'s that is closest to ${\vec {v}}$. Hint. To the illustration done for the prior part, add a vector $d_{1}{\vec {\kappa }}_{1}+d_{2}{\vec {\kappa }}_{2}$ and apply the Pythagorean Theorem to the resulting triangle.

Problem 8

Find a vector in $\mathbb {R} ^{3}$ that is orthogonal to both of these.

${\begin{pmatrix}1\\5\\-1\end{pmatrix}}\qquad {\begin{pmatrix}2\\2\\0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 9

One advantage of orthogonal bases is that they simplify finding the representation of a vector with respect to that basis.

1. For this vector and this non-orthogonal basis for $\mathbb {R} ^{2}$

   ${\vec {v}}={\begin{pmatrix}2\\3\end{pmatrix}}\qquad B=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\right\rangle$

   first represent the vector with respect to the basis. Then project the vector onto the span of each basis vector $[{{\vec {\beta }}_{1}}]$ and $[{{\vec {\beta }}_{2}}]$.
2. With this orthogonal basis for $\mathbb {R} ^{2}$

   $K=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\right\rangle$

   represent the same vector ${\vec {v}}$ with respect to the basis. Then project the vector onto the span of each basis vector. Note that the coefficients in the representation and the projection are the same.
3. Let $K=\left\langle {\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\right\rangle$ be an orthogonal basis for some subspace of $\mathbb {R} ^{n}$. Prove that for any ${\vec {v}}$ in the subspace, the $i$-th component of the representation ${\rm {Rep}}_{K}({\vec {v}}\,)$ is the scalar coefficient $({\vec {v}}\cdot {\vec {\kappa }}_{i})/({\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i})$ from ${\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({{\vec {v}}\,})$.
4. Prove that ${\vec {v}}={\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {\kappa }}_{k}]}({{\vec {v}}\,})$.

Problem 10

Bessel's Inequality. Consider these orthonormal sets

$B_{1}=\{{\vec {e}}_{1}\}\quad B_{2}=\{{\vec {e}}_{1},{\vec {e}}_{2}\}\quad B_{3}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3}\}\quad B_{4}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3},{\vec {e}}_{4}\}$

along with the vector ${\vec {v}}\in \mathbb {R} ^{4}$ whose components are $4$, $3$, $2$, and $1$.

1. Find the coefficient $c_{1}$ for the projection of ${\vec {v}}$ onto the span of the vector in $B_{1}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}$.
2. Find the coefficients $c_{1}$ and $c_{2}$ for the projection of ${\vec {v}}$ onto the spans of the two vectors in $B_{2}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+|c_{2}|^{2}$.
3. Find $c_{1}$, $c_{2}$, and $c_{3}$ associated with the vectors in $B_{3}$, and $c_{1}$, $c_{2}$, $c_{3}$, and $c_{4}$ for the vectors in $B_{4}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{3}|^{2}$ and that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{4}|^{2}$.

Show that this holds in general: where $\{{\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\}$ is an orthonormal set and $c_{i}$ is coefficient of the projection of a vector ${\vec {v}}$ from the space then $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{k}|^{2}$. Hint. One way is to look at the inequality $0\leq |{\vec {v}}-(c_{1}{\vec {\kappa }}_{1}+\dots +c_{k}{\vec {\kappa }}_{k})|^{2}$ and expand the $c$'s.

Problem 11

Prove or disprove: every vector in $\mathbb {R} ^{n}$ is in some orthogonal basis.

Problem 12

Show that the columns of an $n\!\times \!n$ matrix form an orthonormal set if and only if the inverse of the matrix is its transpose. Produce such a matrix.

Problem 13

Does the proof of Theorem 2.2 fail to consider the possibility that the set of vectors is empty (i.e., that $k=0$)?

Problem 14

Theorem 2.7 describes a change of basis from any basis $B=\left\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{k}\right\rangle$ to one that is orthogonal $K=\left\langle {\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\right\rangle$. Consider the change of basis matrix ${\rm {Rep}}_{B,K}({\mbox{id}})$.

1. Prove that the matrix ${\rm {Rep}}_{K,B}({\mbox{id}})$ changing bases in the direction opposite to that of the theorem has an upper triangular shape— all of its entries below the main diagonal are zeros.
2. Prove that the inverse of an upper triangular matrix is also upper triangular (if the matrix is invertible, that is). This shows that the matrix ${\rm {Rep}}_{B,K}({\mbox{id}})$ changing bases in the direction described in the theorem is upper triangular.

Problem 15

Complete the induction argument in the proof of Theorem 2.7.

3 - Projection Onto a Subspace

This subsection, like the others in this section, is optional. It also requires material from the optional earlier subsection on Combining Subspaces.

The prior subsections project a vector onto a line by decomposing it into two parts: the part in the line ${\mbox{proj}}_{[{\vec {s}}\,]}({{\vec {v}}\,})$ and the rest ${\vec {v}}-{\mbox{proj}}_{[{\vec {s}}\,]}({{\vec {v}}\,})$. To generalize projection to arbitrary subspaces, we follow this idea.

Definition 3.1

For any direct sum $V=M\oplus N$ and any ${\vec {v}}\in V$, the projection of ${\vec {v}}$ onto $M$ along $N$ is

${\mbox{proj}}_{M,N}({{\vec {v}}\,})={\vec {m}}$

where ${\vec {v}}={\vec {m}}+{\vec {n}}$ with ${\vec {m}}\in M,\,{\vec {n}}\in N$.

This definition doesn't involve a sense of "orthogonal" so we can apply it to spaces other than subspaces of an $\mathbb {R} ^{n}$. (Definitions of orthogonality for other spaces are perfectly possible, but we haven't seen any in this book.)

Example 3.2

The space ${\mathcal {M}}_{2\!\times \!2}$ of $2\!\times \!2$ matrices is the direct sum of these two.

$M=\{{\begin{pmatrix}a&b\\0&0\end{pmatrix}}\,{\big |}\,a,b\in \mathbb {R} \}\qquad N=\{{\begin{pmatrix}0&0\\c&d\end{pmatrix}}\,{\big |}\,c,d\in \mathbb {R} \}$

To project

$A={\begin{pmatrix}3&1\\0&4\end{pmatrix}}$

onto $M$ along $N$, we first fix bases for the two subspaces.

$B_{M}=\langle {\begin{pmatrix}1&0\\0&0\end{pmatrix}},{\begin{pmatrix}0&1\\0&0\end{pmatrix}}\rangle \qquad B_{N}=\langle {\begin{pmatrix}0&0\\1&0\end{pmatrix}},{\begin{pmatrix}0&0\\0&1\end{pmatrix}}\rangle$

The concatenation of these

$B=B_{M}\!{\mathbin {{}^{\frown }}}\!B_{N}=\langle {\begin{pmatrix}1&0\\0&0\end{pmatrix}},{\begin{pmatrix}0&1\\0&0\end{pmatrix}},{\begin{pmatrix}0&0\\1&0\end{pmatrix}},{\begin{pmatrix}0&0\\0&1\end{pmatrix}}\rangle$

is a basis for the entire space, because the space is the direct sum, so we can use it to represent $A$.

${\begin{pmatrix}3&1\\0&4\end{pmatrix}}=3\cdot {\begin{pmatrix}1&0\\0&0\end{pmatrix}}+1\cdot {\begin{pmatrix}0&1\\0&0\end{pmatrix}}+0\cdot {\begin{pmatrix}0&0\\1&0\end{pmatrix}}+4\cdot {\begin{pmatrix}0&0\\0&1\end{pmatrix}}$

Now the projection of $A$ onto $M$ along $N$ is found by keeping the $M$ part of this sum and dropping the $N$ part.

${\mbox{proj}}_{M,N}({\begin{pmatrix}3&1\\0&4\end{pmatrix}})=3\cdot {\begin{pmatrix}1&0\\0&0\end{pmatrix}}+1\cdot {\begin{pmatrix}0&1\\0&0\end{pmatrix}}={\begin{pmatrix}3&1\\0&0\end{pmatrix}}$

Example 3.3

Both subscripts on ${\mbox{proj}}_{M,N}({{\vec {v}}\,})$ are significant. The first subscript $M$ matters because the result of the projection is an ${\vec {m}}\in M$, and changing this subspace would change the possible results. For an example showing that the second subscript matters, fix this plane subspace of $\mathbb {R} ^{3}$ and its basis

$M=\{{\begin{pmatrix}x\\y\\z\end{pmatrix}}\,{\big |}\,y-2z=0\}\qquad B_{M}=\langle {\begin{pmatrix}1\\0\\0\end{pmatrix}},{\begin{pmatrix}0\\2\\1\end{pmatrix}}\rangle$

and compare the projections along two different subspaces.

$N=\{k{\begin{pmatrix}0\\0\\1\end{pmatrix}}\,{\big |}\,k\in \mathbb {R} \}\qquad {\hat {N}}=\{k{\begin{pmatrix}0\\1\\-2\end{pmatrix}}\,{\big |}\,k\in \mathbb {R} \}$

(Verification that $\mathbb {R} ^{3}=M\oplus N$ and $\mathbb {R} ^{3}=M\oplus {\hat {N}}$ is routine.) We will check that these projections are different by checking that they have different effects on this vector.

${\vec {v}}={\begin{pmatrix}2\\2\\5\end{pmatrix}}$

For the first one we find a basis for $N$

$B_{N}=\langle {\begin{pmatrix}0\\0\\1\end{pmatrix}}\rangle$

and represent ${\vec {v}}$ with respect to the concatenation $B_{M}\!{\mathbin {{}^{\frown }}}\!B_{N}$.

${\begin{pmatrix}2\\2\\5\end{pmatrix}}=2\cdot {\begin{pmatrix}1\\0\\0\end{pmatrix}}+1\cdot {\begin{pmatrix}0\\2\\1\end{pmatrix}}+4\cdot {\begin{pmatrix}0\\0\\1\end{pmatrix}}$

The projection of ${\vec {v}}$ onto $M$ along $N$ is found by keeping the $M$ part and dropping the $N$ part.

${\mbox{proj}}_{M,N}({{\vec {v}}\,})=2\cdot {\begin{pmatrix}1\\0\\0\end{pmatrix}}+1\cdot {\begin{pmatrix}0\\2\\1\end{pmatrix}}={\begin{pmatrix}2\\2\\1\end{pmatrix}}$

For the other subspace ${\hat {N}}$, this basis is natural.

$B_{\hat {N}}=\langle {\begin{pmatrix}0\\1\\-2\end{pmatrix}}\rangle$

Representing ${\vec {v}}$ with respect to the concatenation

${\begin{pmatrix}2\\2\\5\end{pmatrix}}=2\cdot {\begin{pmatrix}1\\0\\0\end{pmatrix}}+(9/5)\cdot {\begin{pmatrix}0\\2\\1\end{pmatrix}}-(8/5)\cdot {\begin{pmatrix}0\\1\\-2\end{pmatrix}}$

and then keeping only the $M$ part gives this.

${\mbox{proj}}_{M,{\hat {N}}}({{\vec {v}}\,})=2\cdot {\begin{pmatrix}1\\0\\0\end{pmatrix}}+(9/5)\cdot {\begin{pmatrix}0\\2\\1\end{pmatrix}}={\begin{pmatrix}2\\18/5\\9/5\end{pmatrix}}$

Therefore projection along different subspaces may yield different results.

These pictures compare the two maps. Both show that the projection is indeed "onto" the plane and "along" the line.

Notice that the projection along $N$ is not orthogonal— there are members of the plane $M$ that are not orthogonal to the dotted line. But the projection along ${\hat {N}}$ is orthogonal.

A natural question is: what is the relationship between the projection operation defined above, and the operation of orthogonal projection onto a line? The second picture above suggests the answer— orthogonal projection onto a line is a special case of the projection defined above; it is just projection along a subspace perpendicular to the line.

In addition to pointing out that projection along a subspace is a generalization, this scheme shows how to define orthogonal projection onto any subspace of ${\mathbb{R}}^n$