Linear Algebra/Length and Angle Measures

We've translated the first section's results about solution sets into geometric terms for insight into how those sets look. But we must watch out not to be mislead by our own terms; labeling subsets of $\mathbb {R} ^{k}$ of the forms $\{{\vec {p}}+t{\vec {v}}\,{\big |}\,t\in \mathbb {R} \}$ and $\{{\vec {p}}+t{\vec {v}}+s{\vec {w}}\,{\big |}\,t,s\in \mathbb {R} \}$ as "lines" and "planes" doesn't make them act like the lines and planes of our prior experience. Rather, we must ensure that the names suit the sets. While we can't prove that the sets satisfy our intuition— we can't prove anything about intuition— in this subsection we'll observe that a result familiar from $\mathbb {R} ^{2}$ and $\mathbb {R} ^{3}$, when generalized to arbitrary $\mathbb {R} ^{k}$, supports the idea that a line is straight and a plane is flat. Specifically, we'll see how to do Euclidean geometry in a "plane" by giving a definition of the angle between two $\mathbb {R} ^{n}$ vectors in the plane that they generate.

Definition 2.1

The length of a vector ${\vec {v}}\in \mathbb {R} ^{n}$ is this.

$|{\vec {v}}\,|={\sqrt {v_{1}^{2}+\cdots +v_{n}^{2}}}$

Remark 2.2

This is a natural generalization of the Pythagorean Theorem. A classic discussion is in (Pólya 1954).

We can use that definition to derive a formula for the angle between two vectors. For a model of what to do, consider two vectors in $\mathbb {R} ^{3}$.

Put them in canonical position and, in the plane that they determine, consider the triangle formed by ${\vec {u}}$, ${\vec {v}}$, and ${\vec {u}}-{\vec {v}}$.

Apply the Law of Cosines, $|{\vec {u}}-{\vec {v}}\,|^{2}=|{\vec {u}}\,|^{2}+|{\vec {v}}\,|^{2}-2\,|{\vec {u}}\,|\,|{\vec {v}}\,|\cos \theta$, where $\theta$ is the angle between the vectors. Expand both sides

$(u_{1}-v_{1})^{2}+(u_{2}-v_{2})^{2}+(u_{3}-v_{3})^{2}$

$=(u_{1}^{2}+u_{2}^{2}+u_{3}^{2})+(v_{1}^{2}+v_{2}^{2}+v_{3}^{2})-2\,|{\vec {u}}\,|\,|{\vec {v}}\,|\cos \theta$

and simplify.

$\theta =\arccos(\,{\frac {u_{1}v_{1}+u_{2}v_{2}+u_{3}v_{3}}{|{\vec {u}}\,|\,|{\vec {v}}\,|}}\,)$

In higher dimensions no picture suffices but we can make the same argument analytically. First, the form of the numerator is clear— it comes from the middle terms of the squares $(u_{1}-v_{1})^{2}$, $(u_{2}-v_{2})^{2}$, etc.

Definition 2.3

The dot product (or inner product, or scalar product) of two $n$-component real vectors is the linear combination of their components.

${\vec {u}}\cdot {\vec {v}}=u_{1}v_{1}+u_{2}v_{2}+\cdots +u_{n}v_{n}$

Note that the dot product of two vectors is a real number, not a vector, and that the dot product of a vector from $\mathbb {R} ^{n}$ with a vector from $\mathbb {R} ^{m}$ is defined only when $n$ equals $m$. Note also this relationship between dot product and length: dotting a vector with itself gives its length squared ${\vec {u}}\cdot {\vec {u}}=u_{1}u_{1}+\cdots +u_{n}u_{n}=|{\vec {u}}\,|^{2}$.

Remark 2.4

The wording in that definition allows one or both of the two to be a row vector instead of a column vector. Some books require that the first vector be a row vector and that the second vector be a column vector. We shall not be that strict.

Still reasoning with letters, but guided by the pictures, we use the next theorem to argue that the triangle formed by ${\vec {u}}$, ${\vec {v}}$, and ${\vec {u}}-{\vec {v}}$ in $\mathbb {R} ^{n}$ lies in the planar subset of $\mathbb {R} ^{n}$ generated by ${\vec {u}}$ and ${\vec {v}}$.

Theorem 2.5 (Triangle Inequality)

For any ${\vec {u}},{\vec {v}}\in \mathbb {R} ^{n}$,

$|{\vec {u}}+{\vec {v}}\,|\leq |{\vec {u}}\,|+|{\vec {v}}\,|$

with equality if and only if one of the vectors is a nonnegative scalar multiple of the other one.

This inequality is the source of the familiar saying, "The shortest distance between two points is in a straight line."

Proof

(We'll use some algebraic properties of dot product that we have not yet checked, for instance that ${\vec {u}}\cdot ({\vec {a}}+{\vec {b}})={\vec {u}}\cdot {\vec {a}}+{\vec {u}}\cdot {\vec {b}}$ and that ${\vec {u}}\cdot {\vec {v}}={\vec {v}}\cdot {\vec {u}}$. See Problem 8.) The desired inequality holds if and only if its square holds.

${\begin{array}{rl}|{\vec {u}}+{\vec {v}}\,|^{2}&\leq (\,|{\vec {u}}\,|+|{\vec {v}}\,|\,)^{2}\\(\,{\vec {u}}+{\vec {v}}\,)\cdot (\,{\vec {u}}+{\vec {v}}\,)&\leq |{\vec {u}}\,|^{2}+2\,|{\vec {u}}\,|\,|{\vec {v}}\,|+|{\vec {v}}\,|^{2}\\{\vec {u}}\cdot {\vec {u}}+{\vec {u}}\cdot {\vec {v}}+{\vec {v}}\cdot {\vec {u}}+{\vec {v}}\cdot {\vec {v}}&\leq {\vec {u}}\cdot {\vec {u}}+2\,|{\vec {u}}\,|\,|{\vec {v}}\,|+{\vec {v}}\cdot {\vec {v}}\\2\,{\vec {u}}\cdot {\vec {v}}&\leq 2\,|{\vec {u}}\,|\,|{\vec {v}}\,|\end{array}}$

That, in turn, holds if and only if the relationship obtained by multiplying both sides by the nonnegative numbers $|{\vec {u}}\,|$ and $|{\vec {v}}\,|$

$2\,(\,|{\vec {v}}\,|\,{\vec {u}}\,)\cdot (\,|{\vec {u}}\,|\,{\vec {v}}\,)\leq 2\,|{\vec {u}}\,|^{2}\,|{\vec {v}}\,|^{2}$

and rewriting

$0\leq |{\vec {u}}\,|^{2}\,|{\vec {v}}\,|^{2}-2\,(\,|{\vec {v}}\,|\,{\vec {u}}\,)\cdot (\,|{\vec {u}}\,|\,{\vec {v}}\,)+|{\vec {u}}\,|^{2}\,|{\vec {v}}\,|^{2}$

is true. But factoring

$0\leq (\,|{\vec {u}}\,|\,{\vec {v}}-|{\vec {v}}\,|\,{\vec {u}}\,)\cdot (\,|{\vec {u}}\,|\,{\vec {v}}-|{\vec {v}}\,|\,{\vec {u}}\,)$

shows that this certainly is true since it only says that the square of the length of the vector $|{\vec {u}}\,|\,{\vec {v}}-|{\vec {v}}\,|\,{\vec {u}}\,$ is not negative.

As for equality, it holds when, and only when, $|{\vec {u}}\,|\,{\vec {v}}-|{\vec {v}}\,|\,{\vec {u}}$ is ${\vec {0}}$. The check that $|{\vec {u}}\,|\,{\vec {v}}=|{\vec {v}}\,|\,{\vec {u}}\,$ if and only if one vector is a nonnegative real scalar multiple of the other is easy.

This result supports the intuition that even in higher-dimensional spaces, lines are straight and planes are flat. For any two points in a linear surface, the line segment connecting them is contained in that surface (this is easily checked from the definition). But if the surface has a bend then that would allow for a shortcut (shown here grayed, while the segment from $P$ to $Q$ that is contained in the surface is solid).

Because the Triangle Inequality says that in any $\mathbb {R} ^{n}$, the shortest cut between two endpoints is simply the line segment connecting them, linear surfaces have no such bends.

Back to the definition of angle measure. The heart of the Triangle Inequality's proof is the "${\vec {u}}\cdot {\vec {v}}\leq |{\vec {u}}\,|\,|{\vec {v}}\,|$" line. At first glance, a reader might wonder if some pairs of vectors satisfy the inequality in this way: while ${\vec {u}}\cdot {\vec {v}}$ is a large number, with absolute value bigger than the right-hand side, it is a negative large number. The next result says that no such pair of vectors exists.

Corollary 2.6 (Cauchy-Schwarz Inequality)

For any ${\vec {u}},{\vec {v}}\in \mathbb {R} ^{n}$,

$|\,{\vec {u}}\cdot {\vec {v}}\,|\leq |\,{\vec {u}}\,|\,|{\vec {v}}\,|$

with equality if and only if one vector is a scalar multiple of the other.

Proof

The Triangle Inequality's proof shows that ${\vec {u}}\cdot {\vec {v}}\leq |{\vec {u}}\,|\,|{\vec {v}}\,|$ so if ${\vec {u}}\cdot {\vec {v}}$ is positive or zero then we are done. If ${\vec {u}}\cdot {\vec {v}}$ is negative then this holds.

$|\,{\vec {u}}\cdot {\vec {v}}\,|=-(\,{\vec {u}}\cdot {\vec {v}}\,)=(-{\vec {u}}\,)\cdot {\vec {v}}\leq |-{\vec {u}}\,|\,|{\vec {v}}\,|=|{\vec {u}}\,|\,|{\vec {v}}\,|$

The equality condition is Problem 9.

The Cauchy-Schwarz inequality assures us that the next definition makes sense because the fraction has absolute value less than or equal to one.

Definition 2.7

The angle between two nonzero vectors ${\vec {u}},{\vec {v}}\in \mathbb {R} ^{n}$ is

$\theta =\arccos(\,{\frac {{\vec {u}}\cdot {\vec {v}}}{|{\vec {u}}\,|\,|{\vec {v}}\,|}}\,)$

(the angle between the zero vector and any other vector is defined to be a right angle).

Thus vectors from $\mathbb {R} ^{n}$ are orthogonal (or perpendicular) if and only if their dot product is zero.

Example 2.8

These vectors are orthogonal.

${\begin{pmatrix}1\\-1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}=0$

The arrows are shown away from canonical position but nevertheless the vectors are orthogonal.

Example 2.9

The $\mathbb {R} ^{3}$ angle formula given at the start of this subsection is a special case of the definition. Between these two

the angle is

$\arccos({\frac {(1)(0)+(1)(3)+(0)(2)}{{\sqrt {1^{2}+1^{2}+0^{2}}}{\sqrt {0^{2}+3^{2}+2^{2}}}}})=\arccos({\frac {3}{{\sqrt {2}}{\sqrt {13}}}})$

approximately $0.94{\text{radians}}$. Notice that these vectors are not orthogonal. Although the $yz$-plane may appear to be perpendicular to the $xy$-plane, in fact the two planes are that way only in the weak sense that there are vectors in each orthogonal to all vectors in the other. Not every vector in each is orthogonal to all vectors in the other.

Exercises

This exercise is recommended for all readers.

Problem 1

Find the length of each vector.

1. ${\begin{pmatrix}3\\1\end{pmatrix}}$
2. ${\begin{pmatrix}-1\\2\end{pmatrix}}$
3. ${\begin{pmatrix}4\\1\\1\end{pmatrix}}$
4. ${\begin{pmatrix}0\\0\\0\end{pmatrix}}$
5. ${\begin{pmatrix}1\\-1\\1\\0\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 2

Find the angle between each two, if it is defined.

1. ${\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}1\\4\end{pmatrix}}$
2. ${\begin{pmatrix}1\\2\\0\end{pmatrix}},{\begin{pmatrix}0\\4\\1\end{pmatrix}}$
3. ${\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}1\\4\\-1\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 3

During maneuvers preceding the Battle of Jutland, the British battle cruiser Lion moved as follows (in nautical miles): $1.2$ miles north, $6.1$ miles $38$ degrees east of south, $4.0$ miles at $89$ degrees east of north, and $6.5$ miles at $31$ degrees east of north. Find the distance between starting and ending positions (O'Hanian 1985).

Problem 4

Find $k$ so that these two vectors are perpendicular.

${\begin{pmatrix}k\\1\end{pmatrix}}\qquad {\begin{pmatrix}4\\3\end{pmatrix}}$

Problem 5

Describe the set of vectors in $\mathbb {R} ^{3}$ orthogonal to this one.

${\begin{pmatrix}1\\3\\-1\end{pmatrix}}$

This exercise is recommended for all readers.

Problem 6

1. Find the angle between the diagonal of the unit square in $\mathbb {R} ^{2}$ and one of the axes.
2. Find the angle between the diagonal of the unit cube in $\mathbb {R} ^{3}$ and one of the axes.
3. Find the angle between the diagonal of the unit cube in $\mathbb {R} ^{n}$ and one of the axes.
4. What is the limit, as $n$ goes to $\infty$, of the angle between the diagonal of the unit cube in $\mathbb {R} ^{n}$ and one of the axes?

Problem 7

Is any vector perpendicular to itself?

This exercise is recommended for all readers.

Problem 8

Describe the algebraic properties of dot product.

1. Is it right-distributive over addition: $({\vec {u}}+{\vec {v}})\cdot {\vec {w}}={\vec {u}}\cdot {\vec {w}}+{\vec {v}}\cdot {\vec {w}}$?
2. Is is left-distributive (over addition)?
3. Does it commute?
4. Associate?
5. How does it interact with scalar multiplication?

As always, any assertion must be backed by either a proof or an example.

Problem 9

Verify the equality condition in Corollary 2.6, the Cauchy-Schwarz Inequality.

1. Show that if ${\vec {u}}$ is a negative scalar multiple of ${\vec {v}}$ then ${\vec {u}}\cdot {\vec {v}}$ and ${\vec {v}}\cdot {\vec {u}}$ are less than or equal to zero.
2. Show that $|{\vec {u}}\cdot {\vec {v}}|=|{\vec {u}}\,|\,|{\vec {v}}\,|$ if and only if one vector is a scalar multiple of the other.

Problem 10

Suppose that ${\vec {u}}\cdot {\vec {v}}={\vec {u}}\cdot {\vec {w}}$ and ${\vec {u}}\neq {\vec {0}}$. Must ${\vec {v}}={\vec {w}}$?

This exercise is recommended for all readers.

Problem 11

Does any vector have length zero except a zero vector? (If "yes", produce an example. If "no", prove it.)

This exercise is recommended for all readers.

Problem 12

Find the midpoint of the line segment connecting $(x_{1},y_{1})$ with $(x_{2},y_{2})$ in $\mathbb {R} ^{2}$. Generalize to $\mathbb {R} ^{n}$.

Problem 13

Show that if ${\vec {v}}\neq {\vec {0}}$ then ${\vec {v}}/|{\vec {v}}\,|$ has length one. What if ${\vec {v}}={\vec {0}}$?

Problem 14

Show that if $r\geq 0$ then $r{\vec {v}}$ is $r$ times as long as ${\vec {v}}$. What if $r<0$?

This exercise is recommended for all readers.

Problem 15

A vector ${\vec {v}}\in \mathbb {R} ^{n}$ of length one is a unit vector. Show that the dot product of two unit vectors has absolute value less than or equal to one. Can "less than" happen? Can "equal to"?

Problem 16

Prove that $|{\vec {u}}+{\vec {v}}\,|^{2}+|{\vec {u}}-{\vec {v}}\,|^{2}=2|{\vec {u}}\,|^{2}+2|{\vec {v}}\,|^{2}.$

Problem 17

Show that if ${\vec {x}}\cdot {\vec {y}}=0$ for every ${\vec {y}}$ then ${\vec {x}}={\vec {0}}$.

Problem 18

Is $|{\vec {u}}_{1}+\cdots +{\vec {u}}_{n}|\leq |{\vec {u}}_{1}|+\cdots +|{\vec {u}}_{n}|$? If it is true then it would generalize the Triangle Inequality.

Problem 19

What is the ratio between the sides in the Cauchy-Schwarz inequality?

Problem 20

Why is the zero vector defined to be perpendicular to every vector?

Problem 21

Describe the angle between two vectors in $\mathbb {R} ^{1}$.

Problem 22

Give a simple necessary and sufficient condition to determine whether the angle between two vectors is acute, right, or obtuse.

This exercise is recommended for all readers.

Problem 23

Generalize to $\mathbb {R} ^{n}$ the converse of the Pythagorean Theorem, that if ${\vec {u}}$ and ${\vec {v}}$ are perpendicular then $|{\vec {u}}+{\vec {v}}\,|^{2}=|{\vec {u}}\,|^{2}+|{\vec {v}}\,|^{2}$.

Problem 24

Show that $|{\vec {u}}\,|=|{\vec {v}}\,|$ if and only if ${\vec {u}}+{\vec {v}}$ and ${\vec {u}}-{\vec {v}}$ are perpendicular. Give an example in $\mathbb {R} ^{2}$.

Problem 25

Show that if a vector is perpendicular to each of two others then it is perpendicular to each vector in the plane they generate. (Remark. They could generate a degenerate plane— a line or a point— but the statement remains true.)

Problem 26

Prove that, where ${\vec {u}},{\vec {v}}\in \mathbb {R} ^{n}$ are nonzero vectors, the vector

${\frac {\vec {u}}{|{\vec {u}}\,|}}+{\frac {\vec {v}}{|{\vec {v}}\,|}}$

bisects the angle between them. Illustrate in $\mathbb {R} ^{2}$.

Problem 27

Verify that the definition of angle is dimensionally correct: (1) if $k>0$ then the cosine of the angle between $k{\vec {u}}$ and ${\vec {v}}$ equals the cosine of the angle between ${\vec {u}}$ and ${\vec {v}}$, and (2) if $k<0$ then the cosine of the angle between $k{\vec {u}}$ and ${\vec {v}}$ is the negative of the cosine of the angle between ${\vec {u}}$ and ${\vec {v}}$.

This exercise is recommended for all readers.

Problem 28

Show that the inner product operation is linear: for ${\vec {u}},{\vec {v}},{\vec {w}}\in \mathbb {R} ^{n}$ and $k,m\in \mathbb {R}$, ${\vec {u}}\cdot (k{\vec {v}}+m{\vec {w}})=k({\vec {u}}\cdot {\vec {v}})+m({\vec {u}}\cdot {\vec {w}})$.

This exercise is recommended for all readers.

Problem 29

The geometric mean of two positive reals $x,y$ is ${\sqrt {xy}}$. It is analogous to the arithmetic mean $(x+y)/2$. Use the Cauchy-Schwarz inequality to show that the geometric mean of any $x,y\in \mathbb {R}$ is less than or equal to the arithmetic mean.

? Problem 30

A ship is sailing with speed and direction ${\vec {v}}_{1}$; the wind blows apparently (judging by the vane on the mast) in the direction of a vector ${\vec {a}}$; on changing the direction and speed of the ship from ${\vec {v}}_{1}$ to ${\vec {v}}_{2}$ the apparent wind is in the direction of a vector ${\vec {b}}$.

Find the vector velocity of the wind (Ivanoff & Esty 1933).

Problem 31

Verify the Cauchy-Schwarz inequality by first proving Lagrange's identity:

$\left(\sum _{1\leq j\leq n}a_{j}b_{j}\right)^{2}=\left(\sum _{1\leq j\leq n}a_{j}^{2}\right)\left(\sum _{1\leq j\leq n}b_{j}^{2}\right)-\sum _{1\leq k<j\leq n}(a_{k}b_{j}-a_{j}b_{k})^{2}$

and then noting that the final term is positive. (Recall the meaning

$\sum _{1\leq j\leq n}a_{j}b_{j}=a_{1}b_{1}+a_{2}b_{2}+\cdots +a_{n}b_{n}$

and

$\sum _{1\leq j\leq n}{a_{j}}^{2}={a_{1}}^{2}+{a_{2}}^{2}+\cdots +{a_{n}}^{2}$

of the $\Sigma$ notation.) This result is an improvement over Cauchy-Schwarz because it gives a formula for the difference between the two sides. Interpret that difference in $\mathbb {R} ^{2}$.

Solutions

References

• O'Hanian, Hans (1985), Physics, vol. 1, W. W. Norton
• Ivanoff, V. F. (proposer); Esty, T. C. (solver) (1933), "Problem 3529", American Mathematical Monthly, 39 (2): 118 {{citation}}: Unknown parameter |month= ignored (help)
• Pólya, G. (1954), Mathematics and Plausible Reasoning: Volume II Patterns of Plausible Inference, Princeton University Press