Linear Algebra/Orthogonal Projection Onto a Line/Solutions

Solutions

edit
This exercise is recommended for all readers.
Problem 1

Project the first vector orthogonally onto the line spanned by the second vector.

  1.  ,  
  2.  ,  
  3.  ,  
  4.  ,  
Answer

Each is a straightforward application of the formula from Definition 1.1.

  1.  
  2.  
  3.  
  4.  
This exercise is recommended for all readers.
Problem 2

Project the vector orthogonally onto the line.

  1.  
  2.  , the line  
Answer
  1.  
  2. Writing the line as   gives this projection.
     
Problem 3

Although the development of Definition 1.1 is guided by the pictures, we are not restricted to spaces that we can draw. In   project this vector onto this line.

 
Answer

 

This exercise is recommended for all readers.
Problem 4

Definition 1.1 uses two vectors   and  . Consider the transformation of   resulting from fixing

 

and projecting   onto the line that is the span of  . Apply it to these vectors.

  1.  
  2.  

Show that in general the projection tranformation is this.

 

Express the action of this transformation with a matrix.

Answer
  1.  
  2.  

In general the projection is this.

 

The appropriate matrix is this.

 
Problem 5

Example 1.5 suggests that projection breaks   into two parts,   and  , that are "not interacting". Recall that the two are orthogonal. Show that any two nonzero orthogonal vectors make up a linearly independent set.

Answer

Suppose that   and   are nonzero and orthogonal. Consider the linear relationship  . Take the dot product of both sides of the equation with   to get that

 

is equal to  . With the assumption that   is nonzero, this gives that   is zero. Showing that   is zero is similar.

Problem 6
  1. What is the orthogonal projection of   onto a line if   is a member of that line?
  2. Show that if   is not a member of the line then the set   is linearly independent.
Answer
  1. If the vector   is in the line then the orthogonal projection is  . To verify this by calculation, note that since   is in the line we have that   for some scalar  .
     
    (Remark. If we assume that   is nonzero then the above is simplified on taking   to be  .)
  2. Write   for the projection  . Note that, by the assumption that   is not in the line, both   and   are nonzero. Note also that if   is zero then we are actually considering the one-element set  , and with   nonzero, this set is necessarily linearly independent. Therefore, we are left considering the case that   is nonzero. Setting up a linear relationship
     
    leads to the equation  . Because   isn't in the line, the scalars   and   must both be zero. The   case is handled above, so the remaining case is that  , and this gives that   also. Hence the set is linearly independent.
Problem 7

Definition 1.1 requires that   be nonzero. Why? What is the right definition of the orthogonal projection of a vector onto the (degenerate) line spanned by the zero vector?

Answer

If   is the zero vector then the expression

 

contains a division by zero, and so is undefined. As for the right definition, for the projection to lie in the span of the zero vector, it must be defined to be  .

Problem 8

Are all vectors the projection of some other vector onto some line?

Answer

Any vector in   is the projection of some other onto a line, provided that the dimension   is greater than one. (Clearly, any vector is the projection of itself into a line containing itself; the question is to produce some vector other than   that projects to  .)

Suppose that   with  . If   then we consider the line   and if   we take   to be any (nondegenerate) line at all (actually, we needn't distinguish between these two cases— see the prior exercise). Let   be the components of  ; since  , there are at least two. If some   is zero then the vector   is perpendicular to  . If none of the components is zero then the vector   whose components are   is perpendicular to  . In either case, observe that   does not equal  , and that   is the projection of   onto  .

 

We can dispose of the remaining   and   cases. The dimension   case is the trivial vector space, here there is only one vector and so it cannot be expressed as the projection of a different vector. In the dimension   case there is only one (nondegenerate) line, and every vector is in it, hence every vector is the projection only of itself.

This exercise is recommended for all readers.
Problem 9

Show that the projection of   onto the line spanned by   has length equal to the absolute value of the number   divided by the length of the vector  .

Answer

The proof is simply a calculation.

 
Problem 10

Find the formula for the distance from a point to a line.

Answer

Because the projection of   onto the line spanned by   is

 

the distance squared from the point to the line is this (a vector dotted with itself   is written  ).

 
Problem 11

Find the scalar   such that   is a minimum distance from the point   by using calculus (i.e., consider the distance function, set the first derivative equal to zero, and solve). Generalize to  .

Answer

Because square root is a strictly increasing function, we can minimize   instead of the square root of  . The derivative is  . Setting it equal to zero   gives the only critical point.

 

Now the second derivative with respect to  

 

is strictly positive (as long as neither   nor   is zero, in which case the question is trivial) and so the critical point is a minimum.

The generalization to   is straightforward. Consider  , take the derivative, etc.

This exercise is recommended for all readers.
Problem 12

Prove that the orthogonal projection of a vector onto a line is shorter than the vector.

Answer

The Cauchy-Schwarz inequality   gives that this fraction

 

when divided by   is less than or equal to one. That is,   is larger than or equal to the fraction.

This exercise is recommended for all readers.
Problem 13

Show that the definition of orthogonal projection onto a line does not depend on the spanning vector: if   is a nonzero multiple of   then   equals  .

Answer

Write   for  , and calculate:  .

This exercise is recommended for all readers.
Problem 14

Consider the function mapping to plane to itself that takes a vector to its projection onto the line  . These two each show that the map is linear, the first one in a way that is bound to the coordinates (that is, it fixes a basis and then computes) and the second in a way that is more conceptual.

  1. Produce a matrix that describes the function's action.
  2. Show also that this map can be obtained by first rotating everything in the plane   radians clockwise, then projecting onto the  -axis, and then rotating   radians counterclockwise.
Answer
  1. Fixing
     
    as the vector whose span is the line, the formula gives this action,
     
    which is the effect of this matrix.
     
  2. Rotating the entire plane   radians clockwise brings the   line to lie on the  -axis. Now projecting and then rotating back has the desired effect.
Problem 15

For   let   be the projection of   onto the line spanned by  , let   be the projection of   onto the line spanned by  , let   be the projection of   onto the line spanned by  , etc., back and forth between the spans of   and  . That is,   is the projection of   onto the span of   if   is even, and onto the span of   if   is odd. Must that sequence of vectors eventually settle down— must there be a sufficiently large   such that   equals   and   equals  ? If so, what is the earliest such  ?

Answer

The sequence need not settle down. With

 

the projections are these.

 

This sequence doesn't repeat.