Engineering Analysis/Linear Independence and Basis

Linear IndependenceEdit

A set of vectors   are said to be linearly dependent on one another if any vector v from the set can be constructed from a linear combination of the other vectors in the set. Given the following linear equation:


The set of vectors V is linearly independent only if all the a coefficients are zero. If we combine the v vectors together into a single row vector:


And we combine all the a coefficients into a single column vector:


We have the following linear equation:


We can show that this equation can only be satisifed for  , the matrix   must be invertable:


Remember that for the matrix to be invertable, the determinate must be non-zero.

Non-Square Matrix VEdit

If the matrix   is not square, then the determinate can not be taken, and therefore the matrix is not invertable. To solve this problem, we can premultiply by the transpose matrix:


And then the square matrix   must be invertable:



The rank of a matrix is the largest number of linearly independent rows or columns in the matrix.

To determine the Rank, typically the matrix is reduced to row-echelon form. From the reduced form, the number of non-zero rows, or the number of non-zero columns (whichever is smaller) is the rank of the matrix.

If we multiply two matrices A and B, and the result is C:


Then the rank of C is the minimum value between the ranks A and B:



A Span of a set of vectors V is the set of all vectors that can be created by a linear combination of the vectors.


A basis is a set of linearly-independent vectors that span the entire vector space.

Basis ExpansionEdit

If we have a vector  , and V has basis vectors  , by definition, we can write y in terms of a linear combination of the basis vectors:




If   is invertable, the answer is apparent, but if   is not invertable, then we can perform the following technique:


And we call the quantity   the left-pseudoinverse of  .

Change of BasisEdit

Frequently, it is useful to change the basis vectors to a different set of vectors that span the set, but have different properties. If we have a space V, with basis vectors   and a vector in V called x, we can use the new basis vectors   to represent x:




If V is invertable, then the solution to this problem is simple.

Grahm-Schmidt OrthogonalizationEdit

If we have a set of basis vectors that are not orthogonal, we can use a process known as orthogonalization to produce a new set of basis vectors for the same space that are orthogonal:

Find the new basis  
Such that  

We can define the vectors as follows:


Notice that the vectors produced by this technique are orthogonal to each other, but they are not necessarily orthonormal. To make the w vectors orthonormal, you must divide each one by its norm:


Reciprocal BasisEdit

A Reciprocal basis is a special type of basis that is related to the original basis. The reciprocal basis   can be defined as: