It is known that the symmetric matrix
A
T
A
{\displaystyle A^{T}A\!\,}
can be factored as
A
T
A
=
V
Λ
V
T
{\displaystyle A^{T}A=V\Lambda V^{T}\!\,}
where the columns of
V
{\displaystyle V\!\,}
are orthonormal eigenvectors of
A
T
A
{\displaystyle A^{T}A\!\,}
and
Λ
{\displaystyle \Lambda \!\,}
is the diagonal matrix containing the corresponding eigenvalues. Using this as a starting point, derive the singular value decomposition of
A
{\displaystyle A\!\,}
. That is show that there is a real orthogonal matrix
U
{\displaystyle U\!\,}
and a matrix
Σ
∈
R
n
×
m
{\displaystyle \Sigma \in R^{n\times m}\!\,}
which is zero except for its diagonal entries
σ
1
≥
σ
2
≥
…
≥
σ
m
>
0
{\displaystyle \sigma _{1}\geq \sigma _{2}\geq \ldots \geq \sigma _{m}>0\!\,}
such that
A
=
U
Σ
V
T
{\displaystyle A=U\Sigma V^{T}\!\,}
We want to show
A
=
U
Σ
V
T
{\displaystyle A=U\Sigma V^{T}\!\,}
which is equivalent to
A
V
=
U
Σ
{\displaystyle AV=U\Sigma \!\,}
Decompose
Λ
{\displaystyle \Lambda \!\,}
into
Σ
T
Σ
{\displaystyle \Sigma ^{T}\Sigma \!\,}
i.e.
[
σ
1
σ
2
⋱
σ
m
]
⏟
Λ
∈
R
m
×
m
=
[
σ
1
0
⋯
0
σ
2
0
⋱
⋮
σ
m
0
⋯
0
]
⏟
Σ
T
∈
R
m
×
n
[
σ
1
σ
2
⋱
σ
m
0
⋯
0
⋮
⋮
⋮
0
⋯
0
0
]
⏟
Σ
∈
R
n
×
m
{\displaystyle \underbrace {\begin{bmatrix}\sigma _{1}&&&\\&\sigma _{2}&&\\&&\ddots &\\&&&\sigma _{m}\end{bmatrix}} _{\Lambda \in R^{m\times m}}=\underbrace {\begin{bmatrix}{\sqrt {\sigma _{1}}}&&&&0&\cdots &0\\&{\sqrt {\sigma _{2}}}&&&0&&\\&&\ddots &&\vdots &&\\&&&{\sqrt {\sigma _{m}}}&0&\cdots &0\end{bmatrix}} _{\Sigma ^{T}\in R^{m\times n}}\underbrace {\begin{bmatrix}{\sqrt {\sigma _{1}}}&&&\\&{\sqrt {\sigma _{2}}}&&\\&&\ddots &\\&&&{\sqrt {\sigma _{m}}}\\0&\cdots &&0\\\vdots &\vdots &&\vdots \\0&\cdots &0&0\end{bmatrix}} _{\Sigma \in R^{n\times m}}\!\,}
We can assume
σ
1
≥
σ
2
≥
…
≥
σ
n
>
0
{\displaystyle \sigma _{1}\geq \sigma _{2}\geq \ldots \geq \sigma _{n}>0\!\,}
since otherwise we could just rearrange the columns of
V
{\displaystyle V\!\,}
.
Let
U
=
A
V
Σ
−
1
{\displaystyle U=AV\Sigma ^{-1}\!\,}
where
Σ
−
1
=
(
1
σ
1
0
⋯
0
1
σ
2
0
⋱
⋮
1
σ
m
0
⋯
0
)
,
{\displaystyle \Sigma ^{-1}=\left({\begin{array}{ccccccc}{\frac {1}{\sqrt {\sigma _{1}}}}&&&&0&\cdots &0\\&{\frac {1}{\sqrt {\sigma _{2}}}}&&&0&&\\&&\ddots &&\vdots &&\\&&&{\frac {1}{\sqrt {\sigma _{m}}}}&0&\cdots &0\end{array}}\right),}
U
U
T
=
A
V
Σ
−
1
Σ
−
T
V
T
A
T
=
A
V
Λ
−
1
V
T
A
T
=
A
A
−
1
A
−
T
A
T
=
I
U
T
U
=
Σ
−
T
V
T
A
T
A
V
Σ
−
1
=
Σ
−
T
V
T
V
Λ
V
T
V
Σ
−
1
=
I
{\displaystyle {\begin{aligned}UU^{T}&=AV\Sigma ^{-1}\Sigma ^{-T}V^{T}A^{T}\\&=AV\Lambda ^{-1}V^{T}A^{T}\\&=AA^{-1}A^{-T}A^{T}\\&=I\\\\U^{T}U&=\Sigma ^{-T}V^{T}A^{T}AV\Sigma ^{-1}\\&=\Sigma ^{-T}V^{T}V\Lambda V^{T}V\Sigma ^{-1}\\&=I\end{aligned}}\!\,}