\section{Eigenvalues and Eigenvectors}
\definition{Let $A$ be a square $n\times n$ matrix. A scalar $\lambda$ is said to be an \textbf{eigenvalue} or a \textbf{characteristic value} of $A$ if there exists a nonzero vector $\lvec{x}$ in $\mathset{R}^n$ such that:
\[\boxed{A\lvec{x} = \lambda \lvec{x}}\,.\]
The nonzero vector $\lvec{x}$ is called an \textbf{eigenvector} of $A$ corresponding to the eigenvalue $\lambda$\,.}

\begin{example}
  Show $\lvec{x} = \begin{bmatrix}
  2\\1
  \end{bmatrix}$ is an eigenvector of $A = \begin{bmatrix*}[r]
  4&-2\\1&1
  \end{bmatrix*}$ and find its corresponding eigenvalue.\\[.5em]
  Solution:
\[ A\lvec{x} = \begin{bmatrix*}[r]
    4&-2\\1&1
    \end{bmatrix*}\begin{bmatrix}
    2\\1
    \end{bmatrix}
    = \begin{bmatrix}
    8-2\\2+1
    \end{bmatrix}
    = \begin{bmatrix}
    6\\3
    \end{bmatrix}
    = 3\begin{bmatrix}
    2\\1
    \end{bmatrix}\]
  Thus $A\lvec{x}=3\lvec{x}$ and therefore $\lvec{x} = \begin{bmatrix}
  2\\1
  \end{bmatrix}$ is an eigenvector of $A$ corresponding to eigenvalue $\lambda = 3$\,. 
\end{example}

\begin{example}
\label{exam:eigenvalue}
  Show $\lvec{x} = (-2,1,1)$ is an eigenvector of $A = \begin{bmatrix*}[r]
  0&0&-2\\1&2&1\\1&0&3
  \end{bmatrix*}$ and find its corresponding eigenvalue.\\[.5em]
  Solution:
\[A\lvec{x} = \begin{bmatrix*}[r]
            0&0&-2\\1&2&1\\1&0&3
            \end{bmatrix*}\begin{bmatrix*}[r]
            -2\\1\\1
            \end{bmatrix*}
            = \begin{bmatrix}
            0+0-2\\-2+2+1\\-2+0+3
            \end{bmatrix}
            = \begin{bmatrix*}[r]
            -2\\1\\1
            \end{bmatrix*}\\
            = (1)\lvec{x}\]
Thus $A\lvec{x}=1\lvec{x}$ and $\lvec{x}=(-2,1,1)$ is an eigenvector of $A$ corresponding to eigenvalue $\lambda = 1$.  
\end{example}

A nonzero scalar multiple of an eigenvector of $A$ also is an eigenvector of $A$.
\begin{example}
In Example~\ref{exam:eigenvalue} it was shown $\lvec{x} = (-2,1,1)$ was an eigenvector of $A$ corresponding to eigenvalue $\lambda = 1$.  Then $2\lvec{x} = (-4,2,2)$ is also an eigenvector corresponding to $\lambda=1$ since
  \[A(2\lvec{x})=\begin{bmatrix*}[r]
  0&0&-2\\1&2&1\\1&0&3
  \end{bmatrix*}\begin{bmatrix*}[r]
  -4\\2\\2
  \end{bmatrix*} = \begin{bmatrix}
  0+0-4\\-4+4+2\\-4+0+6
  \end{bmatrix} = \begin{bmatrix*}[r]
  -4\\2\\2
  \end{bmatrix*} = (1)\begin{bmatrix*}[r]
  -4\\2\\2
  \end{bmatrix*}=(1)(2\lvec{x})\]
  Therefore $2\lvec{x}$ is also an eigenvector of $A$ with the same eigenvalue of $\lambda=1$. 
\end{example}

Additionally if two eigenvectors correspond to the same eigenvalue $\lambda$ of $A$ their sum will also be an eigenvector of $A$.  We summarize these results in the following theorem.

\theorem{\label{thm:eigenspace}If $\lvec{u}$ and $\lvec{v}$ are eigenvectors of $A$ associated with the same eigenvalue $\lambda$, and $c$ is a scalar then $\lvec{u}+\lvec{v}$ and $c\lvec{u}$ are also eigenvectors of $A$ associated with $\lambda$ provided they do not equal the zero vector. (i.e. $\lvec{v}\neq-\lvec{u}$ and $c\neq 0$.)}

\textbook{\newpage}
\textbf{Proof:}\\
Let $\lvec{u}$, $\lvec{v}$ be eigenvectors of $A$ corresponding to eigenvalue $\lambda$ and let $c$ be a scalar. Then
\begin{align*}
A(\lvec{u}+\lvec{v})&=A\lvec{u}+A\lvec{v}=\lambda\lvec{u}+\lambda\lvec{v}=\lambda(\lvec{u}+\lvec{v})\\
A(c\lvec{u})&=c(A\lvec{u})=c(\lambda\lvec{u})=\lambda(c\lvec{u})\,.
\end{align*}

It follows from Theorem~\ref{thm:eigenspace} that the eigenvectors corresponding to a particular eigenvalue $\lambda$ of $A$ combined with the zero vector $\lvec{0}$ form a subspace of $\mathset{R}^n$.

\definition{The subspace of $\mathset{R}^n$ consisting of all eigenvectors $\lvec{x}$ associated with a particular eigenvalue $\lambda$ of $A$ and the zero vector is called the \textbf{eigenspace} of $A$ corresponding to $\lambda$\,.}

\section{Finding Eigenvalues and Eigenvectors}
Recall that if $B$ is a square matrix of order $n$ then the homogeneous system $B\lvec{x}=0$ has the unique trivial solution $\lvec{x}=\lvec{0}$ if $\det B \neq 0$ and it have infinitely many nontrivial solutions if $\det B = 0$\,.  We can cast the problem of finding eigenvalues and eigenvectors of the $n\times n$ square matrix $A$ in this form by solving the eigenvalue equation for $\lvec{x}$\,.

\begin{align*}
  A\lvec{x} &= \lambda \lvec{x}\\
  \Rightarrow A\lvec{x} - \lambda \lvec{x} &= 0\\
  \Rightarrow A\lvec{x}  - \lambda I\lvec{x} &= 0\\
  \Rightarrow (A-\lambda I)\lvec{x} &= 0\\
\end{align*}
Setting  $B=A-\lambda I$ it follows that to have non-trivial $(\lvec{x}\neq\lvec{0})$ solutions to this system of $n$ equations in $n$ unknowns we require
         \[\boxed{\det(A-\lambda I) = 0}\,.\]

\theorem{Let $A$ be an $n\times n$ matrix. The number $\lambda$ is an eigenvalue of $A$ if and only if $\det(A-\lambda I) = 0$.}

Consideration of the form of the determinant shows the left hand side of the equation is a polynomial in $\lambda$ with coefficients determined by $A$.  With that in mind we have the following definition.

\definition{The polynomial $P_A(\lambda) = \det(A-\lambda I)$ is called the \textbf{characteristic polynomial} of $A$ and the equation $\det(A-\lambda I) = 0$ is called its \textbf{characteristic equation}.}

To find the eigenvalues of a matrix $A$ and their associated eigenvectors now becomes a two step process:
\begin{enumerate}
\item Solve the characteristic equation for the eigenvalues of $A$.
\item For each eigenvalue $\lambda$ solve the linear system $\boxed{(A-\lambda I)\lvec{x}=\lvec{0}}$ for $\lvec{x}$ to find its associated eigenvectors.
\end{enumerate}

\begin{example}
\label{exam:eigenvector}
  Find the eigenvalues and eigenvectors of the given matrix.
\[A = \begin{bmatrix}
  2&3\\4&3
  \end{bmatrix}\]
Solution:\\
  \begin{align*}
    \det(A-\lambda I) &= \begin{vmatrix}
    2-\lambda & 3 \\ 4 & 3-\lambda
    \end{vmatrix} = 0\\
    &\Rightarrow (2-\lambda)(3-\lambda) - 12 = 0\\
    &\Rightarrow 6-2\lambda - 3\lambda + \lambda^2 - 12 = 0\\
    &\Rightarrow \lambda^2 - 5\lambda - 6 = 0\\
    &\Rightarrow (\lambda-6)(\lambda+1) = 0\\
    &\Rightarrow \begin{cases}
      \lambda_1 = 6\\
      \lambda_2 = -1\\
    \end{cases}
  \end{align*}
\begin{description}
\item[$\bm{\lambda_1 = 6}$\,:]~\\
  To find the eigenvalues $\lvec{x}_1=(x_1,x_2)$ corresponding to $\lambda_1$ we need to solve:
  \[(A-6I)\lvec{x}_1 = \lvec{0}\]
  Since
  \[A-6I=\begin{bmatrix} 2 & 3 \\ 4 & 3\end{bmatrix}-6\begin{bmatrix}1 & 0 \\ 0 & 1 \end{bmatrix}=\begin{bmatrix*}[r] -4 & 3 \\ 4 & -3\end{bmatrix*}\]
  this can be done using the augmented matrix as follows
  \[\begin{bmatrix}[rr|r] -4&3&0\\4&-3&0\end{bmatrix}\]
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[16.5em]{ \\ \rowop{R_2}{R_2 + R_1}}{\begin{bmatrix}[cr|r]
        \circled{-4}&3&0\\0&0&0
    \end{bmatrix}}\ \Leftrightarrow\ \parbox{5em}{\begin{align*} -4x_1+3x_2 & = 0 \\ 0 &=0 \end{align*}}\hspace{11em}&\end{flalign*}
  Here we have circled the leading entry.  Back-substitution in the linear system gives
  \begin{itemize}
  \item $\boxed{x_2 = t}$\\
  \item $-4x_1 + 3x_2 = 0 \implies -4x_1 + 3t = 0 \implies \boxed{x_1 = \frac{3}{4}t}$
  \end{itemize}
  Thus the eigenvectors are $\lvec{x}_1=\begin{bmatrix}
  \frac{3}{4}t\\t
\end{bmatrix} = t \begin{bmatrix}
  \frac{3}{4}\\1  
\end{bmatrix} = \frac{1}{4}t \begin{bmatrix}
  3\\4
  \end{bmatrix}$
  and $\{(3,4)\}$ is a basis for the $\lambda=6$\\[-.5em] eigenspace.
\item[$\bm{\lambda_2 = -1}$\,:]
  \begin{align*}(A - (-1)I)\lvec{x}_2 &= \lvec{0}\\
   (A+I)\lvec{x}_2 &= \lvec{0}\end{align*}
  \[\begin{bmatrix}[cc|c]3&3&0\\4&4&0\end{bmatrix}\]
    \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[17.5em]{\rowop{R_1}{\frac{1}{3}R_1} \\[.2em] \rowop{R_2}{\frac{1}{4}R_2}}{\begin{bmatrix}[cc|c]
        1&1&0\\[.2em]1&1&0
  \end{bmatrix}}&\end{flalign*}
    \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[17.5em]{\\ \rowop{R_2}{R_2 - R_1}}{\begin{bmatrix}
  [cc|c] \circled{1}&1&0\\0&0&0
    \end{bmatrix}}&\end{flalign*}
  \begin{itemize}
  \item $\boxed{x_2= t}$
  \item $x_1 + x_2 = 0 \implies x_1 + t=0 \implies \boxed{x_1 = -t}$
  \end{itemize}
  The eigenvectors are $\lvec{x}_2 = \begin{bmatrix*}[r]
    -t\\t
  \end{bmatrix*} = t \begin{bmatrix*}[r]
    -1\\1
  \end{bmatrix*}$
  and $\{(-1,2)\}$ is a basis for the $\lambda=-1$ eigenspace.
\end{description}
\end{example}

\textbf{Note:}
\begin{itemize}
\item Since the eigenvalues of a square matrix $A$ are the solutions of the corresponding characteristic equation (i.e. roots or zeroes of the characteristic polynomial), it follows by the Fundamental Theorem of Algebra that an $n\times n$ matrix $A$ has at least one eigenvalue and at most $n$ numerically different eigenvalues.  (So a $3\times3$ matrix has 1, 2, or 3 eigenvalues).  However these eigenvalues may not all be real-valued.
\item Since the eigenvectors corresponding to $\lambda$ are solutions of
\[(A-\lambda I)\lvec{x}=\lvec{0}\]
the eigenspace of $A$ corresponding to eigenvalue $\lambda$ is the null space of the matrix $B=A-\lambda I$\,. i.e. the solutions of $B\lvec{x}=\lvec{0}$\,.  The basic solution eigenvectors will span the eigenspace.  So in Example~\ref{exam:eigenvector} $\{(3,4)\}$ is a basis for the $\lambda=6$ eigenspace and $\{(-1,1)\}$ is a basis for the $\lambda=-1$ eigenspace.
\end{itemize}

\begin{example}
\label{example:eigenspacesdistinct}
Find the eigenvalues and bases for the eigenspaces of the given matrix.
\[A = \begin{bmatrix}
5&-7&7\\4&-3&4\\4&-1&2
\end{bmatrix}\]
Solution:\\
First find the eigenvalues:
\begin{align*}
  \det(A-\lambda I) = 0\\
  &\Rightarrow \begin{vmatrix}
  5-\lambda & -7 & 7\\4&-3-\lambda & 4\\ 4&-1 & 2-\lambda
  \end{vmatrix} = 0\\
  &\Rightarrow (5-\lambda)[(-3-\lambda)(2-\lambda)+4] + 7[4(2-\lambda)-16] + 7[-4-4(-3-\lambda)] = 0\\
  &\Rightarrow (5-\lambda)[\lambda^2+\lambda-2] + 7[-4\lambda-8] + 7[4\lambda+8] = 0\\
  &\Rightarrow 5\lambda^2+5\lambda-10-\lambda^3-\lambda^2+2\lambda=0  ~~(\text{Simplify and multiply both sides by $-1$\,.})\\
  &\Rightarrow \lambda^3 - 4\lambda^2 - 7\lambda + 10 = 0 ~~(\text{Note $\lambda=1$ solves this. Group to get $(\lambda-1)$ factor.})\\
  &\Rightarrow \lambda^3 - 4\lambda^2 - 7\lambda + 7 + 4 -1 = 0\\
  &\Rightarrow (\lambda^3-1) - 4(\lambda^2-1) - 7(\lambda -1) = 0 ~~(\text{Use}~  a^3 - b^3 = (a-b)(a^2+ab+b^2)\,.)\\
  &\Rightarrow(\lambda-1)(\lambda^2 + \lambda + 1) - 4(\lambda +1)(\lambda-1) - 7(\lambda-1) = 0\\
  &\Rightarrow (\lambda-1)[\lambda^2 + \lambda + 1 -4(\lambda+1) - 7]=0\\
  &\Rightarrow (\lambda-1)(\lambda^2 -3\lambda -10)=0\\
  &\Rightarrow (\lambda-1)(\lambda-5)(\lambda+2) = 0\\
  &\Rightarrow \begin{cases}
    \lambda_1 = 1\\
    \lambda_2 = 5\\
    \lambda_3 = -2\\
  \end{cases}
\end{align*}

\begin{description}
\item[$\bm{\lambda_1 = 1}$\,:]
  \[(A-(1)I)\lvec{x}_1 = \lvec{0}\]
  \[\begin{bmatrix}[ccc|c]
  4&-7&7&0\\4&-4&4&0\\4&-1&1&0
  \end{bmatrix}\]
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[16.5em]{ \\ \rowop{R_2}{R_2 - R_1}\\ \rowop{R_3}{R_3 - R_1}}{\begin{bmatrix}
  [rrr|r] 4&-7&7&0\\0&3&-3&0\\0&6&-6&0
  \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[16.5em]{\\ \\ \rowop{R_3}{R_3 - 2R_2}}{\begin{bmatrix}
  [ccr|r] \circled{4}&-7&7&0\\0&\circled{3}&-3&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
  \begin{itemize}
  \item $\boxed{x_3 = t}$
  \item $3x_2 - 3x_3 = 0 \implies 3x_2-3t=0 \implies \boxed{x_2=t}$
  \item $4x_1 - 7x_2 + 7x_3 = 0 \implies 4x_1 - 7t + 7t = 0 \implies \boxed{x_1=0}$
  \end{itemize}
Therefore $\lvec{x}_1= \begin{bmatrix}
    0\\t\\t
  \end{bmatrix} = t \begin{bmatrix}
    0\\1\\1
  \end{bmatrix}$
and so $\{(0,1,1)\}$ is a basis for the $\lambda=1$ eigenspace.

\item[$\bm{\lambda_2 = 5}$\,:]
\[(A-5I)\lvec{x}_2 = \lvec{0}\]
\[\begin{bmatrix}
  [rrr|r] 0&-7&7&0\\4&-8&4&0\\4&-1&-3&0
  \end{bmatrix}\]
  \[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.5em]{\rowswap{R_1}{R_3}\\ \\ }{\begin{bmatrix}[rrr|r]
      4&-1&-3&0\\4&-8&4&0\\0&-7&7&0\\
  \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.5em]{\\ \rowop{R_2}{R_2 - R_1}\\ }{\begin{bmatrix}[rrr|r]
      4&-1&-3&0\\0&-7&7&0\\0&-7&7&0
  \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.5em]{\\ \\ \rowop{R_3}{R_3 - R_2}}{\begin{bmatrix}[ccr|r]
      \circled{4}&-1&-3&0\\0&\circled{-7}&7&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
  \item $\boxed{x_3 = t}$
  \item $-7x_2 + 7x_3 = 0 \implies -7x_2 + 7t = 0 \implies  \boxed{x_2= t}$
  \item $4x_1 - x_2 - 3x_3 = 0 \implies  4x_1 - t - 3t = 0 \implies \boxed{x_1=t}$
\end{itemize}
Therefore $\lvec{x}_2 = \begin{bmatrix}
    t\\t\\t
  \end{bmatrix} = t \begin{bmatrix}
    1\\1\\1
  \end{bmatrix}$
and so $\{(1,1,1)\}$ is a basis for the $\lambda=5$ eigenspace.

\item[$\bm{\lambda_3 = -2}$\,:]
\begin{align*}(A-(-2)I)\lvec{x}_3 &= \lvec{0}\\
  (A+2I)\lvec{x}_3 &= \lvec{0}
  \end{align*}
\[\begin{bmatrix}[rrr|r]
7&-7&7&0\\4&-1&4&0\\4&-1&4&0
  \end{bmatrix}\]
  \[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.5em]{\rowop{R_1}{\frac{1}{7}R_1}\\ \\ \rowop{R_3}{R_3 - R_2}}{\begin{bmatrix}[rrr|r]
      1&-1&1&0\\4&-1&4&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.5em]{\\ \rowop{R_2}{R_2 - 4R_1}\\ }{\begin{bmatrix}[ccr|r]
      \circled{1}&-1&1&0\\0&\circled{3}&0&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
  \item $\boxed{x_3 = t}$
  \item $3x_2 = 0 \implies \boxed{x_2= 0}$
  \item $x_1 - x_2 + x_3 = 0 \implies x_1 - 0 + t = 0 \implies \boxed{x_1 = -t}$
\end{itemize}
Therefore $\lvec{x}_3 = \begin{bmatrix*}[r]
    -t\\0\\t
  \end{bmatrix*} = t \begin{bmatrix*}[r]
    -1\\0\\1  
  \end{bmatrix*}$ and so $\{(-1,0,1)\}$ is a basis for the $\lambda=-2$ eigenspace.
\end{description}
\end{example}

It can occur that the characteristic polynomial $P_A(\lambda)$ contains a particular eigenvalue $\lambda$ more than once as a root. Additionally the corresponding eigenspace of $\lambda$ may have a dimension greater than one.

\definition{The \textbf{algebraic multiplicity} $M$ of an eigenvalue $\lambda$ of square matrix $A$ is the number of times it appears as a root of the characteristic polynomial $P_A(\lambda)$.  The \textbf{geometric multiplicity} $m$ of eigenvalue $\lambda$ of $A$ is the number of linearly independent eigenvectors corresponding to $\lambda$, i.e. the dimension of the eigenspace of $\lambda$.}

The two multiplicities are related as follows.

\theorem{If $\lambda$ is an eigenvalue of $A$ with algebraic multiplicity $M$ then the number $m$ of linearly independent eigenvectors associated with $\lambda$ (its geometric multiplicity) satisfies
\[1\le m\le M\,.\]
}

\textbook{\newpage}
\begin{example}\label{example:eigenspacesmultiplicity}

Find the eigenvalues and eigenvectors of the given matrix. Identify the algebraic and geometric multiplicities of each eigenvalue.
\[A = \begin{bmatrix*}[r]
2&-5&5\\0&3&-1\\0&-1&3
\end{bmatrix*}\]
Solution:
\begin{align*}
\det (A-\lambda I)= 0 \\
&\Rightarrow \begin{vmatrix}
  2-\lambda&-5&5\\0&3-\lambda&-1\\0&-1&3-\lambda
  \end{vmatrix}=0\\
  &\Rightarrow (2-\lambda)\left[(3-\lambda)^2-1\right]=0\\
  &\Rightarrow(2-\lambda)(9-6\lambda+\lambda^2-1)=0\\
  &\Rightarrow(2-\lambda)(\lambda^2-6\lambda+8)=0~~(\text{Multiply both sides by -1 and factor.})\\ 
  &\Rightarrow(\lambda-4)(\lambda-2)(\lambda-2)=0\\
  &\Rightarrow\begin{cases}\lambda_1=4,\  M_1=1 \\
  \lambda_2=2,\  M_2=2
  \end{cases}
\end{align*}
Next find the corresponding eigenvectors.
\begin{description}
\item[$\bm{\lambda_1 = 4}$\,:]
\[(A-4I)\lvec{x}_1 = \lvec{0}\]
\[\begin{bmatrix}[rrr|r]
-2&-5&5&0\\0&-1&-1&0\\0&-1&-1&0
\end{bmatrix}\]
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[15.5em]{ \\ \\ \rowop{R_3}{R_3 - R_2}}{\begin{bmatrix}[ccr|r]
      \circled{-2}&-5&5&0\\0&\circled{-1}&-1&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
  \item $\boxed{x_3= t}$
  \item $-x_2 - x_3 = 0 \implies -x_2-t=0 \implies \boxed{x_2=-t}$
  \item $-2x_1 - 5x_2 + 5x_3 = 0 \implies -2x_1 - 5(-t) + 5t =0 \implies \boxed{x_1 = 5t}$
\end{itemize}
Therefore $\lvec{x}_1 = \begin{bmatrix*}[r]
  5t\\-t\\t
\end{bmatrix*} = t \begin{bmatrix*}[r]
  5\\-1\\1
\end{bmatrix*}$, so $\{(5,-1,1)\}$ is a basis for the $\lambda=4$ eigenspace and the geometric multiplicity of $\lambda_1=4$ is $m_1=1$\,.

\item[$\bm{\lambda_2 = 2}$\,:]
\[(A-2I)\lvec{x}_2=\lvec{0}\]
\[\begin{bmatrix}[rrr|r]
0&-5&5&0\\0&1&-1&0\\0&-1&1&0
\end{bmatrix}\]
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[15.9em]{\rowop{R_1}{-\frac{1}{5}R_1} \\ \\ }{\begin{bmatrix}[rrr|r]
      0&1&-1&0\\0&1&-1&0\\0&-1&1&0
  \end{bmatrix}}&\end{flalign*}
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[15.9em]{\\ \rowop{R_2}{R_2-R_1}\\ \rowop{R_3}{R_3 + R_1}}{\begin{bmatrix}[rcr|r] 0&\circled{1}&-1&0\\0&0&0&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
\item $\boxed{x_1 = t}$
\item $\boxed{x_3 = s}$
\item $x_2 - x_3 = 0 \implies x_2-s=0 \implies \boxed{x_2=s}$
\end{itemize}
Therefore $\lvec{x}_2 = \begin{bmatrix}
  t\\s\\s
\end{bmatrix} = t \begin{bmatrix}
  1\\0\\0
\end{bmatrix} + s \begin{bmatrix}
  0\\1\\1
\end{bmatrix}$, so $\{(1,0,0),(0,1,1)\}$ is a basis (check linear independence) for the $\lambda=2$ eigenspace and the geometric multiplicity of $\lambda_2=2$ is $m_2 = 2$\,.

In this example the geometric multiplicity equalled the algebraic multiplicity for both eigenvalues.
\end{description}
\end{example}

\section{Linear Independence of Eigenvectors}

The following can be proved by induction.
\theorem{If $B=\{\lvec{x}_1,\lvec{x}_2,\ldots,\lvec{x}_k\}$ is a set of eigenvectors of $n\times n$ matrix $A$ corresponding to distinct eigenvalues (so $\lambda_i\neq\lambda_j$ for $i\neq j$) then $B$ is linearly independent.}

% See my handwritten proof.

As a special case when all the eigenvalues of $A$ are distinct we have the following.

\textbf{Corollary:} Let $A$ be a $n \times n$ matrix. If $A$ has $n$ distinct eigenvalues, then $A$ has a set of $n$ linearly independent eigenvectors.

A more general result than the corollary is the following, proved similarly to the original theorem.

\theorem{Let $A$ be an $n\times n$ matrix.  If the geometric multiplicity $m$ of each eigenvalue of $A$ equals its algebraic multiplicity $M$ then $A$ has a set of $n$ linearly independent eigenvectors.}

% See my handwritten proof.

\begin{example}
In Example~\ref{example:eigenspacesdistinct} we found the matrix 
\[A = \begin{bmatrix}
5&-7&7\\4&-3&4\\4&-1&2
\end{bmatrix}\]
had distinct eigenvalues $\lambda_1=1$, $\lambda_2=5$, and $\lambda_3=-1$ with corresponding eigenvectors $\lvec{v}_1=(0,1,1)$, $\lvec{v}_2=(1,1,1)$, and $\lvec{v}_3=(-1,0,1)$.  Evaluating the following determinant along the first row
\[\begin{vmatrix*}[r] 0 & 1 & -1 \\ 1 & 1 & 0 \\ 1 & 1 & 1 \end{vmatrix*}=0+1(-1)(1-0)+(-1)(+1)(1-1)=-1\neq 0\]
shows the set $\{\lvec{v}_1,\lvec{v}_2,\lvec{v}_3\}$ is linearly independent as expected from the corollary.
\end{example}

\begin{comment}
  % This example was left out due to its length and for want of time.
\begin{example}
  Verify the above two properties for $A = \begin{bmatrix}
  3&-1&-1\\-12&0&5\\4&-2&-1
  \end{bmatrix}$

  \begin{align*}\det(A-\lambda I) &= 0\\
  \begin{vmatrix}
    3-\lambda & -1 & -1\\ -12 & -\lambda & 5\\ 4&-2&-1-\lambda
    \end{vmatrix} &= 0\\
    (3-\lambda)[-\lambda(-1-\lambda)+10] +(1)[-2(-1-\lambda)-20]+(-1)[24+4\lambda] &= 0\\
    (\lambda^2 - 1)(\lambda - 2) &= 0\\
    (\lambda+1)(\lambda-1)(\lambda-2) &= 0\\
   &\begin{cases}
      \lambda_1 = 1\\
      \lambda_2 = -1\\
      \lambda_3 = 2\\
    \end{cases}
  \end{align*}

  \underline{$\lambda_1 = 1$}
  \[(A-I)\lvec{x} = 0\]
  \[\begin{bmatrix}[rrr|r]
  2&-1&-1&0\\-12&-1&5&0\\4&-2&-2&0
  \end{bmatrix}\]
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[15em]{ \\ \rowop{R_2}{R_2 + 6R_1} \\ \rowop{R_3}{R_3 - 2R_1}}{\begin{bmatrix}[rrr|r]
        2&-1&-1&0\\0&-7&-1&0\\0&0&0&0
    \end{bmatrix}}&\end{flalign*}
  \begin{align*}
    Let \, \, x_3 &= t\\
    -7x_2 - x_3 &= 0\\
    x_2 &= -\frac{1}{7}t\\
    2x_1-x_2-x_3 &= 0\\
    2x_1 + \frac{1}{7}t - t &= 0\\
    2x_1 &= \frac{6}{7}t\\
    x_1 &= \frac{3}{7}t\\
    \therefore \lvec{x}_1 &= \begin{bmatrix}
      \frac{3}{7}t\\-\frac{1}{7}t\\t
    \end{bmatrix} = \frac{1}{7}t \begin{bmatrix}
      3\\-1\\7
    \end{bmatrix}
  \end{align*}

  \underline{$\lambda_2 = -1$}
  \[(A+I)\lvec{x} = 0\]
  \[\begin{bmatrix}[rrr|r]
  4&-1&-1&0\\-12&1&5&0\\4&-2&0&0
  \end{bmatrix}\]
  \begin{flalign*}&\rowopmatrix[15em]{\\ \rowop{R_2}{R_2+3R_1}\\ \rowop{R_3}{R_3 - R_1}}{\begin{bmatrix}[rrr|r]
        4&-1&-1&0\\0&-2&2&0\\0&-1&1&0
    \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[15em]{ \\ \rowop{R_2}{\frac{1}{2}R_2}\\ }{\begin{bmatrix}[rrr|r]
        4&-1&-1&0\\0&-1&1&0\\0&-1&1&0
    \end{bmatrix}}&\end{flalign*}
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[15em]{ \\ \\ \rowop{R_3}{R_3 - R_2}}{\begin{bmatrix}[rrr|r]
        4&-1&-1&0\\0&-1&1&0\\0&0&0&0
    \end{bmatrix}}&\end{flalign*}
  \begin{align*}
    Let \, \, x_3 &= t\\
    -x_2 + x_3 &= 0\\
    x_2 &= x_3\\
    x_2 &= t\\
    4x_1-x_2-x_3 &= 0\\
    4x_1 - 2t &= 0\\
    x_1 &= \frac{1}{2}t\\
    \therefore \lvec{x}_2 &= \begin{bmatrix}
      \frac{1}{2}t\\t\\t\\
    \end{bmatrix} = \frac{1}{2}t \begin{bmatrix}
      1\\2\\2
    \end{bmatrix}
  \end{align*}  

\underline{$\lambda_3 = 2$}

\[(A-2I)\lvec{x} = 0\]
\[\begin{bmatrix}[rrr|r]
1&-1&-1&0\\-12&-2&5&0\\4&-2&-3&0
\end{bmatrix}\]
\begin{flalign*}&\rowopmatrix[15em]{ \\ \rowop{R_2}{R_2 + 12R_1}\\ \rowop{R_3}{R_3 - 4R_1}}{\begin{bmatrix}[rrr|r]
      1&-1&-1&0\\0&-14&-7&0\\0&2&1&0
  \end{bmatrix}}&\end{flalign*}
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[15em]{ \\ \rowswap{R_2}{R_3}\\ }{\begin{bmatrix}[rrr|r]
      1&-1&-1&0\\0&2&1&0\\0&-14&-7&0
  \end{bmatrix}}&\end{flalign*}
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[15em]{ \\ \\ \rowop{R_3}{R_3 + 7R_2}}{\begin{bmatrix}[rrr|r]
      1&-1&-1&0\\0&2&1&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{align*}
  Let \, \, x_3 &= t\\
  2x_2 + x_3 &= 0\\
  2x_2 &= -t\\
  x_2 &= -\frac{1}{2}t\\
  x_1-x_2-x_3 &= 0\\
  x_1 &= -\frac{1}{2}t + t\\
  x_1 &= \frac{1}{2}t\\
  \therefore \lvec{x}_3 &= \begin{bmatrix}
    \frac{1}{2}t\\-\frac{1}{2}t\\t\\
  \end{bmatrix} = \frac{1}{2}t \begin{bmatrix}
    1\\-1\\2
  \end{bmatrix}
\end{align*}

Form the matrix $\begin{bmatrix}
  \lvec{x}_1 & \lvec{x}_2 & \lvec{x}_3
\end{bmatrix}$ to check for linear independence.

\begin{align*}
  &\Rightarrow \begin{bmatrix}
  3&1&1\\-1&2&-1\\7&2&2
  \end{bmatrix}\\
  &\Rightarrow \begin{vmatrix}
  3&1&1\\-1&2&-1\\7&2&2
  \end{vmatrix}\\
  &= 3(4+2) - (-2+7) + (-2-14)\\
  &= 18-5-16\\
  &= -3 \neq 0
\end{align*}
$\therefore$ the set $\{\lvec{x}_1, \lvec{x}_2, \lvec{x}_3\}$ is linearly independent since $\det \neq 0$.\\
\end{example}
\end{comment}

\section{Diagonalization}

\subsection{Properties of Diagonal Matrices}

Recall that an $n\times n$ matrix $D = [d_{ij}]$ is called diagonal if every entry not on the main diagonal is zero, i.e. if $d_{ij} = 0$ whenever $i\neq j$\,.  In general an $n\times n$ diagonal matrix has the form:
\[D = \begin{bmatrix}
d_1 & 0 &\cdots & 0 \\ 0& d_2 & \cdots & 0\\ \vdots & \vdots & \ddots & \vdots\\ 0&0&\cdots&d_n
\end{bmatrix}\,.\]
Note that some of the $d_1, d_2, \cdots , d_n$ values may be equal to zero.

\begin{example}
The following matrices are diagonal.
\[A = \begin{bmatrix*}[r]
    2&0&0\\0&-1&0\\0&0&5
    \end{bmatrix*}~~~~~~~~~~~~~~
    D = \begin{bmatrix}
    4&0&0&0\\0&1&0&0\\0&0&0&0\\0&0&0&5
    \end{bmatrix}\]
\end{example}

Diagonal matrices have convenient properties.
\theorem{Let $D = \begin{bmatrix}
d_1 & 0 & \cdots & 0\\ 0&d_2&\cdots&0\\ \vdots & \vdots & \ddots & \vdots\\0&0&\cdots&d_n
\end{bmatrix}$ and $W = \begin{bmatrix}
w_1 & 0 & \cdots & 0\\ 0&w_2&\cdots&0\\ \vdots & \vdots & \ddots & \vdots\\0&0&\cdots&w_n
\end{bmatrix}$ be $n\times n$ diagonal matrices.

Then:
\begin{enumerate}
  \item $DW = WD = \begin{bmatrix}
    d_1w_1 & 0 & \cdots&0\\0&d_2w_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&d_nw_n
  \end{bmatrix}$
  \item $\det D = (d_1)(d_2)\cdots (d_n)$
  \item $D$ is nonsingular (invertible) if and only if each entry on the main diagonal is nonzero ($d_i\neq 0$) in which case
  \[D^{-1} = \begin{bmatrix}
  \frac{1}{d_1} & 0 &\cdots&0\\0&\frac{1}{d_2} &\cdots &0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\frac{1}{d_n}\end{bmatrix}\,.\]
  \item The eigenvalues of $D$ are its main diagonal entries: $d_1, d_2, \ldots, d_n$\,.
\end{enumerate}}

\textbook{\newpage}
\begin{example}
  Find the inverse of the given matrix.
  \begin{enumerate}
    \item $D = \begin{bmatrix}
    2&0&0&0\\0&-3&0&0\\0&0&4&0\\0&0&0&-1
    \end{bmatrix}$\\[.5em]
    Solution:\\
    Since $d_i\neq 0$ for $i=1,2,3,4$ then $D^{-1}$ exists with
    \[D^{-1} = \begin{bmatrix}
    \frac{1}{2}&0&0&0\\0&-\frac{1}{3}&0&0\\0&0&\frac{1}{4}&0\\0&0&0&-1
    \end{bmatrix}\]
    \item $D = \begin{bmatrix}
      -1&0&0\\0&4&0\\0&0&0
    \end{bmatrix}$\\[.5em]
    Solution:\\
    $d_3 = 0$ so  $D$ is noninvertible.
  \end{enumerate}
\end{example}

\subsection{Diagonalizable Matrices}

Most matrices are not diagonal, but some are related to diagonal matrices.

\definition{If $A$ and $B$ are $n\times n$ matrices then $A$ is \textbf{similar} to $B$ if there exits an invertible matrix $P$ satisfying
\[\boxed{P^{-1}AP=B}\,.\]}

A matrix $A$ will be diagonalizable if it is similar to a diagonal matrix.

\definition{Let $A$ be a $n\times n$ matrix. Then $A$ is \textbf{diagonizable} if there exists an $n\times n$ invertible matrix $P$ such that $P^{-1}AP$ is a diagonal matrix. When such a $P$ exists we say that $P$ \textbf{diagonalizes} $A$\,.}

\theorem{(Conditions for diagonalizability)\\
Let $A$ be an $n\times n$ matrix with eigenvalues $\lambda_1, \lambda_2, \ldots, \lambda_n$. Let $\lvec{v}_1, \lvec{v}_2, \ldots, \lvec{v}_n$ be eigenvectors of $A$ with $\lvec{v}_j$ associated with $\lambda_j$. Suppose that these eigenvectors are linearly independent and let $P$ be the $n\times n$ matrix having $\lvec{v}_j$ as its $j\textsuperscript{th}$ column, so $\boxed{P=[\lvec{v}_1\lvec{v}_2\cdots\lvec{v}_n]}$. Then $P$ is nonsingular (invertible) and
\[\boxed{P^{-1}AP = \begin{bmatrix}
\lambda_1 &0&\cdots&0\\0&\lambda_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\lambda_n
\end{bmatrix}=D}\]
The diagonal matrix $D$ has the eigenvalues of $A$ along its main diagonal in the same order as the eigenvectors are listed as columns of $P$.}
% I thought the following wasn't needed.
\begin{comment}
If $P = \begin{bmatrix}
\lvec{v_1} & \lvec{v_2} & \ldots & \lvec{v_n}
\end{bmatrix}$ then $D = \begin{bmatrix}
\lambda_1 & 0 &\ldots &0\\0&\lambda_2&\ldots&0\\\vdots&\vdots&\ddots&0\\0&0&0&\lambda_n
\end{bmatrix}$\\

If $P = \begin{bmatrix}
\lvec{v_3} & \lvec{v_2} & \lvec{v_1}&\ldots & \lvec{v_n}
\end{bmatrix}$ then $D = \begin{bmatrix}
\lambda_3 & 0 &\ldots&\ldots &0\\0&\lambda_2&\ldots&\ldots&0\\0&0&\lambda_1&\ldots&0\\
\vdots&\vdots&\vdots&\ddots&0\\0&0&0&0&\lambda_n
\end{bmatrix}$\\
\end{comment}

\textbook{\newpage}
\textbf{Remarks:}
\begin{enumerate}
  \item $P^{-1}$ exists because $\det P \neq 0$ since the eigenvectors are linearly independent.
  \item The eigenvalues of $A$ need not be distinct.  So $\lambda_1$, $\lambda_2$, \ldots, $\lambda_n$ in the theorem includes multiplicity.
  \item The necessary condition for $A$ to be diagonizable is that it have $n$ linearly independent eigenvectors.
  \item If $A$ has $n$ distinct eigenvalues or all the eigenvalues have equal algebraic and geometric multiplicity then $A$ will automatically have $n$ linearly independent eigenvectors and hence be diagonalizable. 
\end{enumerate}

\begin{example}
  Diagonalize the given matrix if possible.
\begin{enumerate}
\item  \[A = \begin{bmatrix}
  2&3\\4&3
\end{bmatrix}\]

Solution:\\
From Example~\ref{exam:eigenvector} matrix $A$ had two distinct eigenvalues $\lambda_1 = 6$, $\lambda_2 = -1$ with corresponding (linearly independent) eigenvectors $\lvec{v}_1=(3,4)$ and $\lvec{v}_2=(-1,1)$.  So $A$ is diagonalizable by
\[P=[\lvec{v}_1\lvec{v}_2]=\begin{bmatrix*}[r]3 & -1 \\ 4 & 1 \end{bmatrix*}\,.\]

Check:
\begin{align*}
P^{-1}AP&=\frac{1}{7}\begin{bmatrix*}[r]1 & 1 \\ -4 & 3\end{bmatrix*}\begin{bmatrix}2&3\\4&3\end{bmatrix}\begin{bmatrix*}[r]3 & -1 \\ 4 & 1 \end{bmatrix*}
=\frac{1}{7}\begin{bmatrix*}[r]1 & 1 \\ -4 & 3\end{bmatrix*}\begin{bmatrix*}[r]
  18&1\\24&-1
\end{bmatrix*}
=\frac{1}{7}\begin{bmatrix*}[r] 42 & 0 \\ 0 & -7 \end{bmatrix*}\\[.2em]
&=\begin{bmatrix*}[r] 6 & 0 \\ 0 & -1 \end{bmatrix*}
=\begin{bmatrix*}[r] \lambda_1 & 0 \\ 0 & \lambda_2 \end{bmatrix*}=D
\end{align*}

\item
\[A = \begin{bmatrix}
5&-7&7\\4&-3&4\\4&-1&2
\end{bmatrix}\]
Solution:\\
From Example~\ref{example:eigenspacesdistinct} matrix $A$ had three distinct eigenvalues $\lambda_1 = 1$, $\lambda_2 = 5$, and $\lambda_3=-2$ with corresponding (linearly independent) eigenvectors $\lvec{v}_1=(0,1,1)$, $\lvec{v}_2=(1,1,1)$, and $\lvec{v}_3=(-1,0,1)$.  So $A$ is diagonalizable by
\[P=[\lvec{v}_1\lvec{v}_2\lvec{v}_3]=\begin{bmatrix*}[r]0 & 1 & -1 \\ 1 & 1 & 0 \\1 & 1 & 1 \end{bmatrix*}\,.\]

Check:
\[P^{-1}AP=\begin{bmatrix*}[r]-1 & 2 & -1\\ 1 & -1 & 1\\ 0 & -1 & 1\end{bmatrix*}
\begin{bmatrix}
5&-7&7\\4&-3&4\\4&-1&2
\end{bmatrix} \begin{bmatrix*}[r]0 & 1 & -1 \\ 1 & 1 & 0 \\1 & 1 & 1 \end{bmatrix*}=\begin{bmatrix*}[r] 1 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & -2 \end{bmatrix*}
=\begin{bmatrix*}[r] \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3\end{bmatrix*}=D\]

\item
\[A = \begin{bmatrix*}[r]
2&-5&5\\0&3&-1\\0&-1&3
    \end{bmatrix*}\]
Solution:\\
From Example~\ref{example:eigenspacesmultiplicity} matrix $A$ had two eigenvalues but each had algebraic multiplicity equalling geometric multiplicity.  Then $\lambda_1 = 4$, $\lambda_2 = 2$, and $\lambda_3=2$ with corresponding (linearly independent) eigenvectors $\lvec{v}_1=(5,-1,1)$, $\lvec{v}_2=(1,0,0)$, and $\lvec{v}_3=(0,1,1)$.  So $A$ is diagonalizable by
\[P=[\lvec{v}_1\lvec{v}_2\lvec{v}_3]=\begin{bmatrix*}[r]5 & 1 & 0 \\ -1 & 0 & 1 \\1 & 0 & 1 \end{bmatrix*}\,.\]

Check:
\[P^{-1}AP=\frac{1}{2}\begin{bmatrix*}[r]0 & -1 & 1\\ 2 & 5 & -5\\ 0 & 1 & 1\end{bmatrix*}
\begin{bmatrix*}[r]
2&-5&5\\0&3&-1\\0&-1&3
    \end{bmatrix*}
    \begin{bmatrix*}[r]5 & 1 & 0 \\ -1 & 0 & 1 \\1 & 0 & 1 \end{bmatrix*}
=\begin{bmatrix*}[r] 4 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{bmatrix*}
=\begin{bmatrix*}[r] \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3\end{bmatrix*}=D\]
\end{enumerate}
\end{example}

Not every matrix is diagonalizable as shown in the next example.
\begin{example}
  Diagonalize the given matrix if possible.
\[A = \begin{bmatrix*}[r]
-2&0&1\\1&1&0\\0&0&-2
\end{bmatrix*}\]

Solution:
\begin{align*}
  \det(A-\lambda I) &= 0\\
  &\Rightarrow \begin{vmatrix}
  -2-\lambda &0&1\\1&1-\lambda&0\\0&0&-2-\lambda
  \end{vmatrix} = 0\\
&\Rightarrow(-2-\lambda)(1-\lambda)(-2-\lambda)=0\\
&\Rightarrow(1-\lambda)(\lambda+2)^2 =0\\
&\Rightarrow\begin{cases}
  \lambda_1 = 1,\ M_1=1\\
  \lambda_2 = -2,\ M_2=2\
\end{cases}
\end{align*}

\begin{description}
\item[$\bm{\lambda_1 = 1}$:]
\[(A-(1)I)\lvec{x}_1 = \lvec{0}\]
\[\begin{bmatrix}[rrr|r]-3&0&1&0\\1&0&0&0\\0&0&-3&0
\end{bmatrix}\]
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16em]{\rowswap{R_1}{R_2} \\ \\ }{\begin{bmatrix}[rrr|r]
      1&0&0&0\\-3&0&1&0\\0&0&-3&0
  \end{bmatrix}}&\end{flalign*}
\textbook{\newpage}
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16em]{ \\ \rowop{R_2}{R_2 +3R_1} \\ }{\begin{bmatrix}[rrr|r]
      1&0&0&0\\0&0&1&0\\0&0&-3&0
  \end{bmatrix}}&\end{flalign*}
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16em]{ \\ \\ \rowop{R_3}{R_3 + 3R_2}}{\begin{bmatrix}[crc|r]
      \circled{1}&0&0&0\\0&0&\circled{1}&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
\item $\boxed{x_2 = t}$
\item $\boxed{x_3 = 0}$
\item $\boxed{x_1 = 0}$
\end{itemize}
  Therefore $\lvec{x}_1 = \begin{bmatrix}
    0\\t\\0
  \end{bmatrix} = t \begin{bmatrix}
    0\\1\\0
  \end{bmatrix}$ and $\{(0,1,0)\}$ is a basis for the $\lambda=1$ eigenspace.

\item[$\bm{\lambda_2=-2}:$]
\begin{align*}(A-(-2)I)\lvec{x}_2 &= \lvec{0}\\
  (A+2I)\lvec{x}_2 &= \lvec{0}
\end{align*}
\[\begin{bmatrix}[rrr|r]
  0&0&1&0\\1&3&0&0\\0&0&0&0
  \end{bmatrix}\]
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.7em]{\rowswap{R_1}{R_2}\\ \\ }{\begin{bmatrix}[crc|r]
      \circled{1}&3&0&0\\0&0&\circled{1}&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}
\begin{itemize}
  \item $\boxed{x_2= t}$
  \item $\boxed{x_3= 0}$
  \item $x_1 + 3x_2 = 0 \implies x_1 +3t = 0 \implies \boxed{x_1 =-3t}$
  \end{itemize}
  Therefore $\lvec{x}_2 = \begin{bmatrix*}[r]
    -3t\\t\\0
  \end{bmatrix*} = t \begin{bmatrix*}[r]
    -3\\1\\0
  \end{bmatrix*}$ and $\{(-3,1,0)\}$ is a basis for the $\lambda=-2$ eigenspace.

  So geometric multiplicity is  $m_2 =1$. Since $m_2 = 1 < M_2 = 2$, $A$ is not diagonalizable.
\end{description}
\end{example}

% These examples were taken out because they would took too long to do
% from scratch.  They could be reintroduced.  The second triangular one is
% interesting.
\begin{comment}
\item $A = \begin{bmatrix}
  1&-1&-1\\1&3&1\\-3&1&-1
  \end{bmatrix}$
  \begin{align*}
    \det(A-\lambda I) &= 0\\
    \begin{vmatrix}
    1-\lambda&-1&-1\\1&3-\lambda&1\\-3&1&-1-\lambda
    \end{vmatrix} &= 0\\
    (1-\lambda)[(3-\lambda)(-1-\lambda)-1]-(-1)[-1-\lambda+3] +(-1)[1+3(3-\lambda)] &= 0\\
    (1-\lambda)[-3-3\lambda+\lambda+\lambda^2-1] + (2-\lambda)-[1+9-3\lambda] &= 0\\
    (1-\lambda)[-4-2\lambda+\lambda^2]+2-\lambda-10+3\lambda &= 0\\
    (1-\lambda)[-4-2\lambda+\lambda^2]-8+2\lambda&=0\\
    (\lambda-3)(\lambda-2)(\lambda+2) &= 0\\
    &\Rightarrow\begin{cases}
      \lambda_1 =3\\
      \lambda_2 =2\\
      \lambda_3 = -2\\
    \end{cases}\\
    D &= \begin{bmatrix}
      3&0&0\\0&2&0\\0&0&-2
    \end{bmatrix}
  \end{align*}

\underline{$\lambda_1 =3$}
\[
  (A-3I)\lvec{x} &= 0\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r] -2&-1&-1&0\\1&0&1&0\\-3&1&-4&0
  \end{bmatrix}\]
  \begin{flalign*}&\rowopmatrix[15em]{\rowswap{R_1}{R_2}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r] 1&0&1&0\\-2&-1&-1&0\\-3&1&-4&0
  \end{bmatrix}}&\end{flalign*}
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_2}{R_2+2R_1}\\
  \rowop{R_3}{R_3+3R_1}\\
  &\Rightarrow\begin{bmatrix}
  [rrr|r] 1&0&1&0\\0&-1&1&0\\0&1&-1&0
  \end{bmatrix}}&\end{flalign*}
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_3}{R_3 + R_2}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]1&0&1&0\\0&-1&1&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}\begin{align*}
  \lvec{x}_1 &= \begin{bmatrix}
    x_1\\x_2\\x_3
  \end{bmatrix}\\
  Let \, \, x_3 &= t\\
  -x_2+x_3&=0\\
  x_2&=x_3\\
  x_2&=t\\
  x_1+x_3&=0\\
  x_1&=-x_3\\
  x_1&=-t\\
  \therefore \lvec{x}_1 &= \begin{bmatrix}
    -t\\t\\t
  \end{bmatrix} = t \begin{bmatrix}
    -1\\1\\1
  \end{bmatrix}
\end{align*}
\underline{$\lambda_2 = 2$}
\[
  (A-2I)\lvec{x}&=0\\
  \begin{bmatrix}
  [rrr|r] -1&-1&-1&0\\1&1&1&0\\-3&1&-3&0  \end{bmatrix}&=0\\
  \Rightarrow \lvec{x}_2 &= t \begin{bmatrix}
    -1\\0\\1
  \end{bmatrix}
\end{align*}

\underline{$\lambda_3 = -2$}
\[
  (A+2I)\lvec{x}&=0\\
  \begin{bmatrix}
  [rrr|r]3&-1&-1&0\\1&5&1&0\\-3&1&1&0
  \end{bmatrix} &= 0\\
  \Rightarrow \lvec{x}_3 &= t \begin{bmatrix}
    1\\-1\\4
  \end{bmatrix}
\end{align*}

Let $P = \begin{bmatrix}
-1&-1&1\\1&0&-1\\1&1&4
\end{bmatrix} \, \, \, , \, \, \, D = \begin{bmatrix}
3&0&0\\0&2&0\\0&0&-2
\end{bmatrix}$\\

\underline{Check:} $P^{-1}AP = D$\\

Let $P = \begin{bmatrix}
1&-1&-1\\-1&1&0\\4&1&1
\end{bmatrix}$ and $D = \begin{bmatrix}
-2&0&0\\0&3&0\\0&0&2
\end{bmatrix}$

\item $A = \begin{bmatrix}
1&0&0\\0&1&2\\0&0&2
\end{bmatrix}$ (Upper triangular matrix)\\

\begin{centering}
  
By finding the determinant of the matrix we are given the $\lambda$ values of:

\end{centering}

\begin{align*}
  \lambda_1 &= 1\\
  \lambda_2 &= 1\\
  \lambda_3 &= 3\\
  \therefore \lambda_{1,2} &= 1 \, \, , \, \, M_\lambda = 2\\
  (A-1I)\lvec{x} &= 0\\
  &\Rightarrow \begin{bmatrix}
    [rrr|r]0&0&0&0\\0&0&2&0\\0&0&1&0
  \end{bmatrix}\\
  \rowswap{R_1}{R_3}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]0&0&1&0\\0&0&2&0\\0&0&0&0
  \end{bmatrix}\\
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_2}{R_2 - 2R_1}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]0&0&1&0\\0&0&0&0\\0&0&0&0
  \end{bmatrix}\\
  Let \, \, x_1 &= t\\ \, \, , \, \, x_2 = s\\
  x_3 &= 0\\
  \therefore \lvec{x}_1 &= \begin{bmatrix}
    t\\s\\0
  \end{bmatrix} = t \begin{bmatrix}
    1\\0\\0
  \end{bmatrix} + s \begin{bmatrix}
    0\\1\\0
  \end{bmatrix}\\
  Let \, \, s=0 , t=1 , \, \, then \, \, \lvec{x}_1 =  \begin{bmatrix}
    1\\0\\0
  \end{bmatrix}\\
   Let \, \, s=1 , t=0 , \, \, then \, \, \lvec{x}_1 =  \begin{bmatrix}
    0\\1\\0
  \end{bmatrix}\\
  \therefore m_\lambda &= M_\lambda = 2
  \end{align*}\\

  \underline{$\lambda_3 = 2$}
  \[
    (A-2I)\lvec{x} &= 0\\
    &\Rightarrow \begin{bmatrix}
    [rrr|r] -1&0&0&0\\0&-1&2&0\\0&0&0&0
    \end{bmatrix}\]
    \Rightarrow \lvec{x}_3 &= t \begin{bmatrix}
      0\\2\\1
    \end{bmatrix}\\
    \therefore P &= \begin{bmatrix}
    1&0&0\\0&1&2\\0&0&1
    \end{bmatrix}\\
    D &= \begin{bmatrix}
    1&0&0\\0&1&0\\0&0&2
    \end{bmatrix}
  \end{align*}
This can be checked by ensuring that $P^{-1}AP = D$.

\end{comment}

\textbook{\newpage}
\subsection{Applications of Diagonalization}

If an $n\times n$ matrix $A$ is diagonalizable, then:
\[P^{-1}AP = D\]
where $D$ is a diagonal matrix.  We can solve for $A$ in terms of $D$ by premultiplying by $P$ and postmultiplying by $P^{-1}$ on both sides to get:
\begin{align*}
P^{-1}AP &= D\\
P(P^{-1}AP)P^{-1} &= PDP^{-1}\\
  (PP^{-1})A(PP^{-1}) &= PDP^{-1}\\
  IAI &= PDP^{-1}\\
  A &= PDP^{-1}
\end{align*}

Recalling that the $m\textsuperscript{th}$ power of square matrix $A$ is just the product of $A$ multiplied $m$ times we have
\begin{align*}
  A^m &= \underbrace{AA\cdots A}_{m\ \text{times}}\\
  &= (PDP^{-1})(PDP^{-1})\cdots (PDP^{-1})\\
  &= PD(P^{-1}P)D(P^{-1}P)\cdots (P^{-1}P)DP^{-1}\\
  &= PDIDI\cdots DP^{-1}\\
  &= P\underbrace{DD\cdots D}_{m\ \text{times}}P^{-1}\\
  &= PD^mP^{-1}
\end{align*}
where
\[D^m=\begin{bmatrix}
  \lambda_1 & 0 &\cdots &0\\0&\lambda_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\lambda_n
  \end{bmatrix}^m= \begin{bmatrix}
  \lambda_1^m & 0 &\cdots &0\\0&\lambda_2^m&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&\lambda_n^m
\end{bmatrix}\,.\]
We observe that the same $P$ that diagonalizes $A$ also diagonalizes $A^m$ and furthermore that the eigenvalues of $A^m$ appearing in its diagonal matrix are $\lambda_i^m$. This is the case since if $\lvec{x}$ is an eigenvector of $A$ corresponding to eigenvalue $\lambda$ we have that $\lvec{x}$ is also an eigenvector of $A^m$ corresponding to eigenvalue $\lambda^m$ since
\[A^m\lvec{x}=A^{m-1}A\lvec{x}=A^{m-1}\lambda\lvec{x}=\lambda A^{m-1}\lvec{x}=\ldots=\lambda^m\lvec{x}\,.\]
More generally yet, the matrix $P$ will diagonalize any polynomial function of diagonalizable matrix $A$, $p(A)$.  It will have the same eigenvectors as $A$ and its eigenvalues will be $p(\lambda_i)$\,.

A direct application of the result $A^m=PD^mP^{-1}$ is the simplicity of taking large powers of a diagonalizable matrix.

\textbook{\newpage}
\begin{example}
  Find $A^{10}$ if $A=\begin{bmatrix*}[r]
  3&-2\\-4&1
  \end{bmatrix*}$\,.\\[.5em]
  Solution:
\begin{align*}
  \det (A-\lambda I) &= 0\\
  \begin{vmatrix}
  3-\lambda &-2\\-4&1-\lambda
  \end{vmatrix} &= 0\\
  (3-\lambda)(1-\lambda)-8 &= 0\\
  3-3\lambda-\lambda+\lambda^2-8 &= 0\\
  \lambda^2 -4\lambda - 5 &= 0\\
  (\lambda-5)(\lambda+1) &= 0\\
  \begin{cases}
    \lambda_1 &= 5\\
    \lambda_2 &= -1
  \end{cases}
\end{align*}

\begin{description}
\item[$\bm{\lambda_1 = 5}$:]
  \[(A-5I)\lvec{x}_1 = \lvec{0}\]
  \[\begin{bmatrix}
  [rr|r]-2&-2&0\\-4&-4&0
  \end{bmatrix}\]
  \[\Downarrow\]
  \begin{flalign*}&\rowopmatrix[16.7em]{ \rowop{R_1}{-\frac{1}{2}R_1}\\[.2em] \rowop{R_2}{R_2-2R_1}}{\begin{bmatrix}[cr|r]
        \circled{1}&1&0\\[.2em]0&0&0
    \end{bmatrix}}&\end{flalign*}
  \begin{itemize}
  \item $\boxed{x_2= t}$
  \item $x_1+x_2 = 0 \implies x_1 +t = 0 \implies \boxed{x_1= -t}$
  \end{itemize}  
  Therefore $\lvec{x}_1 = \begin{bmatrix*}[r]
    -t\\t
  \end{bmatrix*} = t \begin{bmatrix*}[r]
    -1\\1
  \end{bmatrix*}$ so $\{\lvec{v}_1\}=\{(-1,1)\}$ is a basis for the $\lambda=5$ eigenspace.

\item[$\bm{\lambda_2 = -1}$:]
\[(A+I)\lvec{x}_2 = \lvec{0}\]
\[\begin{bmatrix}[rr|r]
4&-2&0\\-4&2&0
\end{bmatrix}\]
\[\Downarrow\]
\begin{flalign*}&\rowopmatrix[16.7em]{ \rowop{R_1}{\frac{1}{4}R_1}\\[.2em] \rowop{R_2}{R_2 + R_1}}{\begin{bmatrix}[cr|r]
        \circled{1}&-\frac{1}{2}&0\\[.2em]0&0\,\,&0
      \end{bmatrix}}&\end{flalign*}
\begin{itemize}
\item $\boxed{x_2= t}$
\item $x_1 - \frac{1}{2}x_2= 0 \implies x_1 - \frac{1}{2}t= 0 \implies \boxed{x_1 = \frac{1}{2}t}$
 \end{itemize}
 Therefore $\lvec{x}_2 = \begin{bmatrix}
   \frac{1}{2}t\\t
 \end{bmatrix} = \frac{1}{2}t \begin{bmatrix}
   1\\2
 \end{bmatrix}$ so $\{\lvec{v}_2\}=\{(1,2)\}$ is a basis for the $\lambda=-1$ eigenspace.
\end{description}

Then $P = [\lvec{v}_1\lvec{v}_2]=\begin{bmatrix*}[r]
-1&1\\1&2
\end{bmatrix*}$ and $D = \begin{bmatrix}
5&0\\0&-1
\end{bmatrix}$.

We find $P^{-1} = \frac{1}{3}\begin{bmatrix*}[r]
-2&1\\1&1
\end{bmatrix*}$\,.

Finally we have
\begin{align*}
  A^m &= PD^mP^{-1}\\
  A^{10} &= PD^{10}P^{-1}\\
  &= \begin{bmatrix*}[r]
  -1&1\\1&2
  \end{bmatrix*}\begin{bmatrix}
  5^{10}&0\\0&(-1)^{10}
  \end{bmatrix} \frac{1}{3} \begin{bmatrix*}[r]
  -2&1\\1&1
  \end{bmatrix*}\\
  &= \frac{1}{3} \begin{bmatrix}
  -(5)^{10} & 1 \\ 5^{10} & 2
  \end{bmatrix}\begin{bmatrix*}[r]
  -2&1\\1&1
  \end{bmatrix*}\\
  &= \frac{1}{3}\begin{bmatrix}
  2(5^{10})+1 & -5^{10}+1 \\ -2(5^{10})+2 & 5^{10}+2
  \end{bmatrix}
\end{align*}
\end{example}


\section{Properties of Eigenvalues}

\definition{Let $A=[a_{ij}]$ be an $n\times n$ matrix. Then the \textbf{trace of matrix $\bm{A}$}, denoted by $\bm{\ltracef{A}}$ is the sum of the diagonal entries of $A$,
\[\boxed{\ltracef{A} = a_{11} + a_{22} + \ldots + a_{nn}}\,.\]}

\begin{example}
  Find the trace of $A = \begin{bmatrix*}[r]
  5&4&-3\\2&1&0\\-1&4&-2
  \end{bmatrix*}$
  
Solution:
  \[\ltracef{A} = 5+1+(-2) = 4\]
\end{example}

\theorem{Let $A_1$, $A_2$, \ldots $A_k$ be square matrices of order $n$.  Then the trace of the product, $\ltracef{A_1A_2\cdots A_k}$ is invariant under cyclic permutations,
\[\ltracef{A_1A_2\cdots A_k}=\ltracef{A_2A_3\cdots A_kA_1}=\ltracef{A_3A_4\cdots A_kA_1A_2}=\ldots\]}

\theorem{\label{thm:tracedet}Let $A$ be an $n\times n$ matrix with eigenvalues $\lambda_1 , \lambda_2 , \ldots , \lambda_n$, where here we are allowing for multiplicity. Then:\\
\begin{enumerate}
  \item $\det A = \lambda_1 \lambda_2 \cdots \lambda_n$
  \item $\ltracef{A} = \lambda_1 + \lambda_2 + \cdots + \lambda_n$
\end{enumerate}}
\textbf{Proof:}\\
We prove the theorem in the event that matrix $A$ is diagonalizable.  It is more generally true.\footnote{In the more general case it can be shown that any square matrix $A$ is similar to an upper triangular matrix with the eigenvalues of $A$ along the diagonal.  This is called the \textbf{Jordan normal form} of $A$.  Because the latter matrix has the same determinant and trace as the special case diagonal matrix, the proof above is essentially the same.} Let $A$ be diagonalized by matrix $P$ so $P^{-1}AP=D$ where $D$ is diagonal with eigenvalues $\lambda_i$ on the diagonal.  Then $A=PDP^{-1}$ and
\begin{enumerate}
  \item $\det A = \detf{PDP^{-1}}=\detf{PP^{-1}D}=\detf{ID}=\det D=\lambda_1 \lambda_2 \cdots \lambda_n$
  \item $\ltracef{A}=\ltracef{PDP^{-1}}=\ltracef{DP^{-1}P}=\ltracef{DI}=\ltracef{D} = \lambda_1 + \lambda_2 + \cdots + \lambda_n$
\end{enumerate}
where here we used that we can commute matrices under a determinant as well as the invariance of the trace under cyclic permutation of the matrices.


\begin{example}
  Verify the properties of Theorem~\ref{thm:tracedet} for the following matrix.
  \[A = \begin{bmatrix*}[r]
  2&-5&5\\0&3&-1\\0&-1&3
  \end{bmatrix*}\]

Solution:\\
From Example~\ref{example:eigenspacesmultiplicity} we had the following eigenvalues with the given algebraic multiplicities.
  \begin{align*}
    \lambda_1 &= 4,\  M_1=1 \\
    \lambda_2 &= 2,\  M_2=2\,.
  \end{align*}
  So $\lambda_1=4$, $\lambda_2=2$, $\lambda_3=2$ allowing for multiplicities.

Then using Theorem~\ref{thm:tracedet}
\begin{align*}  
    \det A&=\lambda_1 \lambda_2\lambda_3 = (4)(2)(2) = 16\\
    \ltracef{A}&=\lambda_1 + \lambda_2 + \lambda_3 = 4+2+2 = 8\,.     
\end{align*}     

Direct calculation using the matrix $A$ gives
\begin{align*}
\det A &= 2(+1)(9-1)+0+0=16\\
\ltracef{A} & = 2+3+3 = 8\,.
\end{align*}
in agreement with the theorem.
\end{example}

Theorem~\ref{thm:tracedet} can be used in reverse to find eigenvalues in limited cases.
\begin{example}
  Find the eigenvalues of a $2\times2$ matrix $A$ with $\det A = -4$ and $\ltracef{A} = -3$.

Solution:\\
  Let $\lambda_1$ and $\lambda_2$ be the eigenvalues of the matrix $A$.\\
  We know that:
  \begin{align*}
    \lambda_1 + \lambda_2 &= \ltracef{A}=-3\\
    \lambda_1\lambda_2 &= \det A= -4
  \end{align*}
  Therefore
\begin{align*}
  \lambda_2 &= -3-\lambda_1\\
  \Rightarrow \lambda_1 (-3-\lambda_1) &= -4\\
  -3\lambda_1 - \lambda_1^2 &= -4\\
  \lambda_1^2 + 3\lambda_1 -4 &= 0\\
  (\lambda_1 + 4)(\lambda_1-1) &= 0\,.
\end{align*}
So $\lambda_1=-4$ (in which case $\lambda_2 = -3-(-4) = 1$) or $\lambda_1=1$ (in which case $\lambda_2 = -3-1 = -4$).  Therefore the eigenvalues of $A$ are $\lambda = -4$ and $\lambda = 1$\,.
\end{example}

\section{Interpreting Eigenvalues and Eigenvectors}
We have seen in Section~\ref{sec:linearoperators} that a linear operator $L(\lvec{x})$ on $\mathset{R}^n$ has a square matrix representation so that $L(\lvec{x})=L_A(\lvec{x})=A\lvec{x}$.  We can cast the discussion of eigenvalues and eigenvectors into linear operator terminology as follows.
\definition{Let $L$ be a linear operator on $\mathset{R}^n$. A scalar $\lambda$ is said to be an \textbf{eigenvalue} of $L$ if there exists a nonzero vector $\lvec{x}$ in $\mathset{R}^n$ such that:
\[\boxed{L(\lvec{x}) = \lambda \lvec{x}}\,.\]
The nonzero vector $\lvec{x}$ is called an \textbf{eigenvector} of $L$ corresponding to the eigenvalue $\lambda$\,.}

Now consider a linear operator such as $S_{\lvec{i},1/2}$ on $\mathset{R}^3$.  This is a compression by a factor of one half along the direction of the $x$-axis.  What can we say about the eigenvalues and eigenvectors of this operator?  The eigenvalue problem geometrically asks us to find those nonzero vectors (eigenvectors) that, when operated upon by $L$, return a scalar multiple (the eigenvalue) times themselves.  Most vectors in $\mathset{R}^3$ will not be eigenvectors of $S_{\lvec{i},1/2}$ since their $x$-components will be compressed by a factor 1/2 while their orthogonal components will remain unchanged.  However if one considers a vector that lies only along the $x$-axis, such as $\lvec{u}=u_1\lvec{i}$ then $S_{\lvec{i},1/2}(\lvec{u})=\frac{1}{2}u_1\lvec{i}=\frac{1}{2}\lvec{u}$.  We thus conclude that such vectors are eigenvectors of $S_{\lvec{i},1/2}$ with eigenvalue $\lambda=1/2$.  Indeed the eigenspace corresponding to $\lambda=1/2$ will be the vectors directed along the $x$-axis (i.e. the line) with basis $\{\lvec{i}\}$.

Are there other eigenvalues and eigenvectors of $S_{\lvec{i},1/2}$?  Suppose a vector has zero component along the $x$-axis, so $\lvec{u}=u_2\lvec{j}+u_3\lvec{k}$.  Then this vector would be its own orthogonal projection and it would be unaffected by the compression. i.e. $S_{\lvec{i},1/2}(\lvec{u})=u_2\lvec{j}+u_3\lvec{k}=\lvec{u}=1\lvec{u}$.  In other words such a vector will be an eigenvector corresponding to the eigenvalue $\lambda=1$.  The eigenspace corresponding to $\lambda=1$ will just be these vectors lying in the $y$-$z$ plane, a basis of which is just $\{\lvec{j},\lvec{k}\}$.  As such we see that the eigenvalues and eigenvectors of a linear operator have important physical interpretations.  Moreover, cast in this linear operator form, we realize that, quite generally, they will be independent of choice of coordinates.  Any compression $S_{\lvec{n},k}$ will have an eigenvalue $\lambda=k$ corresponding to eigenspace with basis $\{\lvec{n}\}$  and eigenvalue $\lambda=1$ corresponding to the subspace of dimension $n-1$ of vectors in $\mathset{R}^n$ orthogonal to $\lvec{n}$.

\begin{example}
The matrix transformation $L_A$ on $\mathset{R}^3$ has
\[A=\frac{1}{5}\begin{bmatrix*}14 & 12 & 0 \\ 12 & 21 & 0 \\ 0 & 0 & 5\end{bmatrix*}\,.\]
If $A$ has eigenvalue $\lambda_1=1$ with eigenspace basis $\{(4,-3,0),(0,0,1)\}$ and eigenvalue $\lambda_2=6$ with eigenspace basis $\{(3,4,0)\}$, describe the linear operator $L_A$.

Solution:\\
Vectors parallel to $(3,4,0)$ are scaled by a factor of $\lambda_2=6$.  The eigenvector $(3,4,0)$ is orthogonal to vectors in the eigenspace of $\lambda_1=1$ since $(3,4,0)\cdot(4,-3,0)=0$ and $(3,4,0)\cdot(0,0,1)=0$.  Vectors in the eigenspace of $\lambda_1$ remain unchanged by $L_A$ since the eigenvalue is 1.  We conclude that $L_A$ is an expansion by factor $k=6$ along the direction of $(3,4,0)$.  A unit vector in that direction is
\[\lvec{n}=\frac{1}{\sqrt{3^2+4^2+0^2}}(3,4,0)=(3/5,4/5,0)\,.\]
Thus $L_A=S_{\lvec{n},6}$\,.
\end{example}


\begin{example}
What eigenvalues and eigenvectors does the rotation operator $R_{\lvec{n},\theta}$ in $\mathset{R}^3$ have?

Solution:\\
The only vector that remains unchanged, short of becoming a scalar multiple of itself, is a vector directed along the axis of rotation, namely $\lvec{u}=u\lvec{n}$.  For such a vector we have
\[R_{\lvec{n},\theta}(\lvec{u})=u\lvec{n}=\lvec{u}=1\lvec{u}\,.\]
It follows that $\lambda=1$ is the eigenvalue corresponding to this eigenvector and that the eigenspace corresponding to $\lambda=1$ is the line directed along the axis of rotation with basis $\{\lvec{n}\}$\,.  A rotation matrix has additional complex-valued eigenvalues and eigenvectors.  While the latter do not represent physical vectors as they are not real-valued, they nevertheless provide useful information about the rotation as one may determine the plane of rotation from them.  The complex eigenvalues similarly determine the angle of rotation.  Complex eigenvalues and eigenvectors will be discussed in Section~\ref{subsec:complexeigenvalues}\,.
\end{example}

We note that for matrices arising from linear operators in physical problems, the eigenvalues and corresponding eigenvectors typically have physical meaning as in our geometrical examples.  In $\mathset{R}^3$ two observers working with the same linear operator but potentially in coordinate systems rotated with respect to each other would find the same eigenvalues for the operator despite the fact their matrix representations would be different.  Similarly they would find that the corresponding eigenvectors despite having different coordinates in their respective systems would represent the same vectors in physical space.  Due to this coordinate system independence, eigenvalues of operators arising from physical problems can represent observable properties and we expect them to show up in analysis of such problems.  Since by Theorem~\ref{thm:tracedet} the determinant and trace of a matrix depends only on its eigenvalues, these too are properties of the operator that are independent of the matrix representation used by each observer and thus typically have a physical meaning.

\section{Interpreting Diagonalization}
Continuing the discussion from the previous section we saw that the compression $S_{\lvec{i},1/2}$ had eigenvalue $\lambda_1=1/2$ with corresponding eigenvector $\lvec{i}$ and eigenvalue $\lambda_2=1$ with corresponding eigenvectors $\lvec{j}$ and $\lvec{k}$.  Since $S_{\lvec{i},1/2}(\lvec{i})=\frac{1}{2}\lvec{i}$ and $S_{\lvec{i},1/2}(\lvec{j})=\lvec{j}$ and $S_{\lvec{i},1/2}(\lvec{k})=\lvec{k}$ it follows that the operator equals $L_A$ where
\[A=\begin{bmatrix}\frac{1}{2} & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{bmatrix}=\begin{bmatrix}\lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3\end{bmatrix}\,.\]
If we had the general operator $S_{\lvec{n},k}$ and we chose our coordinate axes so that $\lvec{n}=\lvec{i}$ then the matrix $A$ would also be diagonal.  For a general choice of axes, as we saw in Section~\ref{subsec:symmetricoperators}, the matrix representation of $S_{\lvec{n},k}$ would be symmetric but not necessarily diagonal.

So if we are working with a linear operator $L$ represented by matrix $A$ in $\mathset{R}^3$ (or more generally $\mathset{R}^n$) one might wonder if there is some set of coordinate axes that we might have chosen in which the operator's matrix representation would be diagonal.  Questions of coordinates in physical space imply a notion of distance which, in $\mathset{R}^n$, implies the use of an inner product which we take to be our usual Euclidean dot product.  In that case the answer to the question is that such a system of coordinate axes exists if the matrix is orthogonally diagonalizable.
\definition{Square matrix $A$ is \textbf{orthogonally diagonalizable} if there exists an orthogonal matrix $P$ such that
\[P^{-1}AP=D\,,\]
where $D$ is a diagonal matrix.}
Recall that $P^{-1}=P^T$ for an orthogonal matrix so $P^TAP=D$.  The orthogonal matrix $P$ provides the information required to relate the coordinate axes in which the matrix representation is diagonal to the original axes.  As an orthogonal matrix it will be composed of reflections, rotations, and inversions.  The following theorem characterizes all orthogonally diagonalizable matrices.
\theorem{An $n\times n$ matrix $A$ is orthogonally diagonalizable if and only if it is symmetric.}

Many useful operators on $\mathset{R}^n$ are symmetric as we have seen and therefore are orthogonally diagonalizable.  Working in a coordinate system in which the operator is diagonal often simplifies calculations.  Such coordinate system transformations requires a more general discussion of vector spaces and their bases than will be provided at this juncture.\footnote{We note that the more general criteria of a matrix being merely diagonalizable only guarantees that new coordinate axes can be chosen in which the matrix representation is diagonal but these axes are no longer necessarily mutually orthogonal.  Lengths of vectors transformed into those coordinates would not, in general, be preserved.}

% The following was taken out as an example.  Fotini's handwritten notes
% use this to illustrate the orthogonal diagonalization of a symmetric matrix.
% This requires discussion of Gram-Schmidt process.  See other texts for
% the discussion of introduction of the dot product and how symmetric matrices
% can be diagonalized with orthogonal matrices.
\begin{comment}
\begin{example}
Diagonalize the symmetric matrix.
$A = \begin{bmatrix}
2&-2&1\\-2&-1&2\\1&2&2
\end{bmatrix}$ (A is symmetric, $A^T = A$)
\begin{align*}
  \det(A-\lambda I)  &= 0\\
  \begin{vmatrix}
  2-\lambda &-2&1\\-2&-1-\lambda&2\\1&2&2-\lambda
  \end{vmatrix} &= 0\\
  (2-\lambda)[(-1-\lambda)(2-\lambda)-4]-(-2)[-2(2-\lambda)-2]+(1)[-4-(-1-\lambda)] &= 0\\
  (\lambda-3)^2(\lambda+3) &= 0\\
      &\Rightarrow \begin{cases}
    \lambda_{1,2} = 3\\
    \lambda_3 = -3\\
  \end{cases}\\
  M_\lambda &= 2
\end{align*}

\underline{$\lambda_{1,2} = 3$}
\[
  (A-3I)\lvec{x} &= 0\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]-1&-2&1&0\\-2&-4&2&0\\1&2&-1&0
  \end{bmatrix}\]
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_2}{R_2 - 2R_1}\\
  \rowop{R_3}{R_3 + R_1}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]-1&-2&1&0\\0&0&0&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}\begin{align*}
  Let \, \, x_2 &= t \, \, , \, \, x_3 = s\\
  -x_1 - 2x_2 + x_3 &= -\\
  x_1 &= -2t + s\\
  \therefore \lvec{x}_1 &= \begin{bmatrix}
    -2t+s\\t\\s
  \end{bmatrix} = t \begin{bmatrix}
    -2\\1\\0
  \end{bmatrix} + s \begin{bmatrix}
    1\\0\\1
  \end{bmatrix}\\
  If \, \, s &= 0 \, \, , \, \, t=1 \, \, , \, \, \lvec{x}_1 = \begin{bmatrix}
    -2\\1\\0
  \end{bmatrix}\\
    If \, \, s &= 1 \, \, , \, \, t=0 \, \, , \, \, \lvec{x}_1 = \begin{bmatrix}
   1\\0\\1
  \end{bmatrix}\\
  \therefore m_\lambda &= 2
\end{align*}

\underline{$\lambda_3 = -3$}

\[
  (A+3I)\lvec{x} &= 0\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r] 5&-2&1&0\\-2&2&2&0\\1&2&5&0
  \end{bmatrix}\]
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_2}{-\frac{1}{2}R_2}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r] 5&-2&1&0\\-1&1&1&0\\1&2&5&0
  \end{bmatrix}}&\end{flalign*}
  \begin{flalign*}&\rowopmatrix[15em]{\rowswap{R_1}{R_2}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]1&-1&-1&0\\5&-2&1&0\\1&2&5&0
  \end{bmatrix}}&\end{flalign*}
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_2}{R_2 - 5R_1}\\
  \rowop{R_3}{R_3 - R_1}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]1&-1&-1&0\\0&3&6&0\\0&3&6&0
  \end{bmatrix}}&\end{flalign*}
  \begin{flalign*}&\rowopmatrix[15em]{\rowop{R_3}{R_3-R_2}\\
  &\Rightarrow \begin{bmatrix}
  [rrr|r]1&-1&-1&0\\0&3&6&0\\0&0&0&0
  \end{bmatrix}}&\end{flalign*}\begin{align*}
  Let \, \, x_3 &= t\\
  3x_2 + 6x_3 &= 0\\
  x_2 &= -2t\\
  x_1 - x_2 - x_3 &= 0\\
  x_1 &= x_2 + x_3\\
  x_1 &= -t\\
  \therefore \lvec{x}_3 &= \begin{bmatrix}
    -t\\-2t\\t
  \end{bmatrix} = t \begin{bmatrix}
    -1\\-2\\1
  \end{bmatrix}
\end{align*}

Then we can form $P = \begin{bmatrix}
-2&1&-1\\1&0&-2\\0&1&1
\end{bmatrix}$ and $D = \begin{bmatrix}
3&0&0\\0&3&0\\0&0&-3
\end{bmatrix}$
\end{example}
\end{comment}

%TODO
% Use row/column operations like in assignment to show how eigenvalue
% calculations can be simplified.
%
% See my proof in my notes for linear independence of eigenvalues, etc.
%
% Uncomment example in the linear independence of eigenvectors section or
% at least move it into assignment since it is largely typeset.
%
% Uncomment from-scratch example on diagonalization.
%
% Add material at end about symmetric matrices and orthogonal diagonalization
% if time.  Do the final commented example but with orthonormal vectors.
%
% Maybe use lower case p for polynomial function of a matrix back in the
% power of a matrix section to make it easier to use the same symbol here
% where we need $P$.  Also put a reference back to the power of a matrix
% section.
%
% Add the word Euclidean earlier in the book or check it is there.
%
% Rewrite the last section as it seems a bit circuitous.  Argue that the
% matrices A and D therefore have the same eigenvalues.
%
% Also should we mention that the eigenvectors are just the elementary vectors
% in the diagonalized space - likely.
%
