\section{A Survey of Linear Transformations}

The student will have encountered the idea of real-valued functions of real variables, such as $f(x)=\sin(x)$\,.  Similarly one can define real-valued functions of vectors in $\mathset{R}^n$.  The length of a vector,
\[f(\lvec{u})=\lnorm{\lvec{u}}\,,\]
would be such a function.   In this chapter we will go one step further and look at functions of vectors whose result is a vector, that is mappings from $\mathset{R}^n\to\mathset{R}^m$.  Such a function can be thought of transforming a vector into a new vector.

\definition{A \textbf{transformation} from $\mathset{R}^n$ to $\mathset{R}^m$, written $T:\mathset{R}^n\to\mathset{R}^m$, is a mapping that assigns a unique vector $\lvec{v}=T(\lvec{u})$ in $\mathset{R}^m$ to each vector $\lvec{u}$ in $\mathset{R}^n$.}

The important class of transformations under consideration here will be linear transformations.  We will see that these are ultimately representable in terms of matrices and their action on vectors with matrix multiplication.

\definition{A \textbf{linear transformation} is a transformation $L(\lvec{u})$ from $\mathset{R}^n$ to $\mathset{R}^m$ satisfying
  \leqnomode
  \begin{subequations}
    \renewcommand{\theequation}{\arabic{equation}}
  \begin{align}
    L(\lvec{u}+\lvec{v})&=L(\lvec{u})+L(\lvec{v}) \\
    L(c\lvec{u})&=cL(\lvec{u})
  \end{align}\end{subequations}\reqnomode
  for any vectors $\lvec{u}$ and $\lvec{v}$ in $\mathset{R}^n$ and scalar $c$\,.}

\begin{example}
  The \textbf{identity transformation}, which will be denoted by $\mathbbm{1}$\,, takes a vector $\lvec{u}$ in $\mathset{R}^n$ to itself and is thus defined by $\boxed{\mathbbm{1}(\lvec{u})=\lvec{u}}$\,.  It is a linear transformation from $\mathset{R}^n$ to $\mathset{R}^n$ since
  \begin{align*}
        \mathbbm{1}(\lvec{u}+\lvec{v})&=\lvec{u}+\lvec{v}=\mathbbm{1}(\lvec{u})+\mathbbm{1}(\lvec{v})\\
        \mathbbm{1}(c\lvec{u})&=c\lvec{u}=c\mathbbm{1}(\lvec{u})\,.
  \end{align*}
To more specifically identify the identity transformation from $\mathset{R}^n$ to $\mathset{R}^n$ we may write $\mathbbm{1}_n$\,.
\end{example}

\begin{example}
  The \textbf{zero transformation}, which will be denoted by $\mathset{O}$, takes any vector $\lvec{u}$ in $\mathset{R}^n$ to the zero vector in $\mathset{R}^m$, and is thus defined by $\boxed{\mathset{O}(\lvec{u})=\lvec{0}}$.  It is a linear transformation since
    \begin{align*}
        \mathset{O}(\lvec{u}+\lvec{v})&=\lvec{0}=\lvec{0}+\lvec{0}=\mathset{O}(\lvec{u})+\mathset{O}(\lvec{v})\\
        \mathset{O}(c\lvec{u})&=\lvec{0}=c\lvec{0}=c\,\mathset{O}(\lvec{u})\,.
  \end{align*}
  To be more specific we may write $\mathset{O}_{mn}$ to identify the zero transformation from $\mathset{R}^n$ to $\mathset{R}^m$\,.
\end{example}

A class of non-trivial linear transformations is as follows.
\textbook{\newpage}
\begin{example}
  Suppose $k$ is a scalar constant.  Define the \textbf{scalar transformation} $C_k$ from $\mathset{R}^n$ to $\mathset{R}^n$ by
  \[\boxed{C_k(\lvec{u})=k\lvec{u}}\,.\]
  Then $C_k$ is a linear transformation since
    \begin{align*}
        C_k(\lvec{u}+\lvec{v})&=k(\lvec{u}+\lvec{v})=k\lvec{u}+k\lvec{v}=C_k(\lvec{u})+C_k(\lvec{v})\\
        C_k(c\lvec{u})&=k(c\lvec{u})=c(k\lvec{u})=cC_k(\lvec{u})
    \end{align*}
For real-valued $k$ geometrically the effect of the transformation is as follows:
    \begin{description}
    \item[Case $\bm{1<k}$\,:] $C_k(\lvec{u})$ is a \textbf{dilation}.  The vector retains its direction but is lengthened by a factor of~$k$.
      \begin{center}
        \includegraphics[height=1.75in]{lineartransformations/dilation}
      \end{center}

    \item[Case $\bm{k=1}$\,:] $C_1(\lvec{u})=1\lvec{u}=\lvec{u}$ and we recover the identity transformation $\mathbbm{1}_n$.
  \begin{center}
    \includegraphics[height=1.75in]{lineartransformations/identity}
  \end{center}  
    \item[Case $\bm{0<k<1}$\,:] $C_k(\lvec{u})$ is a \textbf{contraction}.  The vector retains its direction but is shortened by a factor of~$k$.
      \begin{center}
        \includegraphics[height=1.75in]{lineartransformations/contraction}
      \end{center}
    \item[Case $\bm{k=0}$\,:] $C_0(\lvec{u})=0\lvec{u}=\lvec{0}$ and we recover the zero transformation $\mathset{O}_{nn}$.
    \item[Case $\bm{k=-1}$\,:] $C_{-1}(\lvec{u})=-\lvec{u}$ is \textbf{inversion through the origin $\bm{O}$}.
      The vector retains its length but points in the opposite direction.  We will denote inversion by $D$ so $\boxed{D(\lvec{u})=-\lvec{u}}$\,.
      \begin{center}
        \includegraphics[height=1.75in]{lineartransformations/inversion}
      \end{center}
      Inversion through the original is sometimes called \textbf{reflection through the origin} though we will avoid that terminology.
    \item[Case $\bm{-1<k<0}$\,:] A combination of inversion through the origin and contraction by $|k|$.
    \item[Case $\bm{k<-1}$\,:] A combination of inversion through the origin and dilation by $|k|$.
    \end{description}
\end{example}

One observes that the previous linear transformations written in terms of scalar multiplication are independent of choice of coordinate system (except for the choice of origin $O$ upon which they do depend).  Secondly one often considers the transformation as acting on the terminal points of the vectors themselves.

\begin{example}
  Let $\lvec{n}$ be a unit vector in $\mathset{R}^n$.  The \textbf{projection} onto $\lvec{n}$, denoted by $P_{\lvec{n}}$, and given by
  \[\boxed{P_{\lvec{n}}(\lvec{u})=\vectorproj{\lvec{n}}{\lvec{u}}=\frac{\lvec{u}\cdot\lvec{n}}{\lnorm{\lvec{n}}^2}\lvec{n}=(\lvec{u}\cdot\lvec{n})\lvec{n}}\]
  is a linear transformation from $\mathset{R}^n$ to $\mathset{R}^n$.  It is linear due to the linearity of the dot product:
  \begin{align*}
    P_{\lvec{n}}(\lvec{u}+\lvec{v})&=\vectorprojplain{\lvec{n}}{(\lvec{u}+\lvec{v})}=[(\lvec{u}+\lvec{v})\cdot\lvec{n}]\lvec{n}=(\lvec{u}\cdot\lvec{n}+\lvec{v}\cdot\lvec{n})\lvec{n}\\
    &=(\lvec{u}\cdot\lvec{n})\lvec{n}+(\lvec{v}\cdot\lvec{n})\lvec{n}=\vectorproj{n}{u}+\vectorproj{n}{\lvec{v}}\\
    &=P_{\lvec{n}}(\lvec{u})+P_{\lvec{n}}(\lvec{v})\\
    P_{\lvec{n}}(c\lvec{u})&=\vectorprojplain{\lvec{n}}{(c\lvec{u})}=[(c\lvec{u})\cdot\lvec{n}]\lvec{n}=c(\lvec{u}\cdot\lvec{n})\lvec{n}=c\,\vectorproj{n}{u}\\
    &=cP_{\lvec{n}}(\lvec{u})
  \end{align*}
\end{example}

\subsection{Sum and Scalar Product of Transformations}
Just as we define the sum of two real-valued functions $f+g$ to be the result when we add the action of each function separately, i.e. $(f+g)(x)=f(x)+g(x)$ we can define the sum of two transformations $S(\lvec{u})$ and $T(\lvec{u})$ by
\[(S+T)(\lvec{u})=S(\lvec{u})+T(\lvec{u})\,.\]
Similarly a scalar multiple $c$ times a transformation $T$ can be defined by multiplying the scalar times the result of acting $T$ on the vector,
\[(cT)(\lvec{u})=cT(\lvec{u})\,.\]
\theorem{Let $K$ and $L$ be linear transformations from $\mathset{R}^n$ to $\mathset{R}^m$ and $c$ a scalar then the transformations $K+L$ and $cL$ are themselves linear.}

More generally one can prove inductively that an arbitrary linear combination of a finite number of such linear transformations will be linear.
\begin{example}
  Suppose $S=\{\lvec{n}_1,\ \lvec{n}_2,\ \ldots,\ \lvec{n}_k\}$ is a set of unit vectors in $\mathset{R}^n$ that are mutually orthogonal. The \textbf{orthogonal projection} onto the span of these vectors will be denoted by $P_{\lvec{n}_1,\lvec{n}_2,\ldots,\lvec{n}_k}$\, and is given by
  \[\boxed{P_{\lvec{n}_1,\lvec{n}_2,\ldots,\lvec{n}_k}(\lvec{u})=\vectorprojplain{\lvec{n}_1}{\lvec{u}}+\vectorprojplain{\lvec{n}_2}{\lvec{u}}+\cdots+\vectorprojplain{\lvec{n}_k}{\lvec{u}}}\,.\]
  It is a linear transformation since it is a sum of linear transformations,
\[P_{\lvec{n}_1,\lvec{n}_2,\ldots,\lvec{n}_k}=P_{\lvec{n}_1}+P_{\lvec{n}_2}+\cdots+P_{\lvec{n}_k}\,.\]
 As a specific example, $P_{\lvec{i},\lvec{j}}$ in $\mathset{R}^3$ is the projection of $\lvec{u}$ onto the $x$-$y$ plane.
  \begin{center}
    \includegraphics[height=1.75in]{lineartransformations/projection}
  \end{center}
\end{example}

\begin{example}
We have seen how a vector $\lvec{u}$ can be broken into a vector component parallel to a given direction $\lvec{n}$ and a component orthogonal to it by
\[\lvec{u}=\lvec{u}_1+\lvec{u}_2=\vectorproj{n}{u}+(\lvec{u}-\vectorproj{n}{u})\]
If we transform the vector by multiplying only the parallel component $\lvec{u}_1$ by a constant scalar $k$ to get $k\lvec{u}_1+\lvec{u}_2$ we have the transformation from $\mathset{R}^n$ to $\mathset{R}^n$ given by
\[\boxed{S_{\lvec{n},k}(\lvec{u})=k\,\vectorproj{n}{u}+(\lvec{u}-\vectorproj{n}{u})=\lvec{u}+(k-1)\,\vectorproj{n}{u}}\]
The transformation is linear since it can be written as a linear combination of linear transformations:
    \[S_{\lvec{n},k}=\mathbbm{1}+(k-1)P_{\lvec{n}}\,.\]
    Important special cases are as follows.
    \textbook{\newpage}
    \begin{description}
      \item[Case $\bm{k>1}$\,:]
        The component along $\lvec{n}$ expands by a factor of $k$ and we call $S_{\lvec{n},k}$ an \textbf{expansion}.
    \begin{center}
      \includegraphics[width=3in]{lineartransformations/expansion2d}
    \end{center}
  \item[Case $\bm{k=1}$\,:] We see $S_{\lvec{n},1}=\mathbbm{1}$\,, the identity transformation.
\item[Case $\bm{0<k<1}$\,:] The component along $\lvec{n}$ is compressed by the factor $k$ and we call $S_{\lvec{n},k}$ a \textbf{compression}.
    \begin{center}
      \includegraphics[width=3in]{lineartransformations/compression2d}
    \end{center}
        If we imagine an image made up of points considered to be terminal points of their respective vectors from the origin then for $k$ positive $S_{\lvec{n},k}$ will stretch ($k>1$) or squash ($0<k<1$) the entire image along the direction of $\lvec{n}$ measured from the line (in $\mathset{R}^2$) or plane (in $\mathset{R}^3$) going through the origin orthogonal to $\lvec{n}$\,.
      \item[Case $\bm{k=0}$\,:] We see $S_{\lvec{n},0}=\mathbbm{1}-P_{\lvec{n}}$ which returns the component of $\lvec{u}$ orthogonal to $\lvec{n}$, i.e. $\lvec{u}_2$\,.
    \item[Case $\bm{k=-1}$\,:] The linear transformation $S_{\lvec{n},-1}$ which will be denoted $Q_{\lvec{n}}$, is given by
    \[\boxed{Q_{\lvec{n}}(\lvec{u})=-\vectorproj{n}{u}+(\lvec{u}-\vectorproj{n}{u})=\lvec{u}-2\,\vectorproj{n}{u}}\]
    and is called a \textbf{reflection}.
    In $\mathset{R}^2$ it represents a \textbf{reflection about the line} through the \textbf{origin} with normal $\lvec{n}$.  In $\mathset{R}^3$ it is a \textbf{reflection across the plane} through the \textbf{origin} with normal $\lvec{n}$.  It is called a reflection because a set of points (tips of their respective vectors) will be transformed to their mirror images across the line or plane.
  \begin{center}
    \includegraphics[width=2.5in]{lineartransformations/reflection2d}
  \end{center}
    \end{description}
\end{example}
Note that expansion/compression should be compared with dilation/contraction seen earlier and reflection should be compared with inversion about the origin.  In dilation/contraction and inversion the entire vector is modified, not simply a projection of it.

\begin{example}
  As an important example we consider rotation in the plane $\mathset{R}^2$ and more generally in $\mathset{R}^3$.  In the plane a counterclockwise rotation is determined by an angle $\theta$.  Such a transformation will be written $R_{\theta}(\lvec{u})$.
  \begin{center}
    \includegraphics[height=2in]{lineartransformations/rotation2d}
  \end{center}
  In three dimensions a rotation is determined by a direction, given by unit vector $\lvec{n}$, which determines the \textbf{axis of rotation} through the origin,  as well as the angle of rotation, $\theta$ about that axis.  The right-hand rule with thumb pointing in the direction of $\lvec{n}$ is used to determine the direction of rotation.  The projection of $\lvec{u}$ in the direction of $\lvec{n}$ is unaffected by the rotation.  The orthogonal component which lies in the plane through the origin perpendicular to $\lvec{n}$ is rotated.
  \begin{center}
    \includegraphics[height=2in]{lineartransformations/rotation3d}
  \end{center}
If we denote the three-dimensional rotation by $R_{\lvec{n},\theta}$ then it can be given explicitly in $\mathset{R}^3$ by \textbf{Rodrigues' rotation formula}:
\[\boxed{R_{\lvec{n},\theta}(\lvec{u})=(\cos\theta)\lvec {u} +(\sin\theta)(\lvec{n}\times\lvec{u})+(1-\cos\theta)(\lvec{n}\cdot\lvec{u})\lvec{n}}\,.\]
Recalling the linear properties of the cross and dot product (Theorems~\ref{thm:crossproperties} and \ref{thm:dotproperties} respectively):
\begin{align*}
  \lvec{n}\times(\lvec{u}+\lvec{v}) & = \lvec{n}\times\lvec{u}+\lvec{n}\times\lvec{v} 
  &\lvec{n}\cdot(\lvec{u}+\lvec{v}) & = \lvec{n}\cdot\lvec{u}+\lvec{n}\cdot\lvec{v}\\
  \lvec{n}\times(c\lvec{u}) & = c(\lvec{n}\times\lvec{u})
  &\lvec{n}\cdot(c\lvec{u}) & = c(\lvec{n}\cdot\lvec{u})
\end{align*}
it follows that $R_{\lvec{n},\theta}$ is a linear transformation.  Rotations in higher dimensions can similarly be determined by using a normal direction $\lvec{n}$ and a rotation in its orthogonal plane. The absence of the cross product for general $\mathset{R}^n$ does not hinder this.
\end{example}  

\begin{example}\label{exam:lineartransformations}%
  Given the vectors $\lvec{u}=(2,0,3)$ and $\lvec{v}=(1,3,5)$ in $\mathset{R}^3$ apply the following linear transformations. Describe each transformation.
\begin{multicols}{2}
  \begin{enumerate}
    \item $\mathbbm{1}(\lvec{u})$
    \item $\mathset{O}_{3,3}(\lvec{u})$
    \item $C_2(\lvec{u})$
    \item $C_{\frac{1}{2}}(\lvec{u})$
    \item $D(\lvec{u})$
    \item $P_{\lvec{i}}(\lvec{u})$
    \item $P_{\lvec{i},\lvec{k}}(\lvec{v})$
    \item $S_{\lvec{k},2}(\lvec{u})$
    \item $S_{\lvec{i},\frac{1}{2}}(\lvec{u})$
    \item $Q_{\lvec{i}}(\lvec{u})$
    \item $R_{\lvec{k},\frac{\pi}{2}}(\lvec{u})$
    \end{enumerate}
\end{multicols}
Solution:
\begin{enumerate}
\item $\mathbbm{1}(\lvec{u})=\lvec{u}=(2,0,3)$, the identity transformation.
\item $\mathset{O}_{3,3}(\lvec{u})=\lvec{0}=(0,0,0)$, the zero transformation.
\item $C_2(\lvec{u})=2\lvec{u}=(4,0,6)$, dilation by a factor of $2$\,.
\item $C_{\frac{1}{2}}(\lvec{u})=\frac{1}{2}\lvec{u}=\left(1,0,\frac{3}{2}\right)$, contraction by a factor $1/2$\,.
\item $D(\lvec{u})=-\lvec{u}=(-2,0,-3)$, inversion about the origin $O$\,.
\item $P_{\lvec{i}}(\lvec{u})=(\lvec{u}\cdot\lvec{i})\lvec{i}=(2,0,3)\cdot(1,0,0)\lvec{i}=(2+0+0)\lvec{i}=2\lvec{i}=(2,0,0)$, projection along the direction of $\lvec{i}$, the $x$-axis.
\item $P_{\lvec{i},\lvec{k}}(\lvec{v})=P_{\lvec{i}}(\lvec{v})+P_{\lvec{k}}(\lvec{v})=[(1,3,5)\cdot\lvec{i}]\lvec{i}+[(1,3,5)\cdot\lvec{k}]\lvec{k}=1\lvec{i}+5\lvec{k}=(1,0,5)$, projection onto the $x$-$z$ plane.
\item $S_{\lvec{k},2}(\lvec{u})=\lvec{u}+(2-1)\vectorprojplain{\lvec{k}}{\lvec{u}}=(2,0,3)+1[(2,0,3)\cdot\lvec{k}]\lvec{k}=(2,0,3)+3\lvec{k}=(2,0,6)$, expansion in $z$ direction by a factor of $3$.
\item $S_{\lvec{i},\frac{1}{2}}(\lvec{u})=\lvec{u}+(1/2-1)\vectorprojplain{\lvec{i}}{\lvec{u}}=(2,0,3)-\frac{1}{2}\underbrace{[(2,0,3)\cdot\lvec{i}]}_{=2}\lvec{i}=(2,0,3)-\lvec{i}=(1,0,3)$, compression\\[-1em] in $x$ direction by a factor of $1/2$.
\item $Q_{\lvec{i}}(\lvec{u})=\lvec{u}-2\vectorprojplain{\lvec{i}}{\lvec{u}}=(2,0,3)-2[(2,0,3)\cdot\lvec{i}]\lvec{i}=(2,0,3)-4\lvec{i}=(-2,0,3)$, reflection across plane with normal $\lvec{i}$ (the $y$-$z$ plane).
\item Noting that $\lvec{k}\cdot\lvec{u}=\lvec{k}\cdot(2,0,3)=3$ and 
  \[\lvec{k}\times\lvec{u}=\begin{vmatrix} \lvec{i} & \lvec{j} & \lvec{k} \\ 0 & 0 & 1 \\ 2 & 0 & 3 \end{vmatrix}=\lvec{i}(0-0)-\lvec{j}(0-2)+\lvec{k}(0-0)=2\lvec{j}\]
we have
\begin{align*}
  R_{\lvec{k},\frac{\pi}{2}}(\lvec{u})&=\left(\cos\frac{\pi}{2}\right)\lvec{u}+\left(\sin\frac{\pi}{2}\right)\lvec{k}\times\lvec{u}+\left(1-\cos\frac{\pi}{2}\right)(\lvec{k}\cdot\lvec{u})\lvec{k}\\
  & = (0)\lvec{u}+(1)(2\lvec{j})+(1-0)(3)\lvec{k} \\
  & = 2\lvec{j}+3\lvec{k}=(0,2,3),
\end{align*}
a rotation of $90^\circ$ about the positive $z$-axis.
\end{enumerate}
The student is encouraged to plot and label all the points (vector tips) that lie in the $x$-$z$ plane (i.e. with $y$-component equal to zero) by using $x$ as the horizontal axis and $z$ as the vertical axis, to help visualize the effect of the transformations.
\end{example}
In Example~\ref{exam:lineartransformations} we transformed a single vector, or equivalently a single terminal point.  In general we are often interested in transforming a set of points which constitute an image or object in two or three dimensions.  It is worth considering the effect of such transformations on multiple points whereby their meaning and application becomes clearer.

Rodrigues' rotation formula can be used to determine $R_\theta$ in $\mathset{R}^2$ by associating the vector $(x,y)$ in $\mathset{R}^2$ with the vector $(x,y,0)$ in the $x$-$y$ plane of $\mathset{R}^3$ and setting $\lvec{n}=\lvec{k}$.

\begin{example}
  \label{exam:twodrotation}%
  Let $\lvec{x}=(x,y)$ in $\mathset{R}^2$.  Derive the formula for the linear transformation $R_\theta(\lvec{x})$ by considering the effect of rotating the vector $\lvec{u}=(x,y,0)$ in the $x$-$y$ plane of $\mathset{R}^3$ by the angle $\theta$ about the $z$-axis, i.e. about the direction $\lvec{n}=\lvec{k}$.  Show, by inspection, that the result can be written as the left-multiplication of $\lvec{x}=\begin{bsmallmatrix}x\\y\end{bsmallmatrix}$ by an appropriate $2\times2$ matrix.

  Solution:\\
  Since
  \[\lvec{k}=\begin{bmatrix} 0 \\ 0 \\ 1\end{bmatrix}\ \ \text{and}\ \ \lvec{u}=\begin{bmatrix}x\\y\\0\end{bmatrix}=x\lvec{i}+y\lvec{j}\,.\]
  this implies
  \begin{align*}
    \lvec{k}\cdot\lvec{u}&=\begin{bmatrix}0\\0\\1\end{bmatrix}\cdot\begin{bmatrix}x\\y\\0\end{bmatrix}=0+0+0=0,\ \text{and}\\
    \lvec{k}\times\lvec{u}&=\begin{vmatrix}\lvec{i} & 0 & x\\ \lvec{j} & 0 & y \\ \lvec{k} & 1 & 0\end{vmatrix}=\lvec{i}(0-y)-\lvec{j}(0-x)+\lvec{k}(0-0)=-y\lvec{i}+x\lvec{j}\,.
  \end{align*}
  The rotation formula gives
  \begin{align*}
    R_{\lvec{k},\theta}(\lvec{u})&=(\cos\theta)\lvec {u} +(\sin\theta)(\lvec{k}\times\lvec{u})+(1-\cos\theta)(\lvec{k}\cdot\lvec{u})\lvec{k}\\
    &=\cos\theta(x\lvec{i}+y\lvec{j})+\sin\theta(-y\lvec{i}+x\lvec{j})+0\\
    &=\left((\cos\theta)x-(\sin\theta)y\right)\lvec{i}+\left((\sin\theta)x+(\cos\theta)y\right)\lvec{j}
  \end{align*}
  Next by identifying $\lvec{x}=\begin{bsmallmatrix}x\\y\end{bsmallmatrix}$, $\lvec{i}=\begin{bsmallmatrix}1\\0\end{bsmallmatrix}$, and $\lvec{j}=\begin{bsmallmatrix}0\\1\end{bsmallmatrix}$ in $\mathset{R}^2$, the two dimensional result is
      \begin{align*}
        R_{\theta}(\lvec{x})&=\left((\cos\theta)x-(\sin\theta)y\right)\lvec{i}+\left((\sin\theta)x+(\cos\theta)y\right)\lvec{j}\\
        &=\left((\cos\theta)x-(\sin\theta)y\right)\begin{bmatrix}1\\0\end{bmatrix}+\left((\sin\theta)x+(\cos\theta)y\right)\begin{bmatrix}0\\1\end{bmatrix}\\
            &=\begin{bmatrix}(\cos\theta)x-(\sin\theta)y\\(\sin\theta)x+(\cos\theta)y\end{bmatrix}\ \ \ \ (\leftarrow\text{a vector in $\mathset{R}^2$})
      \end{align*}
      The last expression can be rewritten using matrix multiplication as
      \[R_\theta(\lvec{x})=\begin{bmatrix*}[r] \cos\theta & -\sin\theta \\
      \sin\theta & \cos\theta\end{bmatrix*}\begin{bmatrix} x \\ y\end{bmatrix}\,.\]
\end{example}

Example~\ref{exam:twodrotation} raises two interesting questions.  We were able to represent a linear transformation by a matrix multiplying the vector. Does  multiplying a vector by some other matrix produce a linear transformation?  Secondly, can  other linear transformations be similarly represented by multiplication by a matrix?


\section{Matrix Transformations}
Turning to our first question raised at the end of the last section we have the following definition.
\definition{Let $A$ be an $m\times n$ matrix.  A \textbf{matrix transformation} is the transformation from $\mathset{R}^n$ to $\mathset{R}^m$ given by the matrix multiplication of vector $\lvec{u}$ in $\mathset{R}^n$ (written as a column matrix) on the left by $A$\,: 
  \[\boxed{L_A(\lvec{u})=A\lvec{u}}\,.\]}

That a matrix transformation is a linear transformation follows from the linearity of matrix multiplication.
\theorem{The matrix transformation $L_A(\lvec{u})$ from $\mathset{R}^n$ to $\mathset{R}^m$ is a linear transformation.}

Proof:\\
Let $L_A$ be a matrix transformation with $A$ the $m\times n$ matrix.  Suppose $\lvec{u}$ and $\lvec{v}$ are vectors in $\mathset{R}^n$ and $c$ is a scalar, then we have the following:
\begin{align*}
  L_A(\lvec{u}+\lvec{v})&=A(\lvec{u}+\lvec{v})=A\lvec{u}+A\lvec{v}=L_A(\lvec{u})+L_A(\lvec{v})\\
    L_A(c\lvec{u})&=A(c\lvec{u})=c(A\lvec{u})=cL_A(\lvec{u})
\end{align*}


\begin{example}Let $A=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}$ and let $\lvec{u}=\begin{bmatrix}1 \\ 1\end{bmatrix}$  and $\lvec{v}=\begin{bmatrix}2\\2\end{bmatrix}$ be vectors.  Find
    \begin{enumerate}
    \item $L_A(\lvec{u})$
    \item $L_A(\lvec{v})$
    \item $L_A(\lvec{u}+\lvec{v})$
    \item $L_A(5\lvec{u})$
    \end{enumerate}

    Solution:
    \begin{enumerate}
    \item $L_A(\lvec{u})=A\lvec{u}=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}\begin{bmatrix}1 \\ 1\end{bmatrix}=\begin{bmatrix}1+3 \\0+1\end{bmatrix}=\begin{bmatrix}4\\ 1\end{bmatrix}$
    \item $L_A(\lvec{v})=A\lvec{v}=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}\begin{bmatrix}2 \\ 2\end{bmatrix}=\begin{bmatrix}2+6 \\0+2\end{bmatrix}=\begin{bmatrix}8\\ 2\end{bmatrix}$
    \item $L_A(\lvec{u}+\lvec{v})=A(\lvec{u}+\lvec{v})=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}\begin{bmatrix}1+2 \\ 1+2\end{bmatrix}=\begin{bmatrix} 1& 3 \\ 0 & 1\end{bmatrix}\begin{bmatrix}3\\3\end{bmatrix}=\begin{bmatrix}3+9 \\0+3\end{bmatrix}=\begin{bmatrix}12\\ 3\end{bmatrix}$\\[.5em]
        which equals $L_A(\lvec{u})+L_A(\lvec{v})$ as expected.
      \item $L_A(5\lvec{u})=A(5\lvec{u})=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}\left(5\begin{bmatrix}1 \\ 1\end{bmatrix}\right)=\begin{bmatrix}1 & 3 \\ 0 & 1\end{bmatrix}\begin{bmatrix}5 \\ 5\end{bmatrix}=\begin{bmatrix}5+15 \\0+5\end{bmatrix}=\begin{bmatrix}20\\ 5\end{bmatrix}$\\[.5em]
        which equals $5L_A(\lvec{u})$ as expected.
    \end{enumerate}
\end{example}

Next let us consider the question of which linear transformations may be represented by matrix transformations.  It turns out that all of them are.  To see why observe, the following important property of linear transformations.  First we need our definition of linear combination with matrices applied to vectors.

\definition{Let $\lvec{v}_1$, $\lvec{v}_2$, \ldots, $\lvec{v}_k$ be vectors in $\mathset{R}^n$ and $c_1$, $c_2$, \ldots, $c_k$ be scalars.  Then the vector
  \[c_1\lvec{v}_1+c_2\lvec{v}_2+\cdots+c_k\lvec{v}_k\]
  is a \textbf{linear combination} of $\lvec{v}_1$, $\lvec{v}_2$, \ldots, $\lvec{v}_k$\,.}

\theorem{Let $L(\lvec{u})$ be a linear transformation from $\mathset{R}^n$ to $\mathset{R}^m$.  If $\lvec{u}$ is a linear combination of vectors $\lvec{v}_1$, $\lvec{v}_2$,\ldots $\lvec{v}_k$ in $\mathset{R}^n$, so that
  \[\lvec{u}=c_1\lvec{v}_1+c_2\lvec{v}_2+\cdots+c_k\lvec{v}_k\,,\]
  then
  \[\boxed{L(\lvec{u})=c_1L(\lvec{v}_1)+c_2L(\lvec{v}_2)+\cdots+c_kL(\lvec{v}_k)}\,.\]}

The implication of the theorem is that if you can decompose a vector into a linear combination of other vectors, then knowledge of how the linear transformation acts on those vectors is sufficient to determine how it acts on the original vector.

\begin{example}
  Suppose $\lvec{u}=\begin{bmatrix}1\\0\\3\end{bmatrix}$, $\lvec{v}=\begin{bmatrix}2\\2\\0\end{bmatrix}$ and $\lvec{w}=\begin{bmatrix*}[r]0\\-2\\6\end{bmatrix*}$ and let $L(\lvec{u})$ be a linear transformation from $\mathset{R}^3$ to $\mathset{R}^2$.  If $L(\lvec{u})=\begin{bmatrix}1\\1\end{bmatrix}$ and $L(\lvec{v})=\begin{bmatrix*}[r]-2\\3\end{bmatrix*}$, find $L(\lvec{w})$\,.

  Solution:\\
  Our strategy is to write $\lvec{w}$ as a linear combination of $\lvec{u}$ and $\lvec{v}$ (if possible) and then apply the linearity of $L$ to the result.
  If $\lvec{w}=a\lvec{u}+b\lvec{v}$ then constants $a$ and $b$ must satisfy
  \[a\lvec{u}+b\lvec{v}=a\begin{bmatrix}1\\0\\3\end{bmatrix}+b\begin{bmatrix}2\\2\\0\end{bmatrix}=\begin{bmatrix}a\\0\\3a\end{bmatrix}+\begin{bmatrix}2b\\2b\\0\end{bmatrix}=\begin{bmatrix}a+2b\\2b\\3a\end{bmatrix}=\lvec{w}=\begin{bmatrix*}[r]0\\-2\\6\end{bmatrix*}\]
  Equality of the vectors implies solving the system of equations
  \begin{align*}
    a+2b&=0\\
    2b&=-2\\
    3a&=6
  \end{align*}
  The second equation implies $\boxed{b=-1}$ and the third implies $\boxed{a=2}$.  These values satisfy the first equation, $2+2(-1)=0$, so this is a solution to the system and we have that $\lvec{w}=2\lvec{u}-1\lvec{v}$, which is easily checked.
  The linearity of $L$ implies
  \[L(\lvec{w})=L(2\lvec{u}-1\lvec{v})=L(2\lvec{u})+L(-1\lvec{v})=2L(\lvec{u})+(-1)L(\lvec{v})=2\begin{bmatrix*}[r]1 \\ 1\end{bmatrix*}-1\begin{bmatrix*}[r]-2\\3\end{bmatrix*}=\begin{bmatrix*}[r]2+2\\2-3\end{bmatrix*}=\begin{bmatrix*}[r]4\\-1\end{bmatrix*}\,.\]
\end{example}

\textbook{\newpage}
Recall that $\lvec{e}_1$, $\lvec{e}_2$, \ldots, $\lvec{e}_n$ are the elementary vectors in $\mathset{R}^n$ defined by \mbox{$\lvec{e}_1=(1,0,\ldots,0)$,} \mbox{$\lvec{e}_2=(0,1,\ldots,0)$,} \ldots, \mbox{$\lvec{e}_n=(0,0,\ldots,1)$.}  Then $\{\lvec{e}_1,\lvec{e}_2,\ldots,\lvec{e}_n\}$ is called the \textbf{standard basis} of $\mathset{R}^n$.  With respect to this basis we saw that any vector $\lvec{u}$ has the unique expansion\footnote{Uniqueness follows for if $\lvec{u}$ had some other expansion $\lvec{u}=v_1\lvec{e}_1+v_2\lvec{e}_2+\cdots v_n\lvec{e}_n$ we could take the dot product of each expansion with respect to $\lvec{e}_i$ to get $\lvec{u}\cdot\lvec{e}_i=u_i$ for the first and $\lvec{u}\cdot\lvec{e}_i=v_i$ for the second, showing $u_i=v_i$.  Since $i$ was arbitrary the expansion is unique.}
\[\lvec{u}=(u_1,u_2,\ldots,u_n)=u_1\lvec{e}_1+u_2\lvec{e}_2+\cdots u_n\lvec{e}_n\,.\]
Now consider acting on $\lvec{u}$ by a linear transformation $L$ that takes vectors from $\mathset{R}^n$ to $\mathset{R}^m$.
\begin{align*}
  L(\lvec{u})&=L(u_1\lvec{e}_1+u_2\lvec{e}_2+\cdots +u_n\lvec{e}_n)\\
  &=u_1L(\lvec{e}_1)+u_2L(\lvec{e}_2)+\cdots +u_nL(\lvec{e}_n)
\end{align*}
Defining vectors $\lvec{a}_i=L(\lvec{e}_i)$ in $\mathset{R}^m$ this implies
\[L(\lvec{u}) = u_1\lvec{a_1}+u_2\lvec{a}_2+\cdots+u_n\lvec{a}_n\,. \]
However, when discussing matrix multiplication of a vector (Section~\ref{subsec:matrixmultiplyingvector}) we noted that this right hand side equals $A\lvec{u}$ if $A=[\lvec{a}_1\lvec{a}_2\cdots\lvec{a}_n]$ where $\lvec{a}_i$ is the $i\textsuperscript{th}$ column of $A$.  Hence
\[L(\lvec{u})=A\lvec{u}\]
where
\[A=[\lvec{a}_1\lvec{a}_2\cdots\lvec{a}_n]=[L(\lvec{e}_1)L(\lvec{e}_2)\cdots L(\lvec{e}_n)]\]
  is an $m\times n$ matrix whose $i\textsuperscript{th}$ column is the vector $L(\lvec{e}_i)$\,.  We summarize the result in the following theorem.
  
  \theorem{\label{thm:matrixrepresentation}Let $L$ be a linear transformation from $\mathset{R}^n$ to $\mathset{R}^m$ and let $\{\lvec{e}_1,\lvec{e}_2,\ldots,\lvec{e}_n\}$ be the standard basis of $\mathset{R}^n$ .  Then $L$ equals the matrix transformation $L_A$ where $A$ is the unique $m\times n$ matrix
    \[\boxed{A=[L(\lvec{e}_1)L(\lvec{e}_2)\cdots L(\lvec{e}_n)]}\,.\]
    Here $L(\lvec{e}_i)$ is a vector in $\mathset{R}^m$ written as a column matrix.}

  Proof:\\
  The remaining item to show is that the matrix $A$ is unique.  We have seen that $L(\lvec{e}_i)=\lvec{a}_i$.  Suppose transformation $L$ equalled a second matrix transformation $L_B$.  Then
  \[L(\lvec{e}_i)=L_B(\lvec{e}_i)=B\lvec{e}_i=\lvec{b}_i\,.\]
  where $\lvec{b}_i$ is the $i\textsuperscript{th}$ column of $B$.  The final equality follows due to the components of $\lvec{e}_i$.  Hence $\lvec{b}_i=\lvec{a}_i$.  Since the choice of $i$ was arbitrary, $B=A$\,.

Since we have shown that every matrix transformation is linear and now that every linear transformation equals a matrix transformation we have the following result.
  
\textbf{Corollary:} A transformation $T$ is linear if and only if it equals a matrix transformation.

Theorem~\ref{thm:matrixrepresentation} gives a prescription for finding a matrix to represent any linear transformation $L$ in terms of the action of $L$ on the elementary vectors $\lvec{e}_i$.  The latter can be found using the explicit forms of $L$ of common transformations found earlier, or one can deduce them directly from the transformation the operator presents as the following examples show.

\textbook{\newpage}
\begin{example}
  Find a matrix transformation equal to the linear transformation.
  \begin{enumerate}
  \item The reflection $Q_{\lvec{j}}$ in $\mathset{R}^2$.
  \item The dilation $C_2$ in $\mathset{R}^2$.
  \end{enumerate}

  Solution:\\
  \begin{enumerate}
  \item The reflection $Q_{\lvec{j}}$ in $\mathset{R}^2$ is a reflection across the line with normal $\lvec{j}$.  Since $\lvec{j}$ points along the $y$-axis, the reflection is across the $x$-axis (i.e. the line $y=0$).  Such a reflection leaves the vector $\lvec{i}$ unchanged and takes $\lvec{j}$ to $-\lvec{j}$.  (This can be seen by thinking of the action of the transformation on the endpoints $(1,0)$ and $(0,1)$ of $\lvec{i}$ and $\lvec{j}$ respectively.) In symbols
    \begin{center}
    \parbox{12em}{\includegraphics[height=12em]{lineartransformations/matrixtrans1}}\parbox{20em}{\begin{align*}
      L(\lvec{e}_1)&=L(\lvec{i})=\lvec{i}=\begin{bmatrix}1\\0\end{bmatrix}\\
      L(\lvec{e}_2)&=L(\lvec{j})=-\lvec{j}=-1\begin{bmatrix*}[r]0\\1\end{bmatrix*}=\begin{bmatrix*}[r]0\\-1\end{bmatrix*}
    \end{align*}}
    \end{center}
    Thus $Q_{\lvec{j}}=L_A$ where $A=[L(\lvec{e}_1) L(\lvec{e}_2)]=\begin{bmatrix*}[r]1 & 0 \\ 0 & -1\end{bmatrix*}$\,.
    
  \item The dilation $C_2$ in $\mathset{R}^2$ stretches all vectors by a factor of two, so, in particular the elementary vectors transforms as
    \begin{center}
    \parbox{12em}{\includegraphics[height=12em]{lineartransformations/matrixtrans2}}\parbox{20em}{\begin{align*}
      L(\lvec{e}_1)&=L(\lvec{i})=2\lvec{i}=2\begin{bmatrix}1\\0\end{bmatrix}=\begin{bmatrix}2\\0\end{bmatrix}\\
      L(\lvec{e}_2)&=L(\lvec{j})=2\lvec{j}=2\begin{bmatrix*}[r]0\\1\end{bmatrix*}=\begin{bmatrix*}[r]0\\2\end{bmatrix*}
    \end{align*}}
    \end{center}
        Thus $C_2=L_A$ where $A=[L(\lvec{e}_1) L(\lvec{e}_2)]=\begin{bmatrix*}[r]2 & 0 \\ 0 & 2\end{bmatrix*}$\,.
  \end{enumerate}
  
\end{example}

\textbook{\newpage}
\begin{example}
  Rederive the matrix transformation for $R_\theta$ in $\mathset{R}^2$ using Theorem~\ref{thm:matrixrepresentation}.  Also find the transformation for $R_{\lvec{k},\theta}$ in $\mathset{R}^3$\,.

  Solution:\\
  The following diagram shows the effect of rotating the elementary vectors $\lvec{i}$ and $\lvec{j}$ by an arbitrary counterclockwise rotation $\theta$ in $\mathset{R}^2$.
  \begin{center}
    \includegraphics[height=1.5in]{lineartransformations/rotation2d_proof}
  \end{center}
  Consideration of the triangles shown and noting the hypotenuse of each is 1 since they sit on the unit circle, gives the following components for $R_\theta(\lvec{i})$ and $R_\theta(\lvec{j})$ respectively:
  \begin{align*}
    R_\theta(\lvec{i})&=\begin{bmatrix} \cos\theta \\ \sin\theta \end{bmatrix} 
    & R_\theta(\lvec{j})&=\begin{bmatrix*}[r] -\sin\theta \\ \cos\theta \end{bmatrix*}\,.
  \end{align*}
  These are the columns of the equivalent matrix transformation for $R_\theta$ and we have that $R_\theta$ equals the matrix transformation $L_A$ where
  \[A=\begin{bmatrix*}[r]
  \cos\theta & -\sin\theta \\
  \sin\theta &  \cos\theta
  \end{bmatrix*}\,.\]
  This is the matrix that was found in Example~\ref{exam:twodrotation}\,.
  
  In $\mathset{R}^3$ if we rotate by $\theta$ about the $z$-axis the elementary vectors $\lvec{i}$ and $\lvec{j}$ stay in the plane while $\lvec{k}$ is unchanged  and we have
  \begin{align*}
    R_{\lvec{k},\theta}(\lvec{i})&=\begin{bmatrix} \cos\theta \\ \sin\theta \\ 0 \end{bmatrix} 
    & R_{\lvec{k},\theta}(\lvec{j})&=\begin{bmatrix*}[r] -\sin\theta \\ \cos\theta \\ 0 \end{bmatrix*}
    & R_{\lvec{k},\theta}(\lvec{k})&=\begin{bmatrix*}[r] 0 \\ 0 \\ 1  \end{bmatrix*}\,.
  \end{align*}
  We see that the linear transformation $R_{\lvec{k},\theta}$ equals the matrix transformation $L_B$ with matrix
  \[B=\begin{bmatrix*}[r]
  \cos\theta & -\sin\theta & 0 \\
  \sin\theta &  \cos\theta & 0 \\
  0 & 0 & 1
  \end{bmatrix*}\,.\]
\end{example}

By describing our basic linear transformations by vectors and their operations we were able to show that they were linear by taking advantage of the linear properties of scalar, dot, and cross products. An additional advantage of this approach is that we observe that a linear transformation is independent of choice of Cartesian coordinates (aside from the fixed point origin upon which they all depended) just as vectors are.

When considering matrix transformations we have defined them with respect to the standard basis $\{\lvec{e}_1,\lvec{e}_2,\ldots,\lvec{e_n}\}$ of $\mathset{R}^n$.  When using $\mathset{R}^3$, say, to represent a physical problem then the components of a vector will depend on the orientation of the coordinate axes chosen in physical space which, in turn, determine the directions the elementary vectors $\lvec{e}_i$ represent. The matrix $A$ representing a linear transformation $L(\lvec{u})$ such as a rotation of $\theta$ about a particular direction in physical space will itself depend on this orientation of axes.   Someone else choosing the same origin but a different orientation of coordinate axes would find different components and matrix to represent the same physical vector and linear transformation. It is a standard problem in linear algebra to consider how the components of the same vector in different coordinate systems are related.  Similarly one considers how the matrix representations of the same operator in different systems are related.

Having chosen a particular orientation of coordinate system in space one can \emph{define} a linear transformation by a matrix transformation acting on the standard basis of that system.  However the matrix representation of that transformation in other coordinate systems will be, in general, a different matrix.  With further linear algebra, that matrix can be determined knowing the orientation of the other coordinate system to the initial one.

\section{Composition of Linear Transformations}
If $f(x)=\sin x$ and $g(x)=x^2$, then the composition of functions $f\circ g$ is defined by $(f\circ g)(x)=f(g(x))=\sin(x^2)$, with $g$ applied first and $f$ applied to that result.  Order in application of the functions typically matters, as it does in this example where $g\circ f\neq f\circ g$ since $(g\circ f)(x)=g(f(x))=(\sin x)^2$ which is a different function.  We have seen that linear combinations of transformations from $\mathset{R}^n$ to $\mathset{R}^m$ produced new transformations and that if all the transformations were linear the new transformation was also linear.  To this method of producing new transformation we can add transformation composition.

\definition{If $S$ is a transformation from $\mathset{R}^m$ to $\mathset{R}^l$ and $T$ is a transformation from $\mathset{R}^n$ to $\mathset{R}^m$ then the composition of transformations $S\circ T$ defined by
  \[\boxed{(S\circ T)(\lvec{u})=S(T(\lvec{u}))}\,,\]
  is a transformation from $\mathset{R}^n$ to $\mathset{R}^l$.}

Composition of more that two transformations is similarly accomplished.  In the event the transformations are linear we have the following result.

\theorem{Let $K$ be a linear transformation from $\mathset{R}^m$ to $\mathset{R}^l$ and let $L$ be a linear transformation from $\mathset{R}^n$ to $\mathset{R}^m$ equal to matrix transformations $L_A$ and $L_B$ respectively where $A$ is an $l\times m$ matrix and $B$ is an  $m\times n$ matrix.  Then the composition transformation  $K\circ L$ defined by
  \[\boxed{(K\circ L)(\lvec{u})=K(L(\lvec{u}))}\,,\]
  is a linear transformation from $\mathset{R}^n$ to $\mathset{R}^l$ and equals the matrix transformation given by $L_{AB}$ where $AB$ is the $l\times n$ matrix product of A and B.}

Proof:\\
\[(K\circ L)(\lvec{u})=K[L(\lvec{u})]=K(B\lvec{u})=A(B\lvec{u})=(AB)\lvec{u}=L_{AB}(\lvec{u})\,.\]

Note the following:
\begin{enumerate}
  \item The matrix of the transformation applied first is placed furthest to the right.
  \item Composition of more than one linear transformation is possible (assuming the dimensions of $\mathset{R}$ align appropriately) and the result can be shown to be linear by induction.
  \item The fact that function composition typically depends on the order of application of the functions is mirrored by the fact that matrix multiplication does not, in general, commute ($AB\neq BA$).
\end{enumerate}

\begin{example}
If $K$ is the reflection about the line $y=x$ and $L$ is the rotation $R_{\frac{\pi}{2}}$ in $\mathset{R}^2$, find a matrix transformation equal to the composition $K\circ L$.  What, geometrically, is the new linear transformation?

Solution:\\
The line $y=x$ is the diagonal with slope $m=1$.  Reflection of $\lvec{i}$ across it goes to $\lvec{j}$ and similarly $\lvec{j}$ becomes $\lvec{i}$.  In other words, the endpoints $(0,1)$ and $(1,0)$ exchange locations.  Thus
    \begin{center}
    \parbox{12em}{\includegraphics[height=10em]{lineartransformations/matrixtrans3}}\parbox{20em}{\begin{align*}
  K(\lvec{i})&=\lvec{j}=\begin{bmatrix} 0 \\ 1\end{bmatrix}\\
  K(\lvec{j})&=\lvec{i}=\begin{bmatrix} 1 \\ 0\end{bmatrix}
    \end{align*}}
    \end{center}
so $K=L_A$ where $A=[K(\lvec{i})K(\lvec{j})]=\begin{bmatrix} 0 & 1 \\ 1 & 0\end{bmatrix}$\,.

$L$ is a positive rotation of $\frac{\pi}{2}=90^\circ$ which rotates $\lvec{i}$ to $\lvec{j}$, i.e. point $(1,0)$ goes to $(0,1)$\,.  The vector $\lvec{j}$ is rotated to $-\lvec{i}$, i.e. point $(0,1)$ goes to $(-1,0)$ under the rotation.  Thus
\begin{center}
    \parbox{12em}{\includegraphics[height=10em]{lineartransformations/matrixtrans4}}\parbox{20em}{\begin{align*}
  L(\lvec{i})&=\lvec{j}=\begin{bmatrix} 0 \\ 1\end{bmatrix}\\
  L(\lvec{j})&=-\lvec{i}=-\begin{bmatrix} 1 \\ 0\end{bmatrix}=\begin{bmatrix*}[r] -1 \\ 0\end{bmatrix*}
    \end{align*}}
    \end{center}
so $L=L_B$ where $B=[L(\lvec{i})L(\lvec{j})]=\begin{bmatrix*}[r] 0 & -1 \\ 1 & 0\end{bmatrix*}$\,.

The composition transformation $K\circ L$ is equivalent to the matrix transformation $L_{AB}$ where $AB$ is the product of the matrices
\[AB=\begin{bmatrix} 0 & 1 \\ 1 & 0\end{bmatrix}\begin{bmatrix*}[r] 0 & -1 \\ 1 & 0\end{bmatrix*}=\begin{bmatrix*}[r]1 & 0 \\ 0 & -1 \end{bmatrix*}\,.\]
  The effect of the transformation composition on a vector is thus
  \[K\circ L(\lvec{x})=\begin{bmatrix*}[r]1 & 0 \\ 0 & -1 \end{bmatrix*} \begin{bmatrix}x \\ y \end{bmatrix}=\begin{bmatrix*}[r] x \\ -y \end{bmatrix*}\,.\]
  We see that under $K\circ L$ the $x$-component is unchanged but the $y$-component is flipped.  This a reflection about the $x$-axis.  The normal to that axis is the vector $\lvec{j}$ and so we have the identification
  \[K\circ L=Q_{\lvec{j}}\,.\]
\end{example}

\begin{example}
  In addition to writing an arbitrary rotation in three dimensions by $R=R_{\lvec{n},\theta}$ it is possible to write it as a composition of three angular rotations about the axes of a fixed coordinate system with basis~\{\lvec{i},\lvec{j},\lvec{k}\}, such as
  \[R_{\alpha,\beta,\gamma}=R_{\lvec{k},\gamma}\circ R_{\lvec{j},\beta}\circ R_{\lvec{i},\alpha}\,.\]
  The matrix transformation is then the product of the three corresponding matrices.     The angles $(\alpha,\beta,\gamma)$ are referred to as \textbf{Euler Angles}. There are many conventions for such angles. This formulation of rotation has utility when describing the position of the axes of one coordinate system with respect to another.
\end{example}

\section{Linear Operators}
\label{sec:linearoperators}
In most of the examples we have done so far our linear transformations have been from $\mathset{R}^n$ back to itself.  This important class of transformations has its own name.

\definition{A \textbf{linear operator} is a linear transformation from $\mathset{R}^n$ to $\mathset{R}^n$.}

So dilations, contractions, inversions, expansions, compressions, reflections, and rotations are all linear operators.  The matrix of a linear operator from $\mathset{R}^n$ to $\mathset{R}^n$ will be a square matrix of order $n$ and we can classify our operators according to the type of matrix transformation they equal.

\subsection{Symmetric Operators}
\label{subsec:symmetricoperators}
Recall a symmetric matrix satisfies $A^T=A$.  Consideration of our general transformation $C_k(\lvec{u})=k\lvec{u}$, of which the identity, zero, dilation, contraction, inversion operators are special cases, show they all give rise to symmetric matrix representations.  For instance in $\mathset{R}^3$ we have, by consideration of $C_k(\lvec{i})$, etc., that $C_k$ is equal to $L_A$ where
\[A=\begin{bmatrix*} k & 0 & 0 \\ 0 & k & 0 \\ 0 & 0 & k \end{bmatrix*}\]
which is clearly symmetric.

As another example, the projection operator $P_{\lvec{n}}$, where $\lvec{n}=(n_x,n_y,n_z)$ is a unit vector, has a symmetric matrix representation.  In $\mathset{R}^3$ we find, by evaluating $P_{\lvec{n}}(\lvec{i})$, etc., that $P_{\lvec{n}}$ is equal to $L_A$ where
\[A=\begin{bmatrix*} n_xn_x & n_yn_x & n_zn_x \\
n_xn_y & n_yn_y & n_zn_y \\
n_xn_z & n_yn_z & n_zn_z
\end{bmatrix*}\,,\]
which is symmetric.  Since $(A+B)^T=A^T+B^T=A+B$ for symmetric matrices $A$, $B$,  it follows that the sum of symmetric matrices is also symmetric.  Therefore the more general orthogonal projection $P_{\lvec{n}_1,\lvec{n}_2,\ldots,\lvec{n}_k}$ will also induce a symmetric matrix.

As yet a further example consider the linear operator $S_{\lvec{n},k}$ of which compression, expansion, and reflection are special cases.  We saw that $S_{\lvec{n},k}$  could be written
\[S_{\lvec{n},k}=\mathbbm{1}+(k-1)P_{\lvec{n}}\,.\]
This will also be symmetric since $(k-1)$ is a scalar and $(cA)^T=cA^T=cA$ if $A$ is symmetric.  This implies that $cA$ is symmetric if $A$ is and adding this to symmetric $\mathbbm{1}$ will still be symmetric.  Explicitly in $\mathset{R}^3$ we have that $S_{\lvec{n},k}$ is equal to $L_A$ where
\[A=\begin{bmatrix*} 1+(k-1)n_xn_x & (k-1)n_yn_x & (k-1)n_zn_x \\
(k-1)n_xn_y & 1+(k-1)n_yn_y & (k-1)n_zn_y \\
(k-1)n_xn_z & (k-1)n_yn_z & 1+(k-1)n_zn_z
\end{bmatrix*}\,,\]
which explicitly is symmetric.

\definition{A \textbf{linear operator} on $\mathset{R}^n$ equipped with the usual dot product is called \textbf{symmetric} if it equals a matrix transformation $L_A$ where $A$ is a symmetric matrix.}

The zero, identity, dilations, contraction, inversion, expansion, compression, and reflection operators are therefore symmetric.  The product of symmetric operators is also symmetric if $A$ and $B$ commute since $(AB)^T=B^TA^T=BA=AB$ for two commuting symmetric matrices.  As such we expect an arbitrary symmetric matrix may be decomposable in terms of these types of matrices.

\theorem{A symmetric linear operator $L$ on $\mathset{R}^n$ satisfies
  \[\boxed{\lvec{u}\cdot L(\lvec{v})=L(\lvec{u})\cdot\lvec{v}}\,,\]
  for all $\lvec{u}$ and $\lvec{v}$ in $\mathset{R}^n$\,.}

\textbf{Proof:}\\
$L$ equals $L_A$ where $A$ is a symmetric matrix.  Then for any $\lvec{u}$, $\lvec{v}$ in $\mathset{R}^n$ we have
\[\lvec{u}\cdot L(\lvec{v})=\lvec{u}\cdot (A\lvec{v})=(\lvec{u}^T)A\lvec{v}=(\lvec{u}^T)A^T\lvec{v}=(A\lvec{u})^T\lvec{v}=L(\lvec{u})\cdot\lvec{v}\,.\]

Note that this latter property can also be considered the definition of a symmetric linear operator since it can be shown to imply a symmetric matrix representation.

\subsection{Idempotent Operators}

Recall an idempotent matrix satisfies $A^2=A$. It can be shown that the zero, identity, and projection operators $P_{\lvec{n}}$ and $P_{\lvec{n}_1,\lvec{n}_,\ldots,\lvec{n}_k}$ induce idempotent matrix transformations.  This can be shown explicitly using our $\mathset{R}^3$ version of $P_{\lvec{n}}$ shown previously.

\definition{A \textbf{linear operator} on $\mathset{R}^n$ equipped with the usual dot product is called \textbf{idempotent} if it equals a matrix transformation $L_A$ where $A$ is an idempotent matrix.}

The zero, identity, and projection operators $P_{\lvec{n}}$ and $P_{\lvec{n}_1,\lvec{n}_,\ldots,\lvec{n}_k}$ are therefore idempotent.

\theorem{An idempotent linear operator $L$ satisfies $L\circ L=L$.  That is, for any vector $\lvec{u}$ in $\mathset{R}^n$
  \[\boxed{L(L(\lvec{u}))=L(\lvec{u})}\,.\]}

\textbf{Proof:}
  $L$ equals $L_A$ where $A$ is an idempotent matrix.  Then for any $\lvec{u}$ in $\mathset{R}^n$ we have
  \[L(L(\lvec{u}))=L(A\lvec{u})=A(A\lvec{u})=(A^2)\lvec{u}=A\lvec{u}=L(\lvec{u})\,.\]

Note that this latter property can also be considered the definition of an idempotent linear operator since it can be shown to imply an idempotent matrix representation.

\subsection{Orthogonal Operators}

Inversion, reflections, and rotations fall into an important group of linear operators which we now explore.  Recall an orthogonal matrix satisfies $A^{-1}=A^T$. The identity operator's matrix representation is just $I$, an orthogonal matrix.  The inversion operator has matrix representation $-I$ which is also orthogonal.  The reflection operator $Q_{\lvec{n}}$ also equals a matrix transformation where $A$ is orthogonal.  Finally rotations have orthogonal matrix representations.
\begin{example}
  The $R_{\theta}$ matrix given by  $A=\begin{bmatrix*}[r] \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{bmatrix*}$ is orthogonal since \mbox{$A^T=\begin{bmatrix*}[r] \cos\theta & \sin\theta\\ -\sin\theta & \cos\theta \end{bmatrix*}$} and
  \begin{align*}
    AA^T&=\begin{bmatrix*}[r] \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{bmatrix*}\begin{bmatrix*}[r] \cos\theta & \sin\theta\\ -\sin\theta & \cos\theta \end{bmatrix*}\\
    &=\begin{bmatrix}\cos^2\theta+\sin^2\theta & \cos\theta\sin\theta-\sin\theta\cos\theta \\ \sin\theta\cos\theta-\cos\theta\sin\theta& \cos^2\theta+\sin^2\theta\end{bmatrix}=\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}=I\,.
    \end{align*}
  Similarly $A^TA=I$ and thus $A^{-1}=A^T$\,.
\end{example}

Using Rodrigues' rotation formula to evaluate $R_{\lvec{n},\theta}(\lvec{i})$, etc. one can show that the rotation operator equals $L_A$ where matrix $A$ is:
\[A=\begin{bmatrix} 
\cos \theta +n_x^2 \left(1-\cos \theta\right) & n_x n_y \left(1-\cos \theta\right) - n_z \sin \theta & n_x n_z \left(1-\cos \theta\right) + n_y \sin \theta \\ 
n_y n_x \left(1-\cos \theta\right) + n_z \sin \theta & \cos \theta + n_y^2\left(1-\cos \theta\right) & n_y n_z \left(1-\cos \theta\right) - n_x \sin \theta \\ 
n_z n_x \left(1-\cos \theta\right) - n_y \sin \theta & n_z n_y \left(1-\cos \theta\right) + n_x \sin \theta & \cos \theta + n_z^2\left(1-\cos \theta\right)
\end{bmatrix}\,.\]
This can be shown to be orthogonal as well.

\definition{A \textbf{linear operator} on $\mathset{R}^n$ equipped with the usual dot product is called \textbf{orthogonal} if it equals a matrix transformation $L_A$ where $A$ is an orthogonal  matrix.}

The identity, inverse, reflection and rotation operators are therefore orthogonal.  Orthogonal linear operators have the following important property.

\theorem{If $L$ is an orthogonal linear operator on $\mathset{R}^n$ equipped with the usual dot product then it preserves the dot product.  For any vectors $\lvec{u}$ and $\lvec{v}$ in $\mathset{R}^n$ we have
  \[\boxed{L(\lvec{u})\cdot L(\lvec{v})=\lvec{u}\cdot\lvec{v}}\,.\]}

  \textbf{Proof:}\\
  $L$ equals $L_A$ where $A$ is an orthogonal matrix.  Then for any vectors $\lvec{u}$ and $\lvec{v}$ in $\mathset{R}^n$ we have
  \begin{align*}
    L(\lvec{u})\cdot L(\lvec{v})&=(A\lvec{u})\cdot(A\lvec{v})=(A\lvec{u})^T(A\lvec{v})=(\lvec{u}^TA^T)(A\lvec{v})\\
    &=\lvec{u}^T(A^TA)\lvec{v}=\lvec{u}^T(A^{-1}A)\lvec{v}=\lvec{u}^TI\lvec{v}=\lvec{u}^T\lvec{v}=\lvec{u}\cdot\lvec{v}\,.
    \end{align*}

Setting $\lvec{u}=\lvec{v}$ shows $\lnorm{L(\lvec{u})}^2=L(\lvec{u})\cdot L(\lvec{u})=\lvec{u}\cdot\lvec{u}=\lnorm{\lvec{u}}^2$ from which the following corollary is implied.

\textbf{Corollary:}
An orthogonal linear operator $L$ on $\mathset{R}^n$ preserves the length of a vector, $\boxed{\lnorm{L(\lvec{u})}=\lnorm{\lvec{u}}}$\,.

Note that property is clear for rotations, reflections, and inversions. Since angle between vectors can be written in terms of the dot product and lengths of two vectors it follows that an orthogonal linear operator also preserves angles between vectors.
  
Note that either the preservation of dot product or of length can be used to define an orthogonal linear operator as they can be shown to be equivalent and they imply an orthogonal matrix representation.

Finally a product of orthogonal matrices is orthogonal since $(AB)^{-1}=B^{-1}A^{-1}=B^TA^T=(AB)^T$ for orthogonal matrices $A$ and $B$.  So we expect a general orthogonal matrix to be decomposable in terms of rotations, reflections, and inversions.

\subsection{Polar Decomposition of Operators}

To conclude this chapter we note the following theorem which shows that every matrix $A$ can be decomposed into a product of a symmetric and an orthogonal matrix.

\theorem{Let $A$ be a square matrix with real entries.  Then $A$ can be written as the product of a symmetric matrix $S$ and an orthogonal matrix $O$ as\footnote{In fact the theorem is stronger than stated.  The symmetric matrix found can be restricted to those that are positive semi-definite (i.e. no reflections) and in the event that $A$ is invertible this decomposition is unique.}
  \[\boxed{A=SO}\,.\]
  This is called the \textbf{polar decomposition}\footnote{The term polar decomposition is used as it is analogous to the decomposition of a complex number as $re^{i\theta}$ which, upon multiplication of another complex number will scale it by a factor of $r$ and rotate in the plane by an angle $\theta$.  Complex numbers will be discussed in Chapter~\ref{chap:complexnumbers}\,.}
 of $A$.}

Since any linear operator on $\mathset{R}^n$ equals a matrix operator $L_A$ we have the following corollary.

\textbf{Corollary:} If $L$ is a linear operator on $\mathset{R}^n$ then $L$ can be written as the composition of a symmetric operator~$S$ and an orthogonal operator~$O$ as
\[\boxed{L=S\circ O}\,.\]

These theorems are useful as they characterize all linear operators quite generally as being decomposable into two operations.  The first is a length-preserving orthogonal operation which will consist of a composition of rotations, reflections, and inversions.  This will then be followed by a symmetric operator which will consist of a composition of operations that will typically scale and project the vector along various directions.

%TODO:
% Add pictures for scalar transformations and elsewhere showing a graphic of points undergoing the transformation.  The graphic needs to have broken symmetries to make it clear what is going on - i.e. if it is a happy face the eyes need to be different.  Alternatively it could be made as a problem where the student identifies the transformation.
%
% Fix the zero transformation letter.  Maybe import just that letter from that font without messing up the others.  Currently I'm just using mathset on the O letter which is less than ideal.
%
% Change notation for reflection to a better letter.  Perhaps choose B since the German word for reflection starts with B.  Obviously we'll save R for rotation.
%
% Consider discussing shears (which are linear transformations but not symmetric) and translations which are non-linear transformations. Perhaps in
% assignments
%
% Add about projections that we can think of them as being mapped into lower dimensional subspaces later, such as 1D vector (x) or (x,y) component if projected onto two.  My i,j projection is exactly that and one could comment on how it could be thought of as R^2 - Perhaps in assignment or in later discussion.
%
% Comment on rotations in higher D as having a convention required to determine the direction of rotation. See the article I had.
%
% Add proof of Rodrigues' formula at end.  See my proof in my original notes.
% Add a diagram in the example with the linear transformations showing all the
% points, or at least those that sit in the y-z plane.
%
% I now am using 1 for my identity operator.  I could use that for the identity matrix as well for consistency, though this may be confusing.  This would free up I to be used for inversion which might be good.
%
% Decide on whether to say R^m to R^n or use symbol \to consistently
%
% Add a definition of two transformations being ``equal'' when we introduce transformations because I use equality later.   Refer to equality of functions (same outcome on same domain)
%
% I showed that when using S_k,0 we get u_2.  I should define this to be a special operator as well called P_k\perp.
%
% Maybe use clearer notation (or perhaps the same notation) for matrices and their linear operators.  For instance we could use mathcal for operators.  However this may NOT be desirable as it suggests that the matrix rep is somehow fixed.  I could, perhaps use M_A rather than L_A for matrix transformations as the latter does get a bit messy when I am also using L as one of the linear operators.  Ideally we should have the basis (indeed bases for general linear transformations) associated with the matrix transformation as the thing is meaningless independent of the basis.
%
% Mention convention of just writing the operator to the left like matrix multiplication.
%
% Add graphs showing rotations, etc. of elementary vectors by linear operators
% in the two examples where this is done. See my journal solutions.
%
% Look more closely at Euler angles.  Maybe use clearer notation (or perhaps the same notation) for matrices and their linear operators.  Note my choice of Euler angles just came from the Rotation Matrix entry in wikipedia.  Other choices have angles taken with respect to rotated axes.  This is related to the distinction between intrinsic and extrinsic rotations.
%
% It might make sense to use S rather than C for the scalar operator C_k=k\lvec{u} is actually called a ``scalar'' operator.  This currently requires too many changes.
%
% Comment about matrix shows vector produced is actually a vector despite cross product.
%
% Write the general rotation matrix in terms of $K$ matrix - see my notes
% to make it more compact and provable to be orthogonal.
%
% Move definition of orthogonal matrix into matrices chapter.  Move proof
% that its det is +/-1 into determinants chapter.
