\section{Sequences}\index{sequence}
\definition{An ordered list of numbers:
\[\left\{a_1, a_2, a_3, \ldots, a_n,\ldots \right\}\]
is called a \textbf{sequence}.
The numbers are called \textbf{terms}\index{terms! of a sequence} with $a_1$ here being the \emph{first term}, and, more generally, $a_n$ being the $n^{\text{th}}$ term in the sequence.}

The above sequence may represented by the compact notation 
$\bm{\left\{a_n\right\}}$ or sometimes with the index limits made explicit as $\bm{\left\{a_n\right\}_{n=1}^{\infty}}$.  An explicit index is useful if we start enumerating the sequence from a value other than 1.

Some texts will distinguish \textbf{finite}\index{sequence!finite} and \textbf{infinite}\index{sequence!infinite} sequences depending on whether the sequence terminates or not.  For our purposes we will be assuming infinite sequences unless otherwise noted.

An equivalent way of thinking of a sequence is as a function $f$ whose domain is the positive integers.  In this case $a_n=f(n)$.  Writing $a_n$ as just such a function of the index is a convenient way of representing a sequence.

\begin{example}
The following are several ways to represent the same sequences.
\begin{enumerate}
\item $\left\{1, \frac{1}{2}, \frac{1}{3}, \ldots, \frac{1}{n},\ldots\right\}=\left\{\frac{1}{n}\right\}=\left\{\frac{1}{n}\right\}_{n=1}^\infty$, $a_n=\frac{1}{n}$ \newline
Note this sequence could also have been represented by $\left\{\frac{1}{n+1}\right\}_{n=0}^\infty$
\item $\left\{\frac{1}{2},\frac{-4}{5}, \frac{9}{8}, \ldots,\frac{(-1)^{n+1}n^2}{3n-1},\ldots\right\}=\left\{\frac{(-1)^{n+1}n^2}{3n-1}\right\}_{n=1}^\infty$, $a_n=\frac{(-1)^{n+1}n^2}{3n-1}$
\item $\left\{4,4,4,\ldots,4,\ldots\right\}=\left\{4\right\}_{n=1}^\infty$, $a_n=4$
\end{enumerate}
\end{example}

A sequence may not have a simple defining function in terms of the index $n$.
\begin{example}
The sequence generated by the digits of $\pi=3.14159\ldots$
\[\left\{3,1,4,1,5,9,\ldots\right\}\]
is not representable by a simple function $f(n)$.
\end{example}

Since theoretically any sequence is a function $a_n=f(n)$ on the set of positive integers we can graphically represent it by plotting the coordinate points $(n,a_n)$.

\begin{example}
  A graph of the sequence $\displaystyle\left\{\frac{\cos(n\pi)+n^2}{2n^2}\right\}$ is as follows:
\begin{center}
\includegraphics{graphics/sequences_series1}%
\end{center}
\end{example}

The above graph clearly approaches the value $1/2$ as $n$ gets large.  In symbols we would write
\[\lim_{n\to\infty}\frac{\cos(n\pi)+n^2}{2n^2}=\frac{1}{2}\]
This limit is analagous to the limit of a function $f(x)$ as $x\to\infty$ with the only difference being that $n$ is restricted to positive integers.
This discussion motivates the following definition for the limit of a sequence.\footnote{A more rigorous definition of the limit of a sequence is that $a_n\to L$ as $n\to\infty$ if and only if for any $\epsilon>0$ there exists $m>0$ such that $n>m$ implies $|a_n-L|<\epsilon$.}
\definition{If the terms $a_n$ of sequence $\left\{a_n\right\}$ get arbitrarily close to the value $L$ for sufficiently large~$n$ then we say the sequence \textbf{converges} to \textbf{limit} $\bm{L}$ or \textbf{is convergent} \index{sequence! convergence of} with \textbf{limit} $\bm{L}$.  Symbolically $a_n\to L$ as $n\to\infty$ or
\[\lim_{n\to\infty}a_n=L\ .\]
If a sequence is not convergent (i.e. it has no limit) then the sequence \textbf{diverges} or \textbf{is divergent}.} \index{sequence! divergence of}

A divergent sequence may have a trend to infinity.\footnote{A more rigorous definition for $\displaystyle\lim_{n\to\infty}a_n=\infty$ is that for any $M>0$ there exists an index $m>0$ such that $n>m$ implies $a_n>M$.}

\definition{If the terms $a_n$ of sequence $\{a_n\}$ get arbitrarily large (positively) for sufficiently large $n$ we say that the sequence $\{a_n\}$ \textbf{diverges to infinity} and we write
\[\lim_{n\to\infty} a_n=\infty\]
An analagous definition holds for a sequence to diverge to $-\infty$.}

\begin{example}
The \emph{Fibonacci Sequence} \index{sequence!Fibonacci} satisfies $a_1=1$, $a_2=1$ and $a_n=a_{n-2}+a_{n-1}$ for $n > 2$, i.e.
\[\left\{1,1,2,3,5,8,13,21,\ldots\right\}\]
The sequence diverges to $\infty$ $\left(\displaystyle\lim_{n\to\infty}a_n=\infty\right)$.
\end{example}

The limit of a sequence with $a_n=f(n)$ is essentially the limit of $f(x)$ as $x\to\infty$ with $x$ restricted to the positive integers (instead of the continuous real axis).
\begin{example}
  If we plot $\displaystyle y=f(x)=\frac{\cos(x\pi)+x^2}{2x^2}$ over our earlier sequence we have:
\begin{center}
  \includegraphics{graphics/sequences_series2}
\end{center}
\end{example}

The limit of $f(n)$, with $n$ an integer, clearly cannot differ from that of $f(x)$ if the latter exists, thereby leading to the following theorem.
\theorem{\label{thm:cvgascontinuous}If $\displaystyle\lim_{x\to\infty}f(x)=L$ then the limit of sequence $\left\{a_n\right\}$ with $a_n=f(n)$ is also $L$, 
\[\lim_{n\to\infty}a_n=L\ .\]
}

(Note the converse of this theorem is not true, $\displaystyle\lim_{n\to\infty}a_n=L \not\Rightarrow\lim_{x\to\infty}f(x)=L$ . )

For a sequence which converges to $L=0$ we have the following result:
\theorem{\label{thm:cvgtozero}$\displaystyle\lim_{n\to\infty}a_n=0$ if and only if $\displaystyle\lim_{n\to\infty}|a_n|=0$.}

These theorems are convenient for evaluating the limits of certain sequences.

\begin{example}
  Find the limit of the following sequences:
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \left\{\frac{n^{2}+4}{3n^{2}+5n+1}\right\}$ 
    \item $\displaystyle \left\{\frac{n^{2}+\ln n}{4n^{2}}\right\}$
    \item $\displaystyle \left\{(-1)^{n}\cdot\frac{\ln n}{n^{2}}\right\}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate} 
\item To evaluate the limit pull out the highest power of $n$ (here $n^2$) from each of the numerator and denominator.
\begin{align*}
  \lim_{n \to \infty} \frac{n^{2}+4}{3n^{2}+5n+1}
  =&\lim_{n \to \infty} \frac{n^2}{n^2}\cdot\frac{\frac{n^2}{n^2}+\frac{4}{n^2}}{\frac{3n^2}{n^2}+ \frac{5n}{n^2} +\frac{1}{n^2}} =\lim_{n \to \infty} 1\cdot\frac{1+\frac{4}{n^2}}{3+\frac{5}{n}+\frac{1}{n^2}} 
=\frac{1+0}{3+0+0} =\frac{1}{3}
\end{align*}
\item Here the $n^{\text{th}}$ term $a_n=f(n)$ where $\displaystyle f(n)=\frac{n^2+\ln n}{4n^2}$. Considering the function as a function of a continuous variable $x$, so $\displaystyle f(x)=\frac{x^2+\ln x}{4x^2}$, we have:
\begin{align*}
\lim_{x \to \infty} f(x)&=\lim_{x \to \infty}\frac{x^2+\ln x}{4x^2} \,\,\,
\left(\frac{\infty}{\infty}\ \text{form}\right)\\
&\underset{\lh}{=}\lim_{x \to \infty} \frac{2x+\frac{1}{x}}{8x} \,\,\, \left(\frac{\infty}{\infty}\ \text{form}\right)\\
&\underset{\lh}=\lim_{x \to \infty}\frac{2-\frac{1}{x^2}}{8}=\frac{2-0}{8}=\frac{1}{4} \\
\end{align*}
Therefore $\displaystyle\lim_{n \to \infty} \frac{n^2+\ln n}{4n^2}=\frac{1}{4}$\, by Theorem~\ref{thm:cvgascontinuous}.
\item Consider the limit of the absolute value of $\displaystyle a_n=\frac{(-1)^n\ln n}{n^2}$\,:
\[\lim_{n\to\infty}|a_n|=\displaystyle \lim_{n \to \infty}\left|(-1)^{n}\cdot \frac{\ln n}{n^2}\right|=\lim_{n \to \infty}\frac{\ln n}{n^2} \,\,\, \left(\frac{\infty}{\infty}\ \text{form}\right)\]
Letting $\displaystyle f(x)=\frac{\ln x}{x^2}$, a continuous function of $x$, we have
\begin{align*}
\lim_{x \to \infty} f(x)&=\lim_{x \to \infty}\frac{\ln x}{x^2} \,\,\, \left(\frac{\infty}{\infty}\text{form}\right)
\underset{\lh}=\lim_{x \to \infty} \frac{\frac{1}{x}}{2x}
                           =\lim_{x \to \infty} \frac{1}{2x^2}=0\\
                        & \implies \lim_{n\to\infty}|a_n|= \lim_{n \to \infty} \frac{\ln n}{n^2}=0\ (\text{by\ Theorem~\ref{thm:cvgascontinuous}})\\
&\implies  \lim_{n\to\infty}a_n=\lim_{n \to \infty} (-1)^{n}\frac{\ln n}{n^2}=0\ (\text{by\ Theorem~\ref{thm:cvgtozero}})
\end{align*}
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Find the limits of the following sequences:
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle\left\{\frac{2n}{5n-3}\right\}$
    \item $\displaystyle\left\{\frac{5n}{e^{2n}}\right\}$
    \item $\displaystyle\left\{\frac{(-1)^n(3n+1)}{n^2+5}\right\}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

\theorem{Given sequences $\left\{a_n\right\}$ and $\left\{b_n\right\}$ are convergent, $c$ is a constant, and $f$ a function defined at $a_n$ and continuous at $\displaystyle L=\lim_{n\to\infty} a_n$, then the following hold:
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \lim_{n\to\infty}c = c$
    \item $\displaystyle\lim_{n\to\infty}\left(a_n\pm b_n\right)=\lim_{n\to\infty} a_n\pm\lim_{n\to\infty} b_n $
    \item $\displaystyle\lim_{n\to\infty} ca_n=c\lim_{n\to\infty} a_n$
    \item $\displaystyle\lim_{n\to\infty}\left(a_n\cdot b_n\right)=\left(\lim_{n\to\infty} a_n\right)\cdot\left(\lim_{n\to\infty} b_n\right)$
    \item $\displaystyle\lim_{n\to\infty}\frac{a_n}{b_n}=\frac{\displaystyle\lim_{n\to\infty} a_n}{\displaystyle\lim_{n\to\infty} b_n}$\\[1em] (Here we require $\displaystyle \lim_{n\to\infty} b_n\neq 0$.)
    \item $\displaystyle\lim_{n\to\infty}f\left(a_n\right)=f\left(\lim_{n\to\infty} a_n\right)$
    \end{enumerate}
  \end{multicols}
}
By the last item of the previous theorem applied to $f(x)=x^k$ one has the corollary
\theorem{Given non-negative sequence  $\left\{a_n\right\}$ (i.e. $a_n\ge 0$) and power $k>0$ one has
\[\lim_{n\to\infty}(a_n)^k=\left(\lim_{n\to\infty}a_n\right)^k~.\]}

Here the sequence must have $a_n\ge0$ for $(a_n)^k$ to be defined and the power $k$ cannot be negative to accommodate sequences for which $\displaystyle\lim_{n\to\infty} a_n=0$.

\definition{A sequence is \textbf{geometric} if it has the form $\left\{ar^n\right\}_{n=0}^\infty=\left\{a,ar,ar^2,ar^3,\ldots,ar^n,\ldots\right\}$ ($a\neq 0$). Here $a$ is the first term in the sequence and $r$ is called the common ratio since $\frac{a_{n+1}}{a_n}=r$~(constant) for a geometric sequence.}

One notes that a geometric sequence is of the form $\left\{ar^{n-1}\right\}_{n=1}^\infty$ if we start with index $n=1$.

The following theorem results from consideration of the behaviour of the limit of the exponential function $\lim_{x\to\infty}r^x$ when $r\ge0$ in Theorem~\ref{thm:cvgascontinuous} and use of Theorem~\ref{thm:cvgtozero} noting that $\lim_{n\to\infty}|r^n|=\lim_{n\to\infty}|r|^n$ when $-1<r<0$.   
\theorem{The geometric sequence $\left\{ar^n\right\}_{n=0}^\infty=\left\{a,ar,ar^2,ar^3,\ldots,ar^n,\ldots\right\}$ ($a\neq 0$) is convergent when $-1<r\le1$ with
\[\lim_{n\to\infty}ar^n=\left\{\begin{array}{cl}
0 & \mbox{if $-1<r<1$}\\
a & \mbox{if $r=1$}
\end{array}
\right.\ .\]
It is divergent for all other values of $r$, diverging to infinity ($\infty$) for $r>1$.
}

\begin{example}
  Determine whether the following sequences are divergent or convergent. For convergent sequences determine the limit.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \left\{5\left(\frac{1}{10}\right)^n\right\}_{n=0}^\infty$ 
    \item $\displaystyle \left\{3(-5)^n\right\}$
    \item $\displaystyle \left\{\frac{2n+\ln n}{5n+3\ln n}\right\}$
    \end{enumerate}
  \end{multicols}{3}
\begin{solution*}
\begin{enumerate}
\item This is a geometric sequence with $a=5$ and $\displaystyle r=\frac{1}{10}$. Since $\displaystyle|r|=\left|\frac{1}{10}\right| < 1$ it is convergent with
\begin{align*}
\lim_{n \to \infty}5\left(\frac{1}{10}\right)^{n}=0\,.
\end{align*}
\item Without explicit index labels we infer $\left\{3(-5)^n\right\}=\left\{3(-5)^n\right\}_{n=1}^\infty$.  We can put it in standard form for a geometric sequence starting at $n=1$ by rewriting it:

 \[\left\{3(-5)^n\right\}_{n=1}^\infty=\left\{3(-5)(-5)^{n-1}\right\}_{n=1}^\infty=\left\{-15(-5)^{n-1}\right\}_{n=1}^\infty\,.\]
 This is a geometric sequence with $a=-15$ and  $r=-5$. Since $r=-5<-1 $ it is divergent.

 Alternatively we can show the sequence is geometric by evaluating 
 \[\frac{a_{n+1}}{a_n}=\frac{3(-5)^{n+1}}{3(-5)^n}=-5\]
 proving it has a constant (common) ratio of $r=-5$. The first term is $a=3(-5)^1=-15$\,.
 
\item Let $\displaystyle f(x)=\frac{2x+\ln x}{5x+3\ln x}$
\begin{align*}
\lim_{x \to \infty} f(x)=\lim_{x \to \infty} \frac{2x+\ln x}{5x+3\ln x}
\underset{\lh}=&\lim_{x \to \infty}\frac{2+\frac{1}{x}}{5+\frac{3}{\ln x}}=\frac{2+0}{5+0}=\frac{2}{5}\\
\implies  &\lim_{n \to \infty} \frac{2n+\ln n}{5n+3\ln n} =\frac{2}{5} 
\end{align*}
Therefore the sequence is convergent. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the following sequences are divergent or convergent.  For convergent sequences determine the limit.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle\left\{\left(\frac{n+1}{8n}\right)^{\frac{1}{3}}\right\}$
    \item $\displaystyle\left\{5\left(\frac{1}{2}\right)^n\right\}$
    \item $\displaystyle\left\{2^n\right\}$
    \item $\displaystyle\left\{\left(-\frac{1}{3}\right)^n\right\}$
    \item $\displaystyle\left\{(-3)^n\right\}$
    \end{enumerate}
    \end{multicols}
\end{furtherquestions}
\end{example}

A special class of sequences are those that are \textbf{monotonic}.\index{sequence! monotonic}
\definition{A sequence is \textbf{monotonic} if it is either
\begin{description}
\item[increasing]: $a_n<a_{n+1}$ for all $n$, or
\item[decreasing]: $a_n>a_{n+1}$ for all $n$, or
\item[nondecreasing]: $a_n\le a_{n+1}$ for all $n$, or
\item[nonincreasing]: $a_n\ge a_{n+1}$ for all $n$.
\end{description}
}
Note that increasing sequences are, by definition, nondecreasing as are decreasing sequences nonincreasing.  An example of a nondecreasing sequence that is not an increasing sequence is
\[\left\{1,1,2,2,3,3,4,4,\ldots\right\}\]

\textbook{\clearpage}
\begin{example}
Classify the monotonicity of the following sequences.
\begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle \left\{\frac{n^{2}+3}{n+5}\right\}$ 
\item $\displaystyle \left\{\sqrt{n+2}-\sqrt{n}\right\}$
\end{enumerate}
\end{multicols}
\begin{solution*}
\begin{enumerate}
\item We illustrate two approaches:
  \begin{itemize}
  \item Method 1: Using the Definition
    \begin{align*}
      a_n&<a_{n+1}\ \ (n\ge1)\\
   \iff   \frac{n^2+3}{n+5}&<\frac{(n+1)^2+3}{(n+1)+5} \ \ (n\ge1)\\
   \iff   \frac{n^2+3}{n+5}&<\frac{n^2+2n+4}{n+6} \ \ (n\ge1)\\
   \iff   (n^2+3)(n+6)&<(n^2+2n+4)(n+5) \ \ (n\ge1)\\
   \iff   n^3+6n^2+3n+18&<n^3+5n^2+2n^2+10n+4n+20\ \ (n\ge1)\\
   \iff   0&<n^2+11n+2\ \ (n\ge1)
    \end{align*}
    Since the final statement is true we have proven that $a_n<a_{n+1}$ and  thus $\displaystyle \left\{\frac{n^{2}+3}{n+5}\right\}$ is an increasing sequence.
  \item Method 2: Using the Derivative
    
  Consider $\displaystyle f(x)=\frac{x^2+3}{x+5}$. Then 
\begin{align*}
f'(x)&=\frac{2x(x+5)-(x^2+3)(1)}{(x+5)^2}
=\frac{2x^2+10x-x^2-3}{(x+5)^2}
       =\frac{x^2+10x-3}{(x+5)^2} > 0 \ \text{for} \ x\geq 1\,.
\end{align*}
Therefore, $f(x)$ is an increasing function for $x\ge 1$. Thus  $\displaystyle \left\{\frac{n^{2}+3}{n+5}\right\} $ is an increasing sequence.
\end{itemize}
Since the sequence is increasing it is also nondecreasing.
\item
Considering $f(x)=\sqrt{x+2}-\sqrt{x}$ we have
\begin{align*}
f'(x)&=\frac{1}{2}(x+2)^{-\frac{1}{2}}(1)-\frac{1}{2}x^{-\frac{1}{2}} \\
&=\frac{1}{2\sqrt{x+2}}-\frac{1}{2\sqrt{x}} < 0\ \text{for}\ x\ge1\,.
\end{align*}
Therefore $f(x)$ is a decreasing function for $x\ge 1$. Thus $\displaystyle \left\{\sqrt{n+2}-\sqrt{n}\right\}$ is a decreasing sequence, and also, therefore, a nonincreasing sequence. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Classify the monotonicity of the following sequences.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle\left\{\frac{2}{n+3}\right\}$
    \item $\displaystyle\left\{\frac{2n+3}{3n+5}\right\}$
    \item $\displaystyle\left\{\frac{3n+2}{2n+1}\right\}$
    \end{enumerate}
    \end{multicols}
\end{furtherquestions}
\end{example}

\definition{Sequence $\left\{a_n\right\}$ is \textbf{bounded below} \index{sequence! bounded below} if there exists some $N$ such that $N\le a_n$ for all $n$. The sequence $\left\{a_n\right\}$ is \textbf{bounded above} \index{sequence! bounded above} if there exists some $M$ such that $a_n\le M$ for all $n$.  The sequence $\left\{a_n\right\}$ is \textbf{bounded} \index{sequence! bounded} if it is bounded both below and above.}

\begin{example}
  Determine whether the following sequences are bounded above, bounded below, or bounded.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \{e^n\} $
    \item $\displaystyle \{-n^4\}$
    \item $\displaystyle \left\{\frac{2n^2+1}{n^2+3}\right\}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item  Since $e^{n}>0$ for all $n \geq 1$ the sequence is bounded below. $\displaystyle\lim_{n\to\infty} e^n=\infty$ shows that it is not bounded above.
\item  Since $-n^{4}<0$ for all $n\geq 1$ the sequence is bounded above. $\displaystyle\lim_{n\to\infty}-n^4=-\infty$ shows that it is not bounded below.  
\item Clearly $\displaystyle \frac{2n^2+1}{n^2+3} > 0$ for all n $ \geq 1$ so the sequence is bounded below. Considering the limit of the sequence
  \begin{align*}
  \lim_{n \to \infty}\frac{2n^2+1}{n^2+3}=\lim_{n \to \infty}\frac{n^2}{n^2}\cdot\frac{\frac{2n^2}{n^2}+\frac{1}{n^2}}{\frac{n^2}{n^2}+\frac{3}{n^2}}=\lim_{n \to \infty}1\cdot\frac{2+\frac{1}{n^2}}{1+\frac{1}{n^2}}=\frac{2+0}{1+0}=2
  \end{align*}
  suggests it may have an upper bound around $2$.  The inequality
    \begin{align*}
    \frac{2n^2+1}{n^2+3}<2\iff 2n^2+1<2(n^2+3)\iff 0<5
  \end{align*}
  confirms $2$ is an upper bound for the sequence.
  Thus $\displaystyle 0<\frac{2n^2+1}{n^2+3}<2$ and the sequence is bounded.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the following sequences are bounded above, bounded below, or bounded.\begin{multicols}{2}
\begin{enumerate}
\item $\left\{n^2\right\}$
\item $\displaystyle\left\{\frac{n}{n+1}\right\}$
\end{enumerate}
\end{multicols}
\end{furtherquestions}
\end{example}

The following theorem can be used to determine whether a monotonic sequence converges or not.

\theorem{A monotonic sequence converges if and only if it is bounded.}

\textbook{\clearpage}
\begin{exercise}
\input{SEQUENCES_AND_SERIES/sequences.tex}
\end{exercise}

\section{Series}\index{series}

\definition{Given a sequence $\left\{a_k\right\}$, the summation of its terms,
\[a_1+a_2+a_3+\cdots+a_k+\cdots\ \]
is called an \textbf{(infinite) series}\index{series! infinite}.  A series is abbreviated in \textbf{sigma notation} as $\displaystyle\sum_{k=1}^\infty a_k$ or sometimes without explicit index limits as $\sum a_k$.}

Due to the sum being over an infinite number of terms it need not exist.
\begin{example}
The series
\[\sum_{k=1}^\infty 1=1+1+\cdots+1+\cdots\]
clearly cannot approach a number when added.
\end{example}

To rigorously define what we mean by the value of the sum of a series we introduce the following.

\definition{Given the series $\displaystyle\sum_{k=1}^\infty a_k$ define the sum of the first $n$ terms of the series to be the $\bm{n}^{\mbox{\tiny\textbf{th}}}$~\textbf{partial sum}
\index{series! partial sum of}
$\bm{S_n}$:
\[S_n=\sum_{k=1}^n a_k=a_1+a_2+\cdots+a_n\]
}

\begin{example}
For the series $1+1+1+\cdots+1+\cdots$ the $n^{\mbox{\tiny th}}$ partial sums are
\begin{eqnarray*}
S_1&=&1 \\
S_2&=&1+1=2\\
S_3&=&1+1+1=3\\
\vdots \\
S_n&=&\underbrace{1+1+1+\cdots+1}_{n\ \mbox{\tiny times}}=n
\end{eqnarray*}
\end{example}

The partial sums $S_n$ for a series $\sum a_k$ themselves form a sequence $\{S_n\}$ the limit of which we will consider the sum of the series.

\definition{Let series $\sum a_k$ have $n^{\mbox{\tiny th}}$ partial sums $S_n$.  If the sequence $\{S_n\}$ is convergent, so
\[\lim_{n\to\infty}S_n=S\ ,\]
then we say that the series $\sum a_k$ is \textbf{convergent}\index{series! convergent} and call $S$ the \textbf{sum} \index{series! sum of} of the series,
\[\sum_{k=1}^\infty a_k=a_1+a_2+a_3+\cdots+a_k+\cdots=S\ .\]
If a series is not convergent then it is \textbf{divergent} \index{series! divergent} .}

\begin{example}
The series $\displaystyle\sum_{k=1}^\infty\frac{1}{(k+1)(k+2)}$ can be written using partial fraction decomposition as $\displaystyle\sum_{k=1}^\infty\left(\frac{1}{k+1}-\frac{1}{k+2}\right)$.  The $n^{\mbox{\tiny th}}$ partial sum is therefore
\[S_n=\left(\frac{1}{2}-\frac{1}{3}\right)+\left(\frac{1}{3}-\frac{1}{4}\right)+\cdots+
\left(\frac{1}{n+1}-\frac{1}{n+2}\right)=\frac{1}{2}-\frac{1}{n+2}\]
Since $\displaystyle \lim_{n\to\infty}S_n=\lim_{n\to\infty}\left(\frac{1}{2}-\frac{1}{n+2}\right)=\frac{1}{2}$ the series is convergent with sum $1/2$, i.e.
\[\sum_{k=1}^\infty\frac{1}{(k+1)(k+2)}=\frac{1}{2}\ .\]
Because of the cancellation arising in the partial sum the series is called a \textbf{telescoping series}\index{series! telescoping}. The series collapses like a spyglass in which the tubes disappear into each other to shorten the telescope.
\end{example}

\begin{example}
  We saw the series $\displaystyle \sum_{k=1}^\infty 1$ has partial sum $S_n=n$.  Therefore $\displaystyle\lim_{n\to\infty}S_n=\lim_{n\to\infty}n=\infty$ and so the sequence $\{S_n\}$ and hence the series $\displaystyle \sum_{k=1}^\infty 1$ are divergent (as expected).
\end{example}

\theorem{The \textbf{geometric series} \index{series! geometric}
\[\sum_{k=0}^\infty ar^k=a+ar+ar^2+\cdots+ar^k+\cdots=a\left(1+r+r^2+\cdots+r^k+\cdots\right)\]
is convergent if $-1<r<1$ with sum $\displaystyle\frac{a}{1-r}$ and is otherwise divergent.\footnote{Proof follows by noting that if $S_n=1+ar+\cdots+ar^{n-1}$ then
\[(1-r)S_n=a+ar+ar^2+\cdots+ar^{n-1}-\left(ar+ar^2+ar^3+\cdots+ar^n\right)=a-ar^n,\]
and so $S_n=\frac{a\left(1-r^n\right)}{1-r}$.  Then $\displaystyle S=\lim_{n\to\infty}S_n=\frac{a}{1-r}$ since $\displaystyle\lim_{n\to\infty}r^n=0$ for $-1<r<1$\,.}}

One notes that the geometric series also has the form $\displaystyle \sum_{k=1}^\infty ar^{k-1}$ if the lower limit is taken to be $k=1$ rather than $k=0$\,.

\begin{example}
Determine whether the following series are convergent and, if so, find the sum.
\begin{enumerate}
\item $\displaystyle 2+6+18+54+\cdots$
\item $\displaystyle\sum_{k=0}^\infty \frac{(-1)^k6^{k+1}}{5^k}$
\item $\displaystyle\sum_{n=1}^\infty \left(\frac{1}{4}\right)^n$
\end{enumerate}
\begin{solution*}
  \begin{enumerate}
  \item The series has first term $a=2$ and the ratios of subsequent terms
    \[\frac{6}{2}=\frac{18}{6}=\frac{54}{18}=\cdots\]
    all equal to $3$ shows the series is geometric with common ratio $r=3$. Since $|r|=3>1$ the geometric series is divergent.  Alternatively we note that we can write the series in sigma notation as $\displaystyle\sum_{k=0}^\infty 2(3)^k$ or $\displaystyle\sum_{k=1}^\infty2(3)^{k-1}$ to prove it is geometric and to identify $a=2$ and $r=3$.
  \item The series starts at $k=0$ and so the standard form for that geometric series is $\displaystyle\sum_{k=0}^\infty ar^k$. Rewriting the series
    \[\sum_{k=0}^\infty \frac{(-1)^k6^{k+1}}{5^k}=\sum_{k=0}^\infty 6\frac{(-1)^k6^k}{5^k}=\sum_{k=0}^\infty 6\left(-\frac{6}{5}\right)^k\]
    proves the series is geometric with first term $a=6$ and common ratio $r=-\frac{6}{5}$. Since $|r|=\frac{6}{5}>1$ the geometric series is divergent.
  \item The series starts at $n=1$ so put it in the standard form $\displaystyle\sum_{n=1}^\infty ar^{n-1}$\,:
    \[\sum_{n=1}^\infty \left(\frac{1}{4}\right)^n=\sum_{n=1}^\infty \left(\frac{1}{4}\right)\left(\frac{1}{4}\right)^{n-1}\implies a=\frac{1}{4},\, r=\frac{1}{4}\]
    Then $|r|<1$ implies the series converges to sum
   \[S=\frac{a}{1-r}=\frac{\frac{1}{4}}{1-\frac{1}{4}}=\frac{\frac{1}{4}}{\frac{3}{4}}=\frac{1}{4}\cdot\frac{4}{3}=\frac{1}{3}\]
  \end{enumerate}
\end{solution*}
\begin{furtherquestions}
Determine whether the following series are convergent and, if so, find the sum.
\begin{enumerate}
\item $\displaystyle2+\frac{2}{2}+\frac{2}{4}+\cdots+2\left(\frac{1}{2}\right)^{n-1}+\cdots$
\item $\displaystyle\sum_{n=1}^\infty (3)^{n-1}$
\item $\displaystyle 1+x+x^2+x^3+\cdots$\ \  (for $|x|<1)$
\end{enumerate}
\end{furtherquestions}
\end{example}

\textbook{\clearpage}
A necessary (but not sufficient) requirement for a series to converge is that its terms must approach zero as $n\to\infty$.

\theorem{\label{thm:seriestermsvanish}If series $\displaystyle\sum_{k=1}^\infty a_k$ is convergent then $\displaystyle\lim_{k\to\infty}a_k=0$\,.}

Note the converse of the last theorem, namely that if $\displaystyle\lim_{k\to\infty}a_k=0$ then $\displaystyle\sum_{k=1}^\infty a_k$ is convergent, is \textbf{not} true, as demonstrated in the following example.

\begin{example}
The \textbf{harmonic series},\index{series! harmonic}
\[\sum_{k=1}^\infty\frac{1}{k}=1+\frac{1}{2}+\frac{1}{3}+\cdots+\frac{1}{k}+\cdots\]
is divergent.  To see this note that the harmonic series%Proof was by Nicole Oresme
\[1 \;\;+\;\; \frac{1}{2} \;\;+\;\; \frac{1}{3} \,+\, \frac{1}{4} \;\;+\;\; \frac{1}{5} \,+\, \frac{1}{6} \,+\, \frac{1}{7} \,+\, \frac{1}{8} \;\;+\;\; \frac{1}{9} \,+\, \cdots\]
is strictly greater than
\[\underbrace{\frac{1}{1}}_{>\frac{1}{2}} + \underbrace{\frac{1}{2}}_{=\frac{1}{2}} \,+\;\; \underbrace{\frac{1}{4} \,+\, \frac{1}{4}}_{=\frac{1}{2}} \;\;+\;\; \underbrace{\frac{1}{8} \,+\, \frac{1}{8} \,+\, \frac{1}{8} \,+\, \frac{1}{8}}_{=\frac{1}{2}} \;\;+\;\; \frac{1}{16} \,+\, \cdots,\]

which diverges to infinity as one can exceed any multiple of $1/2$ one wants by taking enough terms.  Specifically, the  partial sums of the harmonic series form an increasing sequence in which $S_{2^n}>\frac{1}{2}(n+1)$ and therefore the sequence $\{S_n\}$ is unbounded (and hence divergent).
\end{example}

The contrapositive of Theorem~\ref{thm:seriestermsvanish} (which logically must be true) provides a useful method to test if some series are divergent:

\theorem{\textbf{The Term Test for Divergence:}\index{Term Test for Divergence}\newline
If the terms $a_k$ of series $\sum a_k$ approach a non-zero limit $\left(\displaystyle\lim_{k\to\infty}a_k=L\neq 0\right)$ or $\displaystyle\lim_{k\to\infty}a_k$ does not exist, then $\sum a_k$ is divergent.
}

We note that by Theorem~\ref{thm:cvgtozero} it follows that if $\displaystyle\lim_{k\to\infty}|a_k|\neq0$ or does not exist then $\displaystyle\lim_{k\to\infty}a_k$ must either be non-zero or not exist and vice versa.  As such, when applying the Term Test for Divergence, it is sufficient to test $\displaystyle\lim_{k\to\infty}|a_k|$ which may be easier to evaluate.

\begin{example}
  Determine whether the series
  \[\sum_{k=1}^\infty\cos\left(\frac{1}{k}\right)\]
  converges or diverges.
  
  \begin{solution}
    Let $a_k=\cos\left(\frac{1}{k}\right)$ be the $k^{\text{th}}$ term in the series. Consider $\cos\left(\frac{1}{x}\right)$, a continuous function of $x$. Then
    \[\lim_{x\to\infty}\cos\left(\frac{1}{x}\right)=\left(\lim_{x\to\infty}\frac{1}{x}\right)=\cos(0)=1\]
    implies, by Theorem~\ref{thm:cvgascontinuous}, $\displaystyle\lim_{k\to\infty}a_k=\lim_{k\to\infty}\cos\left(\frac{1}{k}\right)=1\neq0$. By the Term Test for Divergence the series $\sum a_k$  is divergent.
  \end{solution}

\begin{furtherquestion}
Determine whether the series
\[\frac{1}{3}+\frac{2}{5}+\frac{3}{7}+\cdots+\frac{k}{2k+1}+\cdots\]
converges or diverges.
\end{furtherquestion}
\end{example}

Note that if we find $\displaystyle\lim_{n\to\infty}a_k=0$ we know nothing about the convergence or divergence of series~$\sum a_k$\,.

\theorem{\label{thm:serieslinearity}If $c$ is any constant and $\displaystyle \sum_{k=1}^\infty a_k$, $\displaystyle \sum_{k=1}^\infty b_k$ are convergent series then the following series are convergent with the given results:
\begin{enumerate}
\item $\displaystyle\sum_{k=1}^\infty ca_k=c\sum_{k=1}^\infty a_k$
\item $\displaystyle\sum_{k=1}^\infty(a_k\pm b_k)=\sum_{k=1}^\infty a_k\pm\sum_{k=1}^\infty b_k$
\end{enumerate}
If $\displaystyle\sum_{k=1}^\infty a_k$ is divergent and $c\neq 0$ then $\displaystyle\sum_{k=1}^\infty ca_k$ is divergent.  If one of $\displaystyle \sum_{k=1}^\infty a_k$, $\displaystyle \sum_{k=1}^\infty b_k$ is convergent and one is divergent then $\displaystyle\sum_{k=1}^\infty(a_k\pm b_k)$ is divergent.}

\begin{example}
  Prove that the following series converges and find its sum.
  \[\sum_{i=1}^{\infty} \left(\frac{1}{5^i}+\frac{3}{10^i}\right)\]
  \begin{solution*}
\begin{align*}
\sum_{i=1}^{\infty} \left(\frac{1}{5^i}+\frac{3}{10^i}\right)&=\sum_{i=1}^{\infty}\frac{1}{5^i} +\sum_{i=1}^{\infty}\frac{3}{10^i}\\
&= \underbrace{\sum_{i=1}^{\infty}\frac{1}{5}\cdot\left(\frac{1}{5}\right)^{i-1}}_{\begin{array}{c}\text{geometric series}\\a=\frac{1}{5}, r= \frac{1}{5}\\ |r| < 1\ (\text{convergent})\end{array}}+\underbrace{\sum_{i=1}^{\infty}\frac{3}{10}\cdot\left(\frac{1}{10}\right)^{i-1}}_{\begin{array}{c}\text{geometric series}\\a=\frac{3}{10}, r= \frac{1}{10}\\ |r| < 1\ (\text{convergent})\end{array}} \\
&=\frac{\frac{1}{5}}{1-\frac{1}{5}}+\frac{\frac{3}{10}}{1-\frac{1}{10}}=\frac{\frac{1}{5}}{\frac{4}{5}}+\frac{\frac{3}{10}}{\frac{9}{10}}=\frac{1}{5}\cdot\frac{5}{4}+\frac{3}{10}\cdot\frac{10}{9}=\frac{1}{4}+\frac{1}{3}=\frac{3+4}{12}=\frac{7}{12}
\end{align*}
Thus the given series is convergent with sum $\displaystyle\frac{7}{12}$\,.
  \end{solution*}

\begin{furtherquestion}
Prove that the following series converges and find its sum.
\[\sum_{k=1}^\infty\left[\frac{2}{3^{k-1}}+\frac{7}{k(k+1)}\right]\]
\end{furtherquestion}
\end{example}

\begin{example}
  Determine whether each series is convergent or divergent. For convergent series, find its sum.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{2n^2+5}{6n^2+3n+1} $
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{4}{(n+1)(n+3)}$
    \item $\displaystyle \sum_{n=2}^\infty \left[\left(\frac{3}{4}\right)^n+\frac{n}{\ln n}\right]$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item  Let $a_n$ be the $n^{\text{th}}$ term in the series and consider $\displaystyle\lim_{n\to\infty}a_n$\,:
\begin{align*}
 \lim_{n \to \infty} \frac{2n^2+5}{6n^2+3n+1}&=\frac{n^2}{n^2}\cdot\lim_{n \to \infty}\frac{\frac{2n^2}{n^2} +
 \frac{5}{n^2}}{\frac{6n^2}{n^2} + \frac{3n}{n^2} +\frac{1}{n^2}}
 =\lim_{n \to \infty}1\cdot\frac{2+\frac{5}{n^2}}{6+\frac{3}{n}+ \frac{1}{n^2}} \\
 &=\frac{2+0}{6+0+0}=\frac{2}{6}=\frac{1}{3} \neq 0 \\
\end{align*}
Therefore the given series $\sum a_n$ is divergent by the Term Test for Divergence.
\item Using partial fraction decomposition, the series can be written as
\begin{align*}
  \sum_{n=1}^{\infty}\frac{4}{(n+1)(n+3)}=\sum_{n=1}^{\infty}\left(\frac{2}{n+1}-\frac{2}{n+3}\right)\,.
\end{align*}
The $n^{\text{th}}$ partial sum may be evaluated due to the telescoping terms:
\begin{align*}
S_n&=\left(\frac{2}{2}-\cancel{\frac{2}{4}}\right)+\left(\frac{2}{3}-\cancel{\frac{2}{5}}\right)+\left(\cancel{\frac{2}{4}}-\cancel{\frac{2}{6}}\right)+ \left(\cancel{\frac{2}{5}}-\cancel{\frac{2}{7}}\right) +\ldots+\left(\cancel{\frac{2}{n+1}}-\frac{2}{n+3}\right) \\
   &=\frac{2}{2}+\frac{2}{3}-\frac{2}{n+3}=1+\frac{2}{3}-\frac{2}{n+3}=\frac{5}{3}-\frac{2}{n+3}
\end{align*}
Taking the limit gives
\[S=\lim_{n \to \infty} S_n=\lim_{n \to \infty}\left(\frac{5}{3}-\frac{2}{n+3}\right)=\frac{5}{3}-0=\frac{5}{3}\,,\]
so the given series is convergent with sum $\displaystyle \frac{5}{3}$\,.

Note that the series $\sum\frac{2}{n+1}$ and $\sum\frac{2}{n+3}$ are both separately divergent since they are, up to a scalar multiplier and a few missing initial terms, both harmonic series.  So separating a series into two divergent components does not imply the original series is divergent.  
\item Consider the series as a sum of two series:
  \[\sum_{n=2}^\infty \left[\left(\frac{3}{4}\right)^n+\frac{n}{\ln n}\right]=\sum_{n=2}^\infty \left(\frac{3}{4}\right)^n+\sum_{n=2}^\infty\frac{n}{\ln n}\,.\]
The first series is geometric with $r=\frac{3}{4}$.  Since $|r|<1$ that series converges.  However the second series diverges since
\[\lim_{x\to\infty}\frac{x}{\ln x}\underset{\lh}=\lim_{x\to\infty}\frac{1}{1/x}=\lim_{x\to\infty} x=\infty\,,\]
which implies $\displaystyle\lim_{n\to\infty}\frac{n}{\ln n}=\infty$ and the second series diverges by the Term Test for Divergence.  Since the original series is the sum of a convergent and divergent series, it is divergent.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether each series is convergent or divergent.  For convergent series, find the sum.
  \begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle \sum_{k=1}^\infty \frac{3k}{5k-1}$
\item $\displaystyle \sum_{k=1}^\infty k!$%\ \ \  Here the factorial $k!=k\cdot(k-1)\cdots (1)$ for $k\ge 1$ (and $0!$ is defined to be $1$).
\item $\displaystyle \sum_{k=1}^\infty \left(\frac{1}{3^k}-\frac{1}{4^k}\right)$
\item $\displaystyle \frac{1}{2}+\frac{2}{3}+\cdots+\frac{n}{n+1}+\cdots$
\item $\displaystyle \sum_{k=1}^\infty \left[\left(\frac{3}{2}\right)^k+\left(\frac{2}{3}\right)^k\right]$
\item $\displaystyle \sum_{k=0}^\infty \frac{6^k}{7^{k+1}}$
\item $\displaystyle \sum_{n=3}^\infty \ln\left(\frac{2n}{3n-7}\right)$
\end{enumerate}
\end{multicols}
Here the factorial $k!=k\cdot(k-1)\cdots (1)$ for $k\ge 1$ (and $0!$ is defined to be $1$).
\end{furtherquestions}
\end{example}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/series.tex}
\end{exercise}

\section{Testing Series with Positive Terms}

It is often difficult to find an exact sum of a series.  In most cases a simple formula for the partial sum $S_n$ cannot be found.  It is therefore of interest to develop techniques to test whether a series is convergent or divergent.  We start by considering series $\sum a_k$ with positive ($a_k>0$) terms.  Because the terms are positive the sequence of partial sums, $\left\{S_n\right\}$, is increasing.

\subsection{The Integral Test} \index{Integral Test}

Suppose a series $\sum_{k=1}^\infty a_k$ has terms $a_k=f(k)$ written in terms of a function $f(x)$ that is continuous, positive, and decreasing for $x\ge 1$.  The integral $\int_1^{n} f(x)\,dx$ will be smaller than the partial sum $S_{n-1}$,
\[S_{n-1}=\sum_{k=1}^{n-1}a_k=a_1+a_2+\cdots+a_{n-1}=(a_1)(1)+(a_2)(1)+\cdots+(a_{n-1})(1)\,,\]
since the latter can be considered the total area of rectangles of height $a_k$ and width 1 for $k=1$ to $k=n-1$ as shown in the following diagram:
\begin{center}
\includegraphics{graphics/sequences_series3}%
\end{center}

% The following proof could be changed to not use contradiction.
Here we are considering the $a_k$ to be the height on the left side of the rectangles.  Consider the case that $\int_1^\infty f(x)\,dx$ is divergent.  Since $f(x)$ is positive, $\int_1^t f(x)\,dx $ is an increasing function of $t$ and $\int_1^\infty f(x)\,dx=+\infty$.  Suppose that increasing sequence $\left\{S_n\right\}$ were bounded with upper bound $M$. Then the  relationship $\int_1^n f(x)\,dx<S_{n-1}$ implies that $\int_1^n f(x)\,dx<M$ for any integer $n$ and therefore $\int_1^t f(x)\,dx<M$ for any real $t\ge 1$,  a contradiction to the divergence of $\int_1^\infty f(x)\,dx$.  Hence $\{S_n\}$ must be an unbounded monotonic sequence and therefore is divergent.  Thus if $\int_1^\infty f(x)\,dx$ is divergent then  $\sum a_k$ is divergent.

Alternatively if we consider rectangles with height being $a_k$ on the right (so $k=2$ to $k=n$) we have the following diagram:
\begin{center}
\includegraphics{graphics/sequences_series4}%
\end{center}


In this case it follows that the integral  $\int_1^{n} f(x)\,dx$ must be greater than:
\[(a_2)(1)+(a_3)(1)+\cdots+(a_n)(1)=a_2+a_3+\cdots+a_n=S_n-a_1\]
Consider the case where $\int_1^\infty f(x)\,dx$ is convergent.  Let $M$ be the value of the integral.  Since $f(x)$ is positive, $\int_1^nf(x)\,dx<M$.  From $S_n-a_1<\int_1^n f(x)\,dx$ it follows that for any $n$, $S_n<M+a_1$ and thus monotonic sequence $\{S_n\}$ is bounded and therefore convergent.  Thus if $\int_1^\infty f(x)$ is convergent, then $\sum a_k$ is convergent.

We summarize our result in the following theorem.  

\theorem{\textbf{The Integral Test:}\index{Test for convergence and divergence of a series! Integral Test}\newline
Let $f(x)$ be a continuous positive decreasing function for $x\ge 1$ and $\displaystyle\sum_{k=1}^\infty a_k$ be a series with $a_k=f(k)$.  
\begin{enumerate}
\item If $\int_1^\infty f(x)\,dx$ is convergent then $\displaystyle\sum_{k=1}^\infty a_k$ is also convergent.
\item If $\int_1^\infty f(x)\,dx$ is divergent then $\displaystyle\sum_{k=1}^\infty a_k$ is also divergent.
\end{enumerate}
}

\begin{example}\label{exam:integraltest}
  Determine whether the series converges or diverges.
  \begin{multicols}{2}
    \begin{enumerate}
    \item\label{itm:integraltest1} $\displaystyle \sum_{n=1}^{\infty} \frac{1}{n^2+16}$
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{(\ln n)^2}{n}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item Setting $\displaystyle f(x)=\frac{1}{x^2+16}$ we have that $f(x)$ is continuous and positive for $x\geq 1$.  Also 
\[f'(x)= \frac{-2x}{(x^2+16)^2}<0 \ \text{for}\ x>0 \]
shows $f(x)$ is decreasing for $x\geq 1$\,.
\begin{align*}
\int_{1}^{\infty}\frac{1}{x^2+16}\,dx &=\lim_{t \to \infty}\int^{t}_{1}\frac{1}{x^2+16}\,dx
=\lim_{t \to \infty}\left[\frac{1}{4}\tan^{-1}\left(\frac{x}{4}\right)\right|_{1}^{t}\\
&=\frac{1}{4}\lim_{t \to \infty}\left[\tan^{-1}\left(\frac{t}{4}\right)-\tan^{-1}\left(\frac{1}{4}\right)\right]
=\frac{1}{4}\left[\frac{\pi}{2}-\tan^{-1}\left(\frac{1}{4}\right)\right] 
\end{align*}
Integral $\displaystyle\int_{1}^{\infty}\frac{1}{x^2+16}\,dx \, $ is convergent and therefore the series $\displaystyle \sum_{n=1}^{\infty} \frac{1}{n^2+16}$ is convergent by the Integral Test.
\item Let $\displaystyle f(x)=\frac{(\ln x)^2}{x}$. Then $f(x)$ is continuous and positive for $x \geq 1$. Consider the derivative:
\[
f'(x)=\frac{2(\ln x)\frac{1}{x}\cdot x-(\ln x)^2(1)}{x^2}
=\frac{2(\ln x)-(\ln x)^2}{x^2}
  =\frac{(\ln x)(2-\ln x)}{x^2}
\]
For $x\ge 2$ both $\ln x$ and $x^2$ are positive.  Solve the inequality for the remaining factor to see where it decreases:
\[
  2-\ln x <0 \implies 2 < \ln x \implies e^2 < e^{\ln x}\implies e^2< x \implies x> e^2\approx7.3891\]
(Note here we used that $e^x$ is an increasing function so that the inequality was preserved under exponentiation.) Thus $f(x)$ is decreasing for $x>e^2$. Evaluating the improper integral from $x=8$ onward gives:
\begin{align*}
\int_{8}^{\infty}\frac{(\ln x)^2}{x}\,dx&=\lim_{t \to \infty}\int^{t}_{8}\frac{(\ln x)^2}{x}\,dx \ \ \ (\leftarrow u=\ln x\ \text{so}\ du=\frac{1}{x}dx)\\
& =\lim_{t \to \infty}\int^{\ln t}_{\ln 8} u^2\,du =\lim_{t \to \infty}\left[\frac{1}{3}u^3\right|_{\ln 8}^{\ln t}
  =\lim_{t \to \infty}\left[\frac{1}{3}(\ln t)^3-\frac{1}{3}(\ln 8)^3\right]= \infty
  \end{align*}
  Therefore the improper integral is divergent and the series $\sum_{n=8}^{\infty} \frac{(\ln n)^2}{n}$ must also diverge by the Integral Test.  The original series $\sum_{n=1}^{\infty} \frac{(\ln n)^2}{n}$ differs from the latter by a finite sum of seven terms and so it also diverges.

In practice, when applying the Integral Test and, indeed, other series convergence tests in this chapter, we need only apply the test to the infinite tail of the sequence starting at any finite index (like 8 here).  The requirements of the theorem have to only be met from that index forward.  The convergence (or not) of a series can only depend on the tail of the series as the initial sum of a finite number of terms is necessarily finite and cannot, therefore, impact the convergence properties of the series as a whole.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether each of the following series is convergent or divergent.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle\sum_{n=1}^\infty ne^{-n^2}$
    \item $\displaystyle\sum_{k=1}^\infty\frac{1}{k}$ (The harmonic series)
    \item $\displaystyle\sum_{k=1}^\infty\frac{1}{k^p}$ (The hyperharmonic or $p$-series)
    \item $\displaystyle 1+\frac{1}{2^2}+\frac{1}{3^2}+\cdots+\frac{1}{n^2}+\cdots$
    \item $\displaystyle\sum_{n=1}^\infty\frac{5}{\sqrt{n}}$
    \item $\displaystyle\sum_{n=1}^\infty\frac{\ln n}{n^2}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

We summarize our results for the $p$-series from the Further Questions of the last example in the following theorem.

\theorem{The $\bm{p}$\textbf{-series} $\displaystyle\sum_{k=1}^\infty\frac{1}{k^p}$ is convergent for $p>1$ and divergent otherwise.} \index{series! $\emph{p-}$}

Notes on the Integral Test:
\begin{enumerate}
\item The Integral Test can be relaxed to consider a function $f(x)$ that is continuous positive and decreasing only on $x\ge n$ with the corresponding integral $\int_n^\infty f(x)\,dx$.  This determines convergence or not of the series $\sum_{k=n}^\infty a_k$ but this in turn determines convergence of the entire series $\sum_{k=1}^\infty a_k$ since these two series differ only by a finite number of terms having a finite sum.
\item It follows from the Integral Test that the improper integral and the series either are both convergent or both divergent.  This means that determining the convergence or not of a series can be used to determine the convergence properties of the improper integral of a continuous positive decreasing function $f(x)$ if that were desired.
\end{enumerate}

\subsection*{Estimating the Series Sum} \index{series! estimating the series sum}

Even if a series is convergent it may be impossible to sum due to the impossibility of finding a closed form for the partial sum $S_n$ for which we can take the limit. In that case one may resort to numerically calculating the partial sum $S_n$ itself, for ``large'' $n$ as an approximation for the sum of the series, $S\approx S_n$.  The error in the approximation is the \textbf{remainder}\index{remainder estimates! for the Integral Test} $R_n=S-S_n$ which is the sum of the terms that were not included:
\[S=\sum_{k=1}^\infty a_k=\underbrace{a_1+a_2+\cdots+a_n}_{\displaystyle S_n=\sum_{k=1}^n a_k}+\underbrace{a_{n+1}+a_{n+2}+\cdots}_{\displaystyle R_n=\sum_{k=n+1}^\infty a_k}=S_n+R_n\]

For a convergent series $\sum a_k$ with $a_k=f(k)$ where $f(x)$ is a continuous positive decreasing function one can place bounds on the size of the remainder, thereby estimating the error in the numerical approximation.  In our previous discussion we found that the $n^{\mbox{\tiny th}}$ partial sum satisfied
\[S_n-a_1<\int_1^nf(x)\,dx<S_{n-1}\]
In the case of convergence these inequalities imply
\[S-a_1\le\int_1^\infty f(x)\,dx\le S~.\]
If we start summing at the $n^{\mbox{\tiny th}}$ term instead of the first this generalizes to
\[ \left(a_n+a_{n+1}+a_{n+2}+\cdots\right)-a_{n}\le\int_n^\infty f(x)\,dx\le a_n+a_{n+1}+a_{n+2}+\cdots~,\]
from which it follows that
\[ R_n\le\int_n^\infty f(x)\,dx\le R_{n-1}\ .\]
Thus $\int_n^{\infty}f(x)\,dx$ is an upper bound for the error $R_n$. The substitution $n-1\rightarrow n$ for the inequality on the right implies $\int_{n+1}^\infty f(x)\,dx\le R_n$, thereby providing a lower bound on the remainder (error) as well. We summarize the result in the following theorem:
\theorem{For convergent series $\displaystyle\sum_{k=1}^\infty a_k$ with $a_k=f(x)$ where $f(x)$ is a continuous positive decreasing function, the remainder $\displaystyle R_n=\sum_{k=n+1}^\infty a_k=S-S_n$ satisfies:
  \[\int_{n+1}^\infty f(x)\,dx \le R_n\le\int_n^\infty f(x)\,dx\ ,\]
  and therefore
    \[S_n+\int_{n+1}^\infty f(x)\,dx \le S\le S_n+\int_n^\infty f(x)\,dx\ .\]
}
\begin{example}
  \begin{enumerate}
  \item Estimate the sum of the series $\displaystyle\sum_{n=1}^\infty\frac{1}{n^2+16}$ using the third partial sum $S_3$.
  \item Find bounds on the error (the remainder $R_3$) that you are making by using that approximation.
  \end{enumerate}
  \begin{solution*}
    \begin{enumerate}
    \item From Example~\ref{exam:integraltest} Problem~\ref{itm:integraltest1} we know that the series is convergent.  The sum of the series $S$ is approximately:
      \[S_3=\sum_{n=1}^3\frac{1}{n^2+16}=\frac{1}{(1)^2+16}+\frac{1}{(2)^2+16}+\frac{1}{(3)^2+16}=\frac{1}{17}+\frac{1}{20}+\frac{1}{25}=\frac{253}{1700}\approx0.1488\]
    \item $S=S_3+R_3$ where $R_3$ satisfies
      \[\int_{3+1}^\infty \frac{1}{x^2+16}\le R_3\le \int_{3}^\infty \frac{1}{x^2+16}\,.\]
    \end{enumerate}
    Evaluate the integrals as we did in Example~\ref{exam:integraltest}\,.
    \begin{align*}
\int_{4}^{\infty}\frac{1}{x^2+16}\,dx &=\ldots=
                                        \frac{1}{4}\left[\frac{\pi}{2}-\tan^{-1}\left(\frac{4}{4}\right)\right]=\frac{1}{4}\left[\frac{\pi}{2}-\frac{\pi}{4}\right]=\frac{\pi}{16}\approx0.1963\\
\int_{3}^{\infty}\frac{1}{x^2+16}\,dx &=\ldots=
                                        \frac{1}{4}\left[\frac{\pi}{2}-\tan^{-1}\left(\frac{3}{4}\right)\right]\approx0.2318
    \end{align*}
    Thus $0.1963<R_3<0.2318$. Our estimate of $0.1488$ is therefore too low by somewhere between $0.1963$ and $0.2318$. In other words, the actual sum $S$ of the series must lie between $0.1488+0.1963=0.3451$ and $0.1488+0.2318=0.3806$\,.
  \end{solution*}

\begin{furtherquestions}
Leonhard Euler was able to show the sum of the $p$-series with $p=2$ is
\[\sum_{k=1}^\infty\frac{1}{k^2}=1+\frac{1}{4}+\frac{1}{9}+\frac{1}{25}+\cdots=\frac{\,\pi^2}{6}=1.644934\ldots\]
\begin{enumerate}
\item Find $S_4$\,.
\item Find the remainder $R_4$\,.
\item Show $R_4$ falls within the bounds of the last theorem.
\item What partial sum $S_n$ is required to be in error less than 0.01?
\end{enumerate}
\end{furtherquestions}
\end{example}

\subsection{The Basic Comparison Test}\index{Basic Comparison Test} \index{Test for convergence and divergence of a series! Basic Comparison Test}

We have seen several examples of series with their associated convergence properties:
\begin{description}
\item[geometric:]$\displaystyle \sum_{k=1}^\infty ar^{k-1}$ is convergent for $|r|<1$, divergent for $|r|\ge 1$ \index{series! geometric}
\item[telescoping:] $\displaystyle \sum_{k=1}^\infty \frac{1}{k(k+1)}$ (for example) is convergent \index{series! telescoping}
\item[harmonic:]$\displaystyle \sum_{k=1}^\infty \frac{1}{k}$ is divergent \index{series! harmonic}
\item[$\bm{p}$-series:]$\displaystyle \sum_{k=1}^\infty \frac{1}{k^p}$ is convergent for $p>1$, divergent for $p \le 1$ \index{series! $\emph{p-}$}
\end{description}


We now develop some series convergence tests that use the known convergence properties of one series to determine that of another.  The first test is a discrete analogue to our improper integral test found in Theorem~\ref{thm:impropercomparison}.

\theorem{\textbf{The Basic Comparison Test:}\index{Test for convergence and divergence of a series! Basic Comparison Test}\newline
Let $\sum a_k$ and $\sum b_k$ be series with positive terms satisfying $a_k\le b_k$ for all $k$.
\begin{enumerate}
\item\label{itm:basiccomp1} If $\sum b_k$ is convergent then $\sum a_k$ is convergent.
\item\label{itm:basiccomp2} If $\sum a_k$ is divergent then $\sum b_k$ is divergent.
\end{enumerate}
}

Proof:  Let $S_n=\sum_{k=1}^na_k$ and $T_n=\sum_{k=1}^n b_k$ denote the partial sums of the series $\sum a_k$ and $\sum b_k$ respectively.  Since $a_k\le b_k$ it follows that $S_n\le T_n$ for any $n$.  Furthermore sequences $\left\{S_n\right\}$ and $\{T_n\}$ are both increasing (and hence monotonic) since terms $a_k$ and $b_k$ are positive.

Part~\ref{itm:basiccomp1} of the theorem follows from noting that monotonic sequence $\left\{T_n\right\}$ has upper bound $T=\sum_{k=1}^\infty b_k$ since the series $\sum b_k$, and hence the sequence $\left\{T_k\right\}$ converges.  The sequence $\left\{S_n\right\}$ must also then have this upper bound since $S_n\le T_n\le T$.  Thus $\left\{S_n\right\}$ is a monotonic bounded sequence and hence converges to $S$.  Therefore $\sum a_k$ is convergent.

Part~\ref{itm:basiccomp2} follows from noting that if $\sum a_k$ is divergent then monotonic sequence $\{S_n\}$ is unbounded, which implies it has no upper bound as it is bounded below by 0.  Now $S_n\le T_n$ implies that monotonic sequence $\{T_n\}$ has no upper bound and hence does not converge, thereby proving $\sum b_k$ is divergent.

Note that the \textbf{Basic Comparison Test} is also known as the \textbf{Direct Comparison Test}.
  
\begin{example}\label{exam:basiccomparison}
  Determine whether the series converges or diverges.
  \begin{multicols}{2}
    \begin{enumerate}
    \item \label{itm:basiccomparisonworks} $\displaystyle \sum_{n=1}^{\infty} \frac{4^n}{5^n+n}$
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{4^n(n+2)^2}{n^2-3}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item The given series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_{n}=\frac{4^n}{5^n+n}$ and is therefore a positive series.

  Consider $\displaystyle\sum_{n=1}^\infty b_n$ where $\displaystyle b_{n}=\frac{4^n}{5^n}=\left(\frac{4}{5}\right)^{n}$. Then
  \[\sum_{n=1}^\infty b_n=\sum_{n=1}^{\infty} \left(\frac{4}{5}\right)^n=\sum_{n=1}^{\infty}\left(\frac{4}{5}\right)\left(\frac{4}{5}\right)^{n-1}\]
shows $\sum b_n$ is a geometric series with $\displaystyle r=\frac{4}{5}$.  Since $|r|<1$ it is a convergent series.  We now show that $a_{n}\leq b_{n}$ for $n\geq 1$\,:
\begin{align*}
a_{n} \leq b_{n} &\iff \frac{4^n}{5^n+n} \leq \frac{4^n}{5^n}\\
&\iff (4^n)(5^n) \leq (4^n)(5^n+n)\\
&\iff 0 \leq n(4^n)\ \ \ (\text{which is true for }n\ge1)
\end{align*}
Therefore, by the Basic Comparison Test, the given positive series $\sum a_n$ is convergent because it lies below a convergent series.
\item The given series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_{n}=\frac{4^n(n+2)^2}{n^2-3}$ and is therefore a positive series for $n\ge 2$.

  Noting that for large $n$ both the polynomials in the numerator and denominator will be dominated by their quadratic ($n^2$) term, consider $\displaystyle b_{n}=\frac{4^n\cdot n^2}{n^2}=4^n$. Then
  \[\sum_{n=1}^{\infty} b_{n}= \sum_{n=1}^{\infty}4^n =\sum_{n=1}^{\infty}4(4^{n-1})\]
shows $\sum b_n$ is a geometric series with $a=4$ and $r=4>1$ and therefore is divergent. Now we must show that $a_{n} \geq b_{n}$ for $n\geq 2$\,:
\begin{align*}
a_n\ge b_n &\iff \frac{4^n(n+2)^2}{n^2-3}\geq 4^n 
\iff \frac{(n+2)^2}{n^2-3} \geq 1 \iff (n+2)^2 \geq n^2-3 \\
&\iff n^{2}+4n+4 \geq n^2-3
\iff 4n+7 \geq 0 \\
&\iff n\geq -\frac{7}{4} \,\, (\text{which is true for $n \geq 2$})
\end{align*}
Therefore, by the Basic Comparison Test, the given series $\sum a_n$ is divergent because it lies above a divergent series that is ultimately positive (for $n\ge 2$). Once again one notes that the conditions of the test need only apply for the infinite tail of the given series and one can ignore a finite number of initial terms (here the first term $a_1=-18$) as these do not affect the convergence of the series.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the series converges or diverges.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \sum_{k=1}^\infty\frac{1}{2+5^k}$
    \item $\displaystyle \sum_{n=2}^\infty\frac{3}{\sqrt{n}-1}$
    \item $\displaystyle \sum_{n=1}^\infty\frac{1}{n3^n}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

\subsubsection*{Remainder Estimate} \index{remainder estimates! for the Basic Comparison Test}
Note that if one uses convergent series $\sum b_k$ to show $\sum a_k$ is convergent by the Basic Comparison Test, it follows, since $a_k\le b_k$, that the remainder $R_n=\sum_{k=n+1}^\infty a_k$ is less than or equal to the remainder $\tilde{R}_n=\sum_{k=n+1}^\infty b_k$.  Hence if we have an estimate for the size of the error $\tilde{R}_n\le \epsilon$ for series $\sum b_k$, this implies $R_n\le\epsilon$ for series $\sum a_k$. 

\subsection{The Limit Comparison Test}\index{Limit Comparison Test} \index{Test for convergence and divergence of a series! Limit Comparison Test}

Our next test involves taking the limit of the ratio of terms of two series, one of whose convergence properties are presumed known.

\theorem{\textbf{The Limit Comparison Test:}
  
Let $\sum a_k$ and $\sum b_k$ be series with positive terms.
\begin{enumerate}
\item If $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=L>0$ then both series are convergent or both divergent.
\item If $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=0$ and $\sum b_k$ is convergent then $\sum a_k$ is convergent.
\item If $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=\infty$ and $\sum b_k$ is divergent then $\sum a_k$ is divergent.
\end{enumerate}
}

%TODO: The following is not in the slides.
The convergence conclusions of the Limit Comparison Test can be remembered by noting that the limit condition effectively suggests that the tail of the series satisfies $a_k=Lb_k$, in other words the series tail is effectively a multiple of that of the other series by a constant $c=L$.  For $c=L\neq0$ we saw in Theorem~\ref{thm:serieslinearity} that the new series has the same convergence properties as the original.

Proof: Consider the case where $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=L>0$.  Then there exists $\tilde{M}>0$ and $\tilde{N}>0$ such that
\[\tilde{M}<\frac{a_k}{b_k}<\tilde{N}\ \ \ \mbox{for}\ k>n\]
Let $M$ be the minimum of the finite set of numbers $\left\{\frac{a_k}{b_k}|k\le n\right\}$ and the number $\tilde{M}$.  Similarly let $N$ be the maximum of the finite set of numbers $\left\{\frac{a_k}{b_k}|k\le n\right\}$ and the number $\tilde{N}$.  It follows that for all $k$:
\[M<\frac{a_k}{b_k}<N\]
Since $b_k>0$ we have, for all $k$, 
\[Mb_k<a_k<Nb_k\ .\]
If $\sum b_k$ converges then $\sum Nb_k$ converges and $a_k<Nb_k$ implies $\sum a_k$ converges by the Basic Comparison Test.  Similarly if $\sum b_k$ diverges then $\sum Mb_k$ diverges and $Mb_k<a_k$ implies $\sum a_k$ diverges by the Basic Comparison Test.

In the case $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=0$ we can only argue that $a_k/b_k<N$ and so only $\sum b_k$ convergent implies $\sum a_k$ convergent.  In the case $\displaystyle\lim_{k\to\infty}\frac{a_k}{b_k}=\infty$ we can only argue that $M<a_k/b_k$ and so only $\sum b_k$ divergent implies $\sum a_k$ divergent.

\begin{example}
  Determine whether the following series are convergent or divergent.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{4^n}{5^n-6n}$
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{n^2+2\ln n}{n^4+3n^2+5}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n=\frac{4^n}{5^n-6n}$.  This is a positive series for $n\ge 2$ since the denominator becomes positive at $n=2$ (equal to $13$) and only increases (so remains positive) after that since $\frac{d}{dx}(5^x-6x)=5^x\ln 5-6>0$ when $x\ge2$. Since convergence of a series depends only on its infinite tail we can identify a suitable comparison series by looking at the large $n$ properties of $a_n$.  We note that the denominator is dominated by the exponential $5^n$ which suggests considering $\sum_{n=1}^\infty b_n$ where $b_{n}=\frac{4^n}{5^n}=\left(\frac{4}{5}\right)^n$. Then
\begin{align*}
  \sum_{n=1}^{\infty}b_{n}=\sum_{n=1}^{\infty}\left(\frac{4}{5}\right)^n=\sum_{n=1}^{\infty}\frac{4}{5}\left(\frac{4}{5}\right)^{n-1}
\end{align*}
is a geometric series with $a=\frac{4}{5}$ and $|r|=\left|\frac{4}{5}\right|=\frac{4}{5} <1$ and therefore convergent. 
\begin{align*}
  \lim_{n \to \infty}\frac{a_{n}}{b_n}&=\lim_{n \to \infty} \frac{\frac{4^n}{5^n-6n}}{\frac{4^n}{5^n}}
=\lim_{n \to \infty} \frac{4^n}{5^n-6n}\cdot\frac{5^n}{4^n}
                                        =\lim_{n \to \infty}\frac{5^n}{5^n-6n}\\
  &=\lim_{n \to \infty} \frac{5^n}{5^n}\cdot\frac{\frac{5^n}{5^n}}{\frac{5^n}{5^n}-\frac{6n}{5^n}} =\lim_{n \to \infty} \frac{1}{1-\frac{6n}{5^n}}
    =\lim_{n \to \infty}\frac{1}{1-0}=1
\end{align*}
Here we used L'H\^{o}pital's Rule in evaluating the last step of the limit:
\[\lim_{x \to \infty} \frac{6x}{5^x} \underset{\lh}= \lim_{x \to \infty} \frac{6}{5^x\ln 5}=\frac{6}{\infty}=0\]
The latter limit shows that $5^n$ dominates $6n$ as claimed above and indicates why we factored out that term (and not $6n$) when evaluating our original limit.  
Therefore, since $\displaystyle \lim_{n \to \infty}\frac{a_{n}}{b_n}=1>0$, by the Limit Comparison Test, the given series $\sum a_n$ is also convergent.

As an aside, note that this series is very similar to Problem~\ref{itm:basiccomparisonworks} in Example~\ref{exam:basiccomparison}.  However attempting to use the Basic Comparison Test in the current example using the same comparison series $\sum b_n$ will fail to be conclusive since here, as the reader may confirm, $a_n>b_n$ for $n\ge 2$.  In general the Limit Comparison Test will often work to establish convergence of a series where the Basic Comparison Test will not. 

\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_{n}=\frac{n^2+2\ln n}{n^4+3n^2+5}$ and therefore positive.  Consideration of the limiting behaviour of the terms at large $n$ suggests a comparison with the series $\sum b_n$ where $b_{n}=\frac{n^2}{n^4}$. Then
\begin{align*}
\sum_{n=1}^\infty b_{n}=\sum_{n=1}^\infty\frac{n^2}{n^4}=\sum_{n=1}^{\infty}\frac{1}{n^2}
\end{align*}
is a $p$-series with $p= 2>1$ and therefore convergent. Taking the limit of the ratio of terms gives:
\begin{align*}
  \lim_{n \to \infty} \frac{a_{n}}{b_{n}}&=\lim_{n \to \infty}\frac{\frac{n^2+2\ln n}{n^4+3n^2+5}}{\frac{1}{n^2}} = \lim_{n \to \infty}\frac{(n^2+2\ln n)n^2}{n^4+3n^2+5}=\lim_{n \to \infty} \frac{n^4+2n^2\ln n}{n^4+3n^2+5}\\
                                         &=\lim_{n \to \infty}\frac{n^4}{n^4}\cdot\frac{\frac{n^4}{n^4}+\frac{2n^2\ln n}{n^4}}{\frac{n^4}{n^4}+\frac{3n^2}{n^4}+\frac{5}{n^4}}
=\lim_{n \to \infty}\frac{1+\frac{2\ln n}{n^2}}{1+\frac{3}{n^2}+\frac{5}{n^4}}
=\frac{1+\lim_{n \to \infty}\frac{2\ln n}{n^2}}{1+0+0} \\
&=1 +\lim_{n \to \infty} \frac{2\ln n}{n^2}=1
\end{align*}
Here we evaluated the limit at the last step using L'H\^{o}pital's Rule on the corresponding continuous function $\displaystyle f(x)=\frac{2\ln x}{x^2}$\,:
\begin{align*}
\lim_{x \to \infty} f(x)=\lim_{x \to \infty}\frac{2\ln x}{x^2}=\lim_{x \to \infty}\frac{2\cdot \frac{1}{x}}{2x}=\lim_{x \to \infty}\frac{1}{x^2}=0
\end{align*}
This limit confirms that the power function dominates the logarithm at large $n$ as claimed above and why it is the power, and not the logarithm, which we factor out of the above limit.

Therefore $\displaystyle\lim_{n \to \infty}\frac{a_n}{b_n}=1>0$ and the given series $\sum a_n$ is also convergent by the Limit Comparison Test.

As a final note, the Limit Comparison Test is seen to be superior to the Basic Comparison Test here again as the presence of the logarithm and fourth order polynomial in $a_n$ would make evaluation of the required inequality in the latter test difficult.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the following series are convergent or divergent.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{k=1}^\infty\frac{1}{\sqrt[3]{k^2+1}}$
    \item $\displaystyle \sum_{n=1}^\infty\frac{3n^2+5n}{2^n\left(n^2+1\right)}$
    \item $\displaystyle \sum_{k=1}^\infty\frac{1+2^k}{1+3^k}$
    \item $\displaystyle \sum_{n=1}^\infty\frac{1}{n^2\ln n}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

Note that since convergence is entirely determined by the infinite tail of a series, we can relax the Basic and Limit Comparison Tests to require that they only have positive terms for $k>n$ for some fixed $n$ and, in the case of the Basic Comparison Test, that additionally $a_k\le b_k$ for $k>n$.

\textbook{\clearpage}
\begin{exercise}
\input{SEQUENCES_AND_SERIES/testing-series-with-positive-terms.tex}
\end{exercise}

\section{The Alternating Series Test}\index{Alternating Series Test}

We now consider series where all the terms are not positive.  An alternating series is a special case of such a series.

\definition{An \textbf{alternating series}\index{series! alternating} is a series of either of the forms
\[a_1-a_2+a_3-\cdots+(-1)^{k-1}a_k+\cdots=\sum_{k=1}^\infty(-1)^{k-1}a_k\ ,\]
\[-a_1+a_2-a_3+\cdots+(-1)^{k}a_k+\cdots=\sum_{k=1}^\infty(-1)^ka_k\ ,\ \ \ \ \]
where $a_k$ is positive for all $k$.}

%TODO: Change to $a_k\ge a_{k+1}$ to be consistent with monotonicity defn?
%If done, change alternating harmonic series example accordingly.
%
%TODO: Put limit criteria first because if the limit fails you have also
%immediately proven divergence.  Why? Since we have a theorem that says
%a sequence converges to 0 iff the absolute sequence converges.  So if we
%show the absolute sequence does not cvg to zero then neither does the
%alternating sequence and then, by the term test for divergence the
%alternating sequence diverges.  I do explain this in an example.
%Logically there is no reason to not test the limit first since you have to
%anyway and then this settles it.
%
%TODO: It is problematic to use both (-1)^ka_k for my notation as then it becomes hard to refer to the generic series as sum a_k.  See my strategy summary below.  (Actually it isn't entirely bad but it is awkward.  Some texts define their alternating series using a different letter from a and I could consider that.  If I move to doing all positive series first (which does make sense) I could reserve c_k, say for an arbitrary series everywhere.  Most of the notation would still reserve a_k and b_k for positive series.  I would only need to make changes at the beginning.
\theorem{\textbf{The Alternating Series Test:}\index{Test for convergence and divergence of a series! Alternating Series Test}\newline
If an alternating series of the form $\displaystyle\sum_{k=1}^\infty(-1)^{k-1}a_k$ or $\displaystyle\sum_{k=1}^\infty(-1)^ka_k$ with $a_k>0$ satisfies
\begin{enumerate}
\item $a_{k+1}\le a_k$ for all $k$~,
\item $\displaystyle\lim_{k\to\infty}a_k=0$~,
\end{enumerate}
then the alternating series is convergent.}

Note that $a_{k+1}\le a_k$ is equivalent to $a_{k+1}-a_k\le 0$ and $\displaystyle\frac{a_{k+1}}{a_k}\le 1$\,.

Proof: Suppose we have an alternating series of the form $\sum_{k=1}^\infty(-1)^{k-1}a_k$.  Consider the even partial sums $S_2$, $S_4$,\ldots, where, in general the $(2n)^{\mbox{\tiny th}}$ partial sum is $S_{2n}$ for $n$ a positive integer given by:
\[S_{2n}=a_1-a_2+a_3-a_4+a_5-a_6\cdots+a_{2n-1}-a_{2n}~.\]
Then grouping the terms in pairs one has
\[S_{2n}=(a_1-a_2)+(a_3-a_4)+(a_5-a_6)\cdots+(a_{2n-1}-a_{2n})~,\]
where each term in parentheses is nonnegative since $a_k\ge a_{k+1}$.  This implies $\left\{S_{2n}\right\}$ is a nondecreasing sequence ($S_2\le S_4\le S_6\le\ldots\le S_{2n}\le\ldots$).  The terms of the even partial sum $S_{2n}$ may be regrouped as
\[S_{2n}=a_1-(a_2-a_3)-(a_4-a_5)-\cdots-(a_{2n-2}-a_{2n-1})-a_{2n}~,\]
where, once again, the terms in parentheses are positive.  This shows $S_{2n}<a_1$ for all $n$.  The monotonic sequence of even partial sums $\left\{S_n\right\}$ is bounded and hence has limit $S$.

The odd partial sums are $S_{2n+1}$ for $n$ a positive integer and may be written
\[S_{2n+1}=S_{2n}+a_{2n+1}~.\]
Taking the limit of the odd partial sum sequence $\left\{S_{2n+1}\right\}$ gives
\[\lim_{n\to\infty}S_{2n+1}=\lim_{n\to\infty}\left(S_{2n}+a_{2n+1}\right)=\lim_{n\to\infty}S_{2n}+\lim_{n\to\infty}a_{2n+1}=S+0=S\]
Since both even and odd partial sum sequences approach the same limit $S$, the sequence $\left\{S_n\right\}$ approaches $S$ as well and alternating sequence  $\sum_{k=1}^\infty(-1)^{k-1}a_k$ is convergent.  Since alternating sequence $\sum_{k=1}^\infty(-1)^ka_k=(-1)\sum_{k=1}^\infty(-1)^{k-1}a_k$ this completes the proof for the other possible case.

\begin{example}
The alternating harmonic series
\[\sum_{k=1}^\infty\frac{(-1)^{k-1}}{k}=1-\frac{1}{2}+\frac{1}{3}-\cdots\]
is convergent since $a_k=\frac{1}{k}$ satisfies $a_{k+1}=\frac{1}{k+1}\le\frac{1}{k}=a_k$ and $\lim_{k\to\infty} a_k=\lim_{k\to\infty}\frac{1}{k}=0$.
\end{example}

\begin{example}
  Determine whether the following alternating series converge or diverge.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n-1}\frac{n^2+2}{n^4+1}$
    \item $\displaystyle \sum_{n=2}^{\infty}(-1)^n \frac{n}{\ln n}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item The given series is $\displaystyle \sum_{n=1}^\infty c_n=\sum_{n=1}^\infty(-1)^{n-1}a_n$ where $\displaystyle c_{n}=(-1)^{n-1}a_n$ and $\displaystyle a_{n}=\frac{n^2+2}{n^4+1}>0$.  The series is therefore an alternating series. Consider $\displaystyle f(x)=\frac{x^2+2}{x^4+1}$. Then
  \begin{align*}
    f'(x)&=\frac{2x(x^4+1)-(x^2+2)(4x^3)}{(x^4+1)^2} =\frac{2x^5+2x-4x^5-8x^3}{(x^4+1)^2}= \frac{-2x^5-8x^3+2x}{(x^4+1)^2}
  \end{align*}
  implies $f'(x)<0$ for $x\ge1$ and hence $f(x)$ decreases there. This implies $f(n+1)\leq f(n)$ for $n \geq 1$ and hence $a_{n+1}\le a_n$. Next consider the limit
\begin{align*}
  \lim_{x\to\infty}f(x)=\lim_{x \to \infty} \frac{x^2+2}{x^4+1}\underset{\lh}=\lim_{x \to \infty} \frac{2x}{4x^3}=\lim_{x \to \infty} \frac{1}{2x^2}=0
\end{align*}
where we used  L'H\^{o}pital's Rule on the $\frac{\infty}{\infty}$ indeterminate form. Thus $\displaystyle\lim_{n \to \infty} a_{n}=\lim_{n\to\infty}f(n)=0$\,.

Since both conditions are met, the series $\sum c_n$ is convergent by the Alternating Series Test.
\item  The given series is $\displaystyle \sum_{n=2}^\infty c_n=\sum_{n=2}^\infty(-1)^na_n$ where $\displaystyle c_{n}=(-1)^na_n$ and $\displaystyle a_{n}=\frac{n}{\ln n}>0$.  The series is therefore an alternating series. Consider $\displaystyle f(x)=\frac{x}{\ln x}$. Then
\begin{align*}
\lim_{x \to \infty} f(x)&=\lim_{x \to \infty} \frac{x}{\ln x}\underset{\lh}=\lim_{x \to \infty} \frac{1}{\frac{1}{x}}=\lim_{x \to \infty}x=\infty
\end{align*}
where we used  L'H\^{o}pital's Rule on the $\frac{\infty}{\infty}$ indeterminate form. Thus $\displaystyle\lim_{n \to \infty} a_{n}=\lim_{n\to\infty}f(n)=\infty$. Since $\displaystyle\lim_{n \to \infty} a_{n}=\infty\neq 0$ we cannot draw any conclusions about the convergence or divergence of the given series using the Alternating Series Test.  However since $a_n=|c_n|$ we can conclude by Theorem~\ref{thm:cvgtozero} that $\displaystyle\lim_{n\to\infty}c_n\neq 0$ or does not exist. Therefore, the given series $\sum c_n$  is divergent by the Term Test for Divergence.

As this example illustrates, in practice it makes sense, when applying the Alternating Series Test to series $\sum c_n$, to first determine whether the requirement that the limit of $a_n=|c_n|$ goes to zero holds.  If it does not, then, as shown in this example, the series must diverge by the Term Test for Divergence.  If the limit does go to zero, proceed to determine whether the other condition, that $a_n$ decreases, is met or not. If it is met, then the series converges by the Alternating Series Test.  If it is not met, then that test is inconclusive. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the following alternating series converge or diverge.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^\infty(-1)^{n-1}\frac{2n}{4n^2-3}$
    \item $\displaystyle \sum_{k=1}^\infty(-1)^{k}\frac{2k}{4k-3}$
    \item $\displaystyle \sum_{n=1}^\infty(-1)^{n-1}\frac{\ln n}{n}$
    \item $\displaystyle \sum_{k=1}^\infty\cos\left(k\pi\right)\frac{3k^2+2}{2k^2+1}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

\subsubsection*{Remainder Estimate} \index{remainder estimates! for the Alternating Series}
An estimate of the error made when approximating the sum $S$ of an alternating series with the $n^{\mbox{\tiny th}}$ partial sum $S_n$ is given by the following theorem.
\theorem{For an alternating series with terms of absolute value $a_k>0$ the remainder $R_n=S-S_n$ satisfies $|R_n|<a_{n+1}$\,.}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/the-alternating-series-test.tex}
\end{exercise}

\section{Tests of Absolute Convergence}

For series $\sum a_k$ whose terms have mixed sign, one can consider the convergence properties of the series $\sum|a_k|$ with nonnegative terms generated by taking the absolute value of the terms of the original series.

\subsection{Absolute Convergence}\index{absolute convergence} \index{convergent! absolutely}
\definition{A series $\sum a_k$ is \textbf{absolutely convergent}\index{series! absolutely convergent} if the series $\sum|a_k|$ is convergent.}

A series may be convergent that is not absolutely convergent, prompting the following definition.
\definition{A series $\sum a_k$ that is convergent but not absolutely convergent is called \textbf{conditionally convergent} \index{series! conditionally convergent} \index{convergent! conditionally}
.} 

\begin{example}
  The alternating harmonic series $1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\cdots$ is conditionally convergent because it converges but the harmonic series, which is the series of the absolute values of its terms, does not.
\end{example}

The following theorem shows a convergent series can only be absolutely or conditionally convergent.

\theorem{If a series $\sum a_k$ is absolutely convergent then it is convergent.}

The theorem also shows that convergence of some series $\sum a_k$ may be determined by considering convergence of $\sum|a_k|$.

We remind the reader of some properties of absolute value that will be frequently encountered:

\hfill$|xy|=|x||y|$\hfill$\displaystyle\left|\frac{x}{y}\right|=\frac{|x|}{|y|}$\hfill$|x+y|\le|x|+|y|$\hfill~

and for $a>0$\,:
\begin{align*}
  |x|&<a\iff -a<x<a\\
  |x|&=a\iff x=\pm a\\
  |x|&>a\iff x<-a\ \text{or}\ x>a\\
  |x|&=0\iff x=0
\end{align*}

\begin{example}
  Determine whether the following series are absolutely convergent or conditionally convergent.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} (-1)^ne^{-n}$
    \item $\displaystyle \sum_{n=1}^{\infty}(-1)^{n-1}\frac{\sqrt{n}}{n+1} $
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item Consider the series of absolute values of the given series $\displaystyle \sum_{n=1}^{\infty} (-1)^ne^{-n}$, that is
\begin{align*}
\sum_{n=1}^{\infty}\left|(-1)^ne^{-n}\right|=\sum_{n=1}^{\infty}e^{-n}=\sum_{n=1}^{\infty}\left(\frac{1}{e}\right)^{n}=\sum_{n=1}^{\infty}\frac{1}{e}\left(\frac{1}{e}\right)^{n-1}\,.
\end{align*}
This is a geometric series with $\displaystyle a=\frac{1}{e}$ and $\displaystyle |r|=\left|\frac{1}{e}\right|=\frac{1}{e}<1 $ and therefore convergent. Thus the given series is absolutely convergent and therefore also convergent.
\item The given series is $\displaystyle \sum_{n=1}^\infty c_n$ where $\displaystyle c_n=(-1)^{n-1}\frac{\sqrt{n}}{n+1}$.  Consider the series of absolute values $\sum a_n$ where $\displaystyle a_n=|c_n|=\frac{\sqrt{n}}{n+1}$. Consideration of the large-$n$ behaviour of $a_n$ suggests a comparison with series $\sum b_n$ where
  \[b_{n}=\frac{\sqrt{n}}{n}=\frac{1}{\sqrt{n}}\,.\]
  Then $\displaystyle\sum_{n=1}^{\infty}b_{n}=\sum_{n=1}^{\infty}\frac{1}{\sqrt{n}}$ is a $p$-series with $p=\frac{1}{2}<1$ and therefore divergent. Taking the limit
\begin{align*}
  \lim_{n \to \infty} \frac{a_n}{b_n}=\lim_{n \to \infty} \frac{\frac{\sqrt{n}}{n+1}}{\frac{1}{\sqrt{n}}}=\lim_{n \to \infty} \frac{n}{n+1}=\lim_{n \to \infty}\frac{n}{n}\cdot\frac{\frac{n}{n}}{\frac{n}{n}+\frac{1}{n}} =\lim_{n \to \infty} \frac{1}{1+\frac{1}{n}}=\frac{1}{1+0}=1 >0
\end{align*}
shows the series of absolute values, $\displaystyle\sum_{n=1}^{\infty} a_n$, is also divergent by the Limit Comparison Test.

Returning to the original series we see that it is $\displaystyle\sum_{n=1}^\infty c_n= \sum_{n=1}^\infty (-1)^{n-1}a_n$ with $\displaystyle a_{n}=\frac{\sqrt{n}}{n+1}$ and so is an alternating series. Then
\begin{align*}
\lim_{n \to \infty} a_n=
\lim_{n \to \infty}\frac{\sqrt{n}}{n+1}=\lim_{n \to \infty}\frac{\sqrt{n}}{n}\cdot\frac{1}{1+\frac{1}{n}}=\frac{1}{\sqrt{n}}\cdot\frac{1}{1+\frac{1}{n}}=0\cdot\frac{1}{1+0}=0\,.
\end{align*}
Also for $n\ge 1$ one has
\begin{align*}
a_{n+1}\leq a_{n} &\iff \frac{\sqrt{n+1}}{n+2} \leq \frac{\sqrt{n}}{n+1}\\
&\iff (n+1)^{\frac{3}{2}}\leq \sqrt{n}(n+2)  \ \ (\Leftarrow\text{Cross-multiplying by positive terms)} \\
&\iff (n+1)^3 \leq n(n+2)^2 \ \ (\Leftarrow\text{Squaring of positive sides preserves inequality)} \\
&\iff n^3+3n^2+3n+1 \leq n^3+4n^2+4n \\ 
&\iff -n^2-n+1 \leq 0 \,\, (\text{which is true for all}\ n\geq1)
\end{align*}
Therefore, the given series $\sum c_n$ is convergent by the Alternating Series Test. Since the series of its absolute values $\sum |c_n|=\sum a_n$ is divergent, the convergence of $\sum c_n$ is conditional. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine whether the following series are absolutely convergent or conditionally convergent.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle\sum_{k=1}^\infty (-1)^{k-1}\frac{1}{k^2}$
    \item $\displaystyle\sum_{n=1}^\infty \frac{\sin n}{n^2}$
    \item $\displaystyle\sum_{k=1}^\infty (-1)^{k+1}\frac{1}{\sqrt{k}}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}


%TODO: Some texts introduce ratio/root tests when considering positive series.
% This may be logically a better approach, especially if we isolate
% positive series in their own section.
\subsection{The Ratio Test}\index{Ratio Test} \index{Test for convergence and divergence of a series! Ratio Test}

The following convergence test considers the limit of the ratio of terms within a series.

\theorem{\textbf{The Ratio Test:}\newline 
Suppose the ratio of consecutive terms of series $\displaystyle \sum_{k=1}^\infty a_k$ has limit
\[\lim_{k\to\infty}\left|\frac{a_{k+1}}{a_k}\right|=L\ ,\]
then
\begin{enumerate}
\item If $L<1$ the series $\sum a_k$ is absolutely convergent (and hence convergent).
\item If $L=1$ the test is inconclusive.
\item If $L>1$ the series $\sum a_k$ is divergent.
\end{enumerate}
If $\displaystyle\lim_{k\to\infty}\left|\frac{a_{k+1}}{a_k}\right|=\infty$ then $\sum a_k$ is also divergent.}

Note that if the Ratio Test is inconclusive $(L=1)$ this means that $\sum a_k$ is potentially absolutely convergent, conditionally convergent, or divergent.

%TODO: Not in slides.  Also note this is the same conclusion as the Root Test below.  Both are effectively geometric series.
The convergence conclusions of the Ratio Test can be remembered by noting that the limit condition effectively suggests that the tail of the series satisfies $a_{k+1}=La_k$, in other words it behaves like a geometric series with $r=L$.  From this it follows $r=L<1$ should converge and $r=L>1$ should diverge.


\begin{example}
  Determine whether the following series are absolutely convergent, conditionally convergent or divergent.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n-1}\frac{5^n}{n!(n+4)}$ 
    \item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n}\frac{2^n}{n^4}$
    \item $\displaystyle \sum_{n=1}^{\infty} \frac{1}{\sqrt{n+1}}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item The series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n=(-1)^{n-1}\frac{5^n}{n!(n+4)}$.  The presence of the factorial suggests trying the Ratio Test:
  \begin{align*}
    \lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|&=\lim_{n \to \infty}\left|\frac{\frac{(-1)^n\cdot 5^{n+1}}{(n+1)!(n+5)}}{\frac{(-1)^{n-1}5^n}{n!(n+4)}}\right|
    =\lim_{n \to \infty} \frac{5^{n+1}}{(n+1)!(n+5)}\cdot \frac{n!(n+4)}{5^n}\\
    &=\lim_{n \to \infty} \frac{5^n\cdot 5}{(n+1)n!(n+5)}\cdot \frac{n!(n+4)}{5^n}\ \ (\Leftarrow (n+1)!=(n+1)(n)\cdots (1)=(n+1)n!)\\
    &=\lim_{n \to \infty} \frac{5(n+4)}{(n+1)(n+5)}
    =\lim_{n \to \infty} \frac{5n+20}{n^2+6n+5}=\lim_{n \to \infty} \frac{n}{n^2}\cdot\frac{\frac{5n}{n}+\frac{20}{n}}{\frac{n^2}{n^2}+\frac{6n}{n^2}+\frac{5}{n^2}} \\
&=\lim_{n \to \infty} \frac{1}{n}\cdot\frac{5+\frac{20}{n}}{1+\frac{6}{n}+\frac{5}{n^2}}=0\cdot\frac{5+0}{1+0+0}=0 <1 
\end{align*}
Therefore the given series is absolutely convergent by the Ratio Test.
\item The series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n=(-1)^{n}\frac{2^n}{n^4}$.  Applying the Ratio Test gives:
\begin{align*}
 \lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|&=\lim_{n \to \infty}\left|\frac{(-1)^{n+1}\frac{2^{n+1}}{(n+1)^4}}{(-1)^{n}\frac{2^n}{n^4}}\right|
  =\lim_{n \to \infty}\frac{2^{n+1}}{(n+1)^4}\cdot \frac{n^4}{2^n}
                                                      =\lim_{n \to \infty} \frac{2n^4}{(n+1)^4}\\
  &=2\lim_{n \to \infty}\left(\frac{n}{n+1}\right)^4 =2\left(\lim_{n \to \infty}\frac{n}{n+1}\right)^4
=2\left(\lim_{n \to \infty}\frac{n}{n}\cdot\frac{\frac{n}{n}}{\frac{n}{n}+\frac{1}{n}}\right)^4\\
&=2\left(\lim_{n \to \infty}\frac{1}{1+\frac{1}{n}}\right)^4
=2\left(\frac{1}{1+0}\right)^4=2>1
\end{align*}
Therefore the given series is divergent by the Ratio Test.
\item The series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n=\frac{1}{\sqrt{n+1}}$.
  \begin{align*}
    \lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|&=\frac{1}{\sqrt{n+2}}\cdot\frac{\sqrt{n+1}}{1}=\lim_{n\to\infty}\sqrt{\frac{n+1}{n+2}}=\sqrt{\lim_{n\to\infty}\frac{n+1}{n+2}}\\
    &=\sqrt{\lim_{n\to\infty}\frac{n}{n}\cdot\frac{1+\frac{1}{n}}{1+\frac{2}{n}}}=\sqrt{\frac{1+0}{1+0}}=1
  \end{align*}
  Therefore the Ratio Test is inconclusive and a different test is required.   Since $a_n=\frac{1}{\sqrt{n}}\cdot\frac{1}{\sqrt{1+\frac{1}{n}}}$ shows that $a_n\approx b_n=\frac{1}{\sqrt{n}}$ for large $n$, using the Limit Comparison Test with the divergent ($p=1/2$) $p$-series $\sum b_n$ will show $\sum a_n$ is also divergent.  Quite generally the Ratio Test (and also the Root Test introduced below) will always fail to be conclusive for any series that behaves like a $p$-series at large $n$. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
Determine whether the following series are absolutely convergent, conditionally convergent, or divergent.
  \begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle \sum_{k=1}^\infty(-1)^k\frac{3^k}{k!}$
\item $\displaystyle \sum_{n=1}^\infty\frac{4^n}{n^2}$
\item $\displaystyle \sum_{k=1}^\infty e^{-k}k!$
\item $\displaystyle \sum_{n=1}^\infty\frac{2^n}{2n^2+1}$
\end{enumerate}
\end{multicols}
\end{furtherquestions}
\end{example}
%TODO Add a recursively defined example a_{k+1}=g(a_k)

\subsection{The Root Test}

The next convergence test considers the limit of the $k^{\mbox{\tiny th}}$ root of $|a_k|$.

\theorem{\textbf{The Root Test:}\index{Root Test}\index{Test for convergence and divergence of a series! Root Test}\newline
Suppose the terms of series $\displaystyle \sum_{k=1}^\infty a_k$ satisfy
\[\lim_{k\to\infty}\sqrt[k]{\left|a_k\right|}=L\ ,\]
then
\begin{enumerate}
\item If $L<1$ the series $\sum a_k$ is absolutely convergent (and hence convergent).
\item If $L=1$ the test is inconclusive. 
\item If $L>1$ the series $\sum a_k$ is divergent.
\end{enumerate}
If $\displaystyle\lim_{k\to\infty}\sqrt[k]{\left|a_k\right|}=\infty$ then $\sum a_k$ is also divergent.}

%TODO: Not in slides:
The convergence conclusions of The Root Test can be remembered by noting that the limit condition effectively suggests that the tail of the series behaves like $a_k=L^k$, in other words like a geometric series with $r=L$.  From this it follows that $r=L<1$ should converge and $r=L>1$ should diverge.

\begin{example}
  Determine the convergence or divergence of the following series.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{n=1}^{\infty} \left(\frac{n+1}{2n+3}\right)^n$
    \item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n+1}\frac{(n+1)^n}{e^{2n}}$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with $\displaystyle a_n=\left(\frac{n+1}{2n+3}\right)^n$. The presence of the power $n$ in the term suggests trying the Root Test:
  \begin{align*}
    \lim_{n \to \infty} \sqrt[n]{|a_n|}
    &=\lim_{n \to \infty} \sqrt[n]{\left|\left(\frac{n+1}{2n+3}\right)^{n}\right|}
      =\lim_{n \to \infty}\frac{n+1}{2n+3}\\
    &=\lim_{n \to \infty}\frac{n}{n}\cdot\frac{\frac{n}{n}+\frac{1}{n}}{\frac{2n}{n}+\frac{3}{n}}=\lim_{n \to \infty}\frac{1+\frac{1}{n}}{2+\frac{3}{n}}=\frac{1+0}{2+0}=\frac{1}{2}<1
  \end{align*}
  Therefore the given series is absolutely convergent by the Root Test.
\item The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with $\displaystyle a_n=(-1)^{n+1}\frac{(n+1)^n}{e^{2n}}$. Applying the Root Test gives:
\begin{align*}
\lim_{n \to \infty} \sqrt[n]{|a_n|}
&=\lim_{n \to \infty}\sqrt[n]{\left|(-1)^{n+1}\frac{(n+1)^n}{e^{2n}}\right|}
                                      =\lim_{n \to \infty}\sqrt[n]{\frac{(n+1)^n}{e^{2n}}}
                                      =\lim_{n \to \infty}\left(\frac{(n+1)^n}{e^{2n}}\right)^{\frac{1}{n}}\\
     &=\lim_{n \to \infty}\frac{(n+1)^{\frac{n}{n}}}{e^{\frac{2n}{n}}}
  =\lim_{n \to \infty}\frac{n+1}{e^2}=\infty
\end{align*}
Therefore the given series is divergent by the Root Test.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine the convergence or divergence of the following series.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{k=1}^\infty\frac{2^{3k+1}}{k^k}$
    \item $\displaystyle \sum_{n=2}^\infty\frac{(-1)^n}{(\ln n)^n}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

\subsection{Rearrangement of Series}\index{rearrangement of a series}

For a finite summation of terms the order in which we add the numbers does not matter, i.e. $1+4+2=4+1+2$, or, in symbols $a_1+a_2+a_3=a_2+a_1+a_3$.  The second summation is a \textbf{rearrangement} of the first. To make the idea precise we note that the indices on the second summation, $(2,1,3)$ are a \textbf{permutation} of those on the first $(1,2,3)$.  A permutation on the infinite set of positive indices $(1,2,3,\ldots)$ of a series can similarly be defined thereby making the intuitive definition of a \textbf{rearrangement of a series} precise.

If a series is absolutely convergent then we get the same sum regardless of the order in which the terms are added (as expected), as summarized in the following theorem.

\theorem{Any rearrangment of absolutely convergent series $\sum a_k$ has the same sum as the original series.}

However for a series that is only conditionally convergent the order in which we add the terms does matter.  Indeed we get the following remarkable result:

%TODO: Also Riemann Series Theorem.
\theorem{\textbf{Riemann Rearrangement Theorem:}\index{Riemann Rearrangement Theorem} Let $\sum a_k$ be a conditionally convergent series with sum $S$.  Then for any real number $R$ there exists a rearrangement of series $\sum a_k$ having sum $R$.  Additionally there exist rearrangements of series $\sum a_k$ which diverge to $+\infty$, $-\infty$, and which fail to approach any limit, finite or infinite.}

\begin{example}
  By the latter theorem it follows that the (conditionally convergent) alternating harmonic series can be rearranged to sum to any number or to diverge.
\end{example}
% See wikipedia entry for instance where this is illustrated.

% TODO:  Look up the paper by Keith Taylor where he originally showed that a
% rather important constant of solid state physics which arises due to
% the force on an atom in a lattice of alternating positive and
% negative charge could be shown to be meaningful despite the fact
% that it was conditionally convergent (I believe that was the issue
% as he explained it.)  Keith put it on a firm theoretical foundation.  If I
% am correct in my recall, this would be a good counterexample to a
% conditionally convergent series being entirely meaningless physically.
% Indeed there must be some property of that type of series that makes it
% better in some way than the run of the mill conditionally convergent series.

\begin{exercise}
  \input{SEQUENCES_AND_SERIES/tests-for-absolute-convergence.tex}
\end{exercise}

\section{Procedure for Testing Series}

We have seen several methods for testing for the convergence and divergence of series.  The form of the series should suggest the type of test to be used.  The following steps will be helpful for determining convergence and divergence of series.
\begin{enumerate}
\item Recognize known series with associated convergence and divergence properties:
  \begin{description}
    \item[telesoping series:] \index{series! telescoping} Series for which $n^{\text{th}}$ partial sum $\displaystyle S_n=\sum_{k=1}^na_k$ can be directly evaluated and $\displaystyle\lim_{n\to\infty}S_n$ taken to determine convergence.
\item[geometric series:] \index{series! geometric} $\displaystyle \sum_{k=1}^\infty ar^{k-1}=\sum_{k=0}^\infty ar^k$ is convergent for $|r|<1$ and otherwise divergent.
\item[$\bm{p}$-series:] \index{series! $\emph{p-}$} $\displaystyle \sum_{k=1}^\infty\frac{1}{k^p}$ is convergent for $p>1$ and otherwise divergent.\newline
(Note that $p=1$ is $\sum\frac{1}{k}$ the (divergent) \textbf{harmonic} series.) \index{series! harmonic}
\end{description}
\item If $\displaystyle\lim_{k\to\infty}a_k\neq 0$ or that limit does not exist then the series is divergent by the \textbf{Term Test for Divergence}\index{Term Test for Divergence}.
\item If $\displaystyle\lim_{k\to\infty}a_k=0$ then proceed as follows:
\begin{enumerate}
\item\label{itm:positiveseries} If the terms of the series are \textbf{positive}, use one of the following tests.
\begin{description}
\item[Basic Comparison Test:]\index{Basic Comparison Test} \index{Test for convergence and divergence of a series! Basic Comparison Test} Useful when $a_k$ is a rational or algebraic function of $k$ (i.e. involving roots of polynomials).  Consider a suitable geometric or $p$-series for comparison.  Remember any comparison series must be positive.
\item[Limit Comparison Test:] \index{Limit Comparison Test} \index{Test for convergence and divergence of a series!Limit Comparison Test} Same criteria as the Basic Comparison Test.  Choose this one if evaluating the limit of the ratio of comparing terms is easier than proving an inequality between them as required in the Basic Comparison Test.
\item[Ratio Test:] \index{Ratio Test} \index{Test for convergence and divergence of a series! Ratio Test} Useful for series involving factorials or other products (including a constant raised to power $k$). Do not use this test for rational or algebraic functions of $k$ as these result in inconclusive ($L=1$) results.
\item[Root Test:]\index{Root Test} \index{Test for convergence and divergence of a series! Root Test} Useful if $a_k$ may be written $a_k=(b_k)^k$.
\item[Integral Test:] \index{Integral Test} \index{Test for convergence and divergence of a series! Integral Test} Useful if $a_k=f(k)$ for positive, continuous, decreasing $f(x)$ and $\int_1^\infty f(x)\,dx$ is easily evaluated. 
\end{description}
\item If the series is \textbf{alternating} (either $\sum (-1)^ka_k$ or $\sum (-1)^{k-1}a_k$ for $a_k>0$) either:
\begin{enumerate}
\item Use the \textbf{Alternating Series Test} \index{Alternating Series Test} \index{Test for convergence and divergence of a series! Alternating Series Test} .
\item Apply a positive series test from \ref*{itm:positiveseries} above to the absolute value of the alternating series (either $\sum\left|(-1)^ka_k\right|=\sum a_k$ or $\sum\left|(-1)^{k-1}a_k\right|=\sum a_k$) since the convergence of the latter implies the convergence of the alternating series.
\end{enumerate}
\item If the terms of the series $\sum a_k$ are neither positive nor alternating, apply a test from \ref*{itm:positiveseries} above to the absolute value of the series, $\sum|a_k|$. If $\sum |a_k|$ is convergent then $\sum a_k$ is also convergent.
\item If the series only satisfies theorem criteria (positivity, decreasing, alternating, etc.) after a certain point (i.e. for $k\ge n$ for some $n$) apply the above steps to the tail series $\sum_{k=n}^\infty a_k$.  The convergence or divergence of the entire series will be the same as that of the tail series since they only differ by a finite number of terms of finite sum.
\end{enumerate}
\end{enumerate}

\textbook{\clearpage}
\begin{example}
Determine the convergence (absolute or conditional) or divergence of the following series. 
\begin{multicols}{2} 
\begin{enumerate}
\item $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^nn!}{(2n+1)^5}$
\item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n}\frac{n}{e^n}$
\item $\displaystyle \sum_{n=1}^{\infty} \left(\frac{1}{10^n}+\frac{5}{n^3}\right) $
\item $\displaystyle \sum_{n=1}^{\infty} \frac{(n+2)^3}{(n^3+5)^2}$
\item $\displaystyle \sum_{n=1}^{\infty} \left(\frac{2n+1}{2n+5}\right)^n$ 
\end{enumerate}
\end{multicols}
\begin{solution*}
\begin{enumerate}
\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with $\displaystyle a_n=\frac{(-1)^{n}n!}{(2n+1)^5}$. The presence of the factorial suggests trying the Ratio Test:
  \begin{align*}
    \lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right| &=\lim_{n \to \infty}\left|\frac{\frac{(-1)^{n+1}(n+1)!}{(2n+3)^5}}{\frac{(-1)^nn!}{(2n+1)^5}}\right|
                                                           =\lim_{n \to \infty}\frac{(n+1)!(2n+1)^5}{n!(2n+3)^5}\\
    &=\lim_{n \to \infty} \frac{(n+1)n!(2n+1)^5}{n!(2n+3)^5}
                                                         =\lim_{n \to \infty}\frac{(n+1)(2n+1)^5}{(2n+3)^5}\\
&=\lim_{n \to \infty} (n+1)\lim_{n \to \infty}\left(\frac{2n+1}{2n+3}\right)^5 =\lim_{n \to \infty} (n+1)\left(\lim_{n \to \infty}\frac{2n+1}{2n+3}\right)^5 \\
&=\lim_{n \to \infty}(n+1)\left(\lim_{n \to \infty}\frac{n}{n}\cdot\frac{\frac{2n}{n}+\frac{1}{n}}{\frac{2n}{n}+\frac{3}{n}}\right)^5
=\lim_{n \to \infty}(n+1)\lim_{n \to \infty}\left(\frac{2+\frac{1}{n}}{2+\frac{3}{n}}\right)^5 \\
&=\infty\cdot 1=\infty
  \end{align*}
Therefore the given series is divergent by the Ratio Test.
\item  The given series is $\displaystyle \sum_{n=1}^\infty c_n=\sum_{n=1}^\infty(-1)^na_n$ where $\displaystyle c_{n}=(-1)^na_n$ and $\displaystyle a_{n}=\frac{n}{e^n}>0$. So the series is an alternating series. Let $\displaystyle f(x)=\frac{x}{e^x}$, then
\begin{align*}
  \lim_{x \to \infty} f(x)&=\lim_{x \to \infty} \frac{x}{e^x} \underset{\lh}= \lim_{x \to \infty} \frac{1}{e^x}=\frac{1}{\infty}=0\\
  &\implies \lim_{n\to\infty}a_n=\lim_{n\to\infty}f(n)=0\\
  f'(x)& =\frac{(1)e^x-xe^x}{e^{2x}}=\frac{e^x(1-x)}{e^{2x}}=\frac{1-x}{e^x}<0 \ \  \text{for}\ \  x>1 \\
                        &\implies a_{n+1}=f(n+1)\le f(n)=a_n\  \  \text{for}\ \  n\ge 1
\end{align*}
Therefore the given series $\sum c_n$ is convergent by the Alternating Series Test.

Now, consider the series of absolute values:
$\displaystyle \sum_{n=1}^\infty a_n=\sum_{n=1}^\infty\frac{n}{e^n}$\,:
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|&=\lim_{n \to \infty}\left|\frac{\frac{n+1}{e^{n+1}}}{\frac{n}{e^n}}\right|=\lim_{n \to \infty}\left|\frac{(n+1)e^n}{ne^{n+1}}\right|
                                                     =\lim_{n \to \infty}\frac{(n+1)e^n}{n e^n e}=\lim_{n \to \infty}\frac{n+1}{n e}\\
  &=\frac{1}{e}\lim_{n \to \infty}\frac{n}{n}\cdot\frac{\frac{n}{n}+\frac{1}{n}}{\frac{n}{n}}
=\frac{1}{e}\lim_{n \to \infty}\frac{1+\frac{1}{n}}{1}=\frac{1}{e}\cdot \frac{1+0}{1}=\frac{1}{e}<1 
\end{align*}
Therefore the series $\sum a_n=\sum |c_n|$ is convergent by the Ratio Test. Thus, the series \mbox{$\displaystyle \sum c_n=\sum_{n=1}^{\infty} (-1)^{n}\frac{n}{e^n}$} is absolutely convergent.

The astute reader will notice that because the series turned out to be absolutely convergent it was unnecessary to apply the Alternating Series Test. Had we just applied the Ratio Test to $\sum c_n$ directly we could have concluded that series absolutely converged (and hence converged).  As a general rule, if you must determine not only the convergence of a series but the type of convergence (conditional or absolute) always consider the convergence of the absolute series first.  If that converges the original series is absolutely convergent and no further consideration of the series is required.

\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with The given series can be written as:
  \begin{align*}
\sum_{i=1}^{\infty} \left(\frac{1}{10^n}+\frac{5}{n^3}\right)&=\sum_{n=1}^{\infty}\frac{1}{10^n} +\sum_{n=1}^{\infty}\frac{5}{n^3}\\
&= \underbrace{\sum_{n=1}^{\infty}\frac{1}{10}\cdot\left(\frac{1}{10}\right)^{n-1}}_{\begin{array}{c}\text{geometric series}\\a=\frac{1}{10}, r= \frac{1}{10}\\ |r| < 1\ (\text{convergent})\end{array}}+\underbrace{5\sum_{n=1}^{\infty}\frac{1}{n^3}}_{\begin{array}{c}p\text{-series}\\p=3\\ p>1\ (\text{convergent})\end{array}}
\end{align*}
Thus the given series is convergent as it is the sum of two convergent series.  The convergence is absolute as the series is already positive. 
\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with $\displaystyle a_{n}= \frac{(n+2)^3}{(n^3+5)^2}$.  Consideration of the order of the polynomials suggests a comparison with $\sum b_n$ where $\displaystyle b_{n}= \frac{n^3}{(n^3)^2}= \frac{n^3}{n^6}=\frac{1}{n^3}$. Then
$\displaystyle \sum_{n=1}^{\infty} b_{n}= \sum_{n=1}^{\infty} \frac{1}{n^3}$
is a convergent p-series since $p=3>1$. Also, 
\begin{align*}
\lim_{n \to \infty} \frac{a_n}{b_n}&=\lim_{n \to \infty} \frac{\frac{(n+2)^3}{(n^3+5)^2}}{\frac{1}{n^3}} 
=\lim_{n \to \infty} \frac{n^3(n+2)^3}{(n^3+5)^2}
=\lim_{n \to \infty} \frac{n^3(n^3+6n^2+12n+8)}{n^6+10n^3+25} \\
&=\lim_{n \to \infty} \frac{n^6+6n^5+12n^4+8}{n^6+10n^3+25}
=\lim_{n \to \infty} \frac{n^6}{n^6}\cdot\frac{\frac{n^6}{n^6}+ \frac{6n^5}{n^6}  +\frac{12n^4}{n^ 6}  +\frac{8}{n^6} }{\frac{n^6}{n^6} + \frac{10n^3}{n^6} +  \frac{25}{n^6} } \\
& =\lim_{n \to \infty} \frac{1+ \frac{6}{n} +\frac{12}{n^2} +\frac {8}{n^6} }{1+ \frac{10}{n^3} + \frac{25}{n^6}}
=\frac{1+0+0+0}{1+0+0}= 1>0
\end{align*}
Therefore the given series is also convergent by the Limit Comparison Test. Since the series is positive it is absolutely convergent. 
\item  The given series is $\displaystyle\sum_{n=1}^\infty a_n$ with $a_n=\left(\frac{2n+1}{2n+5}\right)^n$.  The power in the term suggests trying the Root Test:
\begin{align*}
\lim_{n \to \infty} \sqrt[n]{|a_n|}&=\lim_{n\to\infty}\sqrt[n]{\left|\left(\frac{2n+1}{2n+5}\right)^n\right|} =\lim_{n \to \infty} \frac{2n+1}{2n+5}
  =\lim_{n \to \infty} \frac{n}{n}\cdot\frac{\frac{2n}{n}+ \frac{1}{n}}{\frac{2n}{n}+ \frac{5}{n}}\\
  & =\lim_{n \to \infty} \frac{2+ \frac{1}{n}}{2+ \frac{5}{n}}= \frac{2+0}{2+0}= 1 
\end {align*}
Therefore the Root Test is inconclusive. Next consider trying the Term Test for Divergence.  To evaluate the limit let $\displaystyle f(x)=\left( \frac{2x+1}{2x+5}\right)^x$. Then $\displaystyle\lim_{x\to\infty}f(x)$ is the indeterminate form~$1^\infty$. Taking the logarithm gives $\displaystyle \ln f(x)=x\ln  \left(\frac{2x+1}{2x+5}\right)$ and so
\begin{align*}
\lim_{x \to \infty} \ln f(x)&= \lim_{x \to \infty} x \ln \left(\frac{2x+1}{2x+5}\right) \ \  (\infty \cdot 0\ \text{form})  \\
&=\lim_{x \to \infty} \frac{\ln (\frac{2x+1}{2x+5})}{x^{-1}} \,\,  \left(\frac{0}{0}\,\, \text{form}\right) \\
&\underset{\lh}=\lim_{x \to \infty} \frac{\frac{1}{\frac{2x+1}{2x+5}} \cdot \frac{2(2x+5)-2(2x+1)}{(2x+5)^2} }{-x^{-2}}
=\lim_{x \to \infty} \frac{\frac{4x+10-4x-2}{(2x+5)(2x+1)}}{-x^{-2}} \\
&=-\lim_{x \to \infty} \frac{8x^2}{(2x+5)(2x+1)} 
=-\lim_{x \to \infty} \frac{8x^2}{4x^2+12x+5} \\
&\underset{\lh}=-\lim_{x \to \infty} \frac{16x}{8x+12} \underset{\lh}= -\lim_{x \to \infty} \frac{16}{8}= -2\,.
\end{align*}
Thus
$\displaystyle \lim_{x \to \infty} f(x)=e^{-2}$ and therefore $\displaystyle\lim_{n\to\infty}a_n=\lim_{n\to\infty}f(n)=e^{-2}\neq0$\,.\\[1em]
Thus the given series is divergent by the Term Test for Divergence. 
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Determine the convergence (absolute or conditional) or divergence of the following series.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \sum_{k=1}^\infty \frac{2k^2}{k^2+1}$
    \item $\displaystyle \sum_{n=1}^\infty (-1)^n\frac{n}{n^2+1}$
    \item $\displaystyle \sum_{n=1}^\infty \frac{1000-n}{n!}$
    \item $\displaystyle \sum_{k=1}^\infty e^{-2k}$
    \item $\displaystyle \sum_{n=1}^\infty \frac{5^n}{n^6\,3^{n+1}}$
    \item $\displaystyle \sum_{k=1}^\infty \sin\left(\frac{\pi}{2}+\frac{1}{k}\right)$
    \item $\displaystyle \sum_{k=1}^\infty \frac{k^k}{10^k}$
    \item $\displaystyle \sum_{n=1}^\infty \left(-1\right)^{n-1}\frac{\sqrt{n}}{n^2+1}$
    \item $\displaystyle \sum_{n=1}^\infty \left(\frac{1}{3^n}+\frac{5}{\sqrt{n}}\right)$
    \end{enumerate}
    ~%ensures ragged bottom second column
  \end{multicols}
\end{furtherquestions}
\end{example}

\textbook{\clearpage}
\begin{exercise}
\input{SEQUENCES_AND_SERIES/procedure-for-testing-series.tex}
\end{exercise}

\section{Power Series}\index{power series} \index{series! power}
We have seen that in many cases the terms of the series $\sum a_k$ may be written as a function of the summation index, namely $a_k=f(k)$, for some function $f$. However if the series $\sum a_k$ is convergent its value is a number, namely its sum.  The final sum does not depend on the index $k$ in the same way that a definite integral $\int_a^b f(t)\,dt$ results in a number independent of the dummy variable $t$.

Consider, however, the situation where the terms $a_k$ depend additionally on an actual variable, say $x$, different from the summation index, present in the series (i.e. $a_k=a_k(x)$). In this case the sum of the series (and indeed its convergence) depends on the value of $x$.

\begin{example}
The geometric series with $a=1$ and $r=x$ is given by

\[\sum_{k=0}^\infty x^k=1+x+x^2+\cdots+x^k+\cdots\]

Here $a_k(x)=x^k$.  The sum is now a function of $x$, namely $\displaystyle\frac{1}{1-x}$, and is valid for $|x|<1$ for which the series is convergent.
\end{example}

We could introduce the variable $x$ into the terms of a series in many ways, for instance
\[\sum_{k=1}^\infty \frac{\sin(kx)}{k!}\,.\]

If we choose to introduce it as in our geometric series above, namely so that the terms of the series look like terms in a polynomial, $c_k x^k$, we have a \textbf{power series}.

\definition{Let $x$ be a variable.  A series of the form
\[\sum_{k=0}^\infty c_k x^k=c_0+c_1x+c_2x^2+\cdots+c_kx^k+\cdots\,,\]
where $c_k$ are real constants (for $k=0,1,2\ldots$) is a \textbf{power series in} $\bm{x}$ \index{power series! in $\emph{x}$}.  The constants $c_k$ are called the \textbf{coefficients}\index{power series! coefficients of} of the series.}

\begin{example}
  The geometric series with $r=x$ above,  $1+x+x^2+\cdots+x^k+\cdots$,   is a power series in $x$ with coefficients $c_k=1$ for all $k$.
\end{example}

If we choose to make the $k^{\mbox{\tiny th}}$ term of the series have the more general form $c_k(x-a)^k$ we get the following.

\definition{Given real constant coefficients $c_k$ and real constant $a$ the \textbf{power series in} $\bm{(x-a)}$ is
\[\sum_{k=0}^\infty c_k(x-a)^k=c_0+c_1(x-a)+c_2(x-a)^2+\cdots+c_k(x-a)^k+\cdots~.\]
The series is also known as the \textbf{power series about} $\bm{a}$ or \textbf{centred on} $\bm{a}$ \index{power series! centred on a}.}

A power series in $x$ is just a special case of this last definition with $a=0$.

Since the convergence of a power series (and indeed its sum should it converge) will, in general, depend on the value of the variable $x$, an obvious question is to find the values of $x$ for which the power series is convergent.

\begin{example}\label{exam:powerseriesconvergence}
Find the values of $x$ for which the following power series are convergent. 
\begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle  \sum_{n=1}^{\infty} \frac{(-1)^n}{(n+1)!}x^{n}$
\item $\displaystyle  \sum_{n=1}^{\infty} \frac{2n+1}{3^n}x^{n}$
\item  $\displaystyle  \sum_{n=2}^{\infty} \frac{x^n}{\ln n}$
\item  $\displaystyle  \sum_{n=1}^{\infty} n^n(x+3)^n$
\end{enumerate}
\end{multicols}
\begin{solution*}
  \begin{enumerate}
    \item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= \frac{(-1)^n}{(n+1)!}x^{n}$. Consider the following Ratio Test limit:
\begin{align*}
  \lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n} \right|&=\lim_{n\to\infty}\left|\frac{\frac{(-1)^{n+1}}{(n+2)!}x^{n+1}}{\frac{(-1)^n}{(n+1)!}x^{n}}\right|=\lim_{n\to\infty}\left|\frac{x^{n+1}}{(n+2)!}\frac{(n+1)!}{x^n}\right|\\
&=\lim_{n\to\infty}\frac{(n+1)!}{(n+2)(n+1)!}|x|=\lim_{n\to\infty}\frac{|x|}{n+1}=0
\end{align*}
Note that here, since the limit is in $n$, the value $x$ is treated as a constant for the purpose of evaluating the limit. Since the limit of $0$ is less than $1$ we have, by the Ratio Test, that the power series converges for all real values of $x$. 
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= \frac{2n+1}{3^n}x^{n}$. The Ratio Test limit is:
\begin{align*}
\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n} \right|
&=\lim_{n \to \infty} \left|\frac{\frac {2n+3}{3^{n+1}}x^{n+1}}{\frac{2n+1}{3^n}x^n}\right| 
=\lim_{n \to \infty}\left|\frac{(2n+3)x^{n+1}}{3^{n+1}}\frac{3^n}{(2n+1)x^n}\right| 
=\lim_{n \to \infty} \frac{2n+3}{3(2n+1)}|x| \\
&=\lim_{n \to \infty} \frac{n}{n}\cdot\frac{ \frac{2n}{n} + \frac{3}{n} }{3\left(\frac{2n}{n} + \frac{1}{n}\right) }\,\, |x| 
=\lim_{n  \to \infty} \frac{2+ \frac{3}{n} }{3+\left(2 +\frac{1}{n}\right)}\,\, |x|
= \frac{2+0}{3(2+0)}\,\,|x|
= \frac{1}{3}\,\,|x|
\end{align*}
Thus, by the Ratio Test, the series is convergent if the limit satisfies
\[\frac{1}{3}|x|<1 \implies |x|< 3 \implies -3<x<3\,.\]
The Ratio Test is inconclusive if $\displaystyle\frac{1}{3}|x|=1\implies |x|=3$ (so $x=3$ or $x=-3$). We must study these two values separately.
\begin{itemize}
\item If $x=3$ then the series becomes:
\begin{align*}
\sum_{n=1}^{\infty} \frac{2n+1}{3^n}3^n &= \sum_{n=1}^{\infty} (2n+1)\,.
\end{align*}
Then $\displaystyle \lim_{n \to \infty} (2n+1) = \infty$ shows that  
when $x=3$ the series diverges by the Term Test for Divergence. 
  \item If $x=-3$ then the series becomes:
\begin{align*}
\sum_{n=1}^{\infty} \frac{2n+1}{3^n}(-3)^n &= \sum_{n=1}^{\infty}(-1)^n(2n+1)\,.
\end{align*}
Then the fact that $\displaystyle\lim_{n \to \infty} (-1)^n(2n+1)$ does not exist shows that when $x=-3$ the series is divergent by the Term Test for Divergence.
\end{itemize}
Therefore the given power series is convergent if $-3<x<3$ and divergent otherwise.
\item The power series is $\displaystyle\sum_{n=2}^\infty a_n$ where $\displaystyle a_n= \frac{x^n}{\ln n}$. The Ratio Test limit is:
\begin{align*}
\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|=\lim_{n \to \infty} \left|\frac{ \frac{x^{n+1}}{\ln (n+1)} }{\frac{x^n}{\ln n} }\right| 
=\lim_{n \to \infty} \left|\frac{x^{n+1}}{\ln (n+1)} \cdot \frac{\ln n}{x^n} \right| 
=\lim_{n \to \infty} \frac{\ln n}{\ln(n+1)}|x| 
\end{align*}
Again $|x|$ here is just a constant with respect to the limit and we need to evaluate $\displaystyle\lim_{n \to \infty} \frac{\ln n}{\ln(n+1)}$.  
If $\displaystyle f(y)= \frac{\ln y}{\ln (y+1)}$, then
\begin{align*}
\lim_{y \to \infty} f(y)=&\lim_{y \to \infty} \frac{\ln y}{\ln (y+1)} \ \ \left(\frac{\infty}{\infty}\ \text{form}\right)\\ 
\underset{\lh}=&\lim_{y \to \infty} \frac{\frac{1}{y} }{\frac{1}{y+1} }=\lim_{y \to \infty} \frac{y+1}{y}
=\lim_{y \to \infty} \left(1+ \frac{1}{y}\right)= 1+0=1\,.
\end{align*} 
Thus, 
\begin{align*}
\lim_{x \to \infty} \left|\frac{a_{n+1}}{a_n}\right|= \lim_{n \to \infty} \frac{\ln n}{\ln (n+1)}|x|=(1)\,\,|x|= |x|
\end{align*}
Therefore, the series is convergent if $|x|<1 \implies -1<x<1$ by the Ratio Test. The Ratio Test is inconclusive when $|x|=1$ (so $x=1$ and $x=-1$) and we must study these two values separately.
\begin{itemize}
  \item If $x=1$ then the series becomes:
\begin{align*}
\sum_{n=2}^{\infty} \frac{1^n}{\ln n}= \sum_{n=1}^{\infty} \frac{1}{\ln n}\,.
\end{align*}
Let $\displaystyle a_n= \frac{1}{\ln n}$, $\displaystyle b_n= \frac{1}{n}$. Then $\displaystyle \sum_{n=2}^{\infty} b_{n}= \sum_{n=2}^{\infty} \frac{1}{n} $ is a divergent harmonic series. \\
Also for $n\ge2$ we have 
\begin{align*}
0 < \ln n \leq n \implies \frac{1}{\ln n} \geq \frac{1}{n} \,\, \text{for} \,\, n \geq 2 
\end{align*}
Thus when $x=1$ the series is divergent by the Basic Comparison Test. 
\item If $x=-1$  then the series becomes an alternating series:
  \begin{align*}
    \sum_{n=2}^{\infty} \frac{1}{\ln n}(-1)^n=\sum_{n=2}^\infty (-1)^na_n \ \text{where}\  a_n= \frac{1}{\ln n}>0
  \end{align*}
  Then
  \begin{align*}
    \lim_{n \to \infty} a_n= \lim_{n \to \infty} \frac{1}{\ln n}= \frac{1}{\infty}= 0
  \end{align*}
  If $\displaystyle f(y)=\frac{1}{\ln y}$,  then 
$\displaystyle f’(y)= -\frac{1}{y(\ln y)^2} <0 $ for $y \geq 2$ so $a_n=f(n)$ is a decreasing sequence. Thus when $x=-1$ the series is convergent by the Alternating Series Test.
\end{itemize}
Therefore the given power series is convergent if $-1 \leq x < 1$\,.
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= n^n(x+3)^{n}$. Since $(x+3)=(x-(-3))$ this is a power series centred at $a=-3$.  Consider the following Root Test limit:
  \begin{align*}
    \lim_{n\to\infty}\sqrt[n]{|a_n|}&=\lim_{n\to\infty}\sqrt[n]{\left|n^n(x+3)^{n}\right|}=\lim_{n\to\infty}\sqrt[n]{n^n}\sqrt[n]{|x+3|^{n}}=\lim_{n\to\infty}n|x+3|\\
    &=\left\{\begin{array}{c}0\ \ \text{if}\ x=-3\\ \infty\ \text{if}\ x\neq-3\end{array}\right.
  \end{align*}
  Notice that while $x$ and hence $x+3$ are effectively constants when evaluating the limit, the case where $|x+3|=0$ (so $x=-3$) does result in a unique limit of zero since $a_n=0$ when $x=-3$.  By the Root Test only in that case is the limit less than $1$.  Hence the power series converges only for $x=-3$.

  As an aside, while the Root Test can be used to analyze power series convergence, the Ratio Test is almost always preferred.  The Root Test usually results in limits with indeterminate forms requiring evaluation. This example was an exception.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
Find the values of $x$ for which the following power series are convergent.
\begin{multicols}{3}
\begin{enumerate}
\item $\displaystyle \sum_{k=0}^\infty\frac{x^k}{k!}$
\item $\displaystyle \sum_{n=0}^\infty\frac{n^2}{2^n}x^n$
\item $\displaystyle \sum_{n=1}^\infty\frac{\ln n}{e^n}(x-e)^n$
\end{enumerate}
\end{multicols}
\end{furtherquestions}
\end{example}

As suggested by the previous example a power series about $a$ will converge on an interval centred on $a$ as detailed in the following theorem.

\theorem{\label{thm:powerradius}The power series $\displaystyle\sum_{k=0}^\infty c_k(x-a)^k$ will either:
\begin{enumerate}
\item\label{itm:radconverge1} Converge only at $x=a$\,.
\item\label{itm:radconverge2} Converge for $|x-a|<R$ and diverge for $|x-a|>R$ for some positive real number $R$\,.
\item\label{itm:radconverge3} Converge for all $x$\,.
\end{enumerate}
}

\definition{The \textbf{radius of convergence} \index{radius of convergence}\index{power series! radius of convergence} $\bm{R}$ for a power series $\sum_{k=0}^\infty c_k(x-a)^k$ is the value $R$ if Part~\ref{itm:radconverge2} of Theorem~\ref{thm:powerradius} applies.  For Part~\ref{itm:radconverge1} the radius of convergence is defined to be $R=0$ and for Part~\ref{itm:radconverge3} the radius of convergence is defined to be $R=\infty$.}

% TODO: Give a note about what |x|<R -R<x<R so |x-a|<R implies -R<x-a<R implies
% a-R<x<a+R.
% TODO: Add a diagram showing the Part 2 interval centred on a.
% TODO: Incidentally the reason ratio and root are taught with absolute values
% would appear to be their application to finding intervals of convergence
% If I put everything within positive series I should perhaps restate
% these theorems (and others?) for convenience in their absolute value forms.
% Note that for some of these the case where the term is 0 may be problematic
% i.e. if the original series is positive then taking absolute values still
% means that x \neq 0.

\textbook{\clearpage}
\definition{The \textbf{interval of convergence} \index{power series! interval of convergence} \index{interval of convergence} $\bm{I}$ of a power series $\sum_{k=0}^\infty c_k(x-a)^k$ is the set of values of $x$ for which the series converges.  For the three possibilities of convergence one has:
\begin{enumerate}
\item The set containing the single value $x=a$ (i.e. $\left\{a\right\}$) for $R=0$\,.
\item One of $[a-R,a+R]$, $[a-R,a+R)$, $(a-R,a+R]$, or $(a-R,a+R)$ for $R>0$ finite.
\item $(-\infty,\infty)$ for $R=\infty$\,.
\end{enumerate}
The choice of interval in the second case depends upon the convergence or not of the series at the interval endpoint values $x=a\pm R$\,.
}

\begin{example}
  For Example~\ref{exam:powerseriesconvergence} find the radii and intervals of convergence of each series.
  
  \begin{solution}
    Consideration of the solutions of Example~\ref{exam:powerseriesconvergence} shows the radii and intervals of convergence are:
    \begin{multicols}{2}
    \begin{enumerate}
    \item $R=\infty$, $I=(-\infty,\infty)$
    \item $R=3$, $I=(-3,3)$
    \item $R=1$, $I=[-1,1)$
    \item $R=0$, $I=\{-3\}$
    \end{enumerate}
    \end{multicols}
  \end{solution}

  \begin{furtherquestions}
    For Example~\ref{exam:powerseriesconvergence} find the radii and intervals of convergence of each series in the Further Questions.
%For the previous example one has the following radii and intervals of convergence:
%\begin{enumerate}
%\item $R=\infty$, $I=(-\infty,\infty)$
%\item $R=2$, $I=(-2,2)$
%\item $R=e$, $I=(0,2e)$
%\end{enumerate}
\end{furtherquestions}
\end{example}

\begin{example}
Find the radius and interval of convergence of each of the following series. 
\begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle \sum_{n=1}^{\infty} \frac{x^n3^n}{(n!)^2}$
\item $\displaystyle \sum_{n=1}^{\infty} \frac{n!(x-2)^n}{5^n}$
\item $\displaystyle \sum_{n=1}^{\infty} \frac{(-5)^n(3x-2)^n}{n^2}$
\item $\displaystyle \sum_{n=1}^{\infty} (-1)^{n+1} \frac{(x-4)^n}{\sqrt{n+1}} $
\end{enumerate}
\end{multicols}
\begin{solution*}
\begin{enumerate}
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n=\frac{x^n3^n}{(n!)^2}$. The Ratio Test limit is
\begin{align*}
\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|=\lim_{n \to \infty} \left|\frac{ \frac{x^{n+1}3^{n+1} }{[(n+1)! ]^2} }{ \frac{x^n3^n}{(n!)^2} } \right|
&=\lim_{n \to \infty} \frac{ (n!)^2(3)}{  [(n+1)!]^2}|x| 
=\lim_{n \to \infty}\frac{3n!n!}{[(n+1)n!]^2}|x|\\
&=\lim_{n \to \infty} \frac{3}{(n+1)^2}|x| = 0 \cdot |x|= 0< 1 \,\, \text{for all}\ x\,.
\end{align*}
Therefore, the power series converges for all $x$. Thus the radius of convergence is $R=\infty$ and the interval of convergence is $I=(-\infty, \infty)$.
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= \frac{n!(x-2)^n}{5^n} $. The Ratio Test limit is
\begin{align*}
\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|&=\lim_{n \to \infty} \left|\frac { \frac{(n+1)!(x-2)^{n+1}}{ 5^{n+1} }} { \frac{n! (x-2)^n }{5^n} }\right| 
=\lim_{n \to \infty} \frac{5^n(n+1)n!} {5^{n+1}n!}|x-2| 
=\lim_{n \to \infty} \frac{n+1}{5}|x-2|\\
  &=\left\{\begin{array}{c}0\ \ \text{if}\ x=2\\ \infty\ \text{if}\ x\neq 2\end{array}\right.
\end{align*}
This power series is convergent only if $x=2$.
Therefore, the radius of convergence is $R=0$ and the interval of convergence is $I=\{2\}$. Note that the \emph{interval} is centred at $2$ as expected for a power series about $a=2$\,.
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= \frac{(-5)^n(3x-2)^n}{n^2} $. The Ratio Test limit is
\begin{align*}
 \lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|
 &=\lim_{n \to \infty} 
\left|\frac{ \frac{(5)^{n+1}(3x-2)^{n+1}} { (n+1)^2 }}{\frac{(-5)^n(3x-2)^n }{n^2}}\right|
=\lim_{n \to \infty} \frac{5n^2}{(n+1)^2} |3x-2|  
=5\lim_{n \to \infty} \left(\frac{n}{n+1}\right)^2 |3x-2| \\
&=5 \lim_{n \to \infty} \left(\frac{n}{n}\cdot\frac{ \frac{n}{n}} { \frac{n}{n}+ \frac{1}{n}  }\right)^2 |3x-2|  
                                                               =5 \lim_{n \to \infty} \left(\frac{1}{1 + \frac{1}{n} } \right)^2 |3x-2|\\
  &=5 \left(\frac{1}{1+0}\right)^2 |3x-2|=5|3x-2|
\end{align*}
The series is convergent by the Ratio Test if:
\begin{align*}
 5 |3x-2| <1 
 &\implies 15\left|x- \frac{2}{3}\right| <1
\implies \left|x -\frac{2}{3}\right | < \frac{1}{15} \\
&\implies -\frac{1}{15} < x- \frac{2}{3} < \frac{1}{15} \implies -\frac{1}{15}+\frac{2}{3} < x < \frac{1}{15}+\frac{2}{3} \\
&\implies \frac{3}{5} < x < \frac{11}{15}
\end{align*}
The Ratio Test is inconclusive (limit equals $1$) at the endpoints $\displaystyle x=\frac{3}{5}$ and $\displaystyle x=\frac{11}{15}$. We must test these two values separately.
\begin{itemize}
  \item If $\displaystyle x =\frac{3}{5}$ then the series becomes:
\begin{align*}
\sum_{n=1}^{\infty} \frac{(-5)^n\left(\frac{9}{5}-2\right)^n}{n^2}=\sum_{n=1}^{\infty} \frac{(-5)^n(-\frac{1}{5})^n}{n^2} = \sum_{n=1}^{\infty} \frac{1}{n^2}
\end{align*}
This is a convergent $p$-series since  $p=2>1$.
\item If $\displaystyle x=\frac{11}{15}$ then the series becomes:
  \begin{align*}
   \sum_{n=1}^{\infty} \frac{(-5)^n\left(\frac{33}{15}-2\right)^n}{n^2} \sum_{n=1}^{\infty} \frac{ (-5)^n(\frac{1}{5} )^n}{n^2} = \sum_{n=1}^{\infty} \frac{(-1)^n}{n^2}
  \end{align*}
 Consider $\displaystyle a_n=\frac{1}{n^2}$ and define $\displaystyle f(y)= \frac{1}{y^2}$ then:
\begin{align*}
   \lim_{y \to \infty} f(y)&= \lim_{y \to \infty} \frac{1}{y^2}=0 \implies \lim_{n\to\infty} a_n=\lim_{n\to\infty}f(n)=0\,.\\
   f'(y)&= -\frac{2}{y^3} < 0\ \text{for}\ y>0 \implies\ a_n=f(n)\ \text{decreases}\,.
\end{align*}
Thus the series $\displaystyle \sum_{n=1}^{\infty} \frac{(-1)^n}{n^2} $ is convergent by the Alternating Series Test.
\end{itemize}
Therefore the radius of convergence of the power series is $\displaystyle R= \frac{1}{15}$ and the interval of convergence is $I= \left[\frac{3}{5}, \frac{11}{15}\right]$.  Notice that the interval is centred at $a=\frac{2}{3}$.  That the original power series is centred at this value can be seen by writing $(3x-2)^n=3^n(x-2/3)^n$ in the original series.
\item The power series is $\displaystyle\sum_{n=1}^\infty a_n$ where $\displaystyle a_n= (-1)^{n+1} \frac{(x-4)^n}{\sqrt{n+1}}$. The Ratio Test limit is
\begin{align*}
\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|&= \lim_{n \to \infty} \left|\frac{ \frac{ (-1)^{n+2}(x-4)^{n+1} }{\sqrt{n+2} }}{ \frac{ (-1)^{n+1}(x-4)^n} {\sqrt{n+1}}}\right|
=\lim_{n \to \infty} \frac{ \sqrt{n+1}}{ \sqrt{n+2}} |x-4|
=\lim_{n \to \infty} \sqrt{ \frac{n+1}{n+2}} |x-4|\\
&=\lim_{n \to \infty} \sqrt{  \frac{n}{n}\cdot\frac{ \frac{n}{n} + \frac{1}{n} } {\frac{n}{n} + \frac{2}{n} } } |x-4|
=\lim_{n \to \infty} \sqrt{ \frac{1 + \frac{1}{n} } {  1 + \frac{2}{n} }  } |x-4|= |x-4|
\end{align*}
The series is convergent by the Ratio Test if $|x-4|<1=R$.  Then
\[|x-4|<1\implies -1<x-4<1\implies 3<x<5\]
which is as expected since the series is centred at $a=4$ with radius $1$.  The Ratio Test is inconclusive if $x=3$ or $x=5$ and we test these values separately.
\begin{itemize}
\item If $x=3$ then the series becomes
  \[\sum_{n=1}^\infty\frac{(-1)^{n+1}(-1)^n}{\sqrt{n+1}}=\sum_{n=1}^\infty\frac{(-1)^{2n+1}}{\sqrt{n+1}}=-\sum_{n=1}^\infty\frac{1}{\sqrt{n+1}}\]
  Let $\displaystyle a_n=\frac{1}{\sqrt{n+1}}$ and $\displaystyle b_n=\frac{1}{\sqrt{n}}$. Then $\displaystyle \sum_{n=1}^\infty b_n=\sum_{n=1}^\infty \frac{1}{\sqrt{n}}$ is a $p$-series with $\displaystyle p=\frac{1}{2}\le 1$ and hence divergent. Taking the limit
  \begin{align*}
    \lim_{n\to\infty}\frac{a_n}{b_n}&=\lim_{n\to\infty}\frac{\frac{1}{\sqrt{n+1}}}{\frac{1}{\sqrt{n}}}=\lim_{n\to\infty}\frac{\sqrt{n}}{\sqrt{n+1}}=\lim_{n\to\infty}\sqrt{\frac{n}{n+1}}=\sqrt{\lim_{n\to\infty}\frac{n}{n+1}}\\
    &=\sqrt{\lim_{n\to\infty}\frac{n}{n}\cdot\frac{\frac{n}{n}}{\frac{n}{n}+\frac{1}{n}}}=\sqrt{\lim_{n\to\infty}\frac{1}{1+\frac{1}{n}}}=\sqrt{\frac{1}{1+0}}=1>0
  \end{align*}
  shows the series $\displaystyle \sum_{n=1}^\infty\frac{1}{\sqrt{n+1}}$ is divergent by the Limit Comparison Test.
\item If $x=5$ then the series becomes
  \[\sum_{n=1}^\infty\frac{(-1)^{n+1}(1)^n}{\sqrt{n+1}}=\sum_{n=1}^\infty\frac{(-1)^{n+1}}{\sqrt{n+1}}\]
  Then $\displaystyle \lim_{n\to\infty}\frac{1}{\sqrt{n+1}}=\frac{1}{\infty}=0$.
  Let $\displaystyle f(y)=\frac{1}{\sqrt{y+1}}$ then: 
  \[f'(y)=-\frac{1}{2(y+1)^{\frac{3}{2}}}<0\ \text{for}\ x\ge 1\]
  shows the magnitude of the terms, $f(n)$, is decreasing.  Therefore the series  $\displaystyle\sum_{n=1}^\infty\frac{(-1)^{n+1}}{\sqrt{n+1}}$ converges by the Alternating Series Test.
\end{itemize}
Therefore the radius of convergence of the power series is $R=1$ and the interval of convergence is $I=(3,5]$\,.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
Find the radius and interval of convergence of each of the following series.
\begin{multicols}{2}
\begin{enumerate}
\item $\displaystyle\sum_{k=0}^\infty(-1)^k\frac{1}{k+1}(x-3)^k$
\item $\displaystyle\sum_{n=0}^\infty n^3(x-5)^n$
\item $\displaystyle\sum_{k=0}^\infty k!(2x-1)^k$
\item $\displaystyle\sum_{n=1}^\infty\frac{(2x-3)^n}{n\,3^n}$
\end{enumerate}
\end{multicols}
\end{furtherquestions}
\end{example}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/power-series.tex}
\end{exercise}

\section{Representing Functions with Power Series}\index{power series! representing functions with}

The sum of a power series for a given $x$ from its interval of convergence $I$ results in a number.  As such it is natural to consider the power series as defining a function $f(x)$ of $x$ on $I$ with the value of the function being the sum.  We write
\[f(x)=\sum_{k=0}^\infty c_k x^k\ \ (x\in I)\]
If the sum of the power series can be written in a closed form, then the power series can be considered a representation of that function valid on $I$. 

\begin{example}
The power series $\sum_{k=0}^\infty x^k=1+x+x^2+\cdots$ converges for $|x|<1$, and therefore is a function of $x$ on $I=(-1,1)$.  On that interval the sum for given $x$ is $\frac{1}{1-x}$.  The power series thus is a representation of the function $\frac{1}{1-x}$ on this restricted domain:
\[\frac{1}{1-x}=1+x+x^2+\cdots+x^k+\cdots=\sum_{k=0}^\infty x^k\ \ \mbox{for}\ |x|<1\]
\end{example}

One can find power series representations of other functions using known power series.
\begin{example}
  Find representations of the following functions with power series. Indicate the values of $x$ for which the representatives are valid.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \frac{1}{2-5x} $
    \item $\displaystyle \frac{x}{1+4x^2} $
    \item $\displaystyle \frac{x^2}{4+3x^2}$
    \end{enumerate}
  \end{multicols}
  \begin{solution*}
\begin{enumerate}
\item The given functions can be written as:
\begin{align*}
\frac{1}{2-5x}=\frac{1}{2(1-\frac{5}{2}x)}\,.
\end{align*}
The power series representation of $\displaystyle \frac{1}{1-x}$ is: 
\begin{align*}
\frac{1}{1-x}= \sum_{n=0}^{\infty}x^n,\  |x|<1\,.
\end{align*}
Thus
\begin{align*}
\frac{1}{2-5x}=\frac{1}{2(1-\frac{5}{2}x)}&=\frac{1}{2}\sum_{n=0}^{\infty}\left(\frac{5}{2}x\right)^n=\frac{1}{2}\sum_{n=0}^{\infty}\left(\frac{5}{2}\right)^nx^n    
=\sum_{n=0}^{\infty}\frac{5^n}{2^{n+1}}x^n\,,
\end{align*}
which converges when
\[\left|\frac{5}{2}x\right|<1 \implies |x| < \frac{2}{5} \implies -\frac{2}{5}<x<\frac{2}{5}\,.\]
\item This function can be written as 
\begin{align*}
\frac{x}{1+4x^2}=x\cdot\frac{1}{1-(-4x^2)}&=x\sum_{n=0}^{\infty}\left(-4x^2\right)^n,\ |-4x^2| <1\\ 
& = \sum_{n=0}^{\infty}(-1)^n4^nx^{2n+1}\,. 
\end{align*}
Note here that the variable $x$ is effectively a constant with respect to the series in $n$ and can therefore be brought in and out of the summation without affecting its convergence properties.

The series converges if
\begin{align*}
|-4x^2| <1 &\implies |4x^2| <1
\implies 4x^2 <1
\implies x^2 < \frac{1}{4}
\implies |x| < \frac{1}{2}\,.
\end{align*}
\item This function can be written as
\begin{align*}
\frac{x^2}{4+3x^2}&= \frac{x^2}{4(1+\frac{3}{4}x^2)}=\frac{x^2}{4}\cdot \frac{1}{1-\left(-\frac{3}{4}x^2\right)}  \\
&=\frac{x^2}{4}\sum_{n=0}^{\infty}\left(-\frac{3}{4}x^2\right)^n= \sum_{n=0}^{\infty}(-1)^n\frac{3^n}{4^{n+1}}x^{2n+2}\,.
\end{align*}
The power series converges if
\begin{align*}
\left|-\frac{3}{4}x^2\right|<1 \implies \frac{3}{4}x^2<1 \implies x^2 < \frac{4}{3} 
\implies |x| < \frac{2}{\sqrt{3}}\,.
\end{align*}
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Find representations of the following functions with power series.  Indicate the values of $x$ for which the representations are valid.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle \frac{1}{1+x}$
    \item $\displaystyle \frac{1}{1-x^3}$
    \item $\displaystyle \frac{x}{2-3x}$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

As functions power series behave like polynomials.  They are continuous and can be termwise differentiated and integrated to produce new functions of $x$ as detailed in the following theorem.

\theorem{\label{thm:diff_int_power}Suppose $f(x)$ is defined by the power series
\[f(x)=\sum_{k=0}^\infty c_k(x-a)^k=c_0+c_1(x-a)+c_2(x-a)^2+\cdots+c_k(x-a)^k+\cdots\,\]
with radius of convergence $R>0$ (finite or infinite).  Then on the interval $(a-R,a+R)$\,:
\begin{enumerate}
\item $f(x)$ is continuous. 
\item $f(x)$ is differentiable with derivative \index{power series! differentiation of}
\[f'(x)=\sum_{k=0}^\infty kc_k(x-a)^{k-1}=c_1+2c_2(x-a)+3c_3(x-a)^2+\cdots+kc_k(x-a)^{k-1}+\cdots\]
\item $f(x)$ is integrable with integral \index{power series! integration of}
\[\int f(x)\,dx=C+\sum_{k=0}^\infty\frac{c_k}{k+1}(x-a)^{k+1}=C+c_0(x-a)+\frac{c_1}{2}(x-a)^2+\frac{c_2}{3}(x-a)^3+\cdots+\frac{c_k}{k+1}(x-a)^{k+1}+\cdots\]
\end{enumerate}
The series resulting from differentiation and integration both have radius of convergence $R$\,.
}

Note that the above theorem shows that for power series the calculus operations of differentiation and integration can be exchanged with summation, just as occurs with finite sums.
\[\frac{d}{dx} \sum c_k(x-a)^k=\sum\frac{d}{dx}\left[c_k(x-a)^k\right]\]
\[\int\left[\sum c_k(x-a)^k\right]\,dx=\sum\left[\int c_k(x-a)^k\,dx\right]\]

If the power series is a representation of a function with a closed form, differentiation and integration can be used to find power series representations of other functions.

\begin{example}\label{exam:seriesdiffint}
  Find a power series representation for each of the following functions. Indicate the interval for which the representation is valid.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \frac{1}{(4-x)^2}$ 
    \item $\displaystyle \ln(1-x)$ \label{itm:seriesdiffintlog}
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item To find the power series representation of $\displaystyle\frac{1}{(4-x)^2}$ we note that it is just the derivative of the function $\displaystyle \frac{1}{4-x}$ which has the representation
\begin{align*}
 \frac{1}{4-x}& = \frac{1}{4(1-\frac{x}{4})}
 = \frac{1}{4}\sum_{n=0}^{\infty}\left(\frac{x}{4}\right)^n 
 = \sum_{n=0}^{\infty} \frac{x^n}{4^{n+1}}  \\
 &\implies \frac{1}{4-x} 
   = \sum_{n=0}^{\infty}\frac{x^n}{4^{n+1}},\ \left|\frac{x}{4}\right|<1\,.
\end{align*}
Differentiating both sides with respect to $x$ we have
\begin{align*}
\frac{d}{dx}\left(\frac{1}{4-x}\right) 
   &= \frac{d}{dx}\sum_{n=0}^{\infty}\frac{x^n}{4^{n+1}}= \sum_{n=0}^{\infty}\frac{d}{dx}\left(\frac{x^n}{4^{n+1}}\right)\\
  \frac{1}{(4-x)^2} &=\sum_{n=0}^{\infty}\frac{1}{4^{n+1}}nx^{n-1}=\sum_{n=1}^{\infty}\frac{n}{4^{n+1}}x^{n-1}\,,
\end{align*}
where we changed the starting index to $n=1$ since the $n=0$ term equals zero.  The power series converges if
\[\left|\frac{x}{4}\right|<1 \implies |x| < 4\,.\]

\item For the geometric series we have
  \[\frac{1}{1-x}= \sum_{n=0}^{\infty} x^n,\ |x| < 1\,.\]
  We desire a power series representation of $\ln|1-x|$ which is, up to a factor of $-1$,  just the integral of this function. Integrating both sides with respect to $x$ we obtain 
\begin{align*}
\underbrace{\int \frac{1}{1-x}\,dx}_{u=1-x, du=-dx}&= \int\left(\sum_{n=0}^{\infty}x^n\right)dx= \sum_{n=0}^{\infty}\int x^n\,dx\\
 \implies -\ln |1-x|&= \sum_{n=0}^{\infty} \frac{x^{n+1}}{n+1} + C\\ 
  \implies \ln|1-x| &=-\sum_{n=0}^{\infty} \frac{x^{n+1}}{n+1} - C
\end{align*}
For fixed $x$ the function on the left is a number and so the series on the right cannot contain an arbitrary constant.  To determine what it equals evaluate both sides at $x=0$ to get
\[\ln|1|=-0-C \implies C =0\,.\]
Thus
\[\ln(1-x)= -\sum_{n=0}^{\infty} \frac{x^{n+1}}{n+1}\,,\]
which converges for $|x|<1$ as follows by Theorem~\ref{thm:diff_int_power}. Since $1-x>0$ in that interval the absolute value bars have been dropped.

The series representation can be simplified by changing to a new index.\index{series! changing index}  Similar to introducing a new variable in an integral, here let $m=n+1$.  The new limits of the series then become
\begin{align*}
  n=\infty\ &\implies m=\infty+1=\infty \\
  n=0 & \implies m=0+1=1
\end{align*}
and our series representation simplifies to
\[\ln(1-x)= -\sum_{m=1}^{\infty} \frac{x^{m}}{m},\ |x|<1\,.\]
Just like an integration variable we can change the index name back to $n$ now if we so desire.  Reindexing in this manner has several uses.  In addition to simplifying a series, one can also compare two series whose lower limits are not the same by reindexing one to have the same limits as the other.  So for example one can show the following two forms of the geometric series are equivalent,
\[\sum_{n=0}^\infty ar^n=\sum_{m=1}^\infty ar^{m-1}\,,\]
by letting $m=n+1$ in the original sequence, changing the limits as above, and noting that $m=n+1\implies n=m-1$ when rewriting the general term of the first series in terms of the new index $m$.  Reindexing also allows one to align the limits of two series to allow their addition.  Alternatively one may reindex power series so that they all have the same power of $x^n$ so that they may be added and simplified.

Our new version of the representation $\ln(1-x)$ shows that at $x=1$ the series representation diverges as it reduces to the harmonic series. This is unsurprising as $\ln(1-1)=\ln(0)$ is undefined as well.  At $x=-1$ the series becomes the alternating harmonic series and therefore converges conditionally at that value.  The representation is, in fact, valid at $x=-1$ as its sum may be shown to be $\ln(1-(-1))=\ln 2$.  As such the series representation of $\ln(1-x)$ is valid on~$[-1,1)$\,.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Find a power series representation for each of the following functions.  Indicate the interval for which the representation is valid.
  \begin{multicols}{3}
    \begin{enumerate}
    \item$\displaystyle\frac{1}{(1+x)^2}$
    \item$\displaystyle\ln(1+x)$
    \item$\displaystyle\tan^{-1}\!x$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

Because power series representations are easy to integrate, they may be used as a means to integrate difficult functions.  The integral will only be valid for $x$ lying within $(a-R,a+R)$.
\begin{example}
  Integrate each of the following using a power series.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle \int \frac{1}{1+x^6}\, dx$ 
    \item $\displaystyle \int \frac{\ln (1-x)}{x}\, dx$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item  Starting with the geometric series $\displaystyle \frac{1}{1-x}= \sum_{n=0}^{\infty} x^n,\ |x| < 1$ first get a representation for the integrand:
\begin{align*}
\frac{1}{1+x^6}=\frac{1}{1-(-x^6)}=\sum_{n=0}^{\infty}(-x^6)^n= \sum_{n=0}^{\infty}(-1)^nx^{6n}\,,
\end{align*}
which converges for $|-x^6|<1\implies|x|<1\,.$
Thus
\begin{align*}
 \int \frac{1}{1+x^6}\, dx =\int \sum_{n=0}^{\infty}(-1)^nx^{6n}\, dx   
=\sum_{n=0}^{\infty}(-1)^n\frac{x^{6n+1}}{6n+1} + C 
=\sum_{n=0}^{\infty}\frac{(-1)^n}{6n+1}x^{6n+1}+ C\,,
\end{align*}
which is valid for $|x| < 1\,$.
\item In Example~\ref{exam:seriesdiffint} Problem~\ref{itm:seriesdiffintlog} we found
 $\displaystyle \ln (1-x) =-\sum_{n=1}^{\infty} \frac{x^n}{n},\ |x|<1\,.$
Thus
\begin{align*} 
\int \frac{\ln(1-x)}{x}\, dx & =-\int \frac{1}{x}\sum_{n=1}^{\infty}\frac{x^n}{n}\, dx=-\sum_{n=1}^{\infty}\frac{1}{n}\int x^{n-1}\, dx = -\sum_{n=1}^{\infty} \frac{1}{n}\frac{x^{(n-1)+1}}{(n-1)+1} + C \\
&= -\sum_{n=1}^{\infty} \frac{1}{n^2}x^n + C\,,
\end{align*}
which is valid for $|x|<1$\,.
\end{enumerate}
These indefinite integrals could be used to approximate a definite integral by suitably truncating the series at large $n$ and subtracting its evaluation at the limits of the integral as we would for any antiderivative.  The integration limits would need to be within the interval $(-1,1)$ due to the representation of these antiderivatives only being valid there.
\end{solution*}

\textbook{\clearpage}
\begin{furtherquestions}
  Integrate each of the following using a power series.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle\int\frac{1}{1+x^3}\,dx$
    \item $\displaystyle\int\frac{x}{1-x^4}\,dx$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/representing-functions-with-power-series.tex}
\end{exercise}

\section{Maclaurin Series}\index{Maclaurin Series} \index{series! Maclaurin}
We expect some functions $f(x)$ can be represented by an infinite series $\sum_{k=0}^{\infty}c_kx^k$ for a certain, potentially restricted, domain.  While we verified that $\frac{1}{1-x}$ could be represented by the geometric series, we would like a general mechanism for determining the coefficients of the power series that correspond to an arbitrary $f(x)$.  Suppose $f(x)$ has a power series expansion:
\[f(x)=c_0+c_1x+c_2x^2+c_3x^3+c_4x^4+\cdots~.\]
One observes that if we set $x=0$ that $f(0)=c_0$.  In other words, we can determine $c_0$ by evaluating $f$ at $x=0$.  We can differentiate the power series above term by term to get:
\[f'(x)=c_1+2c_2x+3c_3x^2+4c_4x^3+\cdots\]
If we evaluate this at $x=0$ we get $c_1=f'(0)$.  The next derivative is:
\[f''(x)=2c_2+6c_3x+12c_4x^2+\cdots\]
and so $c_2=\frac{f''(0)}{2}$.  Repeated differentiation and evaluation relates the $k^{\mbox{\tiny th}}$ coefficient $c_k$ to the derivative $f^{(k)}$ evaluated at $x=0$ as follows:
\[c_k=\frac{f^{(k)}(0)}{k!}\]
where recall the \textbf{factorial} \index{factorial} is defined by $\bm{k!=k\cdot(k-1)\cdots2\cdot1}$.  The formula is true for $k=0$ as well with the convention $f^{(0)}(x)=f(x)$ and noting that $\bm{0!=1}$ by definition.  Plugging our $c_k$ into the original power series we get the following definition.

\definition{Given function $f(x)$ differentiable to all orders at $x=0$ the power series in $x$ given by
\[f(x)=\sum_{k=0}^{\infty}\frac{f^{(k)}(0)}{k!}x^k=f(0)+f'(0)x+\frac{f''(0)}{2!}x^2+\frac{f'''(0)}{3!}x^3+\cdots\frac{f^{(k)}(0)}{k!}x^k+\cdots\]
is the \textbf{Maclaurin series} for $f(x)$.}

As a power series, the Maclaurin series must converge for $|x|<R$ for some radius of convergence $R$ dependent upon the function.\footnote{We will discuss shortly whether the Maclaurin series of $f(x)$ actually is a valid representation of the function on this interval.}
\begin{example}
  Find the Maclaurin series and its integral of convergence for each of the following functions.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle f(x)=e^{2x}$ 
    \item $\displaystyle f(x)=\cos x$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item Since the Maclaurin series is $f(x)=\displaystyle\sum_{n=0}^\infty\frac{f^{(n)}(0)}{n!}x^n$ we compute the following derivatives and evaluate them at zero:
\begin{align*}
f(x)&=e^{2x} \,\,, & f^{(0)}(0)=f(0)& =e^0=1    \\
f'(x)&=2e^{2x}\,\,, & f'(0)&=2e^0=2 \\
f''(x)&=4e^{2x}\,\,, & f''(0)&=4e^0=4 \\
f'''(x)&=8e^{2x}\,\,, & f'''(0)&=8e^0=8 \\
f^{(4)}(x)&=16e^{2x}\,\,, & f^{(4)}(0)&=16e^0=16 \\
  f^{(5)}(x)&=32e^{2x}\,\,, & f^{(5)}(0)&=32e^0=32 
\end{align*}
Since $0!=1$ and $1!=1$ the Maclaurin series is given by:
\begin{align*} 
f(x)&=f(0)+ f'(0)x + \frac{1}{2!}f''(0)x^2 + \frac{1}{3!}f'''(0)x^3 +\frac{1}{4!}f^{(4)}(0)x^4+ \frac{1}{5!}f^{(5)}(0)x^5 + \ldots\\
\implies e^{2x}&= 1+ 2x +\frac{1}{2!}(4)x^2 + \frac{1}{3!}(8)x^3 +\frac{1}{4!}(16)x^4 + \frac{1}{5!}(32)x^5 + \ldots \\
\implies e^{2x}&= 1 + 2x + \frac{2^2}{2!}x^2 + \frac{2^3}{3!}x^3 + \frac{2^4}{4!}x^4 + \frac{2^5}{5!}x^5 + \ldots \\
\implies e^{2x} &= \sum_{n=0}^{\infty}\frac{2^n}{n!}x^n
\end{align*}
Alternatively if one only wants the series in sigma notation one can observe that $f^{(n)}(0)=2^n$ and substitute that immediately into our Maclaurin formula.

To find the interval of convergence we will use the Ratio Test.
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|
&=\lim_{n \to \infty}\left|\frac{ \frac{2^{n+1}x^{n+1}}{(n+1)!}}{\frac{2^nx^n}{n!}      }\right| =\lim_{n \to \infty}\left|\frac{2^{n+1}x^{n+1}}{(n+1)n!}\frac{n!}{2^nx^n}\right| 
=2\lim_{n \to \infty} \frac{1}{n+1}|x| \\
&= 2(0)|x|= 0<1\ \ \text{for all}\ x
\end{align*}
Thus the integral of convergence of the Maclaurin series is $I=(-\infty, \infty)$\,.
\item We compute the following derivatives and evaluate them:
\begin{align*}
f(x)&=\cos x \,\,, & f(0)& =\cos 0=1 \\
f'(x)&=-\sin x \,\,, & f'(0)&=-\sin0=0 \\
f''(x)&=-\cos x\,\,, & f''(0)&=-\cos 0=-1 \\
f'''(x)&=\sin x\,\,, & f'''(0)&=\sin 0= 0\\
  f^{(4)}(x)&=\cos x\,\,, & f^{(4)}(0)&=\cos 0=1 \\
  &~~\vdots & & ~~\vdots\ \ \ \ (\text{pattern repeats}) 
\end{align*}
Thus the Macluarin series is
\begin{align*}
\cos x&= 1+ \frac{1}{2!}(-1)x^2 + \frac{1}{4!}(1)x^4 + \frac{1}{6!}(-1)x^6 + \ldots \\
\implies \cos x&= 1 -\frac{1}{2!}x^2 + \frac{1}{4!}x^4- \frac{1}{6!}x^6 + \dots \\
\implies \cos x&= \sum_{n=0}^{\infty}(-1)^n \frac{x^{2n}}{(2n)!}\,.
\end{align*}
To find the interval of convergence, we will use the Ratio Test. 
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right| 
&=\lim_{n \to \infty}\left|\frac{ \frac{(-1)^{n+1}x^{2n+2}}{(2n+2)!}}{ \frac{(-1)^nx^{2n}}{(2n)!}}\right|= \lim_{n \to \infty}\frac{(2n)!}{(2n+2)!}x^2
=\lim_{n \to \infty}\frac{(2n)!}{(2n+2)(2n+1)(2n)!}x^2\\
&=\lim_{n \to \infty} \frac{1}{(2n+2)(2n+1)}x^2  
=0\cdot x^2= 0<1 \ \ \text{for all $x$}
\end{align*}
Therefore the interval of convergence is $I=(-\infty, \infty)$.

One notes that the fact that cosine is an even function is reflected in the disappearance of odd powers of $x$ in its Maclaurin series which make it look like an even polynomial with an infinite number of terms.

Also note that the $n$ index in our found series is not the same as the $n$ that appears in the Maclaurin formula.  The series found skips the terms with zero coefficient.  This is not merely for convenience. It was required so that the Ratio Test could be applied to the series to find convergence.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Find the Maclaurin series and its interval of convergence for each of the following functions.
  \begin{multicols}{2}
    \begin{enumerate}
    \item $\displaystyle f(x)=\frac{1}{1-x}$
    \item $\displaystyle f(x)=e^x$
    \item $\displaystyle f(x)=\sin x$
    \item $\displaystyle f(x)=\ln(1+x)$
      % \item $\displaystyle f(x)=\frac{1}{\sqrt{1-x}}$
      % \item $\displaystyle f(x)=\cos x$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

Some important Maclaurin series and their domains of validity are: \index{Maclaurin Series! table of}
\begin{eqnarray*}
\frac{1}{1-x} & = & \sum_{k=0}^\infty x^k=1+x+x^2+x^3+\cdots\ \ \ ;\ (-1,1) \\
e^x & = & \sum_{k=0}^\infty\frac{x^k}{k!}=1+\frac{x}{1!}+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots\ \ \ ;\ (-\infty,\infty) \\
\sin x & = & \sum_{k=0}^\infty(-1)^k\frac{x^{2k+1}}{(2k+1)!}=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\cdots\ \ \ ;\ (-\infty,\infty) \\
\cos x & = & \sum_{k=0}^\infty(-1)^k\frac{x^{2k}}{(2k)!}=1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\cdots\ \ \ ;\ (-\infty,\infty) \\
\tan^{-1}x & = & \sum_{k=0}^\infty\frac{x^{2k+1}}{2k+1}=x-\frac{x^3}{3}+\frac{x^5}{5}-\frac{x^7}{7}+\cdots\ \ \ ;\ [-1,1] \\
\ln(1+x) & = & \sum_{k=0}^\infty(-1)^k\frac{x^{k+1}}{k+1}=x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\cdots\ \ \ ;\ (-1,1]
\end{eqnarray*}

With these series representations we now have a means of numerically calculating these functions for any value of $x$ in the interval of convergence to arbitrary precision!

\textbook{\clearpage}
As we saw with the more general power series, we can derive series of new functions from known Maclaurin series.

\begin{example}
For the following functions we have the power series:
\begin{enumerate}
\item $\displaystyle e^{x^3}=\sum_{k=0}^\infty\frac{\left(x^3\right)^k}{k!}=\sum_{k=0}^\infty\frac{x^{3k}}{k!}$

Series is valid for $x^3$ in $(-\infty,\infty)$ which implies for $x$ in $(-\infty,\infty)$.
\item \begin{align*} \sin^2\!x& =\frac{1}{2}\left(1-\cos2x\right)=\frac{1}{2}-\frac{1}{2}\sum_{k=0}^\infty(-1)^k\frac{(2x)^{2k}}{(2k)!}=\frac{1}{2}+\sum_{k=0}^\infty(-1)^{k+1}\frac{2^{2k-1}x^{2k}}{(2k)!}\\
                              & =\sum_{k=1}^\infty(-1)^{k+1}\frac{2^{2k-1}x^{2k}}{(2k)!}
      \end{align*}

(Here we simplified our expression in the last line by noting the $k=0$ term in the series equals $-1/2$ and so cancels the $1/2$.)  Series is valid for $2x$ in $(-\infty,\infty)$ and so for $x$ in $(-\infty,\infty)$.
\end{enumerate}
These series can be confirmed to be the Maclaurin series of the given functions by direct computation.
\end{example}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/maclaurin-series.tex}
\end{exercise}

\section{Taylor Series}\index{series! Taylor} \index{Taylor series}
Maclaurin series can be generalized by expanding in powers of $(x-a)^k$ rather than $x^k$ for some constant~$a$.  A similar argument to that before gives the following definition.

\definition{Given function $f(x)$ differentiable to all orders at $x=a$ the power series in $(x-a)$ given by 
\[f(x)=\sum_{k=0}^{\infty}\frac{f^{(k)}(a)}{k!}(x-a)^k=f(a)+f'(a)(x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f'''(a)}{3!}(x-a)^3+\cdots\frac{f^{(k)}(a)}{k!}(x-a)^k+\cdots\]
is the \textbf{Taylor Series} for the function $f(x)$ \textbf{at} $\bm{a}$ (or about $a$ or centred on $a$).}
The Taylor series expansion at $a$ will converge within some radius $R$ about $a$, i.e. for $|x-a|<R$.

We note that Maclaurin series is just a special case of the Taylor series when $a=0$.
\begin{example}
  Find the Taylor series of the given function at the specified value and determine the interval of convergence.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle f(x)=\cos x$ at $\displaystyle a=\frac{\pi}{2}$ 
    \item $\displaystyle f(x)= e^{-2x}$ at $a=-1$ 
    \item $\displaystyle f(x)= \frac{1}{x}$ at $a=1$
    \end{enumerate}
  \end{multicols}
\begin{solution*}
\begin{enumerate}
\item Since the Taylor series about $x=a$ is $f(x)=\displaystyle\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n$ we compute the following derivatives and evaluate them at $\displaystyle x=a=\frac{\pi}{2}$\,:
\begin{align*}
f(x)&=\cos x \,\,, & f\left(\frac{\pi}{2}\right)& =\cos \frac{\pi}{2}=0    \\
f'(x)&=-\sin x \,\,, & f'\left(\frac{\pi}{2}\right)&=-\sin \frac{\pi}{2}=-1 \\
f''(x)&=-\cos x\,\,, & f''\left(\frac{\pi}{2}\right)&=-\cos \frac{\pi}{2}=0 \\
f'''(x)&=\sin x\,\,, & f'''\left(\frac{\pi}{2}\right)&=\sin \frac{\pi}{2}= 1\\
  f^{(4)}(x)&=\cos x\,\,, & f^{(4)}\left(\frac{\pi}{2}\right)&=\cos \frac{\pi}{2}=0 \\
    &~~\vdots & & ~~\vdots\ \ \ \ (\text{pattern repeats}) 
\end{align*}
Thus the Taylor series is given by:
\begin{align*} 
f(x)&=f\left(\frac{\pi}{2}\right) +f'\left(\frac{\pi}{2}\right)\left(x-\frac{\pi}{2}\right) + \frac{1}{2!}f''\left(\frac{\pi}{2}\right)\left(x-\frac{\pi}{2}\right)^2 + \frac{1}{3!}f'''\left(\frac{\pi}{2}\right)\left(x-\frac{\pi}{2}\right)^3 \\
&+ \frac{1}{4!}f^{(4)}\left(\frac{\pi}{2}\right)\left(x-\frac{\pi}{2}\right)^4 + \ldots \\
\implies \cos x&=(-1)\left(x-\frac{\pi}{2}\right) + \frac{1}{3!}(1)\left(x-\frac{\pi}{2}\right)^3 + \frac{1}{5!}(-1)\left(x-\frac{\pi}{2}\right)^5 + \ldots \\
\implies \cos x&= -\left(x-\frac{\pi}{2}\right) + \frac{1}{3!}\left(x-\frac{\pi}{2}\right)^3-\frac{1}{5!}\left(x-\frac{\pi}{2}\right)^5 + \ldots \\
\implies \cos x &= \sum_{n=0}^{\infty}\frac{(-1)^{n+1}}{(2n+1)!}\left(x-\frac{\pi}{2}\right)^{2n+1}
\end{align*}
To find the interval of convergence we will use the Ratio Test.
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|
&=\lim_{n \to \infty}\left|\frac{ \frac{(-1)^{n+2} \left(x-\frac{\pi}{2}\right)^{2n+3}}{(2n+3)!}}{\frac{(-1)^{n+1} \left(x-\frac{\pi}{2}\right)^{2n+1}}{(2n+1)!} }\right|
=\lim_{n \to \infty}\frac{1}{(2n+3)(2n+2)} \left(x-\frac{\pi}{2}\right)^2 \\
&=0\cdot\left(x-\frac{\pi}{2}\right)^2=0<1\,\, \text{for all}\,\, x
\end{align*}
Thus the interval of convergence is $I=(-\infty, \infty)$.
\item We compute the following derivatives and evaluate them at $x=a=-1$:
\begin{align*}
f(x)&=e^{-2x} \,\,, & f(-1)& =e^2    \\
f'(x)&=-2e^{-2x}\,\,, & f'(-1)&=-2e^2 \\
f''(x)&=4e^{-2x}\,\,, & f''(-1)&=4e^2 \\
f'''(x)&=-8e^{-2x}\,\,, & f'''(-1)&=-8e^2
\end{align*}
Therefore the Taylor series is
\begin{align*}
  f(x)&=f\left(-1\right) +f'\left(-1\right)\left(x-(-1)\right) + \frac{1}{2!}f''\left(-1\right)\left(x-(-1)\right)^2 + \frac{1}{3!}f'''\left(-1\right)\left(x-(-1)\right)^3+\ldots \\
\implies e^{-2x}&=e^2+\left(-2e^2\right)(x+1) +\frac{1}{2!}\left(4e^2\right)(x+1)^2 + \frac{1}{3!}\left(-8e^2\right)(x+1)^3 + \ldots \\
\implies e^{-2x} &= e^2\left[1-2(x+1)+\frac{2^2}{2!}(x+1)^2-\frac{2^3}{3!}(x+1)^3+\ldots\right] \\
\implies e^{-2x}&=e^2 \sum_{n=0}^{\infty} (-1)^n \frac{2^n}{n!}(x+1)^n\,.
\end{align*}
Alternatively, if only the sigma form of the series is desired, one can substitute the result $f^{(n)}(-1)=(-1)^n(2)^ne^2$ obtained from analyzing the pattern of the derivative evaluation directly into the Taylor series formula.

To find the interval of convergence, we will use the Ratio Test. 
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right|
&=\lim_{n \to \infty}\left|\frac{ \frac{(-1)^{n+1}2^{n+1}(x+1)^{n+1}}{(n+1)!}}{ \frac{(-1)^n 2^n(x+1)^{n}}{n!}}\right|= \lim_{n \to \infty}\frac{2}{n+1}|x+1| \\
&=0 \cdot |x+1|= 0 <1 \,\,\, \text{for all}\,\,\, x
\end{align*}
Therefore the interval of convergence is $I=(-\infty, \infty)$\,.
\item We compute the following derivatives and evaluate them at $x=a=1$:
\begin{align*}
f(x)&=\frac{1}{x} \,\,, & f(1)& =\frac{1}{1}=1   \\
f'(x)&=-\frac{1}{x^2}\,\,, & f'(1)&=-1 \\
f''(x)&= \frac{2}{x^3}    \,\,, & f''(1)&=2\\
f'''(x)&=-\frac{6}{x^4} \,\,, & f'''(1)&=-6 \\
f^{(4)}(x)&=\frac{24}{x^5}\,\,, & f^{(4)}(1)&=24 
\end{align*}
Thus the Taylor series is
\begin{align*}
  f(x)&=f\left(1\right) +f'\left(1\right)\left(x-1\right) + \frac{1}{2!}f''\left(1\right)\left(x-1\right)^2 + \frac{1}{3!}f'''\left(1\right)\left(x-1\right)^3\\
  &+ \frac{1}{4!}f^{(4)}\left(1\right)\left(x-1\right)^4+\ldots \\
\implies\frac{1}{x}&=1+(-1)(x-1) + \frac{1}{2!}(2)(x-1)^2 -\frac{1}{3!}(-6)(x-1)^3 + \frac{1}{4!}(24)(x-1)^4 + \ldots \\
\implies \frac{1}{x}&= 1-(x-1)+(x-1)^2-(x-1)^3+(x-1)^4 + \ldots \\
\implies \frac{1}{x}&= \sum_{n=0}^{\infty}(-1)^n(x-1)^n
\end{align*}
Alternatively one can observe that $f^{(n)}(1)=(-1)^nn!$ directly and place that in the Taylor series formula.  Apply the Ratio Test to find the interval of convergence.
\begin{align*}
\lim_{n \to \infty}\left|\frac{a_{n+1}}{a_n}\right| 
=\lim_{n \to \infty}\left|\frac{(-1)^{n+1}(x-1)^{n+1}}{(-1)^n(x-1)^n}\right| 
=\lim_{n \to \infty} |x-1|=|x-1|
\end{align*}
The Taylor series converges if
\begin{align*}
 |x-1| <1
 \implies -1<x-1< 1
 \implies 0<x<2\,.
\end{align*}
At $x=0$ the function $\frac{1}{x}$ is undefined and the series evaluates to the divergent series $\sum 1$.  At $x=2$ the series evaluates to $\sum (-1)^n$ which also diverges by the Term Test for Divergence.
Therefore the interval of convergence of the Taylor series is $I=(0,2)$.
\end{enumerate}
\end{solution*}

\begin{furtherquestions}
  Find the Taylor Series of the given function at the specified value and determine the interval of convergence.
  \begin{multicols}{3}
    \begin{enumerate}
    \item $\displaystyle f(x)=\sin x$ at $\displaystyle a=\frac{\pi}{2}$
    \item $\displaystyle f(x)=\ln x$ at $\displaystyle a=1$
    \item $\displaystyle f(x)=e^x$ at $\displaystyle a=2$
    \end{enumerate}
  \end{multicols}
\end{furtherquestions}
\end{example}

Assuming convergence to $f(x)$, Taylor series gives us a mechanism for calculating trigonometric and other functions, namely by evaluating the first $n$ terms of the series at $x$.  How many terms of the series are required for a good approximation will depend on the function, the value $a$ about which it is expanded, and $x$.

Tayor series allows expansion of the function $f$ about values other than $a=0$ which is useful for functions that are not defined at 0.  Also in general fewer terms of the expansion will be required for a good approximation if the Taylor series is generated about a value $a$ near the $x$ of interest.  Indeed by truncating the Taylor series at the $k=1$ term
\[f(x)\approx f(a)+f'(a)(x-a)~,\]
or at the $k=2$ term,
\[f(x)\approx f(a)+f'(a)(x-a)+\frac{f''(a)}{2!}(x-a)^2~,\]
we are just reproducing the linear and quadratic approximations of a function at $x=a$ arrived at our previous course.  In general truncating at the $k=n^{\tiny th}$ term,
\[f(x)\approx f(a)+f'(a)(x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f'''(a)}{3!}(x-a)^3+\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n\]
yields increasingly better approximations.  The right hand side is called the  $n^{\mbox{\tiny th}}$ \textbf{Taylor polynomial} \index{Taylor polynomial} of the function $f(x)$ at $a$.  The following diagram shows the first four Taylor polynomials of $\sin x$ at $a=0$ (i.e. the Maclaurin series):
\begin{center}
\includegraphics{graphics/sequences_series5}%
\end{center}
Here are the first four polynomials for the Taylor series of $\sin x$ at $a=\pi/2$:
\begin{center}
\includegraphics{graphics/sequences_series6}%
\end{center}
One observes both the greater accuracy of the higher order approximations as well as the utility of expanding the Taylor series at a value $a$ near the $x$ at which you wish to approximate the function.

One useful result of Theorem~\ref{thm:diff_int_power} is that it justifies our original proof for the coefficients for the Maclaurin series where (recall) we required the power series to be differentiable.  Generalizing this result to Taylor series we have the following result:
\theorem{If function $f(x)$ is represented by the power series
\[f(x)=\sum_{k=0}^\infty c_k (x-a)^k=c_0+c_1(x-a)+c_2(x-a)^2+\cdots\]
on an open interval containing $a$ then the coefficients are the Taylor series coefficients $c_k=f^{(k)}(a)/k!$, (i.e. $f(x)=f(a)+f'(a)(x-a)+\frac{f''(a)}{2!}(x-a)^2+\cdots$).}
 
In other words, if a function $f(x)$ has a power series that converges to $f(x)$ it will be the Taylor series.  The theorem does not say, however, that the Taylor series at $a$ for a given function $f$ will necessarily converge to $f$.  For instance define the continuous piecewise function:
\[f(x)=\left\{\begin{array}{cl} 0 & \mbox{if $x<-\pi/2$} \\  
\cos x & \mbox{if $-\pi/2\le x\le \pi/2$} \\  
0 & \mbox{if $x>\pi/2$}
\end{array}\right.\]
Then $f$ will have the same Taylor series at $a=0$ (i.e. Maclaurin series) as the function $\cos x$.  However that series clearly cannot represent both functions.  One must determine that the Taylor series for $f$ at $a$ really does converge to the function.
\theorem{If a function $f$ has derivatives to all orders in an interval centred on $a$, then
\[f(x)=\sum_{k=0}^\infty\frac{f^{(k)}(a)}{k!}(x-a)^k\]
will hold on the interval if and only if
\[\lim_{n\rightarrow\infty}R_n(x)=0\]
for all $x$ in the interval, where
\[R_n(x)=f(x)-\sum_{k=0}^n\frac{f^{(k)}(a)}{k!}(x-a)^k~.\]}

\begin{exercise}
\input{SEQUENCES_AND_SERIES/taylor-series.tex}
\end{exercise}

\begin{reviewexercise}
\input{SEQUENCES_AND_SERIES/sequences-and-series-review.tex}
\end{reviewexercise}

%TODO Change all tests to either have "The" in front or not, like in
%my handout. Also change slides

%TODO Change geometric series intro to have sum from 0 as well and put |r|<1 instead of -1<r<1 ?

%TODO Change name of The "Basic" comparison test.  Term? comparison test.  May confuse with the term test for divergence.

%TODO Either we should put the ratio and root tests in with the positive series and then deal with non-positive series at the end OR we should really put all the series test with absolute values.  Likely the former way is the way to go.

%TODO: If we wish to show how the various tests (limit comparison, ratio and root) are effectively just geometric series, or multiplicative constants we could use "c" for the limit comparison test and "r" for the root test instead of "L" for the limit.  I don't know if this is preferable or not.  I could use capitals for the limits (C, R, etc.).

%TODO: Currently we use the integral test to arrive at the convergence
%properties of P-series.  This can also be done use the "Cauchy Test"
%or "Cauchy condensation test" which is essentially a version of what
%we do in the notes to prove the divergence of the harmonic series.  One
%could do this at the beginning to prove the p-series properties if one wanted.
%See z_cauchy_to_harmonic_convergence.pdf in my resources folder for details.

%TODO: After an initial test I should explain how the behaviour for the first finite indices does not matter with regard to convergence.  This could be done with the integral test for instance if I keep that early.  Then this leads logically into the idea of the series

%TODO: I should modify the Term Test for Divergence (or add a corollary later) to indicate that using absolute value here is okay as well due to the sequence theorem about things going to zero only if they do in absolute vlue.  This is convenient for testing some divergent alternating series, for instance.

%TODO: In the term test for divergence part a series like (-1)^n/\sqrt{n} can be done using the limit of the absolute value of the sequence to get rid of the -1.  However a problem like (-1)^n/sqrt(n) + 1 cannot be handled in this manner to work out the limit.  In this case one can replace (-1)^n with cos(n\pi) and then apply the squeeze theorem between -1/\sqrt{n} +1 and 1/sqrt{n}+1 to show the limit goes to 1, and hence the series diverges.  This example is useful because it gives a reason to use cos(n\pi) or sin[(n+1/2)\pi], namely to produce a function on which you can use the squeeze theorem.   This could be added as a problem with a hint given to replace with cos and use the squeeze theorem.  Maybe with the squeeze theorem I should add about it working at infinity.

%TODO: Give some examples about converting a pattern into a series and vice-versa.  Also consider changing 110 notes so that series use same indices (k or n) for consistency.

%TODO: Some sources call the Basic Comparison Test the "Direct" comparison test
%which we should adopt if noone uses Basic.  See the wikipedia entry on
%convergence tests for what names are commonly used.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "master"
%%% End:
