Title: Analysis I
Tags: A7.1 A1 A2.3 A3.1 A5.3 A6.1 A4.1 A2.2 A3.2 A5.2 A2.1 A2.4 A3.3 A4.2 A6.0 A2.5 A5.1 A6.3 A7.2 A3.4 A6.2

Q: Field
A: A \emph{field} is a set $\F$ with two binary operations $+$ and $\times$ that satisfies all the familiar properties satisfied by addition and multiplication in $\Q$, namely
  \begin{enumerate}
    \item $(\forall a, b, c)\,a + (b + c) = (a + b) + c$ and $a\times (b\times c) = (a\times b)\times c$ \hfill (associativity)
    \item $(\forall a, b)\,a + b = b + a$ and $a\times b= b\times a$ \hfill (commutativity)
    \item $\exists 0, 1$ such that $(\forall a)a + 0 = a$ and $a\times 1 = a$ \hfill (identity)
    \item $\forall a\,\exists (-a)$ such that $a + (-a) = 0$. If $a\not= 0$, then $\exists a^{-1}$ such that $a\times a^{-1} = 1$. \hfill (inverses)
    \item $(\forall a, b, c)\, a\times (b + c) = (a\times b) + (a\times c)$ \hfill (distributivity)
  \end{enumerate}

Q: Totally ordered set
A: An \emph{(totally) ordered set} is a set $X$ with a relation $<$ which is a partial order and satisfies
  \begin{enumerate}
\item If $x\in X$, then $\neg(x<x)$.
\item If $x,y\in X$, then $\neg(x<y\textrm{ and }y<x)$.
    \item If $x, y, z\in X$, $x < y$ and $y < z$, then $x < z$ \hfill (transitivity)
    \item If $x, y\in X$, exactly one of $x < y, x = y, y < x$ holds \hfill (trichotomy)
  \end{enumerate}

Q: Ordered field
A: An \emph{ordered field} is a field $\F$ with a relation $<$ that makes $\F$ into an ordered set such that
  \begin{enumerate}
    \item if $x, y, z \in \F$ and $x < y$, then $x + z < y + z$
    \item if $x, y, z \in \F$, $x < y$ and $z > 0$, then $xz < yz$
  \end{enumerate}

Q: Positivity of squares
A: Let $\F$ be an ordered field and $x\in \F$. Then $x^2 \geq 0$.
\begin{proof}
  By trichotomy, either $x < 0$, $x = 0$ or $x > 0$.
If $x = 0$, then $x^2 = 0$. So $x^2 \geq 0$.
If $x > 0$, then $x^2 > 0\times x = 0$.
If $x < 0$, then $x - x < 0 - x$. So $0 < -x$. But then $x^2 = (-x)^2 > 0$.
\end{proof}

Q: Least upper bound
A: Let $X$ be an ordered set and let $A\subseteq X$.
  \begin{enumerate}
    \item $(\forall a\in A)\,a \leq x$ (upper bound / bounded above)
    \item $(\forall y < x)(\exists a\in A)\,a \geq y$ (supremum)
  \end{enumerate}
  We write $\sup A$ for the supremum of $A$ when it exists. If $\sup A\in A$, then we call it $\max A$, the maximum of $A$.

Q: Least upper bound property
A: An ordered set $X$ has the \emph{least upper bound property} if every non-empty subset of $\mathbb{X}$ that is bounded above has a supremum.

Q: Real numbers: definition
A: The \emph{real numbers} is an ordered field with the least upper bound property.

Q: Archimedean property
A: Let $\F$ be an ordered field with the least upper bound property. Then the set $\{1, 2, 3, \cdots\}$ is not bounded above.
\begin{proof}
  If it is bounded above, then it has a supremum $x$. But then $x - 1$ is not an upper bound. So we can find $n\in \{1, 2, 3, \cdots\}$ such that $n> x - 1$. But then $n + 1 > x$, but $x$ is supposed to be an upper bound.
\end{proof}
$1/n \to 0$
\begin{proof}
  \textcolor{red}{Let $\varepsilon > 0$}. We want to find an $N$ such that $1/N < \varepsilon$. So \textcolor{red}{pick $N$} such that $N > 1/\varepsilon$. This exists as shown above. Then \textcolor{red}{for all $n > N$}, we have $0 < 1/n < 1/N < \varepsilon$.
\end{proof}

Q: Sequence: definition
A: A function $a: \N \to \R$ (or $\C$). Usually we write $a_n$ instead of $a(n)$. Instead of $a$, we usually write it as $(a_n)$, $(a_n)_1^\infty$ or $(a_n)_{n = 1}^\infty$ to indicate it is a sequence.

Q: Convergence of sequences
A: Let $(a_n)$ be a sequence and $\ell\in \R$. Then $a_n$ \emph{converges to} $\ell$, \emph{tends to} $\ell$, or $a_n \to \ell$ , if
  \[
    (\forall \varepsilon > 0)(\exists N)(\forall n\geq N)\,|a_n - \ell| < \varepsilon.
  \]
  We say $\ell$ is the \emph{limit} of $(a_n)$.
Note that in a field which satisfies the archimedean property, you can let $\varepsilon=\frac{1}{n}$.

Q: Bounded sequence
A: A sequence $(a_n)$ is \emph{bounded} if
  \[
    (\exists C)(\forall n)\,|a_n| \leq C.
  \]
  A sequence is \emph{eventually bounded} if
  \[
    (\exists C)(\exists N)(\forall n\geq N)\, |a_n| \leq C.
  \]
These are actually equivalent.
\begin{proof}
Consider a bounded sequence - let $C$ and $N$ be such that $(\forall n\geq N)\,|a_n| \leq C$. Then $\forall n \in \N$, $|a_n| \leq \max\{|a_1|, \cdots, |a_{N - 1}|, C\}$.
Conversely, any bounded sequence is obviously eventually bounded - just pick $N=1$.
\end{proof}

Q: Sums of sequences
A: If $a_n \to a$ and $b_n \to b$, then $a_n + b_n \to a + b$.
\begin{proof}
Given $\varepsilon > 0$ choose $N = \max\{N_1, N_2\}$ such that
\begin{align*}
&\forall n \geq N_1, |a_n - a| < \varepsilon/2 \\
&\forall n \geq N_2, |b_n - b| < \varepsilon/2.
\end{align*}
Then by the triangle inequality, when $n \geq N$,
  \begin{align*}
    |(a_n + b_n) - (a + b)| &\leq |a_n - a| + |b_n - b| \\
&<\varepsilon/2 + \varepsilon/2 \\
&< \varepsilon.
  \end{align*}
\end{proof}

Q: Every convergent sequence is bounded.
A: \begin{proof}
  Let $a_n \to l$. Then there is an $N$ such that $\forall n \geq N$, $|a_n - l| \leq 1$. So $|a_n| \leq |l| + 1$. So $a_n$ is eventually bounded, and therefore bounded.
\end{proof}

Q: Product of sequences
A: Let $(a_n)\to a$ and $(b_n)\to b$. Then $(a_nb_n)\to ab$.
\begin{proof}
 Observe that $a_nb_n - ab = (a_n - a) b_n + (b_n - b)a$. We know that $a_n - a \to 0$ and $b_n - b\to 0$. Since $(b_n)$ is bounded [e.g. by $|b|+1$], $(a_n - a)b_n + (b_n - b)a \to 0$. So $a_nb_n \to ab$.
\end{proof}

Q: Quotient of sequences
A: Let $(a_n)$ be a sequence such that $(\forall n)\, a_n \not= 0$. Suppose that $a_n \to a$ and $a\not = 0$. Then $1/a_n \to 1/a$.
\begin{proof}
  We have
  \[
    \frac{1}{a_n} - \frac{1}{a} = \frac{a - a_n}{aa_n}.
  \]
Since $a - a_n \to 0$, we just have to show that $1/(aa_n)$ is bounded.
  Since $a_n \to a$, $\exists N$ such that $\forall n\geq N$
\begin{align*}
|a|-|a_n|\leq |a_n - a| &< |a|/2 \\
|a_n| &\geq |a|/2 \\
|1/(a_na)| &\leq 2/|a|^2
\end{align*}
So $1/(a_na)$ is bounded.
\end{proof}
If $a_n \to a, b_n \to b$, ($b_n, b\not= 0$), then $a_n/b_n \to a/b$.
\begin{proof}
  We know that $1/b_n \to 1/b$. So the result follows by the product rule.
\end{proof}

Q: Sandwich rule
A: Let $(a_n)$ and $(b_n)$ be sequences that both converge to a limit $x$. Suppose that $a_n \leq c_n \leq b_n$ for every $n$. Then $c_n \to x$.
\begin{proof}
  Let $\varepsilon > 0$. We can find $N$ such that $\forall n \geq N$
\begin{align*}
&|a_n - x| < \varepsilon \\
&|b_n - x| < \varepsilon
\end{align*}
  Then $\forall n\geq N$, we have \[
x - \varepsilon < a_n \leq c_n \leq b_n < x + \varepsilon.\]
So $|c_n - x| < \varepsilon.$
\end{proof}

Q: Monotone-sequences property
A: An ordered field has the \emph{monotone sequences property} if every increasing sequence that is bounded above converges. We show this $=$ the LUB axiom.
LUB property $\Rightarrow$ MSP.
\begin{proof}
  Let $(a_n)$ be an increasing sequence and let $C$ an upper bound for $(a_n)$. Then $C$ is an upper bound for the set $\{a_n: n \in \N\}$. By the least upper bound property, it has a supremum $s$. We want to show that this is the limit of $(a_n)$.
  Let $\varepsilon > 0$. Since $s = \sup \{a_n: n\in \N\}$, there exists an $N$ such that $a_N > s - \varepsilon$. Then since $(a_n)$ is increasing, $\forall n \geq N$, we have $s - \varepsilon < a_N \leq a_n \leq s$. So $|a_n - s| < \varepsilon$.
\end{proof}
\begin{lemma}
  MSP $\Rightarrow$ Archimedean property v2, i.e. that $1/n \to 0$.
\end{lemma}
\begin{proof}
  Since $1/n > 0$ and is decreasing, by MSP, it converges. Let $\delta$ be the limit. Clearly $\delta \geq 0$.
  If $\delta > 0$, then we can find $N$ such that $1/N < 2\delta$. But then for all $n \geq 4N$, we have $1/n \leq 1/(4N) < \delta/2$. Contradiction. Therefore $\delta = 0$.
\end{proof}
  MSP $\Rightarrow$ least upper bound property.
\begin{proof}
  Let $A$ be a non-empty set that's bounded above. Pick $u_0, v_0$ such that $u_0$ is not an upper bound for $A$ and $v_0$ is an upper bound. Now do a repeated bisection: having chosen $u_n$ and $v_n$ such that $u_n$ is not an upper bound and $v_n$ is, if $(u_n + v_n)/2$ is an upper bound, then let $u_{n + 1} = u_n$, $v_{n + 1} = (u_n + v_n)/2$. Otherwise, let $u_{n + 1} = (u_n + v_n)/2$, $v_{n + 1} = v_n$.
  Then $u_0 \leq u_1 \leq u_2 \leq \cdots$ and $v_0\geq v_1 \geq v_2 \geq \cdots$. We also have
  \[
    v_n - u_n = \frac{v_0 - u_0}{2^n} \to 0.
  \]
  By MSP, $u_n\to s$ (since $(u_n)$ is bounded above by $v_0$). Since $v_n - u_n \to 0$, $v_n \to s$. We now show that $s = \sup A$.
 Suppose $s$ is not an upper bound, then there exists $a\in A$ such that $a>s$. Since $v_n \to s$, then there exists $m$ such that $v_m < a$, contradicting the fact that $v_m$ is an upper bound.
  To show minimality, let $t < s$ be an UB. Then since $u_n \to s$, we can find $m$ such that $u_m > t$. So $t$ is not an upper bound.
\end{proof}

Q: Uniqueness of limits: proof
A: \begin{proof}
Let $(a_n)$ be a sequence, and suppose $a_n \to x$ and $a_n\to y$. Let $\varepsilon > 0$ and pick $N$ such that $\forall n \geq N$, $|a_n - x| < \varepsilon/2$ and $|a_n - y| < \varepsilon/2$. Then $|x - y| \leq |x - a_N| + |a_N - y| < \varepsilon/2 + \varepsilon/2 = \varepsilon$. Since $\varepsilon$ was arbitrary, $x$ must equal $y$.
\end{proof}

Q: Nested intervals property
A: Let $\F$ be an ordered field with the monotone sequences property. Let $I_1\supseteq I_2 \supseteq \cdots$ be closed bounded non-empty intervals. Then $\bigcap_{n = 1}^\infty I_n \not= \emptyset$.
\begin{proof}
  Let $T_n = [a_n, b_n]$ for each $n$. Then $a_1 \leq a_2 \leq\cdots$ and $b_1 \geq b_2\geq \cdots$. For each $n$, $a_n \leq b_n \leq b_1$. So the sequence $a_n$ is bounded above. So by the monotone sequences property, it has a limit $a$. For each $n$, we must have $a_n\leq a$. Otherwise, say $a_n > a$. Then for all $m \geq n$, we have $a_m \geq a_n > a$ so $a\geq a_n > a$.
  Also, for each fixed $n$, we have that $\forall m\geq n$, $a_m \leq b_m \leq b_n$. So $a \leq b_n$. Thus, for all $n$, $a_n \leq a \leq b_n$. So $a\in I_n$. So $a\in \bigcap_{n = 1}^\infty I_n$.
\end{proof}
\begin{prop}
  $\R$ is uncountable.
\end{prop}
\begin{proof}
  Suppose the contrary. Let $x_1, x_2, \cdots$ be a list of all real numbers. Find an interval that does not contain $x_1$. Within that interval, find an interval that does not contain $x_2$. Continue \emph{ad infinitum}. Then the intersection of all these intervals is non-empty, but the elements in the intersection are not in the list. Contradiction.
\end{proof}

Q: Bolzano-Weierstrass theorem
A: Let $\F$ be an ordered field with the monotone sequences property (ie. $\F = \mathbb{R}$).
  Then every bounded sequence has a convergent subsequence.
\begin{proof}
  Let $u_0$ and $v_0$ be a lower and upper bound, respectively, for a sequence $(a_n)$. By repeated bisection, we can find a sequence of intervals $[u_0, v_0] \supseteq [u_1, v_1]\supseteq [u_2,v_2] \supseteq\cdots$ such that $v_n - u_n = (v_0 - u_0)/2^n$, and such that each $[u_n, v_n]$ contains infinitely many terms of $(a_n)$.
  By the nested intervals property, $\bigcap_{n = 1}^\infty [u_n, v_n] \not= \emptyset$. Let $x$ belong to the intersection. Now pick a subsequence $a_{n_1}, a_{n_2}, \cdots$ such that $a_{n_k} \in [u_k, v_k]$. We can do this because $[u_k, v_k]$ contains infinitely many $a_n$, and we have only picked finitely many of them. We will show that $a_{n_k} \to x$.
  Let $\varepsilon > 0$. By the Archimedean property, we can find $K$ such that $v_K - u_K = (v_0 - u_0)/2^K \leq \varepsilon$. This implies that $[u_K, v_K] \subseteq (x - \varepsilon, x + \varepsilon)$ since $x\in [u_K, v_K]$.
  Then $\forall k \geq K$, $a_{n_k}\in [u_k, v_k] \subseteq [u_K, v_K] \subseteq (x - \varepsilon, x + \varepsilon)$. So $|a_{n_k} - x| < \varepsilon$.
\end{proof}
\begin{proof}(alternative)
Define $m\in\N$ to be a 'peak number' of $(x_n)$ if $\forall n>m: x_m>x_n$. Let $P=\{m\in\N | m\textrm{ is a peak number}\}$. If $P$ is infinite, then the sequence of peak numbers is a decreasing subsequence. If $P$ is finite, then every number is not a peak number so we can inductively define an increasing sequence. In either case, there is a monotonic and thus convergent subsequence.
\end{proof}

Q: Cauchy sequence
A: A sequence $(a_n)$ is \emph{Cauchy} if
  \[
    (\forall \varepsilon > 0)(\exists N)(\forall p, q\geq N)\, |a_p - a_q| < \varepsilon.
  \]
  Every convergent sequence is Cauchy.
\begin{proof}
  Let $a_n \to a$. Let $\varepsilon > 0$. Then $\exists N$ such that $\forall n \geq N$, $|a_n - a| < \varepsilon/2$. Then $\forall p, q\geq N$, $|a_p - a_q| \leq |a_p - a| + |a - a_q| < \varepsilon/2 + \varepsilon/2 = \varepsilon$.
\end{proof}
\begin{lemma}
  Let $(a_n)$ be a Cauchy sequence with a subsequence $(a_{n_k})$ that converges to $a$. Then $a_n\to a$.
\end{lemma}
\begin{proof}
  Let $\varepsilon > 0$. Pick $N$ such that $\forall p, q\geq N$, $|a_p - a_q| < \varepsilon/2$. Then pick $K$ such that $n_K \geq N$ and $|a_{n_K} - a| < \varepsilon/2$.
  Then $\forall n \geq N$, we have
  \[
    |a_n - a| \leq |a_n - a_{n_K}| + |a_{n_K} - a| < \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon.
  \]
\end{proof}
\begin{thm}[The general principle of convergence]
  Let $\F$ be an ordered field with the monotone-sequence property. Then every Cauchy sequence of $\F$ converges.
\end{thm}
\begin{proof}
Let $(a_n)$ be a Cauchy sequence. Then it is eventually bounded, since $\exists N$, $\forall n \geq N$, $|a_n - a_N| \leq 1$ by the Cauchy condition. So it is bounded. Hence by Bolzano-Weierstrass, it has a convergent subsequence. Then $(a_n)$ converges to the same limit.
\end{proof}
\begin{proof}(alternative without B-W)
Let $(a_n)$ be Cauchy. Then define inductively $(k_n)$ such that $\forall n\in\N, \exists k_n\in\N$ such that $\forall i,j\geq k_n: |x_i-x_j<\frac{1}{2^n}$. Let $y_n=x_{k_n}-x_{k_{n+1}}$, so $|y_n|<2^{-n}$. By CT, $\sum y_n$ converges absolutely so converges. By telescoping, $\sum_{n=1}^N y_n=x_{k_1}-x_{k_{N+1}}$.  Therefore $(x_{k_n})$ converges. So $(x_n)$ has a convergent subsequence and hence converges to the same limit.
\end{proof}

Q: Complete ordered field
A: An ordered field in which every Cauchy sequence converges is called \emph{complete}.

Q: Complete ordered field with the Archimedean property
A: Let $\F$ be a complete ordered field with the Archimedean property. Then $\F$ satisfies the monotone-sequences property.
\begin{proof}
  Instead of showing `bounded $\implies$ converges (i.e. Cauchy)', we will show the equivalent statement `not Cauchy $\implies$ not bounded'.
  Let $(a_n)$ be an increasing sequence. If $(a_n)$ is not Cauchy, then
  \[
    (\exists \varepsilon > 0)(\forall N)(\exists p, q > N)\,|a_p - a_q| \geq \varepsilon.
  \]
  wlog let $p > q$. Then
  \[
    a_p \geq a_q + \varepsilon \geq a_N + \varepsilon.
  \]
  So for any $N$, we can find a $p > N$ such that
  \[
    a_p \geq a_N + \varepsilon.
  \]
  Then we can construct a subsequence $a_{n_1}, a_{n_2}, \cdots$ such that
  \[
    a_{n_{k + 1}} \geq a_{n_k} + \varepsilon.
  \]
  Therefore
  \[
    a_{n_k} \geq a_{n_1} + (k - 1)\varepsilon.
  \]
  So by the Archimedean property, $(a_{n_k})$, and hence $(a_n)$, is unbounded.
\end{proof}

Q: Limit supremum/infimum
A: Let $(a_n)$ be a bounded sequence. We define the \emph{limit supremum} as
  \[
    \limsup_{n\to \infty} a_n = \lim_{n\to \infty}\left(\sup_{m \geq n} a_m\right).
  \]
  To see that this exists, set $b_n = \sup_{m\geq n}a_m$. Then $(b_n)$ is \textbf{decreasing} since we are taking the supremum of fewer and fewer things, and is \textbf{bounded below} by any lower bound for $(a_n)$ since $b_n \geq a_n$. So it converges.
  Similarly, we define the \emph{limit infimum} as
  \[
    \liminf_{n\to \infty}a_n = \lim_{n\to\infty}\left(\inf_{m\geq n} a_m\right).
  \]

Q: Limit by limit supremum/infimum
A: \begin{lemma}
  Let $(a_n)$ be a sequence. The following two statements are equivalent:
  \begin{itemize}
    \item $a_n\to a$
    \item $\limsup a_n = \liminf a_n = a$.
  \end{itemize}
\end{lemma}
\begin{proof}
  If $a_n \to a$, then let $\varepsilon > 0$. Then
  \[
    (\exists n)(\forall m\geq n)\, a - \varepsilon \leq a_m \leq a + \varepsilon.
  \]
  It follows that
  \[
    a - \varepsilon \leq \inf_{m \geq n}a_m \leq \sup_{m\geq n}a_m \leq a + \varepsilon.
  \]
  Since $\varepsilon$ was arbitrary, it follows that
  \[
    \liminf a_n = \limsup a_n = a.
  \]
  Conversely, if $\liminf a_n = \limsup a_n = a$, then let $\varepsilon > 0$. Then we can find $n$ such that
  \[
    \inf_{m\geq n} a_m > a - \varepsilon\text{ and }\sup _{m \geq n} a_m < a + \varepsilon.
  \]
  It follows that $\forall m\geq n$, we have $|a_m - a| < \varepsilon$.
\end{proof}

Q: Convergence of series
A: Let $(a_n)$ be a real sequence. For each $N$, define
  \[
    S_N = \sum_{n = 1}^N a_n.
  \]
  If the sequence $(S_N)$ converges to some limit $s$, then we say that
  \[
    \sum_{n = 1}^\infty a_n = s,
  \]
  and we say that the series $\displaystyle\sum_{n = 1}^\infty a_n$ \emph{converges}.
  We call $S_N$ the $N$th \emph{partial sum}.

Q: Necessary condition for a series to converge
A: \begin{lemma}
  If $\displaystyle\sum_{n = 1}^\infty a_n$ converges. Then $a_n \to 0$.
\end{lemma}
\begin{proof}
  Let $\displaystyle\sum_{n = 1}^{\infty} a_n = s$. Then $S_n \to s$ and $S_{n - 1} \to s$. Then $a_n = S_n - S_{n - 1} \to 0$.
\end{proof}
However, the converse is false! e.g. Harmonic series

Q: Divergence of Harmonic Series
A: \[
    S_{2^n} - S_{2^{n -1 }} = \frac{1}{2^{n - 1} + 1} + \cdots + \frac{1}{2^n} \geq \frac{2^{n - 1}}{2^n} = \frac{1}{2}.
  \]
  Therefore $S_{2^n} \geq S_1 + n/2$. So the partial sums are unbounded.

Q: Geometric Series
A: Let $|\rho| < 1$. Consider the partial sums:
  \[
    \sum_{n = 0}^N \rho^n = \frac{1 - \rho^{N + 1}}{1 - \rho}.
  \]
  Since $\rho^{N + 1} \to 0$, this tends to $1/(1 - \rho)$.

Q: Sufficient condition for convergence of a series
A: Suppose that $a_n \geq 0$ for every $n$ and the partial sums $S_n$ are bounded above. Then $\sum_{n=1}^\infty a_n$ converges.
\begin{proof}
  The sequence $(S_n)$ is increasing and bounded above. So the result follows form the monotone sequences property.
\end{proof}

Q: Comparison test
A: Let $(a_n)$ and $(b_n)$ be non-negative sequences, and suppose that $\exists C, N$ such that $\forall n\geq N$, $a_n \leq Cb_n$. Then if $\sum b_n$ converges, then so does $\sum a_n$.
\begin{proof}
  Let $M > N$. Say $S_R = \sum_{n = 1}^R a_n$ and $T_R = \sum_{n = 1}^R b_n$. We show $S_R$ is bounded above (and increasing) so converges.
  \[
    S_M - S_N = \sum_{n = N + 1}^M a_n \leq C\sum _{n = N + 1}^M b_n \leq C\sum_{n = N + 1}^\infty b_n.
  \]
  So $\forall M\geq N$, $S_M \leq S_n + C\sum_{n = N + 1}^\infty b_n$.
\end{proof}

Q: Convergence of exponential series
A: If $\alpha > 1$, then $\sum 1/n^\alpha$ converges.
\begin{proof}
      Let $S_N = \sum_{n = 1}^N 1/n^\alpha$. Then
      \begin{align*}
        S_{2^n} - S_{2^{n - 1}} &= \frac{1}{(2^{n - 1} + 1)^\alpha} + \cdots + \frac{1}{(2^{n})^\alpha}\\
        &\leq \frac{2^{n - 1}}{(2^{n - 1})^\alpha}\\
        &= (2^{n - 1})^{1 - \alpha}\\
        &= (2^{1 - \alpha})^{n - 1}.
      \end{align*}
      But $2^{1 - \alpha} < 1$. So
      \[
        S_{2^n} = (S_{2^n} - S_{2^{n - 1}}) + (S_{2^{n - 1}} - S_{2^{n -2 }}) + \cdots (S_2 - S_1) + S_1
      \]
      and is bounded above by comparison with the geometric series $1 + 2^{1 - \alpha} + (2^{1 - \alpha})^2 + \cdots$.
\end{proof}

Q: Absolute convergence
A: A series $\sum a_n$ \emph{converges absolutely} if the series $\sum |a_n|$ converges.

Q: Convergence vs. absolute convergence
A: Let $\sum a_n$ converge absolutely. Then $\sum a_n$ converges.
\begin{proof}
  We know that $\sum |a_n|$ converges. Let $S_n = \sum_{n = 1}^N a_n$ and $T_N = \sum_{n = 1}^N |a_n|$.
  If $p > q$, then
  \[
    |S_p - S_q| = \left|\sum_{n = q + 1}^p a_n\right| \leq \sum_{n = q + 1}^p |a_n| = T_p - T_q.
  \]
  But the $T_n$ converges so $\forall \varepsilon > 0$, we can find $N$ such that $\forall p > q \geq N$, $T_p - T_q < \varepsilon$, which implies $|S_p - S_q| < \varepsilon$.
\end{proof}
The converse is false.

Q: Unconditional convergence
A: A series $\sum a_n$ \emph{converges unconditionally} if the series $\sum_{n = 1}^{\infty} a_{\pi(n)}$ converges for every bijection $\pi: \N \to \N$, ie. no matter how we re-order the elements of $a_n$, the sum still converges.

Q: Conditional convergence vs. absolute convergence
A: If $\sum a_n$ converges absolutely, then it converges unconditionally.
\begin{proof}
  Let $S_n = \sum_{n = 1}^N a_{\pi (n)}$. Then if $p > q$,
  \[
    |S_p - S_q| = \left|\sum_{n = q + 1}^p a_{\pi(n)}\right| \leq \sum_{n = q + 1}^\infty|a_{\pi (n)}|.
  \]
  Let $\varepsilon > 0$. Since $\sum |a_n|$ converges, pick $M$ such that $\sum_{n = M + 1}^\infty|a_n| < \varepsilon$.
  Pick $N$ large enough that $\{1, \cdots, M\}\subseteq \{\pi (1), \cdots, \pi(N)\}$.
  Then if $n > N$, we have $\pi(n) > M$. Therefore if $p > q \geq N$, then
  \[
    |S_p - S_q| \leq \sum_{n = q + 1}^p |a_{\pi(n)}| \leq \sum_{n = M + 1}^\infty |a_n| < \varepsilon.
  \]
\end{proof}
  If $\sum a_n$ converges unconditionally, then it converges absolutely.
\begin{proof}
  We will prove the contrapositive: if it doesn't converge absolutely, it doesn't converge unconditionally.
  Suppose that $\sum |a_n| = \infty$. Let $(b_n)$ be the subsequence of non-negative terms of $a_n$, and $c_n$ be the subsequence of negative terms. Then $\sum b_n$ and $\sum c_n$ cannot both converge, or else $\sum |a_n|$ converges.
  wlog, $\sum b_n = \infty$. Now construct a sequence $0 = n_0 < n_1 < n_2 < \cdots$ such that $\forall k$,
  \[
    b_{n_{k - 1} + 1} + b_{n_{k - 1} + 2} + \cdots + b_{n_k} + c_k \geq 1,
  \]
  This is possible because the $\sum b_n$ is bounded and we can get it as large as we want.
  Let $\pi$ be the rearrangement
  \[
    b_1, b_2, \cdots b_{n_1}, c_1, b_{n_1 + 1}, \cdots b_{n_2}, c_2, b_{n_2 + 1}, \cdots b_{n_3}, c_3,\cdots
  \]
  So the sum up to $c_k$ is at least $k$. So the partial sums tend to infinity.
\end{proof}

Q: Limit of a rearranged absolutely convergent series
A: Let $\sum a_n$ be a series that converges absolutely. Then for any bijection $\pi: \N\to \N$,
 \[
   \sum_{n = 1}^\infty a_n = \sum_{n = 1}^\infty a_{\pi(n)}.
 \]
\begin{proof}
  Let $\varepsilon > 0$. We know that both $\sum |a_n|$ and $\sum|a_{\pi(n)}|$ converge. So let $M$ be such that $\sum_{n > M}|a_n| < \frac{\varepsilon}{2}$ and $\sum_{n > M}|a_{\pi(n)}| < \frac{\varepsilon}{2}$.
  Now let $N$ be large enough such that
  \[
    \{1, \cdots, M\}\subseteq \{\pi(1), \cdots, \pi(N)\},
  \]
  and
  \[
    \{\pi(1), \cdots, \pi(M)\}\subseteq \{1, \cdots, N\}.
  \]
  Then for every $K\geq N$,
  \[
    \left|\sum_{n = 1}^K a_n - \sum_{n = 1}^K a_{\pi(n)}\right| \leq \sum_{n = M + 1}^K |a_n| + \sum_{n = M + 1}^K |a_{\pi(n)}| < \frac{\varepsilon}{2} + \frac{\varepsilon}{2} = \varepsilon.
  \]
  So $\forall K \geq N$, the partial sums up to $K$ differ by at most $\varepsilon$. So $|\sum a_n - \sum a_{\pi(n)}| \leq \varepsilon$.
  Since this is true for all $\varepsilon$, we must have $\sum a_n = \sum a_{\pi(n)}$.
\end{proof}

Q: Alternating sequence test
A: Let $(a_n)$ be a decreasing sequence of non-negative reals, and suppose that $a_n \to 0$. Then $\displaystyle \sum_{n = 1}^\infty (-1)^{n + 1}a_n$ converges, ie. $a_1 - a_2 + a_3 - a_4 + \cdots$ converges.
\begin{proof}
  Let $\displaystyle S_N = \sum_{n = 1}^N(-1)^{n + 1}a_n$. Then
  \[
    S_{2n} = (a_1 - a_2) + (a_3 - a_4) + \cdots (a_{2n - 1} - a_{2n}) \geq 0,
  \]
  and $(S_{2n})$ is an increasing sequence.
  Also,
  \[
    S_{2n + 1} = a_1 - (a_2 - a_3)  - \cdots - (a_{2n} - a_{2n + 1})\leq a_{1},
  \]
  and $(S_{2n + 1})$ is a decreasing sequence. Also $S_{2n + 1} - S_{2n} = a_{2n + 1} \geq 0$.
  Hence we obtain the bounds $0 \leq S_{2n} \leq S_{2n + 1} \leq a_1$. It follows from the monotone sequences property that $(S_{2n})$ and $(S_{2n + 1})$ converge.
  Since $S_{2n + 1} - S_{2n} = a_{2n + 1} \to 0$, they converge to the same limit.
\end{proof}

Q: Ratio test
A: We have three versions:
  \begin{enumerate}
    \item If $\exists c < 1$ such that
      \[
        \frac{|a_{n + 1}|}{|a_n|} \leq c,
      \]
      for all $n$, then $\sum a_n$ converges.
    \item If $\exists c < 1$ and $\exists N$ such that
      \[
        (\forall n \geq N)\, \frac{|a_{n + 1}|}{|a_n|} \leq c,
      \]
      then $\sum a_n$ converges.
    \item (D'Alembert Ratio Test) If $\exists \rho \in (-1, 1)$ such that
      \[
        \frac{a_{n + 1}}{a_n} \to \rho,
      \]
      then $\sum a_n$ converges.
  \end{enumerate}
\begin{proof}\leavevmode
  \begin{enumerate}
    \item $|a_n| \leq c^{n - 1}|a_1|$. Since $\sum c^n$ converges, so does $\sum |a_n|$ by comparison test. So $\sum a_n$ converges absolutely, so it converges.
    \item For all $k\geq 0$, we have $|a_{N + k}|\leq c^k|A_N|$. So the series $\sum |a_{N + k}|$ converges, and therefore so does $\sum |a_k|$.
    \item If $\frac{a_{n + 1}}{a_n} \to \rho$, then $\frac{|a_{n + 1}|}{|a_n|} \to |\rho|$. So (setting $\varepsilon = (1 - |\rho|)/2$) there exists $N$ such that $\forall n \geq N$, $\frac{|a_{n + 1}|}{|a_n|} \leq \frac{1 + |\rho|}{2} < 1$. So the result follows from (ii).
  \end{enumerate}
\end{proof}

Q: Condensation test
A: Let $(a_n)$ be a decreasing non-negative sequence. Then
  \[
    \sum_{n = 1}^\infty a_n < \infty \iff \sum_{k = 1}^\infty 2^k a_{2^k} < \infty.
  \]
\begin{proof}
  \begin{align*}
\sum_{n=1}^{2^k} a_n &= a_1 + a_2 + (a_3 + a_4) + (a_5 + \cdots + a_8)+ \ldots \\
& \quad+ (a_{2^{k-1}+1} + \cdots + a_{2^k})\\
    &\geq  a_1 + a_2 + 2a_4 + 4a_8 \ldots + 2^{k-1} a_{2^k} \\
&\geq \frac{1}{2} \sum_{n=1}^k 2^n a_{2^n}
  \end{align*}
  \begin{align*}
   \sum_{n=1}^{2^k} a_n &= a_1 + (a_2 + a_3) + (a_4 + \cdots + a_7) + \ldots + (a_{2^k}+\ldots + a_{2^k-1})\\
    &\leq  a_1 + 2a_2 + 4a_4 + \ldots + 2^k a_{2^k} \\
&\leq a_1+\sum_{n=1}^k 2^n a_{2^n}
  \end{align*}
\end{proof}

Q: Abel's Test
A: Let $a_1 \geq a_2 \geq \cdots \geq 0$, and suppose that $a_n \to 0$. Let $z\in \C$ such that $|z| = 1$ and $z \not= 1$. Then $\sum a_n z^n$ converges.
\begin{proof}
  We prove that it is Cauchy so it converges.
  \begin{align*}
    \sum_{n = M}^N a_n z^n &= \sum_{n = M}^N a_n\frac{z^{n + 1} - z^n}{z - 1}\\
    &= \frac{1}{z - 1}\sum_{n = M}^N a_n (z^{n + 1} - z^n)\\
    &= \frac{1}{z - 1}\left(\sum_{n = M}^N a_n z^{n + 1} - \sum_{n = M}^N a_n z^n\right)\\
    &= \frac{1}{z - 1}\left(\sum_{n = M}^N a_n z^{n + 1} - \sum_{n = M - 1}^{N - 1} a_{n + 1} z^{n + 1}\right)\\
    &= \frac{1}{z - 1}\left(a_N z^{N + 1} - a_M z^M + \sum_{n = M}^{N - 1} (a_n - a_{n + 1})z^{n + 1}\right)\\
    \therefore \left|\sum_{n = M}^N a_nz^n\right| &\leq \frac{1}{|z - 1|} \left(a_N + a_M + \sum_{n = M}^{N - 1}(a_n - a_{n + 1})\right)\\
    &= \frac{1}{|z - 1|}\left(a_N + a_M + (a_M - a_{M + 1}) + \cdots + (a_{N - 1} - a_{N})\right)\\
    &= \frac{2a_M}{|z - 1|} \to 0.
  \end{align*}
\end{proof}

Q: Continuous function
A: Let $A\subseteq \R$, $a\in A$, and $f: A\to \R$. Then $f$ is \emph{continuous at} $a$ if
  \[
    (\forall \varepsilon > 0)(\exists \delta > 0)(\forall y\in A)\, |y - a| < \delta \Rightarrow |f(y) - f(a)| < \varepsilon.
  \]
  $f$ is \emph{continuous} if it is continuous at each every $a\in A$. That is
  \[
    (\forall a\in A)(\forall \varepsilon > 0)(\exists \delta > 0)(\forall y\in A)\, |y - a| < \delta \Rightarrow |f(y) - f(a)| < \varepsilon.
  \]
Equivalently: if $(a_n)$ is a sequence in $A$ with $a_n \to a$, then $f(a_n) \to f(a)$. ($*$)
\begin{proof}
  Continuous $\Rightarrow$ ($*$): Let $\varepsilon > 0$. Since $f$ is continuous at $a$,
  \[
    (\exists \delta > 0)(\forall y\in A)\, |y-a|< \delta \Rightarrow |f(y) - f(a)| < \varepsilon.
  \]
  Let $y=a_n$ and $\forall n\geq N$: $|a_n - a| < \delta$ (as $a_n\to a)$. Then $\forall n \geq N$, $|f(a_n) - f(a)| < \varepsilon$.
  ($*$) $\Rightarrow$ continuous: We prove the contrapositive: Suppose $f$ is not continuous at $a$. Then
  \[
    (\exists \varepsilon > 0)(\forall \delta > 0)(\exists y\in A)\, |y - a| < \delta \text{ and }|f(y) - f(a)| \geq \varepsilon.
  \]
Letting $\delta=\frac{1}{n}$, $y=a_n$, then $|a_n - a| < \frac{1}{n}$ and $|f(a_n) - f(a)| \geq \varepsilon$. But then $a_n \to a$ (by Archimedean property), but $f(a_n) \not\to f(a)$.
\end{proof}

Q: Sum, product and quotient of continuous functions
A: Let $A\subseteq \R$ and $f, g: A\to \R$ be continuous functions. Then
  \begin{enumerate}
    \item $f + g$ is continuous
    \item $fg$ is continuous
    \item if $g$ never vanishes, then $f/g$ is continuous.
  \end{enumerate}
\begin{proof}\leavevmode
  \begin{enumerate}
    \item Let $a\in A$ and let $(a_n)$ be a sequence in $A$ with $a_n \to a$. Then
      \[
        (f + g)(a_n) = f(a_n) + g(a_n).
      \]
      But $f(a_n) \to f(a)$ and $g(a_n) \to g(a)$. So
      \[
        f(a_n) + g(a_n) \to f(a) + g(a) = (f + g)(a).
      \]
 \item Let $a\in A$ and let $(a_n)$ be a sequence in $A$ with $a_n \to a$. Then
      \[
        (fg)(a_n) = f(a_n) g(a_n).
      \]
      But $f(a_n) \to f(a)$ and $g(a_n) \to g(a)$. So
      \[
        f(a_n) g(a_n) \to f(a) g(a) = (f g)(a).
      \]
 \item Let $a\in A$ and let $(a_n)$ be a sequence in $A$ with $a_n \to a$. Then for $g$ never zero
      \[
        (f/ g)(a_n) = f(a_n) / g(a_n).
      \]
      But $f(a_n) \to f(a)$ and $g(a_n) \to g(a)$. So
      \[
        f(a_n) / g(a_n) \to f(a) / g(a) = (f / g)(a).
      \]
  \end{enumerate}
\end{proof}

Q: Composition of continuous functions
A: Let $A, B\subseteq \R$ and $f: A\to B$, $g: B\to \R$. Then if $f$ and $g$ are continuous, $g\circ f: A\to \R$ is continuous.
\begin{proof}
  We offer two proofs:
  \begin{enumerate}
    \item Let $(a_n)$ be a sequence in $A$ with $a_n \to a\in A$. Then $f(a_n) \to f(a)$ since $f$ is continuous. Then $g(f(a_n)) \to g(f(a))$ since $g$ is continuous. So $g\circ f$ is continuous.
    \item Let $a\in A$ and $\varepsilon > 0$. Since $g$ is continuous at $f(a)$, there exists $\eta > 0$ such that $\forall z\in B$, $|z - f(a)| < \eta \Rightarrow |g(z) - g(f(a))| < \varepsilon$.
      Since $f$ is continuous at $a$, $\exists \delta > 0$ such that $\forall y\in A$, $|y - a| < \delta \Rightarrow |f(y) - f(a)| < \eta$. Therefore $|y - a| < \delta \Rightarrow |g(f(y)) - g(f(a))| < \varepsilon$.
  \end{enumerate}
\end{proof}

Q: Maximum value theorem
A: Let $[a, b]$ be a closed interval in $\R$ and let $f: [a, b] \to \R$ be continuous. Then $f$ is bounded and attains it bounds, ie. $f(x) = \sup f$ for some $x$, and $f(y) = \inf f$ for some $y$.
\begin{proof}
  If $f$ is not bounded above, then for each $n$, we can find $x_n\in [a, b]$ such that $f(x_n) \geq n$ for all $n$.
  By Bolazano-Weierstrass, since $x_n\in [a, b]$ so is bounded, the sequence $(x_n)$ has a convergent subsequence $(x_{n_k})$. Let $x$ be its limit. Then since $f$ is continuous, $f(x_{n_k}) \to f(x)$. But $f(x_{n_k}) \geq n_k \to \infty$. So this is a contradiction.
  Now let $C = \sup\{f(x) \ | \ x\in [a, b]\}$. Then for every $n$, we can find $x_n$ such that $f(x_n) \geq C - \frac{1}{n}$. So by Bolzano-Weierstrass, $(x_n)$ has a convergent subsequence $(x_{n_k})$. Since $C - \frac{1}{n_{k}}\leq f(x_{n_k}) \leq C$, $f(x_{n_k})\to C$. Therefore if $x = \lim x_{n_k}$, then $f(x) = C$.
 Applying the same argument to $-f$ gives the result for the lower bound.
\end{proof}
We can prove the first part (that $f$ is bounded) by Continuous Induction or Heine-Borel.
\begin{proof}
Let $A = \{x\ | \ f\text{ is bounded on }[a, x]\}$. Then $a\in A$. If $x\in A, x\not= b$, then $\exists \delta > 0$ such that $|y - x| < \delta \Rightarrow |f(y) - f(x)| < 1$. So $\exists y > x$ (eg. take $\min\{x + \delta/2, b\}$) such that $f$ is bounded on $[a, y]$, which implies that $y\in A$.
  Now suppose that $\forall \varepsilon > 0$, $\exists y\in (x - \varepsilon, x]$ such that $y\in A$. Again, we can find $\delta > 0$ such that $f$ is bounded on $(x - \delta, x + \delta)$, and in particular on $(x - \delta, x]$. Pick $y$ such that $f$ is bounded on $[a, y]$ and $y > x - \delta$. Then $f$ is bounded on $[a, x]$. So $x\in A$.
\end{proof}
\begin{proof}
By continuity, $\forall x\in [a, b]$:
  \[
   \exists \delta_x > 0: \forall y:  |y - x| < \delta_x\Rightarrow |f(y) - f(x)| < 1.
  \]
 For each $x\in [a,b]$, let $I_x = (x - \delta_x, x + \delta_x)$. So by Heine-Borel, we can find $x_1, \cdots, x_n$ such that $[a, b]\subseteq \bigcup_1^n (x_i - \delta_{x_i}, x_i + \delta_{x_i})$.
  But $f$ is bounded in each interval $(x_i - \delta_{x_i}, x_i + \delta_{x_i})$ by $|f(x_i)| + 1$. So it is bounded on $[a, b]$ by $\max|f(x_i)| + 1$.
\end{proof}

Q: Intermediate value theorem
A: Let $a < b\in \R$ and let $f: [a, b] \to \R$ be continuous. Suppose that $f(a) < 0 < f(b)$. Then there exists an $x\in (a, b)$ such that $f(x) = 0$.
\begin{proof}
  \begin{enumerate}
    \item Let $A = \{x: f(x) < 0\}$ and let $s = \sup A$. We shall show that $f(s) = 0$. If $f(s) < 0$, then setting $\varepsilon = |f(s)|$ in the definition of continuity, we can find $\delta > 0$ such that $\forall y$, $|y - s| < \delta \Rightarrow |f(y)-f(s)|<|f(s)| \Rightarrow f(y) < f(s)+|f(s)|=0$. Then setting, for example, $y=s+\frac{\delta}{2}$ shows $s + \delta/2 \in A$, so $s$ is not an upper bound. Contradiction.
    If $f(s) > 0$, by the same argument, we can find $\delta > 0$ such that $\forall y$, $|y - s| < \delta \Rightarrow f(y) >f(s)-|f(s)|= 0$. So $s - \delta/2$ is a smaller upper bound.
  \item Let $a_0 = a$, $b_0 = b$. By repeated bisection, construct nested intervals $[a_n, b_n]$ such that $b_n - a_n = \frac{b_0 - a_0}{2^n}$ and $f(a_n) < 0 \leq f(b_n)$. Then by the nested intervals property, we can find $x\in \cap_{n = 0}^\infty [a_n, b_n]$. Since $b_n - a_n \to 0$, $a_n, b_n \to x$.
    Since $f(a_n) < 0$ for every $n$, $f(x) \leq 0$. Similarly, since $f(b_n) \geq 0$ for every $n$, $f(x) \geq 0$. So $f(x) = 0$.
\item Suppose that $(\forall x)\, f(x) \not =0$. We use CIv1.
  Let $A = \{x: f(x) < 0\}$ Then $a\in A$ (1.1). If $x\in A$, then $f(x) < 0$, and by continuity, we can find $\delta > 0$ such that $|y - x| < \delta\Rightarrow f(y) < f(x)+|f(x)|=0$. So if $x\not= b$, then we can find $y\in A$, e.g. $y=x+\frac{\delta}{2}$, such that $y > x$ (1.2).
  We contrapose (1.3) twice.
  \begin{align*}
&\forall \delta>0: \exists y\in A: y\in(x-\delta,x] \Rightarrow x\in A \\
    \iff &x\not\in A\Rightarrow \exists \delta > 0: \forall y\in A:  y\not\in(x - \delta, x] \\
\iff &x\not\in A\Rightarrow \exists \delta>0: y\in(x - \delta, x] \implies y\not\in A
  \end{align*}
  If $x\not\in A$, then $f(x) > 0$. Then by continuity, $\exists \delta > 0$ such that $|y - x| < \delta \Rightarrow f(y) >f(x)-|f(x)|= 0 \Rightarrow y\not\in A$.
  Hence by CIv1, $b\in A$. Contradiction.
  \end{enumerate}
\end{proof}
If $f(a) < c < f(b)$, then $\exists x\in (a, b)$ such that $f(x) = c$, so apply the result to $f(x) - c$. If $f(b) < c < f(a)$, then apply the result to $-f(x)$.

Q: Inverse of continuous functions
A: Let $f: [a, b]\to [c, d]$ be a continuous strictly increasing function with $f(a) = c$, $f(b) = d$. Then $f$ is invertible and its inverse is continuous.
\begin{proof}
  Since $f$ is strictly increasing, it is an injection*. Now let $y\in (c, d)$. By the intermediate value theorem, there exists $x\in (a, b)$ such that $f(x) = y$. So $f$ is a surjection. So it is a bijection and hence invertible.
  Let $g$ be the inverse. Let $y\in [c, d]$ and let $\varepsilon > 0$. Let $x = g(y)$. So $f(x) = y$. Let$^\dagger$ $u = f(x - \varepsilon)$ and $v = f(x + \varepsilon)$. Then $u < y < v$. So we can find $\delta > 0$ such that $(y - \delta , y + \delta) \subseteq (u, v)$. Then $|z - y| < \delta \Rightarrow g(z) \in (x - \varepsilon, x + \varepsilon) \Rightarrow |g(z) - g(y)| < \varepsilon$.
\end{proof}
(*) Suppose $x \not= y$. wlog, $x < y$. Then $f(x) < f(y)$ and so $f(x) \not= f(y)$)
($^\dagger$) If $y = c$, then let $u=c$ etc. If $y=d$, then let $y=d$ etc.

Q: Continuous induction
A: \textsc{V1}: Let $a < b$ and let $A\subseteq [a, b]$ have the following properties: \\
 \textbf{1.1} $a\in A$ \\
 \textbf{1.2} If $x\in A$ and $x\not= b$, then $\exists y\in A$ with $y > x$. \\
 \textbf{1.3} If $\forall \varepsilon > 0$, $\exists y\in A: y\in (x - \varepsilon, x]$, then $x\in A$.\\
  Then $b\in A$.
\begin{proof}
  Since $a\in A$ (by (1.1)), $A\not= \emptyset$. $A$ is also bounded above by $b$. So let $s = \sup A$. Then, using (1.2) with $x=s-\varepsilon$, we learn $\forall \varepsilon > 0$, $\exists y\in A$ such that $y > s - \varepsilon$. Therefore, by (1.3), $s\in A$.
  If $s\not= b$, then by (1.2), we can find $y\in A$ such that $y > s$.
\end{proof}
\textsc{V2}: Let $A\subseteq [a, b]$ and suppose that \\
\textbf{2.1} $a\in A$ \\
\textbf{2.2} If $[a, x]\subseteq A$ and $x\not = b$, then there exists $y > x$ such that $[a, y]\subseteq A$. \\
\textbf{2.3} If $[a, x)\subseteq A$, then $[a, x]\subseteq A$. \\
  Then $A = [a, b]$
\begin{proof}
  We prove that version 1 $\Rightarrow$ version 2.
  Suppose $A$ satisfies 2.1-2.3. Let $A' = \{x\in [a, b]: [a, x]\subseteq A\}$. We will show $A'$ satisfies 1.1-1.3.
 (2.1)$\Rightarrow$ $a\in A'$ $\Rightarrow$ (1.1). \\
If $x\in A'$ with $x \not= b$, then $[a, x]\subseteq A$. So, by (2.2), $\exists y > x$ such that $[a, y] \subseteq A$. So $\exists y > x$ such that $y\in A'$, $\Rightarrow$ (1.2). \\
  If $\forall \varepsilon > 0, \exists y\in A':  y\in (x - \varepsilon, x]$, then $[a, y]\subseteq A$, so $[a, x)\subseteq A$. So by (2.3), $[a, x]\subseteq A$, so $x\in A'$, $\Rightarrow$ (1.3). \\
Therefore, by v1, $b\in A'$. So $[a, b]\subseteq A$. So $A = [a, b]$.
\end{proof}

Q: Cover of a set
A: Let $A\subseteq \R$. A \emph{cover} of $A$ by open intervals is a set $\{I_\gamma: \gamma\in \Gamma\}$ where each $I_\gamma$ is an open interval and $A \subseteq \bigcup_{\gamma\in \Gamma}$ $I_\gamma$.
  A \emph{finite subcover} is a finite subset $\{I_{\gamma_1}, \cdots, I_{\gamma_n}\}$ of the cover that is still a cover.

Q: Heine-Borel Theorem
A: Every cover of a closed, bounded interval $[a, b]$ by open intervals has a finite subcover, i.e. is \emph{compact}.
\begin{proof}
  Let $\{I_\gamma: \gamma\in \Gamma\}$ be a cover of $[a, b]$ by open intervals. Let $A = \{x: [a, x]$ can be covered by finitely many of the $I_\gamma\}$.
  Then $a\in A$ (since $a$ must belong to some $I_\gamma$).
  If $x\in A$, then pick $\gamma$ such that $x\in I_\gamma$. Then if $x\not = b$, since $I_\gamma$ is an open interval, it contains $[x, y]$ for some $y > x$. Then $[a, y]$ can be covered by finitely many $I_\gamma$, by taking a finite cover for $[a, x]$ and $I_\gamma$ (which contains $x$).
  Now suppose that $\forall \varepsilon > 0, \exists y\in A$ such that $y\in (x - \varepsilon, x]$.
  Let $I_\gamma$ be an open interval containing $x$. Then it contains $(x - \varepsilon, x]$ for some $\varepsilon > 0$. Now combine $I_\gamma$ with a finite subcover of $[a, y]$ to get a finite subcover of $[a, x]$. So $x\in A$.
  The result follows by continuous induction.
\end{proof}

Q: Limit of a function
A: Let $A\subseteq \R$ and let $f: A\to \R$. We say
  \[
    \lim_{x\to a}f(x) = \ell,
  \]
  or ``$f(x) \to \ell$ as $x \to a$'', if
  \[
    (\forall \varepsilon > 0)(\exists \delta > 0)(\forall x\in A)\, 0 < |x - a| < \delta \Rightarrow |f(x) - \ell| < \varepsilon.
  \]

Q: Limit of sums, products and quotients of functions
A: If $f(x)\to \ell$ and $g(x)\to m$ as $x\to a$, then $f(x) + g(x) \to \ell+ m$, $f(x)g(x) \to \ell m$, and $\frac{f(x)}{g(x)}\to \frac{\ell}{m}$ if $g$ and $m$ don't vanish.
These are proven the same as for sequences, replacing $n>N$ with $|x-c|<\delta$.

Q: Sum rule for differentiation
A: Let $f, g$ be differentiable at $x$. Then $f + g$ is differentiable at $x$, with
  \begin{align*}
    (f + g)'(x) = f'(x) + g'(x)
  \end{align*}
\begin{proof}
  \begin{align*}
    (f + g)(x + h) &= f(x + h) + g(x + h)\\
    &= f(x) +hf'(x) + o(h) + g(x) + hg'(x) + o(h)\\
    &= (f + g)(x) + h(f'(x) + g'(x)) + o(h)
  \end{align*}
\end{proof}

Q: Product rule for differentiation
A: \begin{lemma}[Sum and product rule]
  Let $f, g$ be differentiable at $x$. Then $fg$ is differentiable at $x$, with
  \begin{align*}
    (fg)'(x) = f'(x)g(x) + f(x)g'(x)
  \end{align*}
\end{lemma}
\begin{proof}
  \begin{align*}
    fg(x + h) &= f(x + h)g(x + h)\\
    &= [f(x) + hf'(x) + o(h)][g(x) + hg'(x) + o(h)]\\
    &= f(x)g(x) + h[f'(x)g(x) + f(x)g'(x)]\\
    &+ \underbrace{o(h)[g(x) + f(x) + hf'(x) + hg'(x) + o(h)] + h^2f'(x)g'(x)}_{\text{error term}}\\
    \intertext{By limit theorems, the error term is $o(h)$. So we can write this as}
    &= f(x) + h(f'(x)g(x) + f(x)g'(x)) + o(h).
  \end{align*}
\end{proof}

Q: Chain Rule
A: If $f$ is differentiable at $x$ and $g$ is differentiable at $f(x)$, then $g\circ f$ is differentiable at $x$ with derivative $g'(f(x))f'(x)$.
\begin{proof}
  In this proof, we use $h\varepsilon(h)$ instead of $o(h)$.
  \begin{align*}
    (g\circ f)(x + h) &= g(f(x + h))\\
    &= g[f(x) + \underbrace{hf'(x) + h\varepsilon_1(h)}_{\text{the ``}h\text{'' term}}]\\
    &= g(f(x)) + \big(hf'(x) + h\varepsilon_1(h)\big)g'(f(x))\\
    &+ \big(hf'(x) + h\varepsilon_1(h)\big)\varepsilon_2(hf'(x) + h\varepsilon_1(h))\\
    &= g\circ f(x) + hg'(f(x))f'(x)\\
    &+ \underbrace{h\Big[\varepsilon_1(h)g'(f(x)) + \big(f'(x) + \varepsilon_1(h)\big)\varepsilon_2\big(hf'(x) + h\varepsilon_1(h)\big)\Big]}_{\text{error term}}.
  \end{align*}
  We just need to check that the error term is $o(h)$. So dividing by $h$ and taking the limit as $h\to 0$: $\varepsilon_1(h)g'(f(x))\to 0$, $f'(x) + \varepsilon_1(h)$ is bounded, and $\varepsilon_2(hf'(x) + h\varepsilon_1(h))\to 0$ because $hf'(x) + h\varepsilon_1(h) \to 0$ and $\varepsilon_2(0) = 0$.
\end{proof}

Q: Differentiable vs. continuous
A: If $f$ is differentiable at $x$, then it is continuous at $x$.
\begin{proof}
  As $y\to x$, $\displaystyle \frac{f(y) - f(x)}{y - x} \to f'(x)$. Since, $y - x \to 0$, so $f(y) - f(x) \to 0$ by product theorem of limits. Thus $f(y) \to f(x)$. So $f$ is continuous at $x$.
\end{proof}

Q: Inverse function theorem
A: Let $f:[a, b]\to [c, d]$ be a differentiable on $(a, b)$, continuous on $[a, b]$, and strictly increasing. Suppose that $f'(x)$ never vanishes. Suppose further that $f(a) = c$ and $f(b) = d$. Then $f$ has an inverse $f^{-1}$ and for each $y\in (c, d)$, $f^{-1}$ is differentiable at $y$ with derivative $1/f'(f^{-1}(y))$.
\begin{proof}
 $f$ differentiable $\implies f$ continuous $\implies f^{-1}$  exists and is continuous.
  Let $y, y + k\in (c, d)$. Let $x = f^{-1}(y)$, $x + h = f^{-1}(y + k)$, so $y + k = f(x + h)$.
  Then $k = f(x + h) - y = f(x + h) - f(x)$. So
  \[
    \frac{f^{-1}(y + k) - f^{-1}(y)}{k} = \frac{(x + h) - x}{f(x + h) - f(x)} = \left(\frac{f(x + h) - f(x)}{h}\right)^{-1}.
  \]
  As $k \to 0$, since $f^{-1}$ is continuous, $f^{-1}(y + k) \to f^{-1}(y)$. So $h \to 0$. So
  \[
    \frac{f^{-1}(y + k) - f^{-1}(y)}{k} \to [f'(x)]^{-1} = [f'(f^{-1}(y)]^{-1}.
  \]
\end{proof}

Q: Rolle's Theorem
A: Let $f$ be continuous on a closed interval $[a, b]$ (with $a < b$) and differentiable on $(a, b)$. Suppose that $f(a) = f(b)$. Then there exists $x\in (a, b)$ such that $f'(x) = 0 $.
\begin{proof}
  If $f$ is constant, then we're done.
  Otherwise, there exists $u$ such that $f(u) \not= f(a)$. wlog, $f(u) > f(a)$. Since $f$ is continuous, it has a maximum, and since $f(u) > f(a) = f(b)$, the maximum is not attained at $a$ or $b$.
  Suppose maximum is attained at $x\in (a, b)$. Then for any $h \not = 0$, we have
  \[
    \frac{f(x + h) - f(x)}{h}
    \begin{cases}
      \leq 0 & h > 0\\
      \geq 0 & h < 0.
    \end{cases}
  \]
 By considering the limit $h\to 0$ from both directions, we know that $f'(x) \leq 0$ and $f'(x) \geq 0$. So $f'(x) = 0$.
\end{proof}

Q: Mean Value Theorem: proof
A: Let $f$ be continuous on $[a, b]$ ($a < b$), and differentiable on $(a, b)$. Then there exists $x\in (a, b)$ such that
  \[
    f'(x) = \frac{f(b) - f(a)}{b - a}.
  \]
\begin{proof}
  Let
  \[
    g(x) = f(x) - \frac{f(b) - f(a)}{b - a}x.
  \]
  Then
  \[
    g(b) - g(a) = f(b) - f(a) - \frac{f(b) - f(a)}{b - a}(b - a) = 0.
  \]
  So by Rolle's theorem, we can find $x\in (a, b)$ such that $g'(x) = 0$. So
  \[
    f'(x) = \frac{f(b) - f(a)}{b - a},
  \]
  as required.
\end{proof}

Q: Increasing function vs. positive gradient
A: If $m\leq f'(x) \leq M$ for every $x$ then by the MVT we can find $x\in (a, b)$ such that
  \begin{align*}
    m&\leq f'(x)= \frac{f(b) - f(a)}{b - a}\leq M \\
\therefore m(b-a)+f(a) &\leq f(b) \leq  M(b-a)+f(a)
  \end{align*}
e.g. $f'(x)>0\implies f(b)>f(a)\implies f$ is increasing.
Conversely, if $f$ is increasing over $(x,x+c)$, then \[
\frac{f(x+h)-f(x)}{h}\geq 0
\]
for $0<h<c$ so $f'(x)\geq 0$ if $f$ is differentiable at $x$.

Q: Local version of inverse function theorem
A: Let $f$ be a function with continuous derivative on $(a, b)$.  Let $x\in (a, b)$ and suppose that $f'(x) \not= 0$. Then there is an open interval $(u, v)$ containing $x$ on which $f$ is invertible (as a function from $(u, v)$ to $f((u, v))$). Moreover, if $g$ is the inverse, then $g'(f(z)) = \frac{1}{f'(z)}$ for every $z\in (u, v)$.
\begin{proof}
  wlog, $f'(x) > 0$. By the continuity of $f'$, we can find $\delta > 0$ such that $f'(z) > 0$ for every $z\in (x - \delta, x + \delta)$. So $f$ is strictly increasing on $(x - \delta, x + \delta)$, hence injective. Also, $f$ is continuous on $(x - \delta, x + \delta)$ by differentiability.
  Then done by the inverse function theorem.
\end{proof}

Q: Higher order Rolle's Theorem
A: Let $f$ be continuous on $[a, b]$ ($a < b$) and $n$-times differentiable on an open interval containing $[a, b]$. Suppose that
  \[
    f(a) = f'(a) = f^{(2)}(a) = \cdots = f^{(n - 1)}(a) = f(b) = 0.
  \]
  Then $\exists x\in (a, b)$ such that $f^{(n)}(x) = 0$.
\begin{proof}
  Induct on $n$. The $n = 0$ base case is just Rolle's theorem.
By hypothesis, for $k < n$ we can find $x_k\in (a, b)$ such that $f^{(k)}(x_k) = 0$. Since $f^{(k)}(a) = 0$, we can find $x_{k + 1}\in (a, x_k)$ such that $f^{(k + 1)}(x_{k + 1}) = 0$ by Rolle's theorem.
  So the result follows by induction.
\end{proof}
\begin{cor}
  Suppose that $f$ and $g$ are both differentiable on an open interval containing $[a, b]$ and that $f^{(k)}(a) = g^{(k)}(a)$ for $k = 0, 1, \cdots, n - 1$, and also $f(b) = g(b)$. Then there exists $x\in (a, b)$ such that $f^{(n)}(x) = g^{(n)}(x)$.
\end{cor}
\begin{proof}
  Apply higher order Rolle's to $f - g$.
\end{proof}

Q: Taylor's Theorem with Lagrangian remainder
A: We want an $n$th order polynomial $p$ such that $p^{(k)}(a) = f^{(k)}(a)$ for $k = 0, 1, \cdots, n - 1$ and $p(b) = f(b)$. Observe that
\begin{align*}
  Q_k(x) &= \frac{(x - a)^k}{k!} \\
  \implies Q_k^{(j)}(a) &=
  \begin{cases}
    1 & j = k\\
    0 & j \not= k.
  \end{cases} \\
Q(x) &= \sum_{k = 0}^{n - 1}f^{(k)}(a) Q_k(x) \\
\implies Q^{(j)}(a) &= f^{(j)}(a) \textrm{ for $j = 0, 1, \cdots, n - 1$.}
\end{align*}
To get $p(b) = f(b)$, we use our $n$th degree term:
\[
  p(x) = Q(x) + \frac{(x - a)^n}{(b - a)^n}\big(f(b) - Q(b)\big).
\]
By corollary of higher order Rolle's Theorem, we can find $x\in (a, b)$ such that
\begin{align*}
  f^{(n)}(x) &= p^{(n)}(x) \\
&= \frac{n!}{(b - a)}\big(f(b) - Q(b)\big) \\
  \therefore f(b) &= Q(b) + \frac{(b - a)^n}{n!}f^{(n)}(x) \\
&= f(a) + (b - a)f'(a) + \cdots + \frac{(b - a)^{n - 1}}{(n- 1)!}f^{(n - 1)}(a) + \frac{(b - a)^n}{n!}f^{(n)}(x) \\
 f(a + h) &= \underbrace{f(a) + hf'(a) + \cdots + \frac{h^{n - 1}}{(n - 1)!}f^{(n - 1)}(a)}_{(n - 1)\text{-degree approximation to }f\text{ near }a} + \underbrace{\frac{h^n}{n!}f^{(n)}(x)}_{\text{error term}}.
\end{align*}
for some $x\in (a, a + h)$.
This applies for $h < 0$ too by considering $g(x) = f(-x)$. This is the Lagrange form of the remainder. If $f^{(n)}$ is bounded by $C$ near $a$, then
\[
  \left|\frac{h^n}{n!}f^{(n)}(x)\right| \leq \frac{C}{n!}|h|^n = o(h^{n - 1}).
\]

Q: Eigenfunction of differential operator
A: Let $f: \R \to \R$ be a differentiable function such that $f(0) = 1$ and $f'(x) = f(x)$. Then for every $x$, we have
  \[
    f(x) =e^x= 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots = \sum_{n = 0}^\infty \frac{x^n}{n!}.
  \]
\begin{proof}
  Since $f'(x) =f(x)$, it follows that all derivatives exist. By Taylor's theorem,
  \[
    f(x) = f(0) + f'(0) x + \frac{f^{(2)}(0)}{2!}x^2 + \cdots + \frac{f^{(n - 1)}(0)}{(n - 1)!}x^{n - 1} + \frac{f^{(n)}(u)}{n!}x^n.
  \]
  for some $u$ between $0$ and $x$. So
  \[
    f(x) =\frac{f^{(n)}(u)}{n!}x^n + \sum_{k = 0}^{n - 1}\frac{x^k}{k!}.
  \]
  We just have to show that the remainder term $\frac{f^{(n)}(u)}{n!}x^n \to 0$ as $n \to \infty$. Note here that $x$ is fixed, but $u$ can depend on $n$.
We know $f^{(n)}(u) = f(u)$ and since $f$ is differentiable, it is continuous, and is bounded on $[0, x]$. Suppose $|f(u)| \leq C$ on $[0, x]$. Then
  \[
    \left|\frac{f^{(n)}(u)}{n!}x^n\right| \leq \frac{C}{n!}|x|^n \to 0
  \]
  from limit theorems.
\end{proof}
Conversely, the derivative of $e^z$ is $e^z$.
\begin{proof}
  \begin{align*}
    \frac{e^{z + h} - e^z}{h} &= e^z \left(\frac{e^h - 1}{h}\right)\\
    &= e^z\left(1 + \frac{h}{2!} + \frac{h^2}{3!} + \cdots\right)
  \end{align*}
  But
  \[
    \left|\frac{h}{2!} + \frac{h^2}{3!} + \cdots \right| \leq \frac{|h|}{2} + \frac{|h|^2}{4} + \frac{|h|^3}{8} + \cdots = \frac{|h|/2}{1 - |h|/2} \to 0.
  \]
  So
  \[
    \lim_{h\to 0} \frac{e^{z + h} - e^z}{h} = e^z.
  \]
\end{proof}

Q: Radius of convergence
A: Suppose that $\sum a_nz^n$ converges and $|w| < |z|$, then $\sum a_n w^n$ converges (absolutely).
\begin{proof}
  We know that
  \[
    |a_n w^n| = |a_nz^n|\cdot \left|\frac{w}{z}\right|^n.
  \]
  Since $\sum a_nz^n$ converges, the terms $a_nz^n$ are bounded. So pick $C$ such that
  \[
    |a_nz^n| \leq C
  \]
  for every $n$. Then
  \[
    0 \leq \sum_{n = 0}^\infty |a_nw^n| \leq \sum_{n = 0}^\infty C\left|\frac{w}{z}\right|^n,
  \]
  which converges (geometric series). So by the comparison test, $\sum a_nw^n$ converges (absolutely).
\end{proof}
It follows that if $\sum a_nz^n$ does not converge and $|w| > |z|$, then $\sum a_nw^n$ does not converge.
Now let $R = \sup\{|z|: \sum a_nz^n$ converges $\}$ ($R$ may be infinite but $0\in S$). If $|z| < R$, then we can find $z_0$ with $|z_0|\in (|z|, R]$ such that $\sum_n^\infty a_nz_0^n$ converges. So $\sum a_n z_0^n$ converges by the above result. If $|z| > R$, then $\sum a_nz^n$ diverges by definition of $R$. When $|z| = R$, the series can converge at some points and not the others.
  We call $R$ the \emph{radius of convergence} of the power series and $\{z: |z| < R\}$ the \emph{circle of convergence}.

Q: Radius of convergence by limit supremum
A: The radius of convergence of a power series $\sum a_nz^n$ is
  \[
    R = \frac{1}{\limsup \sqrt[n]{|a_n|}}.
  \]
  Often $\sqrt[n]{|a_n|}$ converges, so we only have to find the limit.
\begin{proof}
  Suppose $|z| < 1/\limsup \sqrt[n]{|a_n|}$. Then $|z| \limsup \sqrt[n]{|a_n|} < 1$. Therefore there exists $N$ and $\varepsilon > 0$ such that
  \[
    \sup_{n \geq N}|z|\sqrt[n]{|a_n|} \leq 1 - \varepsilon
  \]
  by the definition of $\limsup$. Therefore
  \[
    |a_n z^n| \leq (1 - \varepsilon)^n
  \]
  for every $n \geq N$, which implies (by comparison with geometric series) that $\sum a_n z^n$ converges absolutely.
  On the other hand, if $|z|\limsup\sqrt[n]{|a_n|} > 1$, it follows that $|z|\sqrt[n]{|a_n|} \geq 1$ for infinitely many $n$. Therefore $|a_nz^n| \geq 1$ for infinitely many $n$. So $\sum a_nz^n$ does not converge.
\end{proof}

Q: Radius of convergence by ratio
A: Consider $\displaystyle\sum_{n = 0}^\infty a_n z^n$ . The ratio of $(n + 1)$th term to $n$th is
  \[
    \frac{a_{n+1} z^{n + 1}}{a_nz^n } = z\cdot\frac{a_{n+1}}{a_n}=f(n) \to r.
  \]
  So if $|r| < 1$, then the series converges by the ratio test. If $|r| > 1$, then eventually the terms are increasing in modulus.
If $f(n)$ is constant and $|r|=1$, then the terms do not tend to zero, so the series diverges.
If $|r| = 1$, $r\not= 1$, $f(n)$ is decreasing and $f(n)\to 0$ then it converges by Abel's test.
Other cases must be considered separately.

Q: Convolution
A: onsider two sequences $(a_n), (b_n)$. Their \emph{convolution} is the sequence $(c_n)$ defined by
\[
  c_n = a_0b_n + a_1b_{n - 1} + a_2b_{n - 2} + \cdots + a_nb_0.
\]
The relevance of this is that if you take
\[
  \left(\sum_{n = 0}^N a_nz^n\right)\left(\sum_{n = 0}^N b_nz^n\right)\text{ and }\sum_{n = 0}^N c_n z^n,
\]
and equate coefficients of $z^n$, you get
\[
  c_n = a_0b_n + a_1b_{n - 1} + a_2b_{n - 2} + \cdots + a_nb_0.
\]

Q: The exponential function: definition
A: The \emph{exponential function} is
  \[
    e^z = \sum_{n = 0}^\infty \frac{z^n}{n!}.
  \]
We need to check this converges for all of $\C$.
The ratio of $(n + 1)$th term to $n$th is
  \[
    \frac{z^{n + 1}/(n + 1)!}{z^n /n!} = \frac{z}{n + 1}\to 0,
  \]
  so the series converges by the ratio test.

Q: Convolution of absolutely convergent series
A: Let $\sum_{n = 0}^\infty a_n$ and $\sum_{n = 0}^\infty b_n$ be two absolutely convergent series, and let $(c_n)$ be the convolution of the sequences $(a_n)$ and $(b_n)$. Then $\sum_{n = 0}^\infty c_n$ converges (absolutely), and
  \[
    \sum_{n = 0}^{\infty} c_n = \left(\sum_{n = 0}^\infty a_n\right)\left(\sum_{n = 0}^\infty b_n \right).
  \]
\begin{proof}
  We first show that a rearrangement of $\sum c_n$ converges absolutely. Hence it converges unconditionally, and we can rearrange it back to $\sum c_n$.
  Consider the series
  \begin{align*}
    &(a_0b_0) + \\
&(a_0 b_1 + a_1b_1 + a_1b_0)+ \\
&(a_0 b_2 + a_1 b_2 + a_2b_2 + a_2b_1 + a_2b_0) + \cdots\tag{$*$}
  \end{align*}
  Let
  \[
    S_N = \sum_{n = 0}^{N}a_n, \quad T_N = \sum_{n = 0}^N b_n,\quad U_N = \sum_{n = 0}^N | a_n|,\quad V_N \sum_{n = 0}^N|b_n|.
  \]
  Also let $S_N \to S, T_N \to T, U_N \to U, V_N \to V$.
  If we take the modulus of the terms of $(*)$, and consider the first $(N + 1)^2$ terms (ie. the first $N + 1$ brackets), the sum is $U_NV_N$. Hence the series converges absolutely to $UV$. Hence $(*)$ converges.
  The partial sum up to $(N + 1)^2$ of the series $(*)$ itself is $S_NT_N$, which converges to $ST$. So the whole series converges to $ST$.
  Since it converges absolutely, it converges unconditionally. Now consider a rearrangement:
  \[
    a_0 b_0 + (a_0b_1 + a_1b_0) + (a_0b_2 + a_1b_1 + a_2b_0) + \cdots
  \]
  Then this converges to $ST$ as well. But the partial sum of the first $1 + 2 + \cdots + N$ terms is $c_0 + c_1 + \cdots + c_N$. So
  \[
    \sum_{n = 0}^N c_n \to ST = \left(\sum_{n = 0}^\infty a_n\right)\left(\sum_{n = 0}^\infty b_n \right).
  \]
\end{proof}

Q: The exponentiation identity
A: \[
    e^z e^w = e^{z + w}.
  \]
\begin{proof}
  By theorem on convolution of absolutely convergent series (and definition of $e^z$),
  \begin{align*}
    e^z e^w &= \sum_{n = 0}^\infty \left(1\cdot \frac{w^n}{n!} + \frac{z}{1!}\frac{w^{n - 1}}{(n- 1)!} + \frac{z^2}{2!}\frac{w^{n - 2}}{(n- 2)!} + \cdots + \frac{z^{n}}{n!}\cdot 1\right)\\
    e^z e^w &= \sum_{n = 0}^\infty \frac{1}{n!}\left(w^n + \binom{n}{1} zw^{n - 1} + \binom{n}{2}z^2 w^{n - 2} + \cdots + \binom{n}{n}z^n\right)\\
    &= \sum_{n = 0}^\infty \frac{(z + w)^n}{n!} \text{ by the binomial theorem}\\
    &= e^{z + w}.
  \end{align*}
\end{proof}

Q: Sine and Cosine
A: \begin{align*}
    \sin z &= \frac{e^{iz} - e^{-iz}}{2i} = z - \frac{z^3}{3!} + \frac{z^5}{5!} - \frac{z^7}{7!} + \cdots\\
    \cos z &= \frac{e^{iz} + e^{-iz}}{2} = 1 - \frac{z^2}{2!} + \frac{z^4}{4!} - \frac{z^6}{6!}+ \cdots.
  \end{align*}

Q: Approximation of sine and cosine by their power series
A: We know $\sin 0=0$ and $\frac{\d}{\d x} \sin x=\cos x\leq 1$ so by the MVT, $\sin x\leq x$ for $x\geq 0$.
We also know $\cos 0 = 1$, and $\frac{\d}{\d x}\cos x = -\sin x\geq -x$ for $x \geq 0$. From this, it follows that when $x \geq 0$, $\cos x \geq 1 - \frac{x^2}{2}$ (the $1 - \frac{x^2}{2}$ comes from guessing, i.e. ``integrating'' $-x$).
Continuing in this way, we get that for $x \geq 0$, if you truncate the power series for $\sin x$ or $\cos x$, it will be $\geq \sin x, \cos x$ if you stop at a positive term, and $\leq$ if you stop at a negative term.

Q: Pi
A: We know that $\cos 0 = 1$ and
\[
  \cos 2 \leq 1 - \frac{2^2}{2!} + \frac{2^4}{4!} = 1 - 2 + \frac{2}{3} < 0.
\]
 It follows by the intermediate value theorem that there exists some $x\in (0, 2)$ such that $\cos x = 0$. Then $S=\{ x>0 \ | \ \cos x =0\}\neq\emptyset$.
Define $\pi=2\inf S$.
Claim: $\cos\frac{\pi}{2}=0$.
\begin{proof}
For all $n\in\N$: $\frac{\pi}{2}+\frac{1}{n}>\inf A$ so $\frac{\pi}{2}+\frac{1}{n}$ is not a LB, so $\exists x_n\in A: \frac{\pi}{2}\leq x_n < \frac{\pi}{2}+\frac{1}{n}.$
This defines $(x_n)$ such that $\forall n\in N: \cos x_n =0$, $(x_n)\to \frac{\pi}{2}$ and hence $(\cos x_n)\to \cos\frac{\pi}{2}=0$ by continuity.
\end{proof}
Since $\sin^2 z + \cos ^2 z = 1$, it follows that $\sin \frac{\pi}{2} = \pm 1$. Since $\cos x > 0$ on $[0, \frac{\pi}{2}]$, $\sin \frac{\pi}{2} \geq 0$ by the mean value theorem. So $\sin \frac{\pi}{2} = 1$.

Q: Derivative of a power series
A: Let $a_n z^n$ have radius of convergence $R$, and let $|z| < R$. Then $\sum na_n z^{n - 1}$ converges (absolutely).
\begin{proof}
  Pick $r$ such that $|z| < r < R$. Then $\sum a_n r^n$ converges absolutely, so the terms $|a_n|r^n$ are bounded above by, say, $C$. Now
  \[
    \sum n|a_n z^{n - 1}| = \sum n|a_n|r^{n - 1}\left(\frac{|z|}{r}\right)^{n - 1} \leq \frac{C}{r}\sum n\left(\frac{|z|}{r}\right)^{n - 1}.
  \]
  The series $\sum n\left(\frac{|z|}{r}\right)^{n - 1}$ converges, by the ratio test. So $\sum n|a_n z^{n - 1}|$ converges, by the comparison test.
\end{proof}
 Let $\sum a_n z^n$ be a power series with radius of convergence $R$. For $|z| < R$, let
  \[
    f(z) = \sum_{n = 0}^\infty a_n z^n\text{ and }g(z) = \sum_{n = 1}^\infty na_nz^{n - 1}.
  \]
  Then $f$ is differentiable with derivative $g$.
\begin{proof}
Fix $z: |z|<R$, $\delta>0$ such that $|z|+\delta<R$. Then for $|h|<\delta$
\begin{align*}
\left| \frac{f(z+h)-f(z)}{h} -g(z)\right|&=\left| \sum_{n=1}^\infty a_n\left[ \frac{(z+h)^n-z^n}{h}-nz^{n-1}\right] \right| \\
&\leq\sum_{n=1}^\infty |a_n| \left| \frac{(z+h)^n-z^n-hnz^{n-1}}{h}\right| \\
&=\sum_{n=1}^\infty |a_n| \sum_{k=2}^n {n\choose k} z^{n-k}h^{k-2}h\\
&\leq\sum_{n=1}^\infty |a_n| \sum_{k=2}^n {n\choose k} |z|^{n-k}\delta^{k} (|h|/\delta^2) \\
&\leq\sum_{n=1}^\infty \underbrace{|a_n| (|z|+\delta)^n}_{<|a_n|R^n} \underbrace{\frac{|h|}{\delta^2}}_{\to 0} \\
&\to 0\textrm{ as }h\to 0.
\end{align*}
\end{proof}

Q: Hyperbolic trigonometric functions
A: We define
  \begin{align*}
    \cosh z &= \frac{e^z + e^{-z}}{2} = 1 + \frac{z^2}{2!} + \frac{z^4}{4!} + \frac{z^6}{6!} + \cdots\\
    \sinh z &= \frac{e^z - e^{-z}}{2} = z + \frac{z^3}{3!} + \frac{z^5}{5!} + \frac{z^7}{7!} + \cdots
  \end{align*}

Q: Osborne's Rule
A: \begin{prop}
  \begin{align*}
\sinh z &= -i\sin iz \\
    \cosh z &= \cos iz
  \end{align*}
\end{prop}
So to convert an elliptical trigonometric identity to a hyperbolic trigonometric identity, let $\sin\to\sinh,\cos\to\cosh$ and flip the sign of any product of sines.

Q: Dissections
A: Let $[a, b]$ be a closed interval. A \emph{dissection} of $[a, b]$ is sequence $a = x_0 < x_1 < x_2 < \cdots < x_n = b$.

Q: Upper and lower sums
A: Given a dissection $\mathcal{D}$, the \emph{upper sum} and \emph{lower sum} are defined by the formulae
  \begin{align*}
    U_\mathcal{D} (f) &= \sum_{i = 1}^{n}(x_i - x_{i - 1}) \sup_{x\in [x_{i - 1}, x_{i}]}f (x)\\
    L_\mathcal{D} (f) &= \sum_{i = 1}^{n}(x_i - x_{i - 1}) \inf_{x\in [x_{i - 1}, x_{i}]}f (x)
  \end{align*}
  Sometimes we use the shorthand
  \[
    M_i = \sup_{x\in [x_{i - 1}, x_i]} f(x), \quad m_i = \inf_{x\in [x_{i - 1} - x_i]} f(x).
  \]
The upper sum is the total area of the upper rectangles, while the lower sum is the total area of the lower rectangles.
 By definition, clearly $U_{\mathcal{D}}f \geq L_{\mathcal{D}}f$. But in fact, upper sums of a function are \emph{always} larger than lower sums, regardless of how the interval is dissected.
 Let $\mathcal{D}_1$ and $\mathcal{D}_2$ be two dissections of $[a, b]$. Then
  \[
    U_{\mathcal{D}_1}f \geq L_{\mathcal{D}_2}f.
  \]
\begin{proof}
  Let $\mathcal{D}$ be the least common refinement (or indeed any common refinement). Then as $\mathcal{D}$ refines $\mathcal{D}_1$ and $\mathcal{D}$ refines $\mathcal{D}_2$:
  \[
    U_{\mathcal{D}_1}f \geq U_{\mathcal{D}}f \geq L_{\mathcal{D}}f \geq L_{\mathcal{D}_2}f.
  \]
\end{proof}

Q: Refining dissections
A: If $\mathcal{D}_1$ and $\mathcal{D}_2$ are dissections of $[a, b]$, we say that $\mathcal{D}_2$ refines $\mathcal{D}_1$ if every point of $\mathcal{D}_1$ is a point of $\mathcal{D}_2$. We write $\mathcal{D}_1\leq \mathcal{D}_2$.
\begin{lemma}
  If $\mathcal{D}_1\leq \mathcal{D}_2$, then
  \[
    U_{\mathcal{D}_2} f \leq U_{\mathcal{D}_1}f\text{ and }L_{\mathcal{D}_2 f}\geq L_{\mathcal{D}_1}f.
  \]
\end{lemma}
\begin{proof}
  Let $\mathcal{D}$ be $x_0 < x_1 < \cdots < x_n$. Let $\mathcal{D}_2$ be obtained from $\mathcal{D}_1$ by the addition of one point $z$. If $z\in (x_{i - 1}, x_i)$, then
  \begin{align*}
    U_{\mathcal{D}_2}f - U_{\mathcal{D}_1}f &= \left[(z - x_{i - 1}) \sup_{x\in [x_{i - 1}, z]} f(x)\right]\\
    &+ \left[(x_i - z)\sup_ {x\in[z, x_i]}f(x)\right] - (x_i - x_{i - 1})M_i \\
&\leq M_i (z - x_{i - 1}) + M_i (x_i - z)- (x_i - x_{i - 1})M_i \\
&=0 \\
\therefore U_{\mathcal{D}_2} f &\leq U_{\mathcal{D}_1}f.
  \end{align*}
  By induction, the result is true whenever $\mathcal{D}_2$ refines $\mathcal{D}_1$.
  A very similar argument shows that $L_{\mathcal{D}_2f} \geq L_{\mathcal{D}_1}f$.
\end{proof}

Q: Least common refinement
A: Let $\mathcal{D}_1$ and $\mathcal{D}_2$ be dissections of $[a, b]$. Then the least common refinement of $\mathcal{D}_1$ and $\mathcal{D}_2$ is the dissection made out of the points of $\mathcal{D}_1$ and $\mathcal{D}_2$.

Q: Riemann Integral
A: The \emph{upper integral} is
  \[
    \overline{\int_a^b} f(x)\;\d x = \inf_{\mathcal{D}}U_{\mathcal{D}}f.
  \]
  The \emph{lower integral} is
  \[
    \underline{\int_a^b} f(x)\;\d x = \sup_{\mathcal{D}}L_{\mathcal{D}}f.
  \]
  If these are equal, then we call their common value the \emph{Riemann integral} of $f$, and is denoted $\int_a^b f(x)\;\d x$.
  If this exists, we say $f$ is \emph{Riemann integrable}.

Q: Bounded vs. Riemann integrable
A: Boundedness is a necessary condition for Riemann integrability.
\begin{proof}
 If $f$ is unbounded above in $[a, b]$, then for any dissection $\mathcal{D}$, there must be some $i$ such that $f$ is unbounded on $[x_{i - 1}, x_i]$. So $M_i = \infty$. So $U_\mathcal{D} f = \infty$. Similarly, if $f$ is unbounded below, then $L_{\mathcal{D}} f = -\infty$. So unbounded functions are not Riemann integrable.
\end{proof}

Q: Proving integrals from first principles
A: `Guess' the answer.
Write each term in the sum as the correct part of the sum plus an error term.
Let the mesh be $<\delta$ and make the error term smaller than $\varepsilon$ by choosing a post-hoc value for $\delta$.

Q: Mesh
A: The \emph{mesh} of a dissection $\mathcal{D}$ is
\[ \max_i (x_i - x_{i-1}). \]

Q: Riemann's (or Cauchy's) integrability criterion
A: Let $f: [a, b] \to \R$. Then $f$ is Riemann integrable if and only if for every $\varepsilon > 0$, there exists a dissection $\mathcal{D}$ such that
  \[
    U_\mathcal{D} - L_\mathcal{D} < \varepsilon.
  \]
\begin{proof}
  $(\Rightarrow)$ Suppose that $f$ is integrable, so there exist $\mathcal{D}_1$ and $\mathcal{D}_2$ such that
  \[
    U_{\mathcal{D}_1} < \int_a^b f(x)\;\d x + \frac{\varepsilon}{2},
  \]
  and
  \[
    L_{\mathcal{D}_2} > \int_a^b f(x)\;\d x - \frac{\varepsilon}{2}.
  \]
  Let $\mathcal{D}$ be a common refinement of $\mathcal{D}_1$ and $\mathcal{D}_2$. Then
  \[
    U_\mathcal{D} f - L_\mathcal{D} f \leq U_{\mathcal{D}_1} f- L_{\mathcal{D}_2} f < \varepsilon.
  \]
  $(\Leftarrow)$ Conversely, if there exists $\mathcal{D}$ such that
  \begin{align*}
    U_\mathcal{D} f - L_\mathcal{D}f &< \varepsilon, \\
\textrm{then }\inf U_\mathcal{D} f - \sup L_\mathcal{D} f &< \varepsilon, \\
\textrm{i.e. } \overline{\int_a^b} f(x)\;\d x - \underline{\int_a^b} f(x)\;\d x &< \varepsilon.
  \end{align*}
  Since $\varepsilon > 0$ is arbitrary, this gives us that
  \[
    \overline{\int_a^b} f(x)\;\d x = \underline{\int_a^b} f(x)\;\d x.
  \]
  So $f$ is integrable.
\end{proof}

Q: Integral of $-f$
A: Let $f: [a, b] \to \R$ be integrable. Then $-f$ is integrable, and
  \[
    \int_a^b -f(x)\;\d x = -\int_a^bf(x)\;\d x.
  \]
\begin{proof}
  Let $\mathcal{D}$ be a dissection. Then
  \begin{align*}
    \sup_{x\in [x_{i - 1}, x_i]}-f(x) &= -\inf_{x\in [x_{i - 1}, x_i]} f(x)\\
    \inf_{x\in [x_{i - 1}, x_i]}-f(x) &= -\sup_{x\in [x_{i - 1}, x_i]} f(x).
  \end{align*}
  Therefore
  \begin{align*}
    U_\mathcal{D}(-f) = \sum_{i = 1}^n (x_i - x_{i - 1})(-m_i) = -L_\mathcal{D}(f) \\
 L_\mathcal{D}(-f) = \sum_{i = 1}^n (x_i - x_{i - 1})(-M_i) =-U_\mathcal{D}f.
  \end{align*}
  So
  \[
    U_\mathcal{D}(-f) - L_\mathcal{D}(-f) = U_\mathcal{D}f - L_\mathcal{D}f.
  \]
  Hence if $f$ is integrable, then $-f$ is integrable by the Riemann integrability criterion.
\end{proof}

Q: Integral of $\lambda f$, $\lambda>0$.
A: Let $f: [a, b] \to \R$ be integrable, and $\lambda \geq 0$. Then $\lambda f$ is integrable, and
  \[
    \int_a^b \lambda f(x)\;\d x = \lambda\int_a^b f(x)\;\d x.
  \]
\begin{proof}
  Let $\mathcal{D}$ be a dissection of $[a, b]$. First observe
  \begin{align*}
    \sup_{x\in [x_{i - 1}, x_i]}\lambda f(x) &= \lambda\sup_{x\in [x_{i - 1}, x_i]}f(x)\\
\inf_{x\in [x_{i - 1}, x_i]}\lambda f(x) &= \lambda\inf_{x\in [x_{i - 1}, x_i]}f(x).
  \end{align*}
Then
  \begin{align*}
    U_{\mathcal{D}}(\lambda f) &= \lambda U_{\mathcal{D}} f\\
    L_{\mathcal{D}}(\lambda f) &= \lambda L_\mathcal{D} f.
  \end{align*}
  So if we choose $\mathcal{D}$ such that $U_{\mathcal{D}}f - L_\mathcal{D} f < \varepsilon/\lambda$, then $U_\mathcal{D}(\lambda f) - L_\mathcal{D}(\lambda f) < \varepsilon$. So the result follows from Riemann's integrability criterion.
\end{proof}

Q: Integral of $f+g$
A: Let $f, g: [a, b] \to \R$ be integrable. Then $f + g$ is integrable, and
  \[
    \int_a^b(f(x) + g(x))\;\d x = \int_a^b f(x)\;\d x + \int_a^b g(x)\;\d x.
  \]
\begin{proof}
  Let $\mathcal{D}$ be a dissection. Then
  \begin{align*}
    U_\mathcal{D}(f + g) &= \sum_{i = 1}^n (x_i - x_{i - 1})\sup_{x\in [x_{i - 1}, x_i]}(f(x) + g(x))\\
    &\leq \sum_{i = 1}^n (x_i - x_{i - 1}) \left(\sup f(x) + \sup g(x)\right)\\
    &= U_\mathcal{D}f + U_\mathcal{D} g
  \end{align*}
  Therefore,
  \begin{align*}
    \overline{\int_a^b}(f(x) + g(x))\;\d x &\leq \overline{\int_a^b} f(x)\;\d x + \overline{\int_a^b} g(x)\;\d x \\
&= \int_a^b f(x)\;\d x + \int_a^bg(x)\;\d x.
  \end{align*}
  Similarly,
  \[
  \underline{\int_a^b}(f(x) + g(x))\;\d x \geq \int_a^b f(x)\;\d x + \int_a^b g(x)\;\d x.
  \]
  So the upper and lower integrals are equal, and the result follows.
\end{proof}

Q: Integrating inequalities
A: Let $f, g: [a, b] \to \R$ be integrable, and suppose that $f(x) \leq g(x)$ for every $x\in [a,b]$. Then
  \[
    \int_a^b f(x)\;\d x \leq \int_a^b g(x)\;\d x.
  \]
\begin{proof}
 If $f(x) \leq g(x)$  for every $x\in [x_{i-1},x_i]$, then
\begin{align*}
\sup_{x\in [x_{i-1},x_i]} f(x)&\leq\sup_{x\in[x_{i-1},x_i]} g(x) \\
\inf_{x\in [x_{i-1},x_i]} f(x)&\leq\inf_{x\in[x_{i-1},x_i]} g(x).
\end{align*}
So $U_\mathcal{D}(f)\leq U_\mathcal{D}(g)$ and $L_\mathcal{D}(f)\leq L_\mathcal{D}(g)$. Therefore
\begin{align*}
\inf_\mathcal{D}U_\mathcal{D}(f)&\leq \inf_\mathcal{D}U_\mathcal{D}(g) \\
\sup_\mathcal{D}L_\mathcal{D}(f)&\leq \sup _\mathcal{D}L_\mathcal{D}(g).
\end{align*}
The result follows.
\end{proof}
\begin{proof}(alternative) Since $g-f\geq 0$ :
\begin{align*}
\int_a^b g-\int_a^b f&=\int_a^b g-f \\
&\geq (b-a)\inf_{[a,b]} (g-f) \\
&\geq 0.
\end{align*}
\end{proof}

Q: Integrability of $|f|$
A: Let $f: [a, b] \to \R$ be integrable. Then $|f|$ is integrable.
\begin{proof}
  Note that we can write
  \[
    \sup_{x\in [x_{i - 1}, x_i]}f(x) - \inf_{x\in [x_{i - 1}, x_i]}f(x) = \sup_{u, v\in [x_{i - 1}, x_i]}|f(u) - f(v)|.
  \]
  So letting $f\mapsto |f|$:
\begin{align*}
\sup_{x\in [x_{i - 1}, x_i]}|f(x)| - \inf_{x\in [x_{i - 1}, x_i]}|f(x)| &= \sup_{u, v\in [x_{i - 1}, x_i]}||f(u)| - |f(v)|| \\
&\leq \sup_{u, v\in [x_{i - 1}, x_i]}|f(u)| - |f(v)|\\
&= \sup_{x\in [x_{i - 1}, x_i]}f(x) - \inf_{x\in [x_{i - 1}, x_i]} f(x).
\end{align*}
  So for any dissection $\mathcal{D}$, we have
  \[
    U_\mathcal{D} (|f|) - L_\mathcal{D}(|f|) \leq U_\mathcal{D}(f) - L_\mathcal{D}(f).
  \]
  So the result follows from Riemann's integrability criterion.
\end{proof}

Q: Difference in integrals
A: If
\[
  |f(x) - g(x)| \leq C,
\]
for every $x\in[a, b]$, then
\[
  \left|\int_a^bf(x)\;\d x - \int_a^b g(x)\;\d x\right| \leq C(b - a).
\]
\begin{proof}
\begin{align*}
\left|\int_a^bf(x)\;\d x - \int_a^b g(x)\;\d x\right|&=\left|\int_a^bf(x)-g(x)\;\d x\right| \\
&\leq \left|\int_a^b|f(x)-g(x)|\;\d x\right| \\
&\leq \left|\int_a^bC\;\d x\right| \\
&= C\left|\int_a^b\;\d x\right| \\
&= C(b-a) \\
\end{align*}
\end{proof}

Q: Additivity property
A: Let $f$ be a bounded function on $[a,b]$. Then
\begin{enumerate}
\item if $a<c<b$ and $f$ is integrable on $[a,c]$ and $[c,b]$, then it is integrable on $[a,b]$ and $\int_a^b f=\int_a^c f+\int_c^b f$.
\item If $f$ is integrable on $[a,b]$, then $\forall c,d \ ( a\leq c<d\leq b):$ $f$ is integrable on $[c,d]$.
\end{enumerate}
\begin{proof}
\begin{enumerate}
\item Let $\mathcal{D}_n', \mathcal{D}_n''$ be dissections of $[a,c]$, $[c,b]$ respectively such that
\begin{align*}
L_{\mathcal{D}_n'}(f),U_{\mathcal{D}_n'}(f)&\to \int_a^c f \\
L_{\mathcal{D}_n''}(f),U_{\mathcal{D}_n''}(f)&\to \int_c^b f
\end{align*}
as $n\to \infty$. Then $\mathcal{D}_n=\mathcal{D}_n'\cup \mathcal{D}_n''$ is a dissection of $[a,b]$ such that
\begin{align*}
L_{\mathcal{D}_n}(f)&=L_{\mathcal{D}_n'}(f)+L_{\mathcal{D}_n''}(f) \to \int_a^c f+\int_c^b f \\
U_{\mathcal{D}_n}(f)&=U_{\mathcal{D}_n'}(f)+U_{\mathcal{D}_n''}(f) \to \int_a^c f+\int_c^b f.
\end{align*}
\item Given $\varepsilon>0$, $\exists\mathcal{D}:a=x_0<x_1<\ldots x_{j-1}=c<\ldots x_k=d<\ldots x_n=b$ and $U_{\mathcal{D}}(f)-L_{\mathcal{D}}(f) <\varepsilon$. Then $\exists\mathcal{D}: x_{j-1}=c<\ldots x_k=d$ dissects $[c,d]$ such that
\begin{align*}
U_{\mathcal{D'}}(f)-L_{\mathcal{D'}}(f)&=\sum_{i=j}^k (x_i-x_{i-1})\left( \sup_{[x_{i-1},x_i]} f-\inf_{[x_{i-1},x_i]} f \right) \\
&\leq \sum_{i=1}^n (x_i-x_{i-1})\left( \sup_{[x_{i-1},x_i]} f-\inf_{[x_{i-1},x_i]} f \right) \\
&=U_{\mathcal{D}}(f)-L_{\mathcal{D}}(f)\\
&<\varepsilon.
\end{align*}
\end{enumerate}
\end{proof}

Q: Integrability of a product
A: \begin{prop}
  Let $f, g: [a, b] \to \R$ be integrable. Then $fg$ is integrable.
\end{prop}
\begin{proof}
  Let $|f(x)|, |g(x)| \leq C$ for every $x\in [a, b]$. Write $M_i$,$L_i$ and $m_i$, $\ell_i$ for the $\sup$ and $\inf$ of $f(x)$, $g(x)$ in $[x_{i - 1}, x_i]$. Now let $\mathcal{D}$ be a dissection and $u_i,v_i\in [x_{i - 1}, x_i]$. Then
  \begin{align*}
    &\quad \left|\sum_{i = 1}^n \big(x_i - x_{i - 1})(f(v_i)g(v_i) - f(u_i)g(u_i)\big)\right| \\
    &= \left|\sum_{i = 1}^{n}(x_i - x_{i - 1})\big(f(v_i)(g(v_i) - g(u_i)) + (f(v_i) - f(u_i))g(u_i))\big)\right|\\
    &\leq \sum_{i = 1}^n(x_i - x_{i - 1}) \big(C(L_i - \ell_i) + (M_i - m_i)C\big)\\
    &=C(U_\mathcal{D}g - L_\mathcal{D}g + U_\mathcal{D}f - L_\mathcal{D}f).
  \end{align*}
  Since $u_i$ and $v_i$ are arbitrary, this must be true for the sup and inf, i.e.
  \begin{align*}
    U_\mathcal{D}(fg) - L_\mathcal{D}(fg) &\leq C(U_\mathcal{D}f - L_\mathcal{D}f + U_\mathcal{D}g - L_\mathcal{D}g) \\
&<C \left(\frac{\varepsilon}{2C}+\frac{\varepsilon}{2C}\right) \\
&=\varepsilon.
  \end{align*}
\end{proof}

Q: Integrability vs continuity
A: \begin{thm}
  Every continuous function $f$ on a closed bounded interval $[a, b]$ is Riemann integrable.
\end{thm}
\begin{proof}
  wlog assume $[a, b] = [0, 1]$.
  Suppose the contrary. Let $f$ be non-integrable. This means that there exists some $\varepsilon$ such that for every dissection $\mathcal{D}$, $U_{\mathcal{D}} - L_{\mathcal{D}} > \varepsilon$. In particular, for every $n$, let $\mathcal{D}_n$ be the dissection $0, \frac{1}{n}, \frac{2}{n}, \cdots, \frac{n}{n}$.
  Since $U_{\mathcal{D}_n} - L_{\mathcal{D}_n} > \varepsilon$, there exists some interval $\left[\frac{k}{n}, \frac{k + 1}{n}\right]$ in which $\sup f - \inf f > \varepsilon$. Suppose this sup and inf are attained at $u_n$ and $v_n$ respectively. Then we have $|u_n - v_n| < \frac{1}{n}$ and $f(u_n) - f(v_n) > \varepsilon$.
  By Bolzano Weierstrass, $(u_n)$ has a convergent subsequence, say $(u_{n_i})$. Let $u_{n_i}\to \ell$. Since $|u_n - v_n| < \frac{1}{n}\to 0$, we must have $v_{n_i}\to \ell $. By continuity, we must have $f(u_{n_i}) \to f(\ell)$ and $f(v_{n_i}) \to f(\ell)$, but $f(u_{n_i})$ and $f(v_{n_i})$ are always apart by $\varepsilon$. Contradiction.
\end{proof}
\begin{proof}(alternative)
We use uniform continuity. Given $n\in\N$, we can choose $\delta>0$ such that \[
\forall x,y: |x-y|<\delta \implies |f(x)-f(y)|<\frac{1}{n}.
\]
Now choose $\mathcal{D}_n$ such that the mesh $<\delta$. Then for each interval $I\in\mathcal{D}_n$: $\sup_I-\inf_I f\leq \frac{1}{n}.$
Hence $U_{\mathcal{D}_n}f-L_{\mathcal{D}_n}f\leq \frac{1}{n}(b-a)\to 0$.
\end{proof}

Q: Uniform continuity
A: Let $A\subseteq \R$ and let $f: A\to \R$. Then $f$ is \emph{uniformly continuous} if
  \[
    (\forall \varepsilon)(\exists \delta > 0)(\forall x)(\forall y)\;|x - y| < \delta \Rightarrow |f(x) - f(y)| \leq \varepsilon.
  \]

Q: Integrability of uniformly continuous functions
A: Every uniformly continuous function $f$ on a closed bounded interval $[a, b]$ is Riemann integrable.
\begin{proof}
Let the mesh of $\mathcal{D}$ be less than $\delta$ such that $|x-y|<\delta\implies |f(x)-f(y)|<\displaystyle\frac{\varepsilon}{b-a}$. Then
\begin{align*}
U_\mathcal{D}(f)-L_\mathcal{D}(f)&< \sum_{i=1}^n(x_i-x_{i-1})\frac{\varepsilon}{b-a} \\
&=\frac{\varepsilon}{b-a}(b-a) \\
&=\varepsilon.
\end{align*}
\end{proof}

Q: Integrability of monotone functions
A: \begin{thm}
  Let $f: [a, b] \to \R$ be monotone. Then $f$ is Riemann integrable.
\end{thm}
\begin{proof}
  Let $\varepsilon > 0$. Let $\mathcal{D}$ be a dissection of mesh less that $\frac{\varepsilon}{f(b) - f(a)}$. Then
  \begin{align*}
    U_\mathcal{D} f - L_\mathcal{D}f &= \sum_{i = 1}^n (x_i - x_{i - 1})(f(x_i) - f(x_{i - 1}))\\
    &\leq \frac{\varepsilon}{f(b) - f(a)} \sum_{i = 1}^n (f(x_i) - f(x_{i - 1}))\\
    &= \varepsilon.
  \end{align*}
\end{proof}
Pictorially: to calculate the total area, we can stack the error areas together to get something of width $\frac{\varepsilon}{f(b) - f(a)}$ and height $f(b) - f(a)$. So the total area is just $\varepsilon$.

Q: Integrability of piecewise continuous functions
A: Let $a < b$ and let $f$ be a bounded function from $[a, b] \to \R$ that is continuous on $(a, b)$. Then $f$ is integrable.
\begin{proof}
  Let $\varepsilon > 0$. Suppose that $|f(x)| \leq C$ for every $x\in [a, b]$. Pick $x_1\in(a,b)$ such that $x_1 - a < \frac{\varepsilon}{8C}$. Also choose $x_{n-1}\in(a,b)$ such that $b - x_{n-1} < \frac{\varepsilon}{8C}$.
  Then $f$ is continuous on $[x_1, x_{n-1}]$. Therefore it is integrable on $[x_1, x_{n-1}] $. So we can find a dissection $\mathcal{D}'$ with points $x_1 < x_2 < \cdots < x_{n - 1} $ such that
  \[
    U_{\mathcal{D}'}f - L_{\mathcal{D}'}f < \frac{\varepsilon}{2}.
  \]
  Let $\mathcal{D}$ be the dissection $a = x_0 < x_1 < \cdots < x_n = b$. Then
  \[
    U_\mathcal{D} f - L_\mathcal{D} f < \frac{\varepsilon}{8C}\cdot 2C + \frac{\varepsilon}{2} + \frac{\varepsilon}{8C}\cdot 2C = \varepsilon.
  \]
\end{proof}
  Every piecewise continuous and bounded function $f$ on $[a, b]$ is integrable.
\begin{proof}
  Partition $[a, b]$ into intervals $I_1, \cdots, I_k$, on each of which $f$ is (bounded and) continuous. Hence, (as above) for every $I_j$ with end points $x_{j - 1}$, $x_j$, $f$ is integrable on $[x_{j - 1}, x_j]$. But then by the additivity property of integration, we get that $f$ is integrable on $[a, b]$.
\end{proof}

Q: Construction of a dissection whose limit is the Riemann integral
A: Let $f: [a, b] \to \R$ be Riemann integrable, and for each $n$, let $\mathcal{D}_n$ be the dissection $a = x_0 < x_1 < \cdots < x_n = b$, where $x_i = a + \frac{i(b - a)}{n}$ for each $i$. Then
  \[
    U_{\mathcal{D}_n}f \to \int_a^b f(x)\;\d x
  \]
  and
  \[
    L_{\mathcal{D}_n}f \to \int_a^b f(x)\;\d x.
  \]
\begin{proof}
  Let $\varepsilon > 0$. Let $C$ be such that $|f(x)| \leq C$.
  Since $f$ is integrable, there is a dissection $\mathcal{D}$, say $u_0 < u_1 < \cdots < u_m$, such that
  \[
    U_\mathcal{D} f - \int_a^b f(x)\;\d x < \frac{\varepsilon}{2}.
  \]
  For any $n$, let $\mathcal{D}'$ be the least common refinement of $\mathcal{D}_n$ and $\mathcal{D}$. Then $U_{\mathcal{D}'}f \leq U_\mathcal{D} f$.
  Also, the sums $U_{\mathcal{D}_n}f$ and $U_\mathcal{D'}f$ are the same, except that at most $m$ of the intervals $[x_{i - 1}, x_i]$ are subdivided in $\mathcal{D}'$. For each such interval, the upper sum decreases by at most $\frac{b - a}{n}\cdot 2C$ (i.e. max area of the strip). Therefore
  \[
    U_{\mathcal{D}_n}f - U_{\mathcal{D}'}f \leq \frac{b - a}{n}2C\cdot m.
  \]
  Pick $n$ such that $2Cm(b - a)/n < \frac{\varepsilon}{2}$. Then
  \[
    U_{\mathcal{D}_n} f - U_\mathcal{D}f < \frac{\varepsilon}{2}.
  \]
  So
  \[
    U_{\mathcal{D}_n} - \int_a^b f(x)\;\d x < \varepsilon.
  \]
 As $U_{\mathcal{D}_n} f \geq \int_a^b f(x)\;\d x$, therefore
  \[
    U_{\mathcal{D}_n}f \to \int_a^b f(x)\;\d x.
  \]
  Similarly for lower sums.
\end{proof}

Q: Fundamental Theorem of Calculus: proof
A: \textbf{Derivative of an integral.} Let $f: [a, b]\to \R$ be continuous. For $x\in [a, b]$, let $F(x) = \displaystyle\int_a^x f(t)\;\d t$. Then $F$ is differentiable and $F'(x) = f(x)$.
\emph{Proof.}
  \[
    \frac{F(x + h) - F(x)}{h} = \frac{1}{h}\int_x^{x + h}f(t)\;\d t
  \]
  Let $\varepsilon > 0$. Since $f$ is continuous, at $x$, then there exists $\delta$ such that $|y - x| < \delta$ implies $|f(y) - f(x)| < \varepsilon$. So if $|h| < \delta$, then
  \begin{align*}
    \left|\frac{1}{h}\int_x^{x + h}f(t) \;\d t - f(x)\right| &= \left|\frac{1}{h}\int_x^{x + h}(f(t) - f(x))\;\d t\right|\\
    &\leq \frac{1}{|h|}\left|\int_x^{x + h}|f(t) - f(x)|\;\d t\right|\\
    &\leq \frac{\varepsilon|h|}{|h|}= \varepsilon. \qed
  \end{align*}
\textbf{Integral of a derivative} (weak version). If $f$ is continuously differentiable on $[a, b]$, then $\displaystyle\int_a^b f'(t)\;\d t = f(b) - f(a).$
\emph{Proof.}
  Let $g(x) = \displaystyle\int_a^x f'(t)\;\d t$. Then $g'(x)=f'(x)$, i.e. $g'(x) - f'(x) = 0$, so $g(x) - f(x)=c$ by MVT. We know that $g(a) = 0$ so $c=-f(a)$. Then $g(x) = f(x) - f(a)$, in particular for $x = b$. \qed
\textbf{Integral of a derivative}. Let $f: [a, b] \to \R$ be a differentiable function, and suppose that $f'$ is integrable. Then $\displaystyle\int_a^b f'(t)\;\d t = f(b) - f(a)$.
\emph{Proof.}
  Let $\mathcal{D}$ be a dissection $x_0 < x_1 < \cdots < x_n$.
  \[
    f(b) - f(a) = \sum_{i = 1}^n (f(x_i) - f(x_{i - 1}))
  \]
 By MVT,  $f(x_i) - f(x_{i - 1}) = (x_i - x_{i - 1})f'(u_i)$. So
  \begin{align*}
    f(b) - f(a) = \sum_{i = 1}^n (x_i - x_{i - 1}) &\times f'(u_i). \\
\textrm{Clearly } \inf\limits_{x\in[x_i, x_{i - 1}]}f'(x)&<f'(u_i)<\sup\limits_{x\in[x_i, x_{i - 1}]}f'(x), \\
\textrm{so } L_\mathcal{D} f' &\leq f(b) - f(a) \leq U_\mathcal{D} f'.
  \end{align*}
  Since $f'$ is integrable and $\mathcal{D}$ was arbitrary, $L_\mathcal{D}f'$ and $U_\mathcal{D}f'$ can both get arbitrarily close to $\displaystyle\int_a^b f'(t)\;\d t$. So $f(b) - f(a) = \displaystyle\int_a^b f'(t)\;\d t$. \qed

Q: Integration by parts: proof
A: Let $f, g:[a, b]\to \R$ be integrable such that everything below exists. Then
  \[
    \int_a^b f(x)g'(x)\;\d x = f(b)g(b) - f(a)g(a) - \int_a^b f'(x)g(x)\;\d x.
  \]
\begin{proof}
  By the fundamental theorem of calculus,
  \[
    \int_a^b (f(x)g'(x) + f'(x)g(x))\;\d x = \int_a^b(fg)'(x)\;\d x = f(b)g(b) - f(a)g(a).
  \]
  The result follows after rearrangement.
\end{proof}

Q: Taylor's Theorem with the integral form of the remainder
A: Let $f$ be $n + 1$ times differentiable on $[a, b]$ with $f^{(n + 1)}$ continuous. Then
  \begin{align*}
    f(b) &= f(a) + (b - a)f'(a) + \frac{(b - a)^2}{2!}f^{(2)}(a) + \cdots \\
    &+ \frac{(b - a)^n}{n!}f^{(n)}(a) + \int_a^b \frac{(b - t)^n}{n!}f^{(n + 1)}(t)\;\d t.
  \end{align*}
\begin{proof}
  Induction on $n$.
  When $n = 0$, the theorem says
  \[
    f(b) - f(a) = \int_a^b f'(t)\;\d t.
  \]
  which is true by the FTC.
Using integration by parts
  \begin{align*}
    &\int_a^b \underbrace{\frac{(b - t)^n}{n!}}_{v'}\underbrace{f^{(n + 1)}(t)}_{u}\;\d t \\
= &\left[\frac{-(b - t)^{n + 1}}{(n + 1)!}f^{(n + 1)}(t)\right]_a^b+ \int_a^b \frac{(b - t)^{n + 1}}{(n + 1)!}f^{(n + 2)}(t)\;\d t \\
    = &-\frac{(b - a)^{n + 1}}{(n + 1)!}f^{(n+1)}(a) + \int_a^b \frac{(b - t)^{n + 1}}{(n + 1)!}f^{(n + 2)}(t)\;\d t.
  \end{align*}
\end{proof}
We guess the integral form of the remainder by integrating the FTC by parts as follows.
\begin{align*}
  f(b) &= f(a) + \int_a^b 1 \cdot f'(t)\;\d t\\
  &= f(a) + [tf'(t)]_a^b - \int_a^b tf^{(2)}(t)\;\d t\\
  &= f(a) + bf'(b) - af'(a) - \int_a^b tf^{(2)}(t)\;\d t\\
  &= f(a) + (b - a)f'(a) + b(f'(b) - f'(a)) - \int_a^b tf^{(2)}(t)\;\d t\\
&= f(a) + (b - a)f'(a) + b\int_a^b f^{(2)}(t)\;\d t - \int_a^b tf^{(2)}(t)\;\d t\\
  &= f(a) + (b - a)f'(a) + \int_a^b (b - t)f^{(2)}(t)\;\d t.
\end{align*}
Then integrate $(b - t)$ etc.

Q: Integration by substitution
A: Let $f: [a, b] \to \R$ be continuous. Let $g: [u, v] \to \R$ be continuously differentiable, and suppose that $g(u) = a, g(v) = b$, and $f$ is defined everywhere on $g([u, v])$ (and still continuous). Then
  \[
    \int_a^b f(x)\;\d x = \int_u^v f(g(t))g'(t)\;\d t.
  \]
\begin{proof}
  By the fundamental theorem of calculus, $f$ has an anti-derivative $F$ defined on $g([u, v])$. Then
  \begin{align*}
    \int_u^v f(g(t))g'(t) \d t &= \int_u^v F'(g(t))g'(t)\;\d t \\
    &= \int_u^v (F\circ g)'(t)\;\d t \\
    &= F\circ g(v) - F\circ g(u)\\
    &= F(b) - F(a)\\
    &= \int_a^b f(x)\;\d x.
  \end{align*}
\end{proof}

Q: Improper integral
A: Suppose that we have a function $f: [a, b] \to \R$ such that, for every $\varepsilon > 0$, $f$ is integrable on $[a + \varepsilon, b]$ and $\lim\limits_{\varepsilon \to 0}\int_{a + \varepsilon}^b f(x)\;\d x$ exists. Then we define the improper integral
  \[
    \int_a^bf(x)\;\d x=\lim_{\varepsilon \to 0}\int_{a + \varepsilon}^b f(x)\;\d x.
  \]
  We can do similarly for $[a, b - \varepsilon]$, or integral to infinity:
  \[
    \int_a^\infty f(x)\;\d x = \lim_{b \to \infty} \int_a^b f(x)\;\d x.
  \]
  when it exists.

Q: Integral test
A: Let $f: [1, \infty] \to \R$ be a decreasing non-negative function. Then $\sum_{n = 1}^\infty f(n)$ converges iff \[
\int_1^\infty f(x)\;\d x < \infty.\]
\begin{proof}
  For $n>2$ we have
  \[
    \int_n^{n + 1}f(x)\;\d x \leq f(n) \leq \int_{n -1}^n f(x)\;\d x,
  \]
  since $f$ is decreasing. Summing from $n=2$ to $n=N$ and dealing separately with $n=1$ gives
  \[
    \int_1^{N + 1}f(x)\;\d x \leq \sum_{n = 1}^N f(n) \leq \int_1^N f(x)\;\d x + f(1).
  \]
  So if the integral exists, then $\sum f(n)$ is increasing and bounded above by $\int_1^\infty f(x)\;\d x+f(1)$, so converges.
  If the integral does not exist, then $\int_1^N f(x)\;\d x$ is unbounded. Then $\sum_{n = 1}^N f(n)$ is unbounded, hence does not converge.
\end{proof}

Q: Limit of a subsequence
A: If $x_n\to x$, then $x_{k_n}\to x$.
\begin{proof}
Given $\varepsilon>0$, we know \[
\exists N:\forall n\geq N: |x_n-x|<\varepsilon.\]
Since $k_n$ is strictly increasing, we have  \[
k_n\geq n\geq N.\]
Therefore \[
|x_{k_n}-x|<\varepsilon.\]
\end{proof}

Q: Density of the rationals in the reals
A: $\forall x\in \R: \exists (q_n)\in \Q: q_n\to x$
\begin{proof}
\begin{align*}
\lfloor nx \rfloor &\leq nx \leq \lfloor nx \rfloor +1 \\
\frac{\lfloor nx \rfloor}{n} &\leq x \leq \frac{\lfloor nx \rfloor}{n} +\frac{1}{n}
\end{align*}
So let $q_n=\frac{\lfloor nx \rfloor}{n}$. Then \[
q_n\leq x\leq q_n+\frac{1}{n}
\]
so by the Archimedean property \[
|q_n-x|<\frac{1}{n}<\varepsilon
\]
for all $\varepsilon>0$.
\end{proof}
$\forall x<y \in \R: \exists q\in\Q:x<q<y$
\begin{proof}
Pick $z\in\R$ such that $x<z<y$ (e.g. $z=\frac{x+y}{2}$). Then choose $(q_n)\in\Q$ such that $q_n\to z$. Select also $\varepsilon>0$ such that \[
x<z-\varepsilon<z<z+\varepsilon<y, \]
e.g. $\varepsilon=\frac{1}{2}\min(z-x,y-z)$ . Therefore \[
\exists N\in\N:\forall n\geq N: z-\varepsilon<q_n<z+\varepsilon
\]
and in particular
\[
\exists q_N\in\Q: x< z-\varepsilon<q_N<z+\varepsilon<y.
\]
\end{proof}

Q: Density of the irrationals in the reals
A: \[\forall x<y\in\R: \exists r\in R\setminus\Q: x<r<y \]
\begin{proof}
As the rationals are dense in the reals, we can choose $q\in\Q:x<q<y$. Then by the Archimedean property
\[
\exists N: x<q+\frac{\sqrt{2}}{N}<y
\]
by choosing $\frac{\sqrt{2}}{N}<\min(y-q,q-x)$.
Finally we check $r=q+\frac{\sqrt{2}}{N}\not\in\Q$ else $(r-q)N=\sqrt{2}\in\Q$.
\end{proof}

Q: Comparison of sequences
A: Let $(x_n),(y_n)$ be convergent, real sequences such that $\forall n\in\N: x_n\leq y_n$. Then \[
\lim_{n\to\infty} x_n \leq \lim_{n\to\infty} y_n .
\]
\begin{proof}
Let $x=\lim_{n\to\infty} x_n, y=\lim_{n\to\infty} y_n$.
Now suppose otherwise, so $y<x$. Then choose $\varepsilon>0$ s.t. $2\varepsilon< x-y$ (e.g. $\varepsilon=\frac{1}{4}(x-y))$. Then $\exists n\in\N$ s.t. $|x_n-x|<\varepsilon,|y_n-y|<\varepsilon$. Therefore
\begin{align*}
y_n&<y+\varepsilon \\
&<(x-2\varepsilon)+\varepsilon \\
&=x-\varepsilon \\
&<x_n \\
&\leq y_n.
\end{align*}
That is, $y_n<y_n$, which is a contradiction.
\end{proof}

Q: L'H\^opital's Rule: proof
A: Let $a,r\in\R$ ($r>0$). Let $f,g:(a-r,a+r)\setminus \{ a \} \to\R$ be differentiable functions such that \[
0=\lim_{x\to a} f(x)=\lim_{x\to a} g(x).
\]
If $\forall x \in (a-r,a+r): g'(x)\neq 0$ and $\lim_{x\to a} \frac{f'(x)}{g'(x)}$ exists, then \[
\lim_{x\to a} \frac{f(x)}{g(x)}=\lim_{x\to a} \frac{f'(x)}{g'(x)}.
\]
\begin{proof}
Set $f(a)=g(a)=0$. Then $f,g$ are continous on $(a-r,a+r)$.
Given $ x \in (a-r, a+r) \setminus \{ a\}$, assume $ a<x$ ($x<a$ similar).
Since $g$ is continuous on $[a,x]$ and differentiable on a $(a,x)$, then by the mean value theorem
\begin{align*}
\frac{g(x)-g(a)}{x-a}&=g'(c)\textrm{ for some }c\in (a,x) \\
&\neq 0 \\
\therefore g(x)&\neq 0.
\end{align*}
Claim: $\forall x \in (a-r,a+r)\setminus \{ a \}, \exists y$ such that $a<y<x$ and \[
\frac{f(x)}{g(x)}=\frac{f(y)}{g(y)}.
\]
Proof: Consider $h(t)=f(x)g(t)-g(x)f(t)$. $h$ is continuous on $[a,x]$, differentiable on $(a,x)$ and $h(a)=h(x)=0$. By Rolle's Theorem, $\exists y\in (a,x): h'(y)=0,$ as required.
Let $\ell=\lim_{x\to a} \frac{f'(x)}{g'(x)}$.
So, given $\varepsilon>0, \exists\delta>0: 0<|x-a|<\delta \implies \left| \frac{f'(x)}{g'(x)} - \ell \right|<\varepsilon $.
Given $x$ s.t. $0<|x-a|<\delta$, let $y$ be such that $a<y<x$ and \[
\frac{f(x)}{g(x)}=\frac{f'(y)}{g'(y)}
\]
by the claim above.
So
\begin{align*}
\left|\frac{f(x)}{g(x)}- \ell \right| &= \left|\frac{f'(y)}{g'(y)}- \ell \right|  \\
&<\varepsilon.
\end{align*}
\end{proof}

Q: Real-valuedness of the exponential function
A: $\overline{\exp(z)}=\exp(\bar z )$
\begin{proof}
\begin{align*}
\textrm{LHS}&=\left( \lim_{N\to\infty} \sum_{n=0}^N \frac{z^n}{n!}\right)* \\
&= \lim_{N\to\infty} \sum_{n=0}^N \frac{ ( \bar z)^n}{n!} \textrm{ by continuity of conjugation} \\
&=\textrm{RHS}.
\end{align*}
\end{proof}
$\forall x\in\R: \exp(x)\in\R$
\begin{proof}
\begin{align*}
\overline{\exp(x)} &=\exp(\bar x )\\
&=\exp(x) (x\in\R) \\
\therefore\exp(x)&\in\R.
\end{align*}
\end{proof}

Q: Exponential of imaginary numbers
A: $\forall x\in\R: \exp(ix)\in\mathbb{T}=\{ z\in\C: |z|=1\}$
\begin{proof}
\begin{align*}
|\exp (ix)|^2&=\exp(ix)\overline{\exp(ix)} \\
&=\exp(ix)\exp(-ix) \\
&=\exp(0) \\
&=1.
\end{align*}
\end{proof}

Q: Shape of exponential function
A: $\exp:\R\to (0,\infty)$ is a strictly increasing, differentiable bijection. $\exp(0)=1$, $\exp(x)\to\infty$ as $x\to\infty$ and $\exp(x)\to 0$ as $x\to -\infty$.
\begin{proof}
Letting $x=0$ in power series definition gives gives $\exp(0)=1$. Therefore $\forall z\in\C$
\begin{align*}
1&=\exp(0) \\
&=\exp(z-z) \\
&=\exp(z)\exp(-z).
\end{align*}
So $\exp(z)\neq 0$. Thus by IVT $\forall x\in\R: \exp(x)>0$. Then $\exp'(x)=\exp(x)>0$ so $\exp(x)$ is a strictly increasing function by MVT. So $\exp(x)$ is injective.
By contracting the power series $\forall x\geq 0: \exp(x)\geq 1+x$. So by comparison $\exp(x)\to\infty$ as $x\to\infty$. Similarly $\forall x\leq 0: \exp(x)=\frac{1}{\exp(-x)}<\frac{1}{1+x} \to 0$ as $x\to -\infty$.
Given $y\in (0,\infty)$, we can choose $n\in\N$ such that $\frac{1}{n}<y<n$ by Archimedes. Then
\begin{align*}
\exp(n)&\geq 1+n \\
&>y \\
\exp(-n)&=\frac{1}{\exp(n)} \\
&\leq \frac{1}{1+n} \\
&<\frac{1}{n} \\
&<y.
\end{align*}
So by IVT $\exists x\in (-n,n): \exp(x)=y$. Thus $\exp:\R\to (0,\infty)$ is surjective.
\end{proof}

Q: Natural logarithm
A: $\log(0,\infty)\to\R$ (defined as the inverse of the exponential function) is a strictly increasing, differentiable bijection.
$\forall x,y>0: \log'(y)=\frac{1}{y}, \log 1=0, \log(xy)=\log x + \log y,\log x \to\infty$ as $x\to\infty$ and $\log x\to -\infty$ as $x\to 0^+$.
\begin{proof}
If $0<x<y$ and $\log x\geq \log y$, then $x=\exp(\log x) \geq \exp(\log y)=y$, which is a contradiction. So $\log$ is strictly increasing.
Since $\exp'(x)=\exp(x)\neq 0$ by IFT $\log'(y)=\frac{1}{\exp'(\log y)} =\frac{1}{y}$.
$\log 1=0$ follows from $\exp(0)=1$.
$\log x+\log y=\log (xy)$ follows from $\exp(\log x +\log y)= x+y$.
Since $\log$ and $\exp$ are strictly increasing, $\log y>x\iff y>\exp x$ and $\log y< x\iff y<\exp x$. It follows that $\log x \to\infty$ as $x\to\infty$ and $\log x\to -\infty$ as $x\to 0^+$.
\end{proof}

Q: Exponentiation
A: We define $x^\alpha=\exp(\alpha\log x)$. Then
\begin{enumerate}
\item For $n\in\N$ $x^n=x\cdot x \cdots x$ and $y=x^{1/n}$ is the unique positive $y$ such that $y^n=x$.
\item $x^{-\alpha}=\frac{1}{x^\alpha}$ .
\item For $\alpha>0$ ($\alpha <0$), $x^\alpha:(0,\infty)\to (0,\infty)$ is a strictly increasing (decreasing) differentiable bijection with $\frac{\d}{\d x}x^\alpha =\alpha x^{\alpha -1}$.
\item $\forall x,y >0, \forall \alpha, \beta \in \R: (xy)^\alpha=x^\alpha y^\alpha, x^{\alpha+\beta}=x^\alpha x^\beta,(x^\alpha)^\beta=x^{\alpha\beta},1^\alpha=1$.
\end{enumerate}
\begin{proof}
\begin{enumerate}
\item
\begin{align*}
x^n&=\exp(n\log x) \\
&=\exp(\underbrace{\log x+\cdots +\log x}_n)  \\
&=\underbrace{\exp(\log x)\cdots \exp(\log x)}_n \\
&=\underbrace{x\cdot x\cdots x}_n \\
(x^{\frac{1}{n}})^n&=\left[ \exp\left(\frac{1}{n}\log x\right) \right]^n \\
&=\underbrace{\left[ \exp\left(\frac{1}{n}\log x\right) \right]\cdots \left[ \exp\left(\frac{1}{n}\log x\right) \right]}_{n} \\
&=\left[ \exp\left(\underbrace{\frac{1}{n}\log x +\cdots +\frac{1}{n}\log x}_n\right) \right] \\
&=\exp(\log x) \\
&=x.
\end{align*}
\item
\begin{align*}
x^{-\alpha}&=\exp((-\alpha)\log x) \\
&=\exp(-\alpha\log x) \\
&=1/ \exp(\alpha\log x)] \\
&=1/ x^\alpha. \\
\end{align*}
\item Immediate from properties of $\log, \exp$.
\item Immediate from properties of $\log, \exp$.
\end{enumerate}
\end{proof}

Q: Exponential vs polynomial
A: For all $\alpha\in\R$: $\frac{x^\alpha}{\exp x} \to 0$ as $x\to\infty$.
\begin{proof}
Choose $n\in\N$ such that $n>\alpha$. Then, for $x>0$, $\exp(x)>\frac{x^n}{n!}$. Thus
\begin{align*}
\frac{x^\alpha}{\exp x} &< \frac{x^\alpha}{x^n/n!} \\
&= n!x^{\alpha -n} \\
&= n!\exp [(\alpha -n)\log x]\\
&\to 0\textrm{ as }x\to\infty.
\end{align*}
\end{proof}

Q: Logarithm vs polynomial
A: For $\alpha>0$: $\frac{\log x}{x^\alpha}\to 0$ as $x\to\infty$.
\begin{proof}
Let $y=\log (x^\alpha)=\alpha\log x \to\infty$ as $x\infty$. Then
\begin{align*}
\frac{\log x}{x^\alpha} &=\frac{\alpha \log x}{\alpha x^\alpha} \\
&= \frac{y}{\alpha\exp y} \to 0\textrm{ as }x\to\infty.
\end{align*}
\end{proof}

Q: Graphs of $\sin, \cos$
A: We know $\cos x>0$ for $x\in (0,\frac{\pi}{2})$ by definition. Thus $\sin'(x)=\cos x>0$ on $[0,\frac{\pi}{2}]$. But $\sin 0=0$ so $\sin(x)>0$ on $[0,\frac{\pi}{2}]$. Also $\cos'(x)=-\sin x<0$ on $[0,\frac{\pi}{2}]$.
Then $\sin^2\frac{\pi}{2}=1-\cos^2\frac{\pi}{2}=1$, so $\sin\frac{\pi}{2}=1$.
Next $\sin\pi=2\sin\frac{\pi}{2}\cos\frac{\pi}{2}=0$ and $\cos\pi=\cos^2\frac{\pi}{2}-\sin^2\frac{\pi}{2}=-1$.
Now $\sin(\pi-x)=\sin\pi\cos(-x)-\cos\pi\sin(-x)=\sin x$. Letting $x\mapsto \frac{\pi}{2}+x$ and $x\mapsto -x$ give $\sin(\frac{\pi}{2}-x)=\sin(\frac{\pi}{2}+x)$ and $\sin(\pi+x)=\sin(-x)=-\sin(x)=-\sin(\pi-x)$. Letting $x\to x+\pi$ in this last result gives $\sin(2\pi+x)=\sin(x)$. By induction on this result $\forall n\in\Z: \sin(2\pi n+x)=\sin x$.
Finally $\sin(x+\frac{\pi}{2})=\sin x\cos\frac{\pi}{2} +\cos x\sin\frac{\pi}{2}=\cos x$.
This gives all the symmetry and periodicity properties.

Q: Periodicity of the exponential function
A: $\forall z\in \C: e^z=1 \iff z\in 2\pi i\Z$. Obvious corollary: $e^z=e^w \iff z-w\in 2\pi i\Z$.
\begin{proof}
If $z=2\pi i n$ ($n\in Z$), then
\begin{align*}
e^z&=e^{2\pi n} \\
&=\cos(2\pi n)+i\sin(2\pi n) \\
&=\cos 0 + i\sin 0 \\
&=1.
\end{align*}
Conversely, assume $e^z=1$. Write $z=x+iy$. Comparing moduli $e^x=1$ so $x=0$. Now comparing real parts $\cos y=1$.
Since $\cos$ is strictly decreasing on $[0,\pi]$,  $0$ is the only root of $\cos y=1$ in this interval. Since $\cos$ is anti-symmetric in $x=\pi$, $0$ is the only root in $[0,2\pi)$. So, by periodicity, $y\in 2\pi \Z$ and thus $z\in 2\pi i \Z$.
\end{proof}

Q: Taylor's Theorem with Peano form of the remainder
A: Let $n\in N, a\in R, f$ be $n$ times differentiable at $a$. Then $\exists\delta>0:\exists R_n(-\delta, \delta)\to\R: \forall h\in(-\delta,\delta)$: \[
f(a+h)=f(a)+f'(a)h+\frac{f'(a)h^2}{2}+\cdots +\frac{f^{(n)}(a)h^n}{n!}+R_n(h)
\]
where $\frac{R_n(h) }{h^n} \to 0$ as $h\to 0$.
\begin{proof}(by L'H\^opital)
By definition, $\exists\delta>0$ such that $f$ is defined and $n-1$ times differentiable on $(a-\delta, a+\delta)$ and $f^{(n)}(a)$ exists. We define $R_n(h)=f(a+h)-\sum_{k=0}^n \frac{f^{(k)}(a)h^k}{k!}$ for $|h|<\delta$.
($n=1$) $\frac{R_1(h)}{h}=\frac{f(a+h)-f(a)}{h}-f'(a) \to 0$ as $h\to 0$.
($n\geq 2)$ For $k=0,1\ldots n-1$:
\begin{align*}
R_n^{(k)}(h)&= f^{(k)}(a+h)-\sum_{l=k}^n \frac{f^{(l)}(a)}{l!}l(l-1)\cdots (l-k+1)h^{l-k} \\
R_n^{(k)}(0)&=f^{(k)}(a)-f^{(k)}(a) \\
&=0.
\end{align*}
Let $g(h)=h^n$. Then $g^{(k)}(h)=n(n-1)\cdot (n-k+1)h^{n-k}$. So $g^{(k)}(0)=0$ and $g^{(k)}(h)\neq0$ for $h\neq 0$.
\begin{align*}
\frac{R_n^{(n-1)}(h)}{g^{(n-1)}(h)} &=\frac{f^{(n-1)}(a+h)-f^{(n-1)}(a)-hf^{(n-1)}(a)}{n!h} \\
 &=\frac{1}{n!} \left[ \frac{f^{(n-1)}(a+h)-f^{(n-1)}(a)}{h}-f^{(n)}(a) \right]  \\
&\to 0\textrm{ as }h\to 0.
\end{align*}
Finally applying L'H\^opital's Rule $n-1$ times, we find that $\frac{R_n(h) }{h^n} \to 0$ as $h\to 0$.
\end{proof}

Q: Linear approximation to integrals
A: Suppose $f:[a,b]\to R$ is integrable and let $m=\inf_{[a,b]}f,M=\sup_{[a,b]}f.$ Then \[
m(b-a)\leq \int_a^b f\leq M(b-a)
\]
and \[
\left| \int_a^b f\right| \leq (b-a) \sup_{[a,b]}|f|.
\]
\begin{proof}
Consider $\mathcal{D}: a=x_0<x_1=b$. Then \[
L_\mathcal{D}(f)=m(b-a) \leq \int_a^b f\leq M(b-a)=U_\mathcal{D}(f).
\]
For the second part simply note that \[
M\leq \sup_{[a,b]}|f|,  m\geq - \sup_{[a,b]}|f|.
\]
\end{proof}

Q: Continuous functions on a closed, bounded interval are uniformly continuous
A: Let $f:[a,b]\to\R$ be continuous. Then \[
\forall\varepsilon>0,\exists\delta>0:\forall x,y\in[a,b]: |y-x|<\delta \implies |f(y)-f(x)|<\varepsilon.
\]
\begin{proof}
Suppose otherwise, then \[
\exists\varepsilon>0,\forall\delta>0:\exists x,y\in[a,b]: |y-x|<\delta, |f(y)-f(x)|\geq\varepsilon.
\]
In particular  \[
\exists\varepsilon>0,\forall n\in\N:\exists x_n,y_n\in[a,b]: |y_n-x_n|<\frac{1}{n}, |f(y_n)-f(x_n)|\geq\varepsilon.
\]
$(x_n)$ is bounded so by B-W $\exists(k_n)$ such that $(x_{k_n})\to x\in\R$. Since $a \leq x_{k_n}\leq b$, we must have $x\in[a,b]$.
But $|y_{k_n}-x_{k_n}|<\frac{1}{k_n}\leq \frac{1}{n}\to 0$ as $n\to\infty$. So $y_{k_n}=x_{k_n}+(y_{k_n}-x_{k_n})\to x$ as $n\to\infty$.
By continuity, $(f(x_{k_n}))\to f(x)$ and $(f(y_{k_n}))\to f(x)$, contradicting that $|f(y_{k_n})-f(x_{k_n})|\geq\varepsilon$.
\end{proof}

Q: Cauchy-Schwarz inequality for integration
A: Integrating $(f-\lambda g)^2\geq 0$ gives
\begin{align*}
0&\leq \int_a^b (f-\lambda g)^2\\
&=\int_a^b f^2+\lambda^2\int_a^b g^2-2\lambda\int_a^b fg.
\end{align*}
Picking $\lambda=\frac{\int_a^b fg}{\int_a^b g^2}$ gives
\begin{align*}
0&\leq \int_a^b f^2+\left[\frac{\int_a^b fg}{\int_a^b g^2}\right]^2\int_a^b g^2-2\frac{\int_a^b fg}{\int_a^b g^2}\int_a^b fg \\
0&\leq \int_a^b f^2\int_a^b g^2+\left[\int_a^b fg \right]^2-2\left[\int_a^b fg\right]^2 \\
0&\leq \int_a^b f^2\int_a^b g^2-\left[\int_a^b fg\right]^2.
\end{align*}
This leads to the Cauchy-Schwarz inequality \[
\left|\int_a^b fg\right| \leq \sqrt{\int_a^b f^2\int_a^b g^2}.
\]
If $\int_a^b g^2=0$, then \[
0\leq \int_a^b f^2-2\lambda\int_a^b fg
\]
for all $\lambda\in\R$, which forces $\int_a^b fg=0$ so the inequality still holds.

Q: Triangle inequality for integration
A: Integrating $\pm f\leq |f|$ gives \[
\pm\int_a^b f=\int_a^b \pm f\leq \int_a^b |f|.
\]
Hence
 \[
\left| \int_a^b f \right| \leq \int_a^b |f|.
\]

Q: Integral of mostly equivalent functions
A: \begin{enumerate}
\item Let $h:[a,b]\to\R$ satisfy $\forall x\in[a,b]\setminus\{c_0=a,c_1,c_2\ldots c_n=b \}: h(x)=0$. Then $h$ is integrable and $\int_a^b h=0$.
\item Let $f:[a,b]\to\R$ be integrable and $\forall x\in[a,b]\setminus\{c_0=a,c_1,c_2,\ldots c_n=b \}: f(x)=g(x)$. Then $g$ is integrable and $\int_a^b f=\int_a^b g$.
\end{enumerate}
\begin{proof}
\begin{enumerate}
\item If $M=\max_{0\leq i\leq n} |h(c_i)|$, then $\forall x: |h(x)|\leq M$ so $h$ is bounded. Fix $\delta>0$ such that $\forall k: \delta<\frac{1}{2}(c_k-c_{k-1})$. Consider $\mathcal{D}:c_0=a<a+\delta<c_1-\delta<c_1+\delta<c_2-\delta\ldots <b-\delta<b=c_n$. Then for $1\leq k\leq n$: \[
\inf_{[c_{k-1}+\delta,c_k-\delta]} h=\sup_{[c_{k-1}+\delta,c_k-\delta]} h =0.
\]
If $I=[c_k-\delta,c_k+\delta]$, $I=[a,a+\delta]$ or $I=[b-\delta,b]$, then
\begin{align*}
\sup_I h&\leq M \\
\inf_I h&\geq -M.
\end{align*}
Hence
\begin{align*}
L_\mathcal{D}(h)&\geq (n-1) 2\delta (-M) +2\delta (-M) \\
&=-2M\delta n \\
U_\mathcal{D}(h) &\geq (n-1)2\delta (M) +2\delta (M) \\
&-2M\delta n.
\end{align*}
Since this is true for arbitrarily small $\delta$, the result follows.
\item Using the additivity property and the first result \[
\int_a^b g=\int_a^b f + \int_a^b g-f =\int_a^b f. \]
\end{enumerate}
\end{proof}

Q: Continuity of indefinite integrals
A: Let $f$ be integrable on $[a,b]$. Then $F=\int_a^x f(t)\d t$ is continuous on $[a,b]$.
\begin{proof}
Let $M=\sup_{[a,b]}|f|$. Given $\varepsilon>0$ and $\forall x\in[a,b]:$
\begin{align*}
|F(y)-F(x)|&=\left| \int_x^yf(t)\d t\right| \\
&\leq |y-x| \sup_{[a,b]}|f| \\
&=M|y-x|.
\end{align*}
So, given  $y\in[a,b]$ such that $|y-x|<\delta=\frac{\varepsilon}{M}$, \[
|F(y)-F(x)|<\varepsilon.
\]
\end{proof}

Q: Binomial Theorem
A: For $\alpha\in\R$, $|x|<1$ \[
(1+x)^\alpha=\sum_{k=0}^\infty  \frac{\alpha^{\underline{k}}}{k!} x^k. \]
\begin{proof}
Let $f=(1+x)^\alpha$. By Taylor's Theorem
\begin{align*}
(1+x)^\alpha&=\sum_{k=0}^\infty  \frac{f^{(k)}(0)}{k!} x^k+ \frac{1}{n!}\int_0^x (x-t)^n f^{(n+1)}(t)\d t \\
&=\sum_{k=0}^\infty  \frac{\alpha^{\underline{k}}}{k!} x^k+ \frac{\alpha^{\underline{n+1}}}{n!}\int_0^x (x-t)^n (1+t)^{\alpha-n-1}\d t.
\end{align*}
So we are rtp $R_n(x)=\frac{\alpha^{\underline{n+1}}}{n!}\int_0^x (x-t)^n (1+t)^{\alpha-n-1}\d t\to 0$ as $n\to\infty$.
\begin{align*}
|R_n(x)|&\leq \frac{|\alpha^{\underline{n+1}}|}{n!} |x| \sup_{t\in [0,x]} \left|\frac{x-t}{1+t}\right|^n (1+t)^{\alpha -1}  \\
&\leq \frac{|\alpha|(|\alpha| +1) +\ldots +(|\alpha|+n))}{n!} |x| \sup_{t\in [0,x]} (1+t)^{\alpha -1} \sup_{t\in [0,x]}\left|\frac{x-t}{1+t}\right|^n   \\
&\leq \frac{(k+n)!}{k!n!} k|x|  M\sup_{t\in [0,x]}\left|\frac{x-t}{1+t}\right|^n   \\
&\leq \frac{(k+n)^k}{k!} k|x|  M\sup_{t\in [0,x]}\left|\frac{x-t}{1+t}\right|^n   \\
&\leq \frac{2^kn^k}{k!} k|x|  M\sup_{t\in [0,x]}\left|\frac{x-t}{1+t}\right|^n   \\
&\leq C\sup_{t\in [0,x]}\left|\frac{x-t}{1+t}\right|^n   \\
\end{align*}
where $k=\lceil |\alpha| \rceil$, $M=\sup_{t\in [0,x]} (1+t)^{\alpha -1},n>k, C=\frac{2^k}{k!} k|x|.$
If $0\leq x<1$, then $0\leq t<x <1$ so $0\leq \frac{x-t}{1+t}\leq x$. If $-1<x<0$, then $-1<x\leq t\leq 0$, so $0\geq \frac{x-t}{1+t}\geq x$. So in either case \[
|R_n(x)|\leq Cn^k|x|\to 0\textrm{ as }n\to\infty.
\]
\end{proof}

Q: Sequences criterion for improper integrability
A: Let $a\in\R$, $F:[a,\infty]\to\R$. Then \[
\lim_{x\to\infty}F(x) \textrm{ exists }\iff \forall (x_n):  (x_n)\to \infty\textrm{ as }n\to\infty\implies \lim_{n\to\infty} F(x_n)\textrm{ exists }
\]
\begin{proof}
  ('$\Rightarrow$') Let $\lim_{x\to\infty}F(x)=\ell$. Given $\varepsilon > 0$ and $(x_n): (x_n)\to\infty$ as $n\to\infty$:
  \[
    \forall r\in\R: \exists N\in\N: \forall n>N: x_n>r,\textrm{ so } |f(x_n) - \ell| < \varepsilon.
  \]
  So $(F(x_n))\to\ell$ as $n\to\infty$.
  ('$\Rightarrow$') Suppose not. Then $\lim_{n\to\infty} F(x_n)=\ell$ but
  \[
    \exists \varepsilon > 0:\forall r\in\R: \exists x>r: |F(x)-\ell| \geq \varepsilon.
  \]
Thus we can define $(y_n)$ such that $y_n>n$ and $\forall n\in\N:  |F(y_n)-\ell| \geq \varepsilon$. Then $(y_n)\to\infty$ but $(F(y_n))\not\to\ell$. Contradiction.
\end{proof}

Q: Comparison test for integrals
A: Let $a\in\R, f,g:[a,\infty)\to\R$ be functions such that $\forall b>a: f$ is integrable on $[a,b]$. Let $\int_0^\infty g(t)\d t$ exist and $\forall t\in[a,\infty): |f(t)|\leq g(t)$. Then $\int_a^\infty f(t)\d t$ exists.
\begin{proof}
Let $F(x)=\int_a^x f(t)\d t, G(x)=\int_0^x g(t)\d t.$ Let $(x_n)\in[a,\infty]$ such that $(x_n)\to\infty$ as $n\to\infty$. We know $(G(x_n)))$ converges so is Cauchy. Hence, by the triangle inequality, $\forall m\geq n\in\N:$
\begin{align*}
|F(x_m)-F(x_n)|&=\left| \int_{x_n}^{x_m} f(t) \d t\right| \\
&\leq \left| \int_{x_n}^{x_m} |f(t)| \d t\right| \\
&\leq \left| \int_{x_n}^{x_m} g(t) \d t\right| \\
&=|G(x_m)-G(x_n)| \\
&\to 0\textrm{ as }n\to\infty.
\end{align*}
So $(F(x_n)))$ is Cauchy and thus converges.
\end{proof}