Title: Probability and Measure

Q: $\sigma$-algebra
A: A \emph{$\sigma$-algebra} $\mathcal{E}\subseteq \mathcal{P}(E)$ satisfies
\textbf{Empty set}. $\emptyset \in \mathcal{E}$.
\textbf{Complements}. $A \in \mathcal{E}\implies A^C = E \setminus A \in \mathcal{E}$.
\textbf{Countable Unions}. $\forall (A_n)\in\mathcal{E}$:
\[
\bigcup_n A_n \in \mathcal{E}.
\]
The pair $(E, \mathcal{E})$ is called a \emph{measurable space}.
Also closed under countable intersections as $A \cap B = (A^C \cup B^C)^C$.

Q: Measure
A: A \emph{measure} on a measurable space $(E, \mathcal{E})$ is a function $\mu: \mathcal{E} \to [0, \infty]$ such that
\textbf{Empty set}. $\mu(\emptyset) = 0$.
\textbf{Countable additivity}. For any disjoint $(A_n)\in\mathcal{E}$:
\[
\mu\left(\bigcup_n A_n\right) = \sum_{n = 1}^\infty \mu(A_n).
\]

Q: Elementary set
A: Finite union of (disjoint) boxes.

Q: Elementary sets are spanned by disjoint boxes
A: Let $E\subseteq \R^d$ be an elementary set. Then $E$ is a finite union of \emph{disjoint} boxes.
\begin{proof}
Case $d = 1$. Given intervals $I_1, \cdots, I_k$, place the $2k$ endpoints in increasing order (discarding repetitions). Taking the open intervals and points, we have $J_1, \cdots , J_{k_0}$ with $k_0\leq 3k$ such that each of the $I_i$ is a union of some subcollection of the $J_i$ .
Case $d>1$. We have $E = \bigcup_{i=1}^k B_i$ where the $B_i = I_{i,1} \times \cdots \times I_{i,d}$. For each $j = 1,\cdots , d$, by the first case each of $I_{1,j} ,\cdots , I_{k,j}$ is a union of subcollections of $J_{1,j} , \cdots, J_{k_j, j}$ disjoint intervals. Taking Cartesian products, we can express the $B_i$ as finite unions of $J_{i_1, 1} \times\cdots\times J_{i_d,d}$, for $1\leq i_j \leq k_j, 1\leq j \leq d$.
\end{proof}

Q: Elementary measure
A: If $E =\bigsqcup_{i=1}^k B_i$, then $m(E) := |B_1| + \cdots + |B_k|$ is independent
of the partition.
\begin{proof}(discretisation argument)
For each interval $I$ we use the limiting formula
\[
|I| = \lim_{N\to\infty} \frac{1}{N} \left| \left( I \cap \frac{\Z}{N} \right) \right|.
\]
Taking Cartesian products, for any box $B$
\[
|B| = \lim_{N\to\infty} \frac{1}{N^d} \left| \left( B \cap \frac{\Z^d}{N} \right) \right|
\]
and taking unions
\[
|B_1| + \cdots + |B_k| = \lim_{N\to\infty} \frac{1}{N^d} \left| \left( E \cap \frac{\Z^d}{N} \right) \right| =:m(E).
\]
\end{proof}

Q: Discretisation formula for an interval
A: For any interval $I=[a,b]$, $a<b$, we have
\[
b - a = \lim_{N\to\infty} \frac{1}{N}\left|\left( I \cap \frac{\Z}{N} \right) \right|.
\]
\begin{proof}
\[
\frac{1}{N}\left|\left( I \cap \frac{\Z}{N} \right) \right| = \frac{\lfloor Nb \rfloor - \lceil Na \rceil}{N} = b-a + O\left(\frac{1}{N}\right) \to b-a.
\]
\end{proof}

Q: Uniqueness of elementary measure
A: Let $d \geq 1$. Let $m': \mathcal{E}(\R^d) \to \R^+$ obey the non-negativity, finite additivity, and translation invariance properties. Let $m$ be the standard elementary measure. Then $m'=cm$ for some $c\in\R$, i.e. we have uniqueness up to normalisation.
\begin{proof}
Let $m'([0,1)^d)=c$. By finite additivity and translation invariance for $q\in\Q: m'([0, q)^d) = cq^d$. Done by density of $\Q$ in $\R$.
\end{proof}

Q: Jordan measure
A: Let $E\subseteq \R^d$ be a bounded set, $\mathcal{E}$ be the elementary subsets of $\R^d$, $m$ the elementary measure. Then
\begin{enumerate}
\item The Jordan inner measure is
\[
m_{(J)}(E) = \sup_{A\subseteq E, A\in\mathcal{E}} m(A)
\]
\item The Jordan outer measure is
\[
m^{(J)}(E) = \inf_{B\supseteq E, B\in\mathcal{E}} m(B)
\]
\item If $m(E) := m_{(J)}(E) = m^{(J)}(E)$,  then $E$ is Jordan measurable.
\end{enumerate}

Q: Generator of $\sigma$-algebra
A: The $\sigma$-algebra \emph{generated by} $\mathcal{A} \subseteq \mathcal{P}(E)$ is
\[
\sigma(\mathcal{A}) = \{A \subseteq E: A \in \mathcal{E}\text{ for all $\sigma$-algebras $\mathcal{E}\supseteq\mathcal{A}$}\}.
\]
In other words $\sigma(\mathcal{A})$ is the smallest sigma algebra that contains $\mathcal{A}$.

Q: Borel $\sigma$-algebra
A: Let $E$ be a topological space. We define the \emph{Borel $\sigma$-algebra} as
\[
\mathcal{B}(E) = \sigma(\{U \subseteq E: U \text{ is open}\}).
\]
Elements of $ \mathcal{B}(E)$ are called Borel sets. We write $\mathcal{B}=\mathcal{B}(\R)$.
Since open/closed sets are countable unions of boxes, $\mathcal{B}(\R^d)$ is also generated by
\[
\{\text{boxes } \subseteq\R^d\} \text{ or } \{C \subseteq \R^d: C \text{ closed}\}.
\]

Q: Lebesgue Outer Measure
A: For \emph{any} $F\subseteq\R^d$:
\[
m^*(F) = \inf \left\lbrace \sum_{n\geq 1} m(B_n), F\subseteq \bigcup_{n\geq 1} B_n, B_n \text{ boxes}\right\rbrace.
\]
Note LOM$\leq$JOM as here we allow countable unions.

Q: Lebesgue Measurable
A: A set $E\subseteq\R^d$ is LM if $\forall\varepsilon>0$, $\exists$ $(B_i)_{i\geq 1}$ boxes such that $E\subseteq \bigcup_{i=1}^\infty B_i$ and $m^*(\bigcup B_i \setminus E)<\varepsilon$, where $m^*$ is the LOM.

Q: Outer measure
A: An \emph{outer measure} on $(E, \mathcal{E})$, $ \mathcal{E}\subseteq\mathcal{P}(E)$, is a function $\mu: \mathcal{E} \to [0, \infty]$ such that
\begin{enumerate}
\item (Empty set) $\mu(\emptyset) = 0$.
\item (Monotonicity) If $A\subseteq B\subseteq E$, then $m^*(A)\leq m^*(F)$.
\item Countable subadditivity:\index{countable additivity} For any $(A_n)$ in $\mathcal{E}$:
\[
\mu\left(\bigcup_n A_n\right) \leq \sum_{n = 1}^\infty \mu(A_n).
\]
\end{enumerate}

Q: Vitali's set
A: Choose $E\subseteq [0,1]$ to contain exactly one representative of each element of $\R/\Q$. Then $E$ is not LM and $m^*(E)>0$.
\begin{proof}
By construction, $\{ q+E: q\in\Q\}$ partitions $\R$.
Suppose that $m^*$ is additive on $E$. Then for $(q_i)\subseteq \Q\cap [0,1]$ distinct, by translation invariance:
\[
m^*\left( \bigcup_{i=1}^N (q_i + E)\right) = Nm^*(E).
\]
But $\bigcup_{i=1}^N (q_i + E)\subseteq [0,2]$ so by monotonicity
\[
m^*\left(\bigcup_{i=1}^N (q_i + E)\right) \leq m^*([0,2]) = 2.
\]
So $Nm^*(E)\leq 2$ for any $N\in\N$. Thus $m^*(E)=0$.
But as $[0,1]\subseteq \bigcup_{q\in\Q} (q+E)$, subadditivity gives the contradiction
\[
1 = m^*([0,1]) \leq \sum_{q\in\Q} m^*(q+E) = \sum_{q\in\Q} m^*(E) = 0.
\]
\end{proof}

Q: Middle thirds Cantor set
A: Let $I_0=[0,1]$, $I_1=[0,\frac{1}{3}]\cup[\frac{2}{3},1], \cdots I_n \cdots $ and $C:=\bigcap_{n\geq 0} I_n$. Equivalently, $C\subseteq[0,1]$ has no $1$s in its ternary expansion.
$I_n$ is elementary with $2^n$ intervals of length $\frac{1}{3^n}$ so has measure $m(I_n)=\left(\frac{2}{3}\right)^n\to 0$ as $n\to\infty$. Hence $C$ is JM with measure 0.

Q: Real spaces are second countable by boxes
A: Every open set $U\subseteq\R^d$ is a countable union of open boxes.
\begin{proof}
For $r\in\Q>0$, $q\in\Q^d$, define
\[
B_{q,r}=\{ x\in\R^d: |x_i-q_i| < \frac{1}{r} \quad (1\leq i \leq d) \}.
\]
Then $U$ is exactly the union of boxes $B_{q,r}$ which it contains.
\end{proof}

Q: LM sets are a $\sigma$-algebra
A: Let $\mathcal{L}$ be the space of LM sets. Then $\mathcal{L}$ is a $\sigma$-algebra.
\begin{proof}
\textbf{Unions}. Fix $(E_n)\in\mathcal{L}$. Given $\varepsilon>0$, $\exists C_n=\bigcup_i B_i^{(n)}\supseteq E_n$ such that
\[
m^*(C_n\setminus E_n) < \frac{\varepsilon}{2^n}
\]
and $\bigcup_n E_n \subseteq \bigcup_n C_n \subseteq \bigcup_{i,n} B_i^{(n)}$. By subadditivity:
\[
m^*\left(\bigcup_{i,n} B_i^{(n)} \setminus \bigcup_n E_n \right) \leq \sum_n m^*(C_n\setminus E_n) < \sum_n \frac{\varepsilon}{2^n} \leq \varepsilon.
\]
\textbf{Complements}. Fix $E\in\mathcal{L}$. Given $n\in\N$, $\exists C_n=\bigcup_i B_i^{(n)}\supseteq E$ such that
\[
m^*(C_n\setminus E) < \frac{1}{n}.
\]
Wlog, $B_i^{(n)}$ are open so $C_n$ is open, $C_n^C$ is closed so LM. Hence $\bigcup_n C_n^C\in\mathcal{L}$ by the first part. By subadditivity
\[
m^*\left(E^C\setminus \bigcup_n C_n^C\right) \leq m^*(E^C\setminus C_n^C) = m^*(C_n\setminus E) < \frac{1}{n} .
\]
So $E^C\setminus \bigcup_n C_n^C$ is null and thus LM. By unions
\[
E^C=\left(\bigcup_n C_n^C\right) \cup \left(E^C\setminus \bigcup_n C_n^C\right) \in\mathcal{L}.
\]
\end{proof}

Q: Closed sets partition to compact subsets
A: Every closed set $C\subseteq\R^d$ is a countable union of compact subsets.
\begin{proof}
Let
\[
A_n=\{x\in\R^d: n-1 \leq \|x \| \leq n \}
\]
 be the integer annuli. Then
\[
C = \bigcup_{n\geq 1} (A_n\cap C).
\]
\end{proof}

Q: Open/closed sets are LM
A: Every open/closed subset of $\R^d$ is Lebesgue Measurable.
\begin{proof}
By unions, ets for open boxes and compact sets. Boxes are trivial.
Let $E\subseteq\R^d$ be compact. Then $\forall k\geq 1, \exists U_k:=\bigcup_n B_n^{(k)} \supseteq E$ such that
\[
m^*(E) + \frac{1}{2^k} \geq m^*(U_k).
\]
Wlog $B_n^{(k)}$ are open, have diameter $\leq \frac{1}{2^k}$ (by subdivision), intersect $E$ (by removal) and are finite in number (by compactness).
Immediately $E\subseteq \bigcap_{k\geq 1} U_k$. Conversely
\[
x\in\bigcap_{k\geq 1} U_k \implies \forall k \geq 1: \exists n\in\N, x_k \in E \text{ s.t. } x, x_k \in B_n^{(k)}.
\]
Note $|x-x_k| \leq \frac{1}{2^k}\to 0$ so by closedness $x\in E$. Hence $E = \bigcap_{k\geq 1} U_k$.
By compactness/disjointness
\[
m^*(\bar U_k\setminus U_{k+1}) + m^*(E) = m^*\left( (\bar U_k\setminus U_{k+1}) \cup E\right) \leq m^*(\bar U_k) = m^*(U_k).
\]
So
\[
m^*(U_k\setminus U_{k+1}) \leq m^*(U_k) - m^*(E) \leq \frac{1}{2^k}.
\]
Since $E = \bigcap_{k\geq 1} U_k$, by subadditivity
\[
m^*(U_k\setminus E) \leq \sum_{i\geq k} m^*(U_i\setminus U_{i+1}) \leq \sum_{i\geq k} \frac{1}{2^i} \to 0
\]
as $k\to \infty$.
\end{proof}

Q: LOM of a compact union
A: If $A,B\subseteq \R^d$ are disjoint and compact, then
\[
m^*(A\sqcup B) = m^*(A) + m^*(B).
\]
\begin{proof}
Given $\varepsilon >0$, cover $A$ and $B$ each in disjoint boxes of measure within $\frac{\varepsilon}{2}$ of $m^*(A), m^*(B)$ respectively. By subdivision, we can take boxes of diameter less than $d(A,B)$. Then these boxes form a disjoint covering of $A\sqcup B$. So
\[
m^*(A\sqcup B) \leq m^*(A) + m^*(B) + \varepsilon.
\]
\end{proof}

Q: Telescopic union formula
A: For any sets $(U_i)_{i\geq 1}$, we have the identity
\[
U_1 \cap \bigcup_{i \geq 1} (U_i\setminus U_{i+1}) = U_1 \setminus \bigcap_{i\geq 1} U_i.
\]
In particular
\[
U_1 \setminus \bigcap_{i\geq 1} U_i \subseteq \bigcup_{i \geq 1} (U_i\setminus U_{i+1}).
\]
\begin{proof}
\[
\text{LHS} =\bigcup_k \left[ (U_1\cup \cdots \cup U_{k-1})\setminus U_k \right] =  \text{RHS}
\]
\end{proof}

Q: Boolean algebra
A: $\mathcal{A}\subseteq\P(E)$ is a \emph{Boolean algebra} on $E$ if
\begin{enumerate}
\item $\emptyset \in A$
\item $\forall A, B \in \mathcal{A}: A^C \in \mathcal{A}$ and $A \cup B \in \mathcal{A}$.
\end{enumerate}

Q: Set function
A: Let $\mathcal{A}\subseteq\P(E)$ with $\emptyset \in \mathcal{A}$.
A \emph{set function} is a function $\mu: \mathcal{A} \to [0, \infty]$ such that $\mu(\emptyset) = 0$.

Q: Measure space
A: If $X$ is a set, $\mathcal{A}$ a $\sigma$-algebra on $X$, $\mu$ a measure on $\mathcal{A}$, then $(X,\mathcal{A},\mu)$ is a \emph{measure space}.

Q: Lebesgue Measure
A: The LM $m$ of a LM set $E$, defined as $m(E)=m^*(E)$, forms a measure.
\begin{proof}
$m^*(\emptyset)=0$ is clear. Consider $(E_n)\subseteq\R^d$ disjoint.
\textbf{Case 1: compact}. By unions, monotonicity, subadditivity:
\[
\sum_{i=1}^N m^*(E_i) = m^*\left( \bigcup_{i=1}^N E_i\right) \leq m^*\left( \bigcup_{i=1}^\infty E_i\right) \leq \sum_{i=1}^\infty m^*(E_i).
\]
Taking $N\to\infty$, we have equality.
\textbf{Case 2: bounded}. By LM, $\exists F_n\subseteq E_n$ closed ($\Rightarrow$ compact) with $m^*(E_n\setminus F_n)<\frac{\varepsilon}{2^n}$. Also $E_n$ bounded $\Rightarrow$ $F_n$ compact, $(E_n)$ disjoint $\Rightarrow$ $(F_n)$ disjoint. Hence we can reduce to Case 1:
\begin{align*}
m^*(E_n) &\leq m^*(F_n) +m^*(E_n\setminus F_n) \quad \text{(subadditivity)}\\
\sum_{n\geq 1} m^*(E_n) &\leq \sum_{n\geq 1} m^*(F_n) + \varepsilon\sum_{n\geq 1}\frac{1}{2^n} \quad\text{(summation)}\\
&= m^*\left( \bigcup_{n\geq 1} F_n \right) + \varepsilon \quad\text{(Case 1)}\\
&\leq m^*\left( \bigcup_{n\geq 1} E_n \right) + \varepsilon \quad\text{(monotonicity)}.
\end{align*}
The reverse holds by countable subadditivity.
\textbf{Case 3: general}.  Decompose $\R^d = \bigcup_{m\geq 1} A_m$, $A_m=\{x\in\R^d: m-1\leq \|x \| < m\}$. Then $\{A_m\cap E_n : n,m\in\N\}$ are disjoint, bounded and countable. Hence by Case 2:
\[
m^*\left(\bigcup_{n\geq 1} E_n\right) = \sum_{m,n\geq 1} m^*(A_m\cap E_n) = \sum_{n\geq 1} m^*(E_n).
\]
\end{proof}

Q: Lebesgue Measurable using open/closed sets
A: If $E$ is LM, then $\forall\varepsilon>0$, $\exists U\subseteq\R^d$ open such that $E\subseteq U$ and
\[
m^*(U\setminus E) < \varepsilon.
\]
Also $\exists F\subseteq E$ closed with
\[
m^*(E\setminus F) < \varepsilon.
\]
\begin{proof}
First part follows from LM by taking open boxes. Second part follows from the first applied to $E^C$ with $F=U^C$.
\end{proof}

Q: Uniqueness of Lebesgue Measure
A: On $(\R^d, \mathcal{L})$, $\exists ! \mu: \mathcal{L}\to [0,\infty]$ which is translation invariant, normalised measure.
\begin{proof}
By uniqueness of elementary measure, $\mu = m^*$ on elementary sets.
Now for $E\in\mathcal{L}$, by definition $\exists C\supseteq E$ elementary such that $m^*(C\setminus E)<\varepsilon$. Therefore
\begin{align*}
\mu(E) &\leq \mu (E) + \mu (C\setminus E) \\
&=\mu (C) \\
&=m^*(C) \\
&=m^*(E) + m^*(C\setminus E) \\
&< m^*(E) + \varepsilon.
\end{align*}
So $\mu(E)\leq m^*(E)$. The converse holds by the complementary result for LIM.
\end{proof}

Q: Characterisation of LM up to null sets
A: Let $E\subseteq\R^d$. Then TFAE:
\begin{enumerate}
\item $E$ is LM
\item $E$ is a $G_\delta$ set with a null removed
\item $E$ is the union of an $F_\sigma$ set and a null set.
\end{enumerate}
\begin{proof}
If $E$ is LM, then $\exists U_n$ open such that $U_1\supseteq U_2 \supseteq \cdots \supseteq E$ such that $m(\bigcap U_n \setminus E)=0$. Hence $E=\bigcap U_n \setminus N$ where $N=\bigcap_n U_n \setminus E$ is null and $\bigcap_n U_n$ is  $G_\delta$.
Similarly, if $E$ is LM, then $\exists C_n$ closed such that $C_1\subseteq C_2 \subseteq \cdots \subseteq E$ such that $m(E\setminus \bigcup_n C_n)=0$. Hence $E=N \cup \bigcup _nC_n$ where $N=E\setminus \bigcup_n C_n$ is null and $\bigcup_n C_n$ is $F_\sigma$.
\end{proof}

Q: Trivial algebra
A: \[
\{ \emptyset, X \}
\]

Q: Discrete algebra
A: \[
\P(X)
\]

Q: Elementary algebra
A: Finite union of boxes in $\R^d$ (or complements).

Q: Constructible sets
A: For $X$ a topological space:
\[
\mathcal{B}=\bigcup_{U\subseteq X \text{ open, } C\subseteq X \text{ closed }} (U\cap C).
\]

Q: Sub-null
A: A set contained in a null set.

Q: Completion
A: The completion of a measurable space $(X,\mathcal{A},\mu)$ is
\[
\mathcal{A}^* =  \{ A\cup N \text{ or } A\setminus N: A\in\mathcal{A}, N \text{ sub-null wrt } \mu \}.
\]
If all sub-null sets in $X$ belong to $\mathcal{A}$, then $\mathcal{A}$ is complete.
The completion is equivalent to the Caratheodory measurable sets for $\sigma$-finite spaces.

Q: Monotone convergence of sets
A: Let $(X,\mathcal{A},\mu)$ be a measure space.
\begin{enumerate}
\item (Upwards) $\forall E_1\subseteq E_2 \subseteq\cdots \in\mathcal{A}$:
\[
\lim_{n\to\infty} \mu(E_n) = \sup \mu(E_n) = \mu\left(\bigcup_n E_n\right).
\]
\item (Downwards) $\forall F_1\supseteq F_2 \supseteq\cdots \in\mathcal{A}$ with $\mu(F_1)<\infty$:
\[
\lim_{n\to\infty} \mu(F_n) = \inf \mu(F_n) = \mu\left(\bigcap_n F_n\right).
\]
\end{enumerate}
\begin{proof}
Simply take measures in the identities
\begin{align*}
\bigcup_n E_n &= E_1 \sqcup \bigsqcup_{i\geq 1} (E_{i+1}\setminus E_i) \\
 F_1 &= \bigsqcup_{i\geq 1} (F_i\setminus F_{i+1}) \sqcup \bigcap_n F_n.
\end{align*}
\end{proof}

Q: Additive set function
A: A set function $\mu$ on $\mathcal{A}$ is \emph{additive} if
\[
A, B, A \cup B\in \mathcal{A}, A \cap B = \emptyset \implies \mu(A \cup B) = \mu(A) + \mu(B).
\]

Q: Countably additive set function
A: A set function $\mu$ on $\mathcal{A}$ is $\sigma$-additive if
\[
(A_n), \bigcup_n A_n \in \mathcal{A}, A_n \text{ disjoint } \implies \mu\left(\bigcup_n A_n \right) = \sum_n \mu(A_n).
\]

Q: $\sigma$-finite measure
A: Let $(E, \mathcal{E})$ be a measurable space and $\mu$ a measure. We say $\mu$ is \emph{$\sigma$-finite} if $\exists (E_n)\in\mathcal{E}$ such that
\[
\bigcup_n E_n = E \text{ and }\forall n: \mu(E_n) < \infty.
\]

Q: Caratheodory measurable
A: $A \subseteq E$ is $\mu^*$-measurable if $\forall B \subseteq E$:
\[
\mu^*(B) = \mu^*(B \cap A) + \mu^*(B \cap A^C).
\]

Q: Measures extend to outer measures
A: Let $(\mu, \mathcal{A}, E)$ be a measurable space. For $B \subseteq E$, we define the \term{outer measure}
\[
\mu^*(B) = \inf\left\{\sum_n\mu(A_n): (A_n) \in \mathcal{A}\text{ and } B\subseteq \bigcup A_n\right\}.
\]
Define $\inf\emptyset = \infty$ such that $\mu^*(\emptyset) = 0$. Monotonicity is clear.
\textbf{Subadditivity}. Fix $(B_n)\subseteq E$ where wlog $\mu^*(B_n)<\infty$ for all $n$ (else trivial). Given $\varepsilon > 0$, $\exists (B_{n, m})_{m = 1}^\infty \in \mathcal{A}$ such that
\[
B_n \subseteq \bigcup_m B_{n, m}
\]
and
\[
\mu^*(B_n) + \frac{\varepsilon}{2^n} \geq \sum_m \mu(B_{n, m}).
\]
Then we have
\[
\bigcup_n B_n \subseteq \bigcup_{n, m}B_{n, m}.
\]
Thus by monotonicity and $\mathcal{A}$-additivity:
\[
\mu^*\left(\bigcup_n B_n\right) \leq \sum_{n, m}\mu^*(B_{n, m}) \leq \sum_n \left(\mu^*(B_n) + \frac{\varepsilon}{2^n}\right) = \varepsilon + \sum_n \mu^*(B_n).
\]
\textbf{Extension}. Since $A$ covers itself, we have $\mu^*(A) \leq \mu(A)$. Conversely, let $A, (A_n) \in \mathcal{A}$ and $A \subseteq \bigcup_n A_n$. By monotonicity and subadditivity
\[
\mu(A) \leq \sum_n \mu(A \cap A_n) \leq \sum_n \mu(A_n).
\]
Taking the infimum gives $\mu(A) \leq \mu^*(A)$.

Q: Measurable sets are Caratheodory measurable
A: \begin{claim}
For $\mathcal{A}\subseteq\mathcal{P}(E)$, we have that $\mathcal{A}\subseteq \mathcal{A}^*$.
\end{claim}
Suppose that $B \in \mathcal{A}$ and $A \subseteq E$. Rtp that
\[
\mu^*(A) = \mu^*(A \cap B) + \mu^*(A \cap B^C).
\]
($\leq$) Immediate by countable subadditivity.
($\geq$) If $\mu^*(A)=\infty$ is infinite, then trivially done. Otherwise, given $\varepsilon > 0$, $\exists (B_n)\in\mathcal{A}$ such that $A \subseteq \bigcup_n B_n$ and
\[
\mu^*(A) + \varepsilon \geq \sum_n \mu(B_n).
\]
Taking intersections
\begin{align*}
A \cap B &\subseteq \bigcup_n (B_n \cap B)\\
 A \cap B^C &\subseteq \bigcup_n (B_n \cap B^C).
\end{align*}
Summing these inequalities (using subadditivity)
\begin{align*}
\mu^* (A \cap B) + \mu^*(A \cap B^c) &\leq \sum_n \mu(B_n \cap B) + \sum_n \mu(B_n \cap B^C)\\
&= \sum_n (\mu(B_n \cap B) + \mu(B_n \cap B^C))\\
&= \sum_n \mu(B_n) &\\
&\leq \mu^*(A) + \varepsilon.
\end{align*}

Q: Caratheodory measure space is an algebra
A: \begin{claim}
Let $(\mu, \mathcal{A}, E)$ be a measure algebra. Then $\mathcal{B}^*$ is an algebra.
\end{claim}
\textbf{Whole space}. Clearly $E \in \mathcal{A}^*$ since $\forall B \subseteq E$:
\[
\mu^*(B) = \mu^*(B \cap E) + \mu^*(B \cap E^C).
\]
\textbf{Complements}. If $A \in \mathcal{A}^*$, then $\forall B\subseteq E$:
\[
\mu^*(B) = \mu^*(B \cap A) + \mu^*(B \cap A^C).
\]
By symmetry in $A$ and $A^C$, we also have $A^C \in \mathcal{A}^*$.
\textbf{Intersection}. Suppose $A_1, A_2 \in \mathcal{A}^*$ and $B \subseteq E$. Then we have
\begin{align*}
\mu^*(B) ={}& \mu^*(B \cap A_1) + \mu^*(B \cap A_1^C)\\
={}& \mu^*(B \cap A_1 \cap A_2) + \mu^*(B \cap A_1 \cap A_2^C) + \mu^*(B \cap A_1^C)\\
={}& \mu^*(B \cap (A_1 \cap A_2)) + \mu^*(B \cap (A_1\cap A_2)^C \cap A_1) \\
& \qquad\qquad\qquad\qquad + \mu^*(B \cap (A_1 \cap A_2)^C \cap A_1^C)\\
={}& \mu^*(B \cap (A_1 \cap A_2)) + \mu^*(B \cap (A_1 \cap A_2)^C).
\end{align*}
So we have $A_1 \cap A_2 \in \mathcal{A}^*$.

Q: Caratheodory measure space
A: If $(\mu, \mathcal{B}, E)$ is a measure algebra, then $(\mu^*, \mathcal{B}^*, E)$ is a measure $\sigma$-algebra.
\begin{proof}
Let $(B_n) \in \mathcal{B}^*$ be disjoint and $B=\bigcup B_n$.
\textbf{Unions}. Suppose that $A \subseteq E$. Then we have
\begin{align*}
\mu^*(A) &= \mu^*(A \cap B_1) + \mu^*(A \cap B_1^C)\\
&= \mu^*(A \cap B_1) + \mu^*(A \cap B_2) + \mu^*(A \cap B_1^C \cap B_2^C)\\
&= \cdots\\
&= \sum_{i = 1}^n \mu^*(A\cap B_i) + \mu^*(A \cap B_1^C \cap \cdots \cap B_n^C)\\
&\geq \sum_{i = 1}^n \mu^*(A \cap B_i) + \mu^*(A \cap B^C) \\
\mu^*(A) &\geq \sum_{i = 1}^\infty \mu^*(A \cap B_i) + \mu^*(A \cap B^C) \\
 &\geq \mu^*(A \cap B) + \mu^*(A \cap B^C) \quad\text{(subadditivity)}.
\end{align*}
By subadditivity, we have equality. So $B\in\mathcal{B}^*$.
\textbf{Measure}. From the above
\[
\mu^*(A) = \sum_{i = 1}^\infty \mu^*(A \cap B_i) + \mu^*(A \cap B^C).
\]
Taking $A = B$, this gives
\[
\mu^*(B) = \sum_{i = 1}^\infty \mu^*(B \cap B_i) + \mu^*(B \cap B^C) = \sum_{i = 1}^\infty \mu^*(B_i).
\]
\end{proof}

Q: Caratheodory extension theorem
A: If $\mathcal{B}\subseteq\mathcal{P}(X)$ is a ring (or algebra) and $\mu$ is a countably additive set function on $\mathcal{B}$, then $\mu$ extends to a measure on $\sigma(\mathcal{B})$.

Q: $\pi$-system
A: Let $\mathcal{A}$ be a collection of subsets of $E$. Then $\mathcal{A}$ is called a \emph{$\pi$-system} if
\begin{enumerate}
\item $\emptyset \in A$
\item $A, B \in \mathcal{A} \implies A \cap B \in A$.
\end{enumerate}

Q: d-system
A: Let $\mathcal{A} \subseteq \P(E)$. Then $\mathcal{A}$ is called a \emph{d-system} if
\begin{enumerate}
\item $E \in \mathcal{A}$
\item $A\subseteq B \in \mathcal{A} \implies B \setminus A \in \mathcal{A}$
\item $(A_n \in\mathcal{A})\nearrow \implies \bigcup_n A_n \in \mathcal{A}$.
\end{enumerate}

Q: $\pi + d$
A: \begin{prop}
A collection $\mathcal{A}$ is a $\sigma$-algebra iff it is both a $\pi$-system and a $d$-system.
\end{prop}

Q: Ring
A: A collection of subsets $\mathcal{A}$ is a \emph{ring} on $E$ if
\begin{enumerate}
\item $\emptyset \in A$
\item $A, B \in \mathcal{A} \implies B \setminus A, A \cup B \in \mathcal{A}$.
\end{enumerate}

Q: Dynkin's $\pi$-system lemma
A: \begin{lemma}
Let $\mathcal{A}$ be a $\pi$-system. Then any d-system containing $\mathcal{A}$ also contains $\sigma(\mathcal{A})$.
\end{lemma}
\begin{proof}
Let $\mathcal{D}$ be the intersection of all d-systems containing $\mathcal{A}$, i.e.\ the smallest d-system containing $\mathcal{A}$. Rtp $\mathcal{D}\supseteq\sigma(\mathcal{A})$. Ets $\mathcal{D}$ is $\pi$.
Let $\mathcal{A}\subseteq\mathcal{D}' \subseteq  \mathcal{D}$ be closed under $\cap$ with $\mathcal{A}$, i.e.
\[
\mathcal{D}' := \{ B \in \mathcal{D}: B \cap A \in \mathcal{D}\text{ for all }A \in \mathcal{A}\}.
\]
We show $\mathcal{D}'$ is a $d$-system. Clearly $E \in \mathcal{D}'$ and $B_1\subseteq B_2 \in \mathcal{D}', A \in \mathcal{A}$ implies
\[
(B_2 \setminus B_1) \cap A = (\underbrace{B_2 \cap A}_{\in\mathcal{D}}) \setminus (\underbrace{B_1 \cap A}_{\in\mathcal{D}}) \in \mathcal{D}.
\]
Finally, $(B_n\in \mathcal{D}')\nearrow, A \in \mathcal{A} \implies \bigcup B_n \in \mathcal{D}'$ since
\[
\left(\bigcup B_n\right) \cap A = \bigcup (\underbrace{B_n \cap A}_{\in\mathcal{D}}) \in \mathcal{D}.
\]
So $\mathcal{D}' = \mathcal{D}$. Thus $A\subseteq\mathcal{D}'' \subseteq  \mathcal{D}$ where
\[
\mathcal{D}'' := \{B \in \mathcal{D}: B \cap A \in \mathcal{D}\text{ for all }A \in \mathcal{D}\}.
\]
As above, $\mathcal{D}''$ is a d-system, so $\mathcal{D}'' = \mathcal{D}$; i.e. $\mathcal{D}$ is a $\pi$-system.
\end{proof}

Q: Uniqueness of extending measures
A: \begin{thm}
Suppose that $\mu_1, \mu_2$ are finite measures on $(E, \mathcal{E})$. If $\mathcal{A}$ is a $\pi$-system with $\sigma(\mathcal{A}) = \mathcal{E}$, $\mu_1|_\mathcal{A}=\mu_2|_\mathcal{A}$ and $\mu_1(E)=\mu_2(E)$, then $\mu_1 = \mu_2$.
\end{thm}
\begin{proof}
Let $\mathcal{D} \supseteq \mathcal{A}$ be given by
\[
\mathcal{D} := \{A \in \mathcal{E}: \mu_1(A) = \mu_2(A)\}
\]
Rtp $\mathcal{D} \supseteq \sigma (\mathcal{A})$. By DL, ets that $\mathcal{D}$ is a d-system.
\begin{enumerate}
\item $E \in \mathcal{D}$: by assumption.
\item $A\subseteq B \in \mathcal{D} \implies B \setminus A \in \mathcal{D}$:
\begin{align*}
\mu_1(B) &= \mu_1(A) + \mu_1(B \setminus A) < \infty\\
\mu_2(B) &= \mu_2(A) + \mu_2(B \setminus A) < \infty
\end{align*}
and subtracting gives $\mu_1(B \setminus A) = \mu_2(B \setminus A)$.
\item $(A_n \in \mathcal{D}) \nearrow \implies \bigcup A_n \in\mathcal{D}$:
\[
\mu_1\left(\bigcup A_n\right) = \lim_{n \to \infty}\mu_1(A_n) = \lim_{n \to \infty} \mu_2(A_n) = \mu_2\left(\bigcup A_n\right).
\]
\end{enumerate}
\end{proof}

Q: Borel and Radon measure
A: A measure $\mu$ on $(E, \mathcal{B}(E))$ is called a \emph{Borel measure}.
If $\mu(K) < \infty$ for all $K \subseteq E$ compact, then $\mu$ is a \emph{Radon measure}.

Q: Measurable function
A: Let $(E, \mathcal{E})$ and $(G, \mathcal{G})$ be measure spaces. A map $f: E \to G$ is \emph{measurable} if $\forall A \in \mathcal{G}$: $f^{-1}(A) \in \mathcal{E}$.
If $(G, \mathcal{G}) = (\R, \mathcal{B})$, then we just say that $f$ is measurable on $E$.
Immediately all continuous functions are Borel-measurable.
Like continuity, measurability is closed under composition, projection. It forms an $\R$-algebra (vector space with a product).

Q: Borel Function
A: A measurable function on $(E, \mathcal{B}(E))$.

Q: Measurability can be checked on a generator
A: Let $(E, \mathcal{E})$ and $(G, \mathcal{G})$ be measurable spaces where $\mathcal{G} = \sigma(\mathcal{Q})$. If $\forall A \in \mathcal{Q}: f^{-1}(A) \in \mathcal{E}$, then $f$ is measurable.
\begin{proof}
Follows immediately from the fact that
\[
\{A \subseteq G: f^{-1}(A) \in \mathcal{E}\}
\]
is a $\sigma$-algebra on $G$, since
\[
f^{-1}\left(\bigcup_n A_n\right) = \bigcup_n f^{-1}(A_n),\quad f^{-1}(A^C) = (f^{-1}(A))^C,\quad f^{-1}(\emptyset) = \emptyset.
\]
\end{proof}

Q: $\sigma$-algebra generated by functions
A: Let $E$ be a set and $\{f_i: i \in I\}$ functions $E\to\R$. We then define
\[
\sigma(f_i: i \in I) = \sigma(f^{-1}_i(A): A \in \mathcal{B}, i \in I).
\]
This is the smallest $\sigma$-algebra on $E$ which makes all the $f_i$'s measurable.

Q: Product $\sigma$-algebra
A: If $(E_i, \mathcal{E}_i)$ are measure spaces, the \emph{product measure space} is $\prod_i E_i$ with $\sigma$-algebra generated by the projection maps $\pi_i: \prod_j E_j \to E_i$.
For example
\[
\mathcal{E} \otimes \mathcal{G} = \sigma(\{A \times B: A \in \mathcal{E}, B \in \mathcal{G}\}).
\]

Q: Devil's Staircase (Cantor's Function)
A: We define $\varphi: [0,1] \to [0,1]$ continuous and non-decreasing such that $0\mapsto 0, 1\mapsto 1$ as follows.
Let $f_0(x)=x$ and for $n\geq 1$, define $f_n: [0,1]\to\R$ by
\[
f_n(x) = \begin{cases}
\frac{1}{2}f_{n-1}(3x) \quad & x \in [0,\frac{1}{3}] \\
\frac{1}{2} \quad & x \in (\frac{1}{3},\frac{2}{3}) \\
\frac{1}{2} + \frac{1}{2}f_{n-1}(3x-2) \quad & x \in [\frac{2}{3}, 1].
\end{cases}
\]
Define $\varphi = \lim_{n\to\infty }f_n$. We can check by induction that $\|f_{n+1}-f_n\|_\infty\leq \frac{1}{2^n}$ (so $(f_n)$ is Cauchy) and $f_n$ are continuous non-decreasing. Hence the convergence is uniform, $\varphi$ is (non-absolutely) continuous non-decreasing. Also $\varphi$ is constant and differentiable on $[0,1]\setminus C$. $\varphi$ is a counterexample to FTC.

Q: Simple function
A: A \emph{simple function} is a function that can be written as a finite non-negative linear combination of indicator functions of measurable sets, i.e.
\[
f = \sum_{k = 1}^n a_k \mathbbm{1}_{A_k}
\]
for some $A_k \in \mathcal{E}$ and $a_k \geq 0$.

Q: Integral of simple function
A: The integral of $f = \sum_{i=1}^m a_i \mathbbm{1}_{A_i}$ with $a_i\neq 0$ can be defined as
\[
\mu(f) = \sum_{k = 1}^n a_k \mu(A_k).
\]
\begin{proof}
Suppose
\[
\sum_{i=1}^m a_i \mathbbm{1}_{A_i} = \sum_{i=1}^n b_i \mathbbm{1}_{B_i}. \tag{$*$}
\]
Partition the space into the $2^{m+n}$ sets which are intersections of $(A_i)$ and $(B_i)$. Throw away empty sets to leave $k\leq 2^{m+n}$ disjoint, LM sets $C_1, \cdots C_k$. For each $i$, write
\[
A_i = \bigcup_{j\in I_i} C_j, \quad B_i = \bigcup_{j\in J_i} C_j.
\]
Then
\[
m(A_i) = \sum_{j\in I_i} m(C_j), \quad m(B_i) = \sum_{j\in J_i} m(C_j).
\]
Fix $x\in C_j, j\leq k$. On $\{x\}$, we have that for each $i$
\[
 \mathbbm{1}_{A_i}(x) = \mathbbm{1}_{I_i}(j), \quad  \mathbbm{1}_{B_i}(x) = \mathbbm{1}_{J_i}(j)
\]
Substituting into $(*)$:
\[
\sum_{i=1}^m a_i \mathbbm{1}_{I_i}(j) = \sum_{i=1}^n b_i \mathbbm{1}_{J_i}(j).
\]
Multiplying by $m(C_j)$ and summing over $j$ gives
\[
\sum_{i=1}^m  a_i  m(A_i)= \sum_{i=1}^m \sum_{j\in I_i} a_i  m(C_j) = \sum_{i=1}^n \sum_{j\in J_i}  b_i m(C_j) = \sum_{i=1}^n  b_i m(B_i).
\]
\end{proof}

Q: Lebesgue Integral
A: Let $f$ be a measurable function. If $f\geq 0$, then we define
\[
\mu(f) = \sup\{\mu(g): g \leq f, g\text{ is simple}\}.
\]
Otherwise write
\[
f = f^+ - f^- = \max\{f, 0\}  + \min\{f, 0\}
\]
such that $|f| = f^+ + f^-$.
$f$ is \emph{integrable} if $\mu(|f|) < \infty$ and in this case
\[
\mu(f) = \mu(f^+) - \mu(f^-).
\]

Q: Monotone convergence theorem of Beppo-Levi
A: Let $(f_n) \nearrow f$ be non-negative measurable $X\to\R$. Then $\mu(f_n) \nearrow \mu(f)$.
\begin{proof}
Clearly $\mu(f_1) \leq \cdots \leq \mu(f_n) \leq \cdots \leq \mu(f)$.
Given $\varepsilon>0$, let $g$ be a simple function such that $0\leq g \leq f$ and define
\[
E_n=\{ x \in X: f_n(x) \geq (1-\varepsilon)g(x) \}.
\]
 Then $E_n\nearrow X$. Apply MCT for sets to the measure $m_g : E \mapsto \mu(\mathbbm{1}_Eg)$.
Then $m_g(E_n)\nearrow m_g(X)=\mu(g)$ and
\begin{align*}
m_g(E_n) &= \int \mathbbm{1}_{E_n}(x)g(x) \; \d\mu(x) \\
&\leq \frac{1}{1-\varepsilon} \int f_n(x)\; \d\mu(x) \\
&=  \frac{\mu(f_n)}{1-\varepsilon}. \\
\intertext{Multiplying out and taking $\lim\inf$s:}
(1-\varepsilon)\mu(g) &\leq \lim\inf\mu(f_n) \leq \lim\sup\mu(f_n)\leq \mu(f). \\
\intertext{Taking the sup over all simple $g\leq f$:}
\mu(f)(1-\varepsilon) &\leq \lim\inf\mu(f_n) \leq \lim\sup\mu(f_n)\leq \mu(f).
\end{align*}
So $\mu(f)=\lim\inf\mu(f_n) = \lim\sup\mu(f_n)$.
\end{proof}

Q: Linearity of integral
A: Let $f, g$ be measurable, and $\alpha, \beta \geq 0$. Then
\[
\mu(\alpha f + \beta g) = \alpha \mu(f) + \beta \mu(g).
\]
\begin{proof}\leavevmode
First assume $f,g \geq 0$ and let
      \begin{align*}
        f_n = 2^{-n}\lfloor 2^n f\rfloor \wedge n\\
        g_n = 2^{-n}\lfloor 2^n g\rfloor \wedge n.
      \end{align*}
      Then $f_n, g_n$ are simple with $f_n \nearrow f$ and $g_n \nearrow g$.
By MCT $\mu(f_n) \nearrow \mu(f)$, $\mu(g_n) \nearrow g$ and $\mu(\alpha f_n + \beta g_n) \nearrow \mu(\alpha f + \beta g)$.
 By simplicity,
\[
\mu(\alpha f_n + \beta g_n)=\alpha \mu(f_n) + \beta \mu(g_n).
\]
 Letting $n\to\infty$ gives the result.
In the general case, scalar multiplication still works as $\mu(-f)=-\mu(f)$ by definition. Now let $h = f + g$. We can write this as
\begin{align*}
h^+ - h^- &= (f^+ - f^-) + (g^+ - g^-) \\
 h^+ + f^- + g^- &= f^+ + g^+ + h^-. \\
\intertext{Now both sides are non-negative. Taking measures}
\mu(f^+) + \mu(f^-) + \mu(g^-) &= \mu(f^+) + \mu(g^+) + \mu(h^-) \\
\mu(h^+) - \mu (h^-) &= \mu(f^+) - \mu(f^-) + \mu(g^+) - \mu(g^-) \\
\mu(h) &=\mu(f) + \mu(g).
\end{align*}
\end{proof}

Q: Monotonicity of integral
A: Let $f, g$ be measurable. Then $f \leq g$ implies $\mu(f) \leq \mu(g)$.
\begin{proof}
If $f, g$ are non-negative, then
      \begin{align*}
        \mu(g) &= \sup\{\mu(h): h \leq g\text{ simple}\}\\
        &\geq \sup \{\mu(h): h \leq f\text{ simple}\}\\
        &= \mu(f).
      \end{align*}
In the general case, apply this result to $g-f\geq 0$.
\end{proof}

Q: Zero integral
A: Let $f\geq 0$ be measurable. Then $f = 0$ a.e. $\iff$ $\mu(f) = 0$.
N.B. ($\Rightarrow$) still holds for any $f$ measurable (by applying to $f^+$ and $f^-$).
\begin{proof}\leavevmode
($\Rightarrow$) Suppose $f = 0$ a.e. We define the simple functions
      \[
        f_n = 2^{-n} \lfloor 2^n f\rfloor \wedge n.
      \]
Then $f_n \nearrow f$, $\mu(f_n)=0$ and $f_n = 0$ a.e. By MCT
      \[
        \mu(f) = \lim_{n \to \infty}\mu(f_n) = 0.
      \]
($\Leftarrow$)
  Suppose $f \not= 0$ a.e. Let
      \[
        A_n = \left\{x: f(x) > \frac{1}{n}\right\}.
      \]
      Then
      \[
        \{x: f(x) \not= 0\} = \bigcup_n A_n
      \]
      has positive measure. So $\exists N$ such that $\mu(A_N)>0$. Define
      \[
        h = \frac{1}{N} \mathbf{1}_{A_N}.
      \]
By monotonicity $\mu(f) \geq \mu(h) =\frac{1}{N}\mu(A_N)> 0$. So $\mu(f) \not= 0$.
\end{proof}

Q: Exchanging integration and summation
A: Let $(f_n)$ be measurable. If $f_n\geq 0$ or $\sum|f_n|$ is $\mu$-integrable, then
  \[
    \mu\left(\sum_{n = 1}^\infty f_n\right) = \sum_{n = 1}^\infty \mu(f_n).
  \]
\begin{proof}
If $f_n\geq 0$, then let $g_N=\sum_{n=1}^N f_n$. Done by MCT on $(g_N)$. \\
If $g:=\sum |f_n|<\infty$, then let $g_N=\sum_{n=1}^N f_n$. Since $|g_N|\leq g$ and $g_N\to g$, done by DCT.
\end{proof}

Q: Fatou's lemma
A: Let $(f_n)\geq 0$ be msble. Then \[ \mu(\lim\inf f_n) \leq \lim\inf \mu(f_n). \] \begin{proof} If $k\geq n$, then \begin{align*} \inf_{m \geq n} f_m &\leq f_k \\ \mu\left(\inf_{m \geq n} f_m\right) &\leq \mu(f_k) \quad \text{(monotonicity)} \\ \mu\left(\inf_{m \geq n} f_m\right) &\leq \inf_{k \geq n} \mu(f_k) \leq \liminf_m \mu(f_m) \\
\mu\left(\liminf_m f_m\right) &\leq \inf_{k \geq n} \mu(f_k) \leq \liminf_m \mu(f_m) \quad \text{(MCT)}. \end{align*} \end{proof}

Q: Lebesgue dominated convergence theorem
A: Let $(f_n), f$ be measurable where $f=\lim f_n$. Suppose $\exists g$ $\mu$-integrable such that $\sup|f_n| \leq g$, then $ \mu(f_n) \to \mu(f)$.
\begin{proof}
Note that $|f_n|\leq g \implies |f| \leq g \implies \mu(|f|)\leq \mu(g)$. So $f, (f_n)$ are $\mu$-integrable. Since $g \pm f_n \geq 0$, by Fatou
\begin{align*}
\mu(g+f) &\leq \mu(g) + \lim\inf \mu(f_n) \\
\mu(f) & \leq \lim\inf(\mu(f_n)) \qquad (\dagger)\\
\mu(-f) &\leq \lim\inf \mu(-f_n) \qquad (f_n\mapsto -f_n) \\
\mu(f) &\geq \lim\sup\mu(f_n)  \qquad (\ddagger).
\end{align*}
\end{proof}

Q: Differentiation under the integral sign
A: Let $(E, \mathcal{E}, \mu)$ be a measure space, $U \subseteq \R$ open and $f: U \times E \to \R$. Assume
\begin{enumerate}
\item $x \mapsto f(t, x)$ is integrable ($\forall t \in U$);
\item $t \mapsto f(t, x)$ is differentiable ($\forall x \in E$);
\item $\exists g$ $\mu$-integrable such that $\left|\frac{\partial f}{\partial t}(t, x)\right| \leq g(x)$ ($\forall x \in E, t \in U$).
\end{enumerate}
Then $x \mapsto \frac{\partial f}{\partial t}(t, x)$ is $\mu$-integrable ($\forall t\in U$) and
\[
F(t) = \int_E f(t, x) \d \mu
\]
is differentiable with value
\[
F'(t) = \int_E \frac{\partial f}{\partial t}(t, x)\;\d \mu.
\]
\begin{proof}
$x \mapsto \frac{\partial f}{\partial t}(t, x)$ is the limit of measurable functions so is measurable.
Let $(h_n)\searrow 0$ and
\[
g_n(x) = \frac{f(t + h_n, x) - f(t, x)}{h_n}
\]
such that $ g_n \to \frac{\partial f}{\partial t}$.
By MVT, $\exists \theta_n\in[t, t+h_n]$ such that $g_n(x)=\partial_t f(\theta_n,x)$.
Then $|g_n|\leq g$ and $g$ is $\mu$-integrable. By DCT
\[
\mu\left(\frac{\partial f}{\partial t}(t, x)\right)= \lim_{n\to\infty}\left[\frac{F(t + h_n, x) - F(t, x)}{h_n}\right] = F'(t).
\]
\end{proof}

Q: Egoroff Theorem
A: Let $(X,\mathcal{A},\mu)$ be a finite measure space, $(f_n)$ measurable and $f=\lim f_n$. Then $\forall\varepsilon>0, \exists A\in\mathcal{A}$ such that $\mu(A)>\mu(X)-\varepsilon$ and
\[
\sup_{x\in A} \left| f_n(x)-f(x) \right| \to 0.
\]
\begin{proof}
Wlog $(f_n)\searrow 0=f$ (else replace $f_n$ by $\sup_{k\geq n} |f_k(x) - f(x) |$).
Given $k\in \N$, $\forall x\in X, \exists n_k(x)\in\N$ such that $0\leq f_{n_k}(x) \leq \frac{1}{k}$, i.e.
\[
\bigcup_{1\leq T < \infty} \left\{ x: n_k(x) \leq T\right\} = X.
\]
By MCT $\mu\left(\left\{ x: n_k(x) \leq T\right\}\right) \nearrow \mu(X)$ as $T\to\infty$. Hence $\exists T_k\in\N$ such that $\mu\left(\left\{ x: n_k(x) \leq T_k\right\}\right) \geq \mu(X) - \frac{\varepsilon}{2^k}$. Take $A=\bigcap_{k\geq 1}\left\{ x: n_k(x) \leq T_k\right\}$. By subadditivity
\[
\mu(X\setminus A) \leq \sum_{k\geq 1} \frac{\varepsilon}{2^k}=\varepsilon.
\]
Finally $\sup_A f_{T_k} \leq \frac{1}{k}$ so  $\sup_{x\in A}|f_n(x)| \to 0$ as $n\to\infty$.
\end{proof}

Q: Change of variables formula
A: Let $\phi: [a, b] \to \R$ be $C^1$ and increasing. Then $\forall g$ bounded Borel:
\[
\int_{\phi(a)}^{\phi(b)} g(y) \;\d y = \int_a^b g(\phi(x))\phi'(x)\;\d x.\tag{$*$}
\]
\begin{proof}
We let
\[
V = \{\text{Borel functions $g$ such that $(*)$ holds}\}.
\]
By linearity of the integral, $V$ is indeed a vsp. Moreover
\begin{enumerate}
\item By FTC: $\mathbf{1}_A \in V$ for all $A$ in the $\pi$-system $\{ [u, v] \subseteq [\phi (a),\phi (b) ] \}$.
\item If $(g_n)\geq 0 \in V \nearrow g$, then we know
\[
\int_{\phi(a)}^{\phi(b)} g_n(y) \;\d y = \int_a^b g_n(\phi(x))\phi'(x)\;\d x
\]
so by MCT
\[
\int_{\phi(a)}^{\phi(b)} g(y) \;\d y = \int_a^b g(\phi(x))\phi'(x) \;\d x.
\]
\end{enumerate}
\end{proof}

Q: Monotone class theorem
A: Let $(E, \mathcal{E})$ be a msble space and $\mathcal{A} \subseteq \mathcal{E}$ be a $\pi$-system with $\sigma(\mathcal{A}) = \mathcal{E}$. Let $\mathcal{V}$ be a vsp such that
\begin{enumerate}
\item $1 = \mathbf{1}_E \in \mathcal{V}$.
\item $\mathbf{1}_A \in \mathcal{V}$ ($\forall A \in \mathcal{A}$)
\item $\mathcal{V}$ is closed under bounded, monotone limits, i.e. $(f_n)\in \mathcal{V}$ non-negative, $f_n \nearrow f$ bounded $\implies f \in \mathcal{V}$.
\end{enumerate}
Then $\mathcal{V}$ contains all bounded measurable functions. Moreover if $\mathcal{V}$ is closed under unbounded monotone limits, then $\mathcal{V}$ also contains all NNM functions.
\begin{proof}
Let
\[
\mathcal{D} = \{A \in \mathcal{E}: \mathbf{1}_A \in \mathcal{V}\}.
\]
$\mathcal{D}$ is a $d$-system:
\begin{enumerate}
\item $\mathbf{1}_E \in \mathcal{V} \implies E \in \mathcal{D}$.
\item $\mathbf{1}_A \in V \implies 1 - \mathbf{1}_A = \mathbf{1}_{E \setminus A} \in \mathcal{V} \implies E \setminus A \in \mathcal{D}$.
\item $(A_n) \in \mathcal{D} \nearrow \bigcup A_n  \implies \mathbf{1}_{A_n} \nearrow \mathbf{1}_{\bigcup A_n} \implies \bigcup A_n \in \mathcal{D}$.
\end{enumerate}
By Dynkin's Lemma $\mathcal{D} = \mathcal{E}$.
Given $f$ bounded non-negative measurable, by boundedness $f_n = 2^{-n} \lfloor 2^n f\rfloor$ is simple so $f_n\in V$ for each $n\in\N$. But $(f_n)\nearrow f$ so by closedness $f \in \mathcal{V}$.
Hence $\forall f$ bounded measurable
\[
f = (f \vee 0) + (f \wedge 0)  \in V
\]
and $\forall f$ NNM
\[
f \wedge n \nearrow f.
\]
\end{proof}

Q: Pushforward/image measure
A: Let $(E, \mathcal{E}, \mu)$ and $(G, \mathcal{G})$ be measure spaces, and $f: E \to G$ measurable. Then
\[
\nu:=\mu_* f :=  \mu \circ f^{-1}
\]
is a measure on $(G, \mathcal{G})$, known as the \emph{pushforward} or \emph{image} measure.

Q: Pushforward of a function
A: Writing $\nu=\mu\circ f^{-1}$, $\forall g$ NNM
\[
\nu(g) = \mu(g \circ f).
\]
\begin{proof}
By MCT/linearity ets for $g=\mathbbm{1}_{A}$, $A$ msble. Indeed
\[
\nu(\mathbbm{1}_{A}) = \nu(A) = \mu(f^{-1}(A)) = \mu(\mathbbm{1}_{f^{-1}(A)}) = \mu(\mathbbm{1}_{A} \circ f)
\]
\end{proof}

Q: Density
A: Let $(E, \mathcal{E}, \mu)$ be a measure space, and $f$ be non-negative measurable. Then
\[
\nu(A) = \mu(f \mathbf{1}_A)
\]
 is a measure on $(E, \mathcal{E})$.
\begin{proof}\leavevmode
\begin{enumerate}
\item $\nu(\emptyset) = \mu(f\mathbf{1}_\emptyset) = \mu(0) = 0$.
\item If $(A_n)\in \mathcal{E}$ is disjoint, then
\[
\nu\left(\bigcup A_n\right) = \mu(f\mathbf{1}_{\bigcup A_n}) = \mu\left(f \sum\mathbf{1}_{A_n}\right) = \sum \mu\left(f \mathbf{1}_{A_n}\right) = \sum \nu(A).
\]
\end{enumerate}
\end{proof}

Q: Change of variable in higher dimensions
A: Let $U,V\subseteq\R^d$ be open sets, $\phi:U\to V$ a diffeomorphism and $f:V\to\R$ LM. If $f$ is LI or NNM, then
\[
\int f(\phi(x)) |J_\varphi(x)| \; \d x = \int_V f(y) \; \d y
\]
where $J_\varphi(x)=\det (D\varphi)$.

Q: Integrals are measurable
A: Let $E = E_1 \times E_2$, $\mathcal{E} = \mathcal{E}_1 \times \mathcal{E}_2$. If $f: E \to \R$ is $\mathcal{E}$-measurable, then
\begin{enumerate}
\item $x_1 \mapsto f(x_1, x_2)$ is $\mathcal{E}_1$-measurable.
\item If $f$ is bounded or $\geq 0$, then
\[
f_2(x_2) = \int_{E_1} f(x_1, x_2)\; \d\mu_1( x_1)
\]
is $\mathcal{E}_2$-measurable.
\end{enumerate}
\begin{proof}
For the first part $x_1 \mapsto (x_1, x_2) \mapsto f(x_1, x_2)$ is a composition of msble functions.
For the second part, let $V = \left\{ f: \text{(ii) holds} \right\}$. Linearity $\implies$ $V$ vsp and MCT $\implies$ closed monotone limits.
$\mathbf{1}_A \in V$ for all $A \in \mathcal{E}$ as indicator functions integrate to simple functions which are msble.
 By MCl, $V$ contains all BB/NNM functions.
\end{proof}

Q: Product measure: E+U
A: \begin{thm}
If $E$ is $\sigma$-finite, then there exists a unique measure $\mu = \mu_1 \otimes \mu_2$ on $\mathcal{E}$ such that
\[
\mu(A_1 \times A_2) = \mu(A_1) \mu(A_2)
\]
for all $A_1 \times A_2 \in \mathcal{A}$.
\end{thm}
\begin{proof}
(E) We define
\[
\mu(A) = \int_{E_1} \left(\int_{E_2} \mathbf{1}_A(x_1, x_2)\;\mu_2(\d x_2)\right)\mu_1(\d x_1).
\]
This is a measure:
\begin{enumerate}
\item $\mu(\emptyset) = 0$ since $1_{\emptyset} = 0$.
\item If $A = \bigsqcup A_n$, then $\mathbf{1}_A = \sum_n \mathbf{1}_{A_n}$. By non-negativity we can move the sum to the outside so
\begin{align*}
\mu(A) &= \int_{E_1} \left(\int_{E_2} \sum_n \mathbf{1}_{A_n}(x_1, x_2) \;\mu_2(\d x_2)\right)\mu_1(\d x_1)\\
&= \sum_n \int_{E_1} \left(\int_{E_2} \mathbf{1}_{A_n}(x_1, x_2)\;\mu_2(\d x_2)\right)\mu_1(\d x_1)\\
&= \sum_n \mu(A_n).
\end{align*}
\end{enumerate}
Also $\mu(A_1 \times A_2) = \mu(A_1)\mu(A_2)$ as $\mathbf{1}_{A_1\times A_2} =\mathbf{1}_{A_1}\mathbf{1}_{A_2}$.
(U) Immediate by $\sigma$-finiteness and specification of values on $\mathcal{A}$ which generates $\mathcal{E}$.
\end{proof}

Q: Fubini-(Tonneli) Theorem
A: \begin{enumerate}
\item If $f\geq 0$ is msble, then
\[
\mu(f) = \int_{E_1} \left(\int_{E_2} f(x_1, x_2)\; \mu_2(\d x_2)\right)\mu_1(\d x_1).\tag{$*$}
\]
\item If $f$ is $\mu$-integrable and
\[
A_1 = \left\{x_1 \in E_1: \int_{E_2}|f(x_1, x_2)| \mu_2(\d x_2) < \infty\right\}.
\]
then $\mu_1(E_1 \setminus A_1) = 0$ and if we set
\[
f_1(x_1) =
\begin{cases}
\int_{E_2}f(x_1, x_2) \;\mu_2(\d x_2) & x_1 \in A_1\\
0 & x_1 \not\in A_1
\end{cases},
\]
then $f_1$ is $\mu_1$-integrable with $\mu_1(f_1) = \mu(f)$.
\end{enumerate}
\begin{proof}
\begin{enumerate}
\item Let $V=\{f: (*)\text{ holds}\}$. Linearity $\implies$ $V$ vsp. MCT $\implies$ $V$ closed monotone limits. $\mathbf{1}_E, \mathbf{1}_A \in V$ for all $A \in \mathcal{A}$ by definition of product measure. By MCl, $V$ contains all NNM functions.
\item Assume that $f$ is $\mu$-integrable. Then
\[
\phi_1: x_1 \mapsto \int_{E_2} |f(x_1, x_2)|\;\mu(\d x_2)
\]
is $\mathcal{E}_1$-measurable, and, by (i), is $\mu_1$-integrable. So $A_1^C=\varphi_1^{-1}(\{\infty\})\in \mathcal{E}_1$ is null.
Observe that $f_1 = (f_1^+ - f_1^-) \mathbf{1}_{A_1}$, so applying (i) to signed parts:
\[
\mu(f) = \mu(f^+) - \mu(f^-) = \mu_1(f_1^+) - \mu_1(f_1^-) = \mu_1(f_1).
\]
\end{enumerate}
\end{proof}

Q: Random variable distribution
A: A rv is a measurable function $X:\Omega\to\R$ where $(\Omega, \mathcal{F}, \mu)$ is a probability space.
We equip $(\R, \mathcal{B})$ with the image measure $\mu_X=\mu\circ X^{-1}$ which is called the \emph{distribution} or \emph{law} of $X$. $\mu_X$ is determined by its values on the $\pi$-system of intervals $(-\infty, x]$, i.e. by the cdf $F(x)=\mu_X(-\infty, x]$.

Q: Distribution function
A: A \emph{distribution function} is a non-decreasing, right continuous function $\R \to [0, 1]$ satisfying
\[
x\mapsto
\begin{cases}
0 & x \to -\infty\\
1 & x \to +\infty
\end{cases}.
\]

Q: Distribution vs distribution function
A: If $F$ is a distn function, then $\exists$ a prob space $(\Omega, \mathcal{F}, \P)$ and a rv $X$ such that $F_X = F$.
\begin{proof}
Take $(\Omega, \mathcal{F}, \P) = ((0, 1), \mathcal{B}(0, 1), \text{LM})$. We take $X: \Omega \to \R$ to be
\[
X(\omega) = \inf\{x: \omega \leq F(x)\}
\]
such that
\begin{align*}
X(\omega) \leq x &\Longleftrightarrow \omega\leq F(x) \\
\P[X \leq x]& = \P[(0, F(x)]] \\
F_X(x) &= F(x).
\end{align*}
\end{proof}

Q: Independence of events
A: A sequence of events $(A_n)$ is said to be \emph{independent} if
\[
\P\left[\bigcap_{n \in J} A_n\right] = \prod_{n \in J} \P[A_n]
\]
for all finite subsets $J \subseteq \N$.

Q: Independence of $\sigma$-algebras
A: A sequence of $\sigma$-algebras $(\mathcal{A}_n)\subseteq \mathcal{F}$ is said to be independent if $\forall A_n \in \mathcal{A}_n$: $(A_n)$ is independent.

Q: Independence on a generator
A: \begin{thm}
Suppose $\mathcal{A}_1,\mathcal{A}_2\subseteq \mathcal{F}$ are $\pi$-systems. If $\forall A_i\in\mathcal{A}_i$
\[
\P[A_1 \cap A_2] = \P[A_1] \P[A_2],
\]
then $\sigma(\mathcal{A}_1)$ and $\sigma(\mathcal{A}_2)$ are independent.
\end{thm}
\begin{cor}
The $\pi$-systems $(\mathcal{A}_n)$ are independent iff $\sigma(\mathcal{A}_n)$ are independent.
\end{cor}
\begin{proof}
Fix $A_1 \in \mathcal{A}_1$ and define the measures
\[
\mu(A) = \P[A \cap A_1], \quad \nu(A) = \P[A] \P[A_1].
\]
By assumption, they agree on $\mathcal{A}_2$. They are also finite as $\mu(\Omega) = \P[A_1] = \nu(\Omega) \leq 1$, so by uniqueness agree on $\sigma(\mathcal{A}_2)$.
So $\mathcal{A}_1$ and $\sigma(\mathcal{A}_2)$ are independent. Done by symmetry.
\end{proof}

Q: Independent random variables
A: A family $(X_n)$ of RVs is \emph{independent} iff $\sigma(X_n)=\sigma(X_n^{-1}(\mathcal{B}))$ are independent.
\begin{prop}
If $(X_n)$ are real-valued RVs, then they are independent iff
\[
\P[x_1 \leq x_1, \cdots, x_n \leq x_n] = \prod_{j = 1}^n \P[X_j \leq x_j]
\]
for all $n$ and $x_j$.
\end{prop}
\begin{proof}
($\Rightarrow$) Immediate.
($\Leftarrow$) Simply note that $\{(-\infty, x]: x \in \R\}$ is a generating $\pi$-system for $\mathcal{B}$.
\end{proof}

Q: Bernstein's Example
A: If $X,Y\sim B(1,\tfrac{1}{2})$ and $Z=|X-Y|$, then $X,Y,Z$ are pairwise independent but dependent.

Q: Independent random variable distributions
A: $X = (X_1, \cdots, X_n)$ is $\prod\mathcal{F}_i$-measurable and TFAE
\begin{enumerate}
\item $X_1, \cdots, X_n$ are MI.
\item $\mu_X = \mu_{X_1}\otimes \cdots \otimes \mu_{X_n}$.
\item $\forall f_1, \cdots, f_n$ bounded measurable: $\E\left[\prod f_k (X_k)\right] = \prod \E[f_k(X_k)]$.
\end{enumerate}
\begin{proof}\leavevmode
\begin{itemize}
\item (i) $\Rightarrow$ (ii): For any $A =A_1\times \cdots A_n$
\begin{align*}
\mu_X(A) &= \P[X \in A] \\
&= \prod_{k = 1}^n \P[X_k \in A_k] \qquad \text{(MI)}\\
&= (\mu_{X_1} \otimes \cdots \otimes \mu_{X_n})(A).
\end{align*}
\item (ii) $\Rightarrow$ (iii): By assumption
\begin{align*}
\E\left[\prod_{k = 1}^n f_k(X_k)\right] &= \int_E \prod_{k = 1}^n f_k(x_k) \mu(\d x_k)\\
&= \prod_{k = 1}^n \int_{E_k} f_k(x_k) \mu_k(\d x_k) \quad \text{(Fubini)}\\
&= \prod_{k = 1}^n \E[f_k(X_k)].
\end{align*}
\item (iii) $\Rightarrow$ (i): Take $f_k = \mathbf{1}_{A_k}$ for $A_k \in \mathcal{E}_k$. Then
\begin{align*}
\P[X_1 \in A_1, \cdots, X_n \in A_n] &= \E\left[\prod_{k = 1}^n \mathbf{1}_{A_k}(X_k)\right] \\
&= \prod_{k = 1}^n \E[\mathbf{1}_{A_k}(X_k)]\\
&= \prod_{k = 1}^n \P[X_k \in A_k].
\end{align*}
\end{itemize}
\end{proof}

Q: Borel--Cantelli lemma 1
A: If
\[
\sum_n \P[A_n] < \infty,
\]
then
\[
\P[A_n\text{ i.o.}] := \P(\lim\sup A_n)= 0.
\]
\begin{proof}
For each $n$, we have
\begin{align*}
\P[A_n\text{ i.o}] &= \P\left[\bigcap_n \bigcup_{m \geq n} A_m\right]\\
&\leq \P\left[\bigcup_{m \geq n} A_m\right]\\
&\leq \sum_{m = n}^\infty \P[A_m]\\
&\to 0
\end{align*}
as $n \to \infty$.
\end{proof}

Q: Borel--Cantelli lemma II
A: If $(A_n)$ are MI and
\[
\sum_n \P[A_n] = \infty,
\]
then
\[
\P[A_n\text{ i.o.}] = 1.
\]
\begin{proof}
$(A_n)$ MI $\implies$ $(A_n^C)$ MI. Therefore
\begin{align*}
(\lim\sup A_n)^C &=\bigcup_n \bigcap_{m = n}^\infty A_m^C \\
\P\left[\bigcap_{m = n}^\infty A_m^C\right] &= \prod_{m = n}^\infty \P[A_m^C]\\
&= \prod_{m = n}^\infty (1 - \P[A_m])\\
&\leq \prod_{m = n}^\infty \exp(-\P[A_m])\\
&= \exp\left(- \sum_{m = n}^\infty \P[A_m]\right)\\
&= 0.
\end{align*}
\end{proof}

Q: Tail $\sigma$-algebra
A: Let $(X_n)$ be RVs and
\[
\mathcal{T}_n = \sigma(X_{n + 1}, X_{n + 2}, \cdots).
\]
Then
\[
\mathcal{T} = \bigcap_n \mathcal{T}_n
\]
is the \emph{tail $\sigma$-algebra}.

Q: Kolmogorov 0-1 law
A: Let $(X_n)$ be MI RVs. If $A \in \mathcal{T}$, then $\P[A]\in\{0, 1\}$.
Moreover, if $X$ is a $\mathcal{T}$-measurable RV, then $\exists c\in\R$ such that
\[
\P[X = c] = 1.
\]
\begin{proof}
Let
\[
\mathcal{F}_n = \sigma(X_1, \cdots, X_n).
\]
Then $\mathcal{F}_n$ and $\mathcal{T}_n\supseteq\mathcal{T}$ are MI. So $\mathcal{F}_n$ and $\mathcal{T}$ are MI. Since $(\mathcal{F}_k)\nearrow$, $\mathcal{T}$ and $\mathcal{F}_\infty=\bigcup_k \mathcal{F}_k$ are MI. But $\mathcal{T}\subseteq \mathcal{F}_\infty$ so $\mathcal{T}$ is MI from itself. Hence $\forall A\in\mathcal{T}$:
\[
\P[A] = \P[A \cap A] = \P[A] \P[A],
\]
so $\P[A] = 0$ or $1$.
Therefore $\P[X \leq x] \in \{0, 1\}$ for any $x \in \R$. Now take
\[
c = \inf\{x \in \R: \P[X \leq x] = 1\}.
\]
\end{proof}

Q: Strong LLN for finite fourth moments
A: Let $(X_n)$ be MIRVs with a common mean and assume $\exists M > 0$ such that $\E[X_n^4] \leq M$ for all $n$. Then
\[
\frac{S_n}{n} \to \mu\text{ a.s. as }n \to \infty.
\]
\begin{proof}
Wlog $\E[X]=0$ as
\[
\E[(X_n-\E[X])^4] \leq 2^4(\E[X_n^4]+ \E^4[X]]) <\infty.
\]
Then by independence
\begin{align*}
S_n^4 &= \sum_{i,j,k,\ell} X_i X_j X_k X_\ell \\
\E[S_n^4] &= \sum_i \E[X_i^4] + 6 \sum_{i<j} \E[X_i^2X_j^2]  \\
&\leq nM + 6 \left(\frac{n(n-1)}{2} \right)\sqrt{ \E[X_i^4] \E[X_j^4]}  \\
&\leq nM + 3 Mn(n-1) \\
&\leq 3Mn^2 \\
\E \left[\sum_{n = 1}^\infty \left(\frac{S_n}{n}\right)^4\right] &\leq \sum_{n = 1}^\infty \frac{3M}{n^2} < \infty \\
\sum_{n = 1}^\infty \left(\frac{S_n}{n}\right)^4 &< \infty \text{ a.s.}
\end{align*}
So $S_n/n \to 0$ a.s.
\end{proof}

Q: Convergence in measure
A: Let $(E, \mathcal{E}, \mu)$ be a measure space and $(f_n), f$ measurable. We say $f_n \to f$ \emph{in measure} if for each $\varepsilon > 0$, we have
\[
\mu(\{x \in E: |f_n(x) - f(x)| \geq \varepsilon\}) \to 0\text{ as } n \to \infty.
\]

Q: Weak convergence
A: If $(\mu_n), \mu$ are probability measures on $(\R^d, \mathcal{B}(\R^d))$, then $\mu_n\wto\mu$ if
\[
\forall f\in C_b(\R^d): \mu_n(f)\to \mu(f)
\]
as $n\to\infty$.

Q: Convergence in distribution
A: We say $(X_n) \to X$ \emph{in distribution} if $F_{X_n}(x) \to F_X(x)$ for all $x \in \R$ at which $F_X$ is continuous.
Equivalently, $\mu_{X_n}\wto \mu_X$.

Q: Convergence a.e. $\implies$ in measure
A: If $\mu(E) < \infty$, then $f_n \to f$ a.e. implies $f_n \to f$ in measure.
\begin{proof}
By DCT
\[
\mu(\{|X_n - X| \geq \varepsilon\}) = \E[\mathbf{1}[|X_n - X| \geq \varepsilon]] \to 0.
\]
\end{proof}

Q: Convergence in measure implies a.e.
A: If $f_n \to f$ in measure, then $\exists (n_k)$ such that $f_{n_k} \to f$ a.e.
\begin{proof}
Pick $(n_k)$ such that
\[
\mu\left(\left\{x \in E: |f_{n_k}(x) - f(x)| > \frac{1}{k}\right\}\right) \leq 2^{-k}.
\]
Then by Borel--Cantelli
\[
\mu\left(\left\{x \in E: |f_{n_k}(x) - f(x)| > \frac{1}{k} \text{ i.o.}\right\}\right) = 0.
\]
\end{proof}

Q: Convergence in probability $\implies$ in distribution
A: If $X_n \to X$ in probability, then $X_n \to X$ in distribution.
\begin{proof}
Let $f\in C_b(\R^d)$. Given $\varepsilon>0$, by uniform continuity $\exists\delta >0$ such that
\[
\| x \| \leq \frac{1}{\varepsilon}, \|x - y \| \leq  \delta \implies |f(x) - f(y) | < \varepsilon.
\]
Hence
\begin{align*}
|\mu_n(f) - \mu(f) | & \leq \E|f(X_n) - f(X) | \\
& \leq \varepsilon + 2\|f\|_\infty \P(\|X_n-X\|>\delta \text{ or } \|X\|>\tfrac{1}{\varepsilon}) \\
&\leq \varepsilon + 2\|f\|_\infty \left[\P(\|X_n-X\|>\delta) + \P(\|X\|>\tfrac{1}{\varepsilon})\right] \\
&\to 0 \text{ as } n\to\infty, \varepsilon\to 0.
\end{align*}
\end{proof}

Q: Chebyshev's/Markov's inequality
A: Let $f\geq 0$ be measurable and $\varepsilon > 0$. Then
\[
\mu(\{f \geq \varepsilon\}) \leq \frac{1}{\varepsilon} \mu(f).
\]
Note letting $f\mapsto f^n, \varepsilon\mapsto \varepsilon^n$ gives that $\forall n\in\N$:
\[
\mu(\{f \geq \varepsilon\}) \leq \frac{1}{\varepsilon^n} \mu(f^n).
\]
\begin{proof}
Write
\[
f  \geq \varepsilon \mathbf{1}_{f \geq \varepsilon}
\]
and take measures.
\end{proof}

Q: Convergence in measure implies convergence in $L^p$
A: Given $(f_n)$ msble with $|f_n|\leq g\in L^p$ and $(f_n)\to f$ in measure, we have that $(f_n)\to f$ in $L^p$.
\begin{proof}
Suppose not, then $\exists (\varphi(n))$ a subsequence and $\varepsilon >0$ such that $\forall n$:
\[
 \|f_{\varphi(n)}-f\|_p \geq \varepsilon. \tag{$*$}
\]
But $(f_{\varphi(n)})\to f$ in measure so $\exists (\psi(n))\subseteq (\varphi(n))$ a sub-subsequence such that $(f_{\psi(n)})\to f$ a.e. By DCT
\[
 \|f_{\psi(n)}-f\|_p \to 0,
\]
contradicting $(*)$.
\end{proof}

Q: Convergence in mean $\implies$ in probability
A: If $X_n\xrightarrow{L^1}X$, then $X_n\xrightarrow{P} X$.
\begin{proof}
Given $\varepsilon>0$, by Markov's inequality
\[
\varepsilon \P(\|X_n-X\| > \varepsilon) \leq \E[\|X_n-X\|] \to 0
\]
as $n\to\infty$.
\end{proof}

Q: Uniformly integrable
A: The integrable RVs $(X_n)$ are \emph{uniformly integrable} if
\[
\sup_n \E[\|X_n\| \mathbf{1}_{\|X_n\|>M}] \to 0
\]
as $M\to\infty$.

Q: Uniform integrability vs. convergence in mean
A: TFAE:
\begin{enumerate}
\item $(X_n), X \in L^1$ and $X_n \xrightarrow{L^1} X$.
\item $\{X_n\}$ is UI and $X_n \xrightarrow{P} X$.
\end{enumerate}
\begin{proof}
(i) $\Rightarrow$ (ii): Recall $X_n \xrightarrow{L^1} X$ $\implies$ $X_n \xrightarrow{P} X$.
Given $\varepsilon > 0$, $\exists N\in\N$ such that $\forall n \geq N$: $\E[|X - X_n|] \leq \frac{\varepsilon}{2}$. For $n\leq N$, by UI of finite $L^1$ sets, $\exists \delta > 0$ such that
\[
\P[A] < \delta \implies \E[|X |\mathbf{1}_A], \E[|X_n|\mathbf{1}_A] \leq \frac{\varepsilon}{2}.
\]
So $\forall n\geq N$, $\P[A] \leq \delta$:
\begin{align*}
\E[|X_n| \mathbf{1}_A] &\leq \E[|X - X_n| \mathbf{1}_A] + \E[|X|\mathbf{1}_A]\\
&\leq \frac{\varepsilon}{2} + \frac{\varepsilon}{2}.
\end{align*}
(ii) $\Rightarrow$ (i): Let $(X_{n_k}) \to X$ a.s. By Fatou and UI$\implies$UB:
\[
\E[|X|] = \E\left[\liminf_{k \to \infty} |X_{n_k}|\right] \leq \liminf_{k \to \infty} \E[|X_{n_k}|] < \infty.
\]
So $(X_n),X \in L^1$.
Given $\varepsilon > 0$, by UI $\exists K >0$ such that
\[
\E\left[|X| \mathbf{1}_{\{|X| > K\}}\right], \E\left[|X_n| \mathbf{1}_{\{|X_n| > K\}}\right] \leq \frac{\varepsilon}{3}.
\]
The restrictions to $[-K,K]$
\[
X_n^K = (X_n \vee -K) \wedge K,\quad X^K = (X \vee -K)\wedge K
\]
converge in $L^1$ by BCT. So $\exists N\in\N$ such that $\forall n \geq N$:
\[
\E|X_n^K - X^K| \leq \frac{\varepsilon}{3}.
\]
Combining, we have $\forall n \geq N$:
\[
\E[|X_n - X|] \leq \E[|X_n^K - X^K|] + \E[|X|\mathbf{1}_{\{|X| \geq K\}}] + \E[|X_n|\mathbf{1}_{\{|X_n| \geq K\}}] \leq \varepsilon.
\]
\end{proof}

Q: Bounded $\implies$ uniformly integrable
A: Let $\mathcal{X}$ be $L^p$-bounded for $p > 1$, i.e.
\[
C = \sup\{\|X\|_p : X \in \mathcal{X}\} < \infty.
\]
Then $\mathcal{X}$ is UI.
\begin{proof}
For any $X \in \mathcal{X}$, $A \in \mathcal{F}$:
\[
\E[|X|\mathbf{1}_A]  \leq \E[|X|^p]^{1/p} \P[A]^{1/q} \leq C \P[A]^{1/q}.
\]
by H\"older's inequality.
\end{proof}

Q: Equivalent definitions of uniform integrability
A: $\mathcal{X}$ is UI $\iff$ $\mathcal{X}$ is $L^1$-bounded and $\sup_\chi\{\E[|X|\mathbf{1}_A] \}\to 0$ as $\P(A)\to 0$.
\begin{proof}\leavevmode
\begin{itemize}
\item[$(\Rightarrow)$] Boundedness: for $k$ large enough, $\forall  X\in \mathcal{X}$:
\[
\E[|X|] \leq k + \sup_{\mathcal{X}} \E[|X| \mathbf{1}_{|X| > k}] < \infty.
\]
Similarly $\forall A\in\mathcal{F},  X\in \mathcal{X}$:
\[
\E[|X| \mathbf{1}_A]  \leq  \sup_{\mathcal{X}}\E[|X| \mathbf{1}_{|X| > k}] + k \P[A].
\]
Thus, given $\varepsilon > 0$, pick $k$ s.t. the first term is $< \frac{\varepsilon}{2}$ and then $\P[A] < \frac{\varepsilon}{2k}$.
\item[$(\Leftarrow)$]  For any $X\in \mathcal{X}$, by Markov
\[
M\P(\|X\| \geq M) \leq \E\|X\| \leq \sup_{\mathcal{X}} \E\|X\| < \infty
\]
so $\P(\|X\| \geq M)\to 0$ as $M\to\infty$. Hence $\sup_{\mathcal{X}} \E[|X|\mathbf{1}_{\|X\| \geq M}] \to 0$.
\end{itemize}
\end{proof}

Q: Finite $\implies$ uniformly integrable
A: Let $\mathcal{X} = \{X\}$, where $X \in L^1(\P)$. Then $\mathcal{X}$ is uniformly integrable.
Hence, any finite collection of $L^1$ functions is UI.
\begin{proof}
The sum
\[
\E[|X|] = \sum_{k = 0}^\infty \E[|X| \mathbf{1}_{X \in [k, k + 1)}].
\]
is finite so as $K\to\infty$
\[
\E[|X|\mathbf{1}_{|X| \geq K}] = \sum_{k = K}^\infty \E[|X| \mathbf{1}_{X \in [k, k + 1)}] \to 0.
\]
\end{proof}

Q: Young's inequality
A: For $p,q$ H\"older conjugates:
\[
|ab| \leq\frac{|a|^p}{p} + \frac{|b|^q}{q}.
\]
\begin{proof}
If $ab=0$, then it is trivial. So wlog $a,b>0$. By concavity of $\log$ in $[a^p, b^p]$:
\[
\log(ab) = \frac{1}{p} \log a^p + \frac{1}{q} \log b^q \leq \log \left(\frac{a^p}{p} + \frac{b^q}{q}\right).
\]
\end{proof}

Q: H\"older's inequality
A: Let $p, q \in [1, \infty]$ be conjugate. Then for $f, g$ measurable
\[
 \|fg\|_1 \leq \|f\|_p \|g\|_q.
\]
\begin{proof}
$p=1$, $\|f\|_p=0$ are trivial. So wlog $\|f\|_p = \|g\|_q = 1$ and $1<p<\infty$. By Young's inequality
\[
\int |f||g| \;\d \mu \leq \int \left(\frac{|f|^p}{p} + \frac{|g|^q}{q}\right) \;\d \mu = \frac{1}{p} + \frac{1}{q} = 1.
\]
\end{proof}

Q: Bound on sums in $L^p$
A: Let $a, b \geq 0$ and $p \geq 1$. Then
\[
(a + b)^p \leq 2^p(a^p + b^p).
\]
In particular, for any $f,g$ measurable
\[
\|f+g\|_p \leq 2( \|a\|_p^p + \|b\|_p^p)^{1/p}.
\]
\begin{proof}
Wlog $a \leq b$. Then
\[
(a + b)^p \leq (2b)^p \leq 2^p b^p \leq 2^p(a^p + b^p).
\]
\end{proof}

Q: Minkowski inequality
A: Let $p \in [1, \infty]$ and $f, g$ measurable. Then
\[
\|f + g\|_p \leq \|f\|_p + \|g\|_p.
\]
\begin{proof}
If $p \in \{1,\infty\}$, then this is immediate from the triangle inequality on $\R$. Moreover if $\|f + g\|_p = 0$, then the result is trivial and $\|f\|_p,\|g\|_p<\infty \implies \|f + g\|_p\leq 2(\|f\|_p^p + \|g_p^p\|)^{1/p} < \infty$.
Otherwise, by H\"older's Inequality
\begin{align*}
\mu(|f + g|^p) &= \mu(|f + g| |f + g|^{p - 1}) \\
&\leq \mu(|f||f + g|^{p - 1}) + \mu(|g||f + g|^{p - 1})\\
&\leq  (\|f\|_p + \|g\|_p) \mu(|f + g|^{(p - 1)q})^{1/q}\\
&= (\|f\|_p + \|g\|_p) \mu(|f + g|^p)^{1/q} \\
\mu(|f + g|^p)^{1/p} &\leq \|f\|_p + \|g\|_p.
\end{align*}
\end{proof}

Q: $L^p$ spaces
A: Let $(E, \mathcal{E}, \mu)$ be a measurable space. For $1 \leq p < \infty$, we define $\mathcal{L}^p = \mathcal{L}^p(E, \mathcal{E}, \mu)$ to be the set of all measurable functions $f$ such that
\[
\|f\|_p = \left(\int |f|^p \d \mu\right)^{1/p} < \infty.
\]
For $p = \infty$, we let $\mathcal{L}^\infty = \mathcal{L}^\infty(E, \mathcal{E}, \mu)$ be the space of \emph{essentially bounded} functions, i.e.
\[
\|f\|_{\infty} = \inf \{\lambda \geq 0: |f| \leq \lambda \text{ a.e.}\} < \infty.
\]
To make this seminorm a norm, we let $f \sim g$ iff $f - g = 0$ a.e. and define $L^p=\mathcal{L}^p  / \sim$.

Q: Riesz-Fischer Theorem of $L^p$
A: Let $1 \leq p < \infty$. Then $\mathcal{L}^p(X,\mathcal{A},\mu)$ is a Banach space.
\begin{proof}
Given $(f_n)\subseteq \mathcal{L}^p$ Cauchy, pick $\varphi(k)$ inductively such that $\forall k\in\N$
\[
\|f_{\varphi(k + 1)} - f_{\varphi(k)}\|_p \leq 2^{-k}.
\]
By MCT
\[
h:=\sum_{k = 1}^\infty|f_{\varphi(k+1)} - f_{\varphi(k)}|
\]
has $\|h\|_p \leq 1$ and hence is finite a.e. That is
\[
g:=\sum_{k = 1}^\infty \left[ f_{\varphi(k+1)} - f_{\varphi(k)}\right]
\]
converges (absolutely) a.e. with $|g| \leq |h|$. By DCT, the convergence is also in $\mathcal{L}^p$.
By telescoping $f_{\varphi(k)} \to f:= g + f_{\varphi(1)}$ a.e. and in $\mathcal{L}^p$.
\end{proof}

Q: Smooth box cutoff function
A: Let $f(x)=e^{-1/x}\mathbf{1}[x>0]$ be a smooth rise from 0 to 1. Then
\[
\varphi(x) = \int_{-\infty}^x f(u) f(1-u) \; \d u
\]
is a smooth cutoff from 1 to 0.
Next $\varphi_n(x):=\varphi(nx)$ is a cutoff from $\frac{1}{n}$ to 0.
For $I=[a,b], \varepsilon>0$, we can glue two $\varphi_n$ for $\frac{1}{n}<\varepsilon$ to make $\psi_{I,\varepsilon}\in C^\infty$ such that
\[
\mathbf{1}_{[a,b]} \leq \psi_{I,\varepsilon} \leq \mathbf{1}_{[a-\varepsilon, b + \varepsilon]}.
\]
For boxes, simply take products.

Q: Existence of supporting hyperplanes
A: If $c: I \to \R$ is a convex function and $m\in\Int(I)$, then $\exists a, b\in\R$ such that
\[
c(x) \geq ax + b
\]
for all $x \in I$, with equality at $x = m$.
\begin{proof}
Given $x < m < y \in I$, writing $m = tx + (1 - t) y$, by convexity
\begin{align*}
c(m) &\leq t c(x) + (1 - t) c(y) \\
t (c(m) - c(x)) &\leq (1 - t)(c(y) - c(m)) \\
\frac{y-m}{y-x} (c(m) - c(x)) &\leq \frac{m-x}{y-x}(c(y) - c(m)) \\
\frac{c(m) - c(x)}{m - x} &\leq \frac{c(y) - c(m)}{y - m}. \\
\intertext{Since $x,y$ are arbitrary, $\exists a \in \R$ such that $\forall x < m < y$:}
\frac{c(m) - c(x)}{m - x} \leq a &\leq \frac{c(y) - c(m)}{y - m}\\
c(z) &\geq a(z - m) + c(m) \qquad (\forall z \in I).
\end{align*}
\end{proof}

Q: Jensen's inequality
A: Let $X$ be an integrable RV with values in $I\subseteq\R$. If $c: I \to \R$ is convex, then we have
\[
\E[c(X)] \geq c(E[X]).
\]
\begin{proof}
Wlog $X$ is \emph{not} a.s. constant so $\E [X]\in\Int(I)$. Hence $\exists a,b\in\R$ such that
\[
c(x) \geq ax + b
\]
with equality when $x=\E[X]$.
We can also bound
\[
c(X)^-  \leq |a| |X| + |b| \implies \E [ c(X) ^- ] \leq |a| \E |X| + |b| < \infty
\]
so $\E[c(X)]$ is well-defined.
Taking expectations.
\[
\E[c(X)] \geq \E[aX + b] = a\E[X] + b = c(\E[X]).
\]
\end{proof}

Q: Conditional probability
A: Let $\mathcal{G}\subseteq (\Omega, \mathcal{F}, \P)$. Then $\forall A\in\mathcal{F}$, $\exists!\mathcal{G}$-measurable RV $\P(A|\mathcal{G})$ such that $\forall B\in \mathcal{G}$:
\[
\P(A\cap B) = \E[\mathbf{1}_B \P(A|\mathcal{G})].
\]
\begin{proof}
(E) $\mathcal{L}^2(\mathcal{G}) \subseteq \mathcal{L}^2(\mathcal{F})$ is a closed subset of a Hsp so
\[
\mathcal{L}^2(\mathcal{F}) = \mathcal{L}^2(\mathcal{G}) \oplus \mathcal{L}^2(\mathcal{G})^\perp.
\]
Now $\mathbf{1}_A\in\mathcal{L}^2(\mathcal{F})$ so $\exists Y\in \mathcal{L}^2(\mathcal{G})$ such that $\mathbf{1}_A-Y \in \mathcal{L}^2(\mathcal{G})^\perp$. In particular $\forall B\in\mathcal{G}$: $\E[\mathbf{1}_B(\mathbf{1}_A-Y)]=0$, i.e.
\[
\P(A\cap B) = \E[\mathbf{1}_BY].
\]
(U) Suppose $Y,Y'$ are $\mathcal{G}$-measurable with $\E[\mathbf{1}_BY] = \E[\mathbf{1}_BY']$. Then $\E[\mathbf{1}_B(Y-Y')]=0$. Letting $B=\{ Y-Y'>0\}, \{ Y-Y'<0\}$ gives that $Y=Y'$ a.s.
\end{proof}

Q: Convolution of functions
A: If $f,g\in L^1(\R^d)$, then $f*g$ with the LM is finite a.e.
\begin{proof}
By Fubini's Theorem $(x,y)\mapsto f(x-y)g(y)\in L^1(\R^d\times\R^d)$ and
\[
\left|\iint f(x-y) g(y) \; \d y \right| \leq \iint |f(x-y)| |g(y)| \; \d x \d y \leq \|f\|_1 \|g\|_1.
\]
\end{proof}

Q: Convolution of function with measure
A: Let $f \in L^p$ and $\nu$ a probability measure. Then the \emph{convolution} of $f$ with $\nu$ is
\[
f * \nu(x) = \int f(x - y)\;\nu(\d y) \in L^p.
\]
In particular
\[
\|f * \nu\|_p \leq \|f\|_p.
\]
\begin{proof}
Since $\nu$ is a probability measure, Jensen's inequality gives
\begin{align*}
\|f * \nu\|_p^p &= \int \left(\int |f(x - y)| \nu(\d y)\right)^p \;\d x\\
&\leq \iint |f(x - y)|^p \nu(\d y)\;\d x\\
&= \|f\|_p^p \\
&< \infty.
\end{align*}
\end{proof}

Q: Convolution of random variables
A: Let $\mu, \nu$ be PMs on $X,Y$ MI. Let $\mu$ have pdf $f$ wrt LM. Then $\mu * \nu$ is the law of $X + Y$, i.e.
\begin{align*}
\mu * \nu (A) &= \P[X + Y \in A] \\
&= \iint \mathbf{1}_{A}(x + y)\;\mu(\d x)\;\nu(\d y) \\
&= \iint \mathbf{1}_A(x + y) f(x) \;\d x\;\nu(\d y)\\
&= \iint \mathbf{1}_A(x) f(x - y) \;\d x\;\nu(\d y)\\
&= \int \mathbf{1}_A(x) \left(\int f(x - y)\;\nu (\d y)\right)\;\d x \\
&= \int_A (f*\nu).
\end{align*}
So $\mu * \nu$ has density $f* \nu$.

Q: Continuity of translation on $L^p$
A: If $p\in[1,\infty)$, $f\in L^p (\R^d)$, then
\[
\| \tau_h(f) - f \|_p \to 0 \text{ as } h \to 0
\]
where $\tau_h(f)(x) = f(x+h)$.
\begin{proof}
If $f\in C_C(\R^d)$, then this is immediate by DCT.
Otherwise, given $\varepsilon>0$, pick $g \in C_C(\R^d)$ such that $\|f-g\|_p<\varepsilon$. Next pick $h$ small enough that $\| \tau_h(g) - g \|_p<\varepsilon$. Then
\[
\| \tau_h(f) - f \|_p \leq \| \tau_h(f) -\tau_h(g) \|_p + \|\tau_h g - g \|_p + \|g-f \|_p < 3\varepsilon.
\]
\end{proof}

Q: Gaussian convolution
A: Let $f \in L^p(\R^d)$, $p\in[1,\infty)$. Then $f * g_\sigma$ is a \emph{Gaussian convolution} where $g_\sigma$ is the pdf of $N(0, \sigma^2)$.

Q: Gaussian approximation
A: If $p\in [1,\infty)$, $f\in L^p(\R^d)$, then
\[
\| f*g_\sigma - f \|_p \to 0
\]
as $\sigma\to 0$.
\begin{proof}
Using Jensen's inequality
\begin{align*}
f*g_\sigma(x) - f(x) &= \E[f(x-\sigma Z) - f(x)] \\
\| f*g_\sigma - f \|_p^p &= \|  \E[f(x-\sigma Z) - f(x)] \|_p^p\\
&\leq \E \|f(x-\sigma Z) - f(x) \|_p^p \\
&\to 0
\end{align*}
by continuity of translation and DCT.
\end{proof}

Q: Fourier Transform of a function
A: The FT $\hat{f}: \R^d \to \C$ of $f \in L^1(\R^d)$ is given by
\[
\hat{f}(u) = \int_{\R^d}f(x) e^{i (u, x)}\;\d x.
\]
Note $|\hat{f}(u)| \leq \|f\|_1$ is bounded and by DCT is continuous.

Q: Fourier transform of measure
A: The FT of a \emph{finite measure} $\mu$ on $\R^d$ is $\hat{\mu}: \R^d \to \C$ given by
\[
\hat{\mu}(u) = \int_{\R^d} e^{i(u, x)} \mu(\d x).
\]

Q: Characteristic function
A: The \emph{characteristic function} of a RV $X$ is the FT of its law, i.e.
\[
\phi_X(u) = \E[e^{i(u, X)}] = \hat{\mu}_X(u)
\]
where $\mu_X$ is the law of $X$.

Q: Derivative of a Fourier Transform
A: If $f:\R\to\C$ is absolutely integrable, then
\[
 \tilde{f}'=\mathcal{F}[ixf(x)].
\]
\begin{proof}
    \begin{align*}
\tilde{f}'(k) &= \frac{\d}{\d k}\int_{-\infty}^{\infty} e^{ikx} f(x)\;\d x \\
&= i \int_{-\infty}^\infty e^{ikx} xf(x)\;\d x \\
&=   \mathcal{F}[xf(x)]
    \end{align*}
\end{proof}

Q: Fourier Transform of standard normal
A: The Fourier transform $\tilde{f}(k)$ is given by
\[
\mathcal{F}[e^{-\frac{1}{2}x^2}]=\sqrt{2\pi}e^{-\frac{1}{2}k^2}.
\]
\begin{proof}
From differerential properties of the FT
\begin{align*}
\tilde{f}'(k) &= i \mathcal{F}[xe^{-\frac{1}{2}x^2}] \\
&= (-i)(-1)(ik) \tilde{f}(k)\\
&=-k \tilde{f}(k).
\end{align*}
So we obtain the ODE
\[
\tilde{f}(0)=\sqrt{2\pi}, \tilde{f}'(k) =-k \tilde{f}(k).
\]
This has solution
\[
\tilde{f}(k) = \sqrt{2\pi}e^{-\frac{1}{2}k^2}.
\]
\end{proof}

Q: Fourier Transform of a derivative
A: If $f:\R\to\C$ is a differentiable function with $f,f'$ absolutely integrable, then
\[
\mathcal{F}[f'](k)=-ik\mathcal{F}[f](k).
\]
\begin{proof}
Integrating by parts, we have
    \begin{align*}
      \mathcal{F}[f'(x)] &= \int_{\infty}^\infty e^{ikx} \frac{\d f}{\d x}\;\d x\\
      &= [e^{ikx}f(x)]_{-\infty}^\infty -\int_{-\infty}^\infty \frac{\d}{\d x} (e^{ikx}) f(x)\;\d x\\
      &= -ik \int_{-\infty}^\infty e^{ikx} f(x)\;\d x\\
      &= -ik \mathcal{F}[f(x)]
    \end{align*}
where the boundary term vanishes since $f$ is absolutely integrable.
\end{proof}

Q: Fourier transform of a convolution
A: \[
\widehat{f * \nu} = \hat{f} \hat{\nu}.
\]
\begin{proof}
By Fubini's Theorem and change of variables
\begin{align*}
\widehat{f * \nu}(u) &= \int \left(\int f(x - y) \nu(\d y)\right) e^{i(u, x)}\;\d x\\
&= \iint f(x - y) e^{i(u, x)} \;\d x\;\nu (\d y)\\
&= \int \left(\int f(x - y) e^{i(u, x - y)} \;\d (x - y)\right) e^{i(u, y)}\;\nu(\d y)\\
&= \int \hat{f}(u) e^{i(u, y)}\;\nu(\d y)\\
&= \hat{f}(u) \int e^{i(u, y)}\;\nu(\d y)\\
&= \hat{f}(u) \hat{\nu}(u).
\end{align*}
\end{proof}

Q: Fourier Transform of a rescaled function
A: If $f\in L^1(\R^d)$, then
\[
 \mathcal{F}[f(cx)](k)= \frac{1}{|c|^d}\mathcal{F}[f(x)](k/c).
\]
\begin{proof}
    \begin{align*}
      \mathcal{F}[f(cx)](k) &= \int_{-\infty}^\infty e^{i\bra k, x \ket}f(cx)\;\d x\\
      &= \int_{-\infty}^\infty e^{i\bra k, y \ket /c} f(y) \frac{\d y}{|c|^d}\\
      &= \frac{1}{|c|^d}\mathcal{F}[f(x)](k/c).
    \end{align*}
\end{proof}

Q: Fourier Inversion Theorem for a Gaussian
A: Let $g_\sigma \sim N(0,\sigma^2 I)$. Then
\[
g_\sigma(x) = g_\sigma(-x) = (2\pi)^{-d} \hat{\hat{g}}_\sigma (-x).
\]
\begin{proof}
We compute
\begin{align*}
g_\sigma(x) &= (2\pi\sigma^2)^{-\frac{d}{2}} e^{-|x|^2 / 2\sigma^2} = \frac{1}{\sigma^d} g_1(\tfrac{x}{\sigma}) \\
\hat{g}_\sigma(u) &=  \hat g_1(u\sigma) =e^{-\tfrac{1}{2}|u|^2\sigma^2} = (2\pi)^{d/2} \sigma^{-d} g_{1/\sigma}(u) \\
\hat{\hat{g}}_\sigma(x) &= (2\pi)^d g_\sigma(x).
\end{align*}
\end{proof}

Q: Fourier inversion theorem for Gaussian convolutions
A: \[
f * g_\sigma(x) = (2\pi)^{-d} \widehat{\widehat{f*g_\sigma}}(-x).
\]
\begin{proof}
Using IFT of a Gaussian and Fubini's Theorem
\begin{align*}
f * g_\sigma(x) &= \int f(x - y) \left(\frac{1}{(2\pi)^d} \int \hat{g}_\sigma(u) e^{-i(u, y)}\;\d u\right)\;\d y\\
&= \left(\frac{1}{2\pi}\right)^d \int \left(\int f(x - y) e^{i(u, x - y)} \;\d y\right) \hat{g}_\sigma(u) e^{-i(u, x)}\;\d u\\
&= \left(\frac{1}{2\pi}\right)^d \int \hat{f}(u)\hat{g}_\sigma(u) e^{-i(u, x)}\;\d u\\
&= \left(\frac{1}{2\pi}\right)^d \int \widehat{f * g_\sigma}(u) e^{-i(u, x)}\;\d u.
\end{align*}
\end{proof}

Q: Fourier Inversion Theorem for $L^1$
A: If $f,\hat{f}\in L^1$, then
\[
f(x) = (2\pi)^{-d} \int \hat{f}(u) e^{-i\bra u, x \ket} \; \d u.
\]
\begin{proof}
Let $f_\sigma=f*g_\sigma$. Using IFT of Gaussian convolutions
\[
f_\sigma(x) =  (2\pi)^{-d}\int \hat{f}(u) \hat{g}_\sigma(u) e^{-i\bra u, x \ket} \; \d u.
\]
Letting $\sigma\to 0$, Gaussian approximation gives $f_\sigma\to f$ in $L^1$.
RHS $\leq \|\hat{f}\|_1<\infty$ so done by DCT.
\end{proof}

Q: Plancherel identity
A: For any $f \in L^1 \cap L^2$
\[
\|\hat{f}\|_2 = (2\pi)^{d/2} \|f\|_2.
\]
Moreover, by the polarisation identity
\[
\bra \hat{f}, \hat{g} \ket = (2\pi)^d \bra f, g \ket.
\]
\begin{proof}
\textbf{Case} $f, \hat{f} \in L^1$. By IFT
\begin{align*}
\|f\|_2^2 &= \int f(x) \overline{f(x)}\;\d x\\
&= \frac{1}{(2\pi)^d} \int \left(\int \hat{f}(u) e^{-i(u, x)}\;\d u\right)\overline{f(x)}\;\d x\\
&= \frac{1}{(2\pi)^d} \iint \hat{f}(u) \overline{\left(f(x) e^{i(u, x)}\;\d x\right)} \;\d u\\
&= \frac{1}{(2\pi)^d} \int \hat{f}(u) \overline{\hat{f}(u)} \;\d u\\
&= \frac{1}{(2\pi)^d} \|\hat{f}(u)\|_2^2.
\end{align*}
\textbf{Case} $f \in L^1 \cap L^2$. Let $f_\sigma = f * g_\sigma$. Then by the first case and MCT
\begin{align*}
\hat{f}_\sigma(u) &= \hat{f}(u)e^{-|u|^2 \sigma^2/2}  \\
(2\pi)^{d} \|f_\sigma\|_2^2 & = \int |\hat{f}(u)|^2 \underbrace{e^{-|u|^2 \sigma^2}}_{\nearrow 1 \text{ as }\sigma\to 0} \; \d u \\
(2\pi)^{d} \|f\|_2^2 &= \|\hat f\|_2^2.
\end{align*}
\end{proof}

Q: Fourier Transforms in $L^2$
A: There exists a unique Hsp automorphism $F: L^2 \to L^2$ such that
\[
F(f) = (2\pi)^{-d/2}\hat{f}
\]
whenever $f \in L^1 \cap L^2$.
\begin{proof}
We define $F_0: L^1 \cap L^2 \to L^2$ by
\[
F_0(f) = (2\pi)^{-d/2} \hat{f}.
\]
Plancherel $\implies$ $F_0$ is an isometry $\implies$ $F_0$ is injective.
Also $ C_C^\infty \subseteq L^1 \cap L^2  $ is dense in $L^2$. So we know $F_0$ extends uniquely to an isometry $F: L^2 \to L^2$ (defining $F(f)=\lim_n F(f_n)$).
For surjectivity, by IFT, $F$ sends
\[
V = \{f \in L^2: f, \hat{f} \in L^1\}
\]
to itself and $F^4|_V = \id$. So $V\subseteq \im F$ and $V$ contains the Gaussian convolutions so is dense in $\mathcal{L}^2$.
\end{proof}

Q: Uniqueness of characteristic functions
A: If $\mu,\nu$ are PMs on $\R^d$ such that $\hat \mu = \hat \nu$, then $\mu=\nu$.
\begin{proof}
Let $\sigma>0$, $ \sigma Z\sim N(0,\sigma^2 I_d)$ with pdf $G_\sigma$, we can check FIT applies:
\[
\widehat{\mu * G_\sigma}=\underbrace{\hat{\mu}}_{\in L^\infty}\underbrace{\hat{G}_\sigma}_{\in L^1} \in L^1.
\]
Let $g\in C_b(\R^d)$ and $X\sim \mu$. Then by DCT
\[
\mu(g) = \E g(X) = \lim_{\sigma\to 0} \E[g(X+\sigma Z)] =  \lim_{\sigma\to 0} \int g(x) \mu * G_\sigma(x) \; \d x.
\]
By FIT, RHS and thus $\E g(X)$ are determined by $\hat{\mu}$.
\end{proof}

Q: Measures of $C_b$
A: Let $\mu,\nu$ be finite BMs on $(X,d)$ and $\forall f\in C_b(X): \mu(f)=\nu(f)$. Then $\mu=\nu$.
\begin{proof}
Given $C\subseteq X$ closed, let $f_n(x) = (1-nd(x,C))^+$. Then $f_n\in C_b$ and $(f_n) \searrow\mathbf{1}_C$. By DCT $\mu(C)=\nu(C)$.
\end{proof}

Q: Existence of densities
A: If $\mu\in$ BM($\R^d$), $\hat{\mu}\in L^1(\R^d)$, then $\mu$ has density $f\in L^1(\R^d)$ wrt Lebesgue.
\begin{proof}
Let $f_\sigma = \mu * G_\sigma$. Since $f_\sigma, \hat{f}_\sigma\in L^1$, by FIT
\begin{align*}
f_\sigma(x) &= \frac{1}{(2\pi)^d} \int e^{-i\bra u, x \ket} \hat{\mu}(u) e^{- \| \sigma u\|^2}  \; \d u \\
\intertext{Since $\hat{\mu}\in L^1$, by DCT }
f_\sigma(x) &\to \frac{1}{(2\pi)^d} \int e^{-i\bra u, x \ket} \hat{\mu}(u) \; \d u =: f(x).
\end{align*}
Let $X\sim \mu, g\in C_b(\R^d), Z\sim N(0,I_d)$. Then by DCT
\[
\E g(X) = \lim_{\sigma\to 0} \E[g(X+\sigma Z)] = \lim_{\sigma\to 0} \int g(x) f_\sigma(x) \; \d x = \int g(x) f(x) \; \d x.
\]
\end{proof}

Q: Levy's continuity theorem
A: Let $(X_n),X$ be $\R^d$-valued. Then $X_n\xrightarrow{d} X$ $\iff$ $\hat{\mu}_{X_n} \to \hat{\mu}_X$ pointwise.
\begin{proof}
($\Rightarrow$). $x\mapsto e^{i\bra u, x\ket} \in C_b(\R^d)$.
($\Leftarrow$). Ets $\E g(X_n) \to \E g(X)$ for any $g\in C_C^\infty(\R^d)$ (ES4). Using Gaussian apx $\mu_{X_n} * G_\sigma \sim X_n+\sigma Z$, we can bound
\begin{align}
| \E g(X_n) - \E g(X) | \leq\quad &|\E (g(X_n) -  g(X_n+\sigma Z)| \\
+ & |\E (g(X_n+\sigma Z) - g(X+\sigma Z) ) | \\
+ &| \E (g(X+\sigma Z) - g(X)) |.
\end{align}
(1), (3) $\leq \sigma\| \nabla g \|_\infty \E |Z| \to 0$ as $\sigma\to 0$ (uniformly in $n$).
By FIT and DCT, as $n\to\infty$
\[
(2) = \int g(x)(\mu_{X_n}* G_\sigma - \mu * G_\sigma)(x) \; \d x \to 0.
\]
\end{proof}

Q: Projection in $L^2$ for countable disjoint events
A: Let $Y=\E[X | \mathcal{G}]$, $X\in L^2(\P)$, $\mathcal{G} = \sigma(G_n: n \in \N)$ for $G_n$ disjoint and $\bigcup_n G_n = \Omega$. Then $Y$ is the projection of $X$ onto the subspace $L^2(\mathcal{G}, \P)$.
\begin{proof}
Given $W$ a $\mathcal{G}$-msble rv, write
\[
W = \sum_{n = 1}^\infty a_n \mathbf{1}_{G_n}.
\]
where $a_n \in \R$. Then
\begin{align*}
\E[(X - W)^2] &= \E\left[\left(\sum_{n = 1}^\infty (X - a_n) \mathbf{1}_{G_n}\right)^2\right]\\
&= \E\left[\sum_n (X^2 + a_n^2 - 2a_n X)\mathbf{1}_{G_n}\right]\\
&= \E\left[\sum_n (X^2 + a_n^2 - 2a_n \E[X\mid G_n])\mathbf{1}_{G_n}\right]
\end{align*}
is minimised by $a_n = \E[X \mid G_n]$, i.e. $W = Y$.
\end{proof}

Q: Central Limit Theorem
A: Let $(X_n)$ be iid $\R^d$-valued rvs with $\E X^2<\infty$. Then
\[
\sqrt{n}(\bar X_n - \E X) \xrightarrow{d} N(0, \Sigma)
\]
where $\Sigma = \cov X$.
\begin{proof}
Wlog $d=1$ (by Levy), $\E X=0,\E X^2=1$.
 Differentiating under the expectation twice
\[
\phi(u)= \E[e^{iuX}],\quad \phi'(u) = \E[iX e^{iuX}],\quad \phi''(u) = \E[-X^2 e^{iuX}].
\]
Evaluating at $0$
\[
\phi(0) = 1,\quad \phi'(0) = 0,\quad \phi''(0) = -1.
\]
Taylor expanding $\phi$ at $0$
\[
\phi(u) = 1 - \frac{u^2}{2} + o(u^2).
\]
The MGF of $S_n/\sqrt{n}$ is
\begin{align*}
\phi_n(u) &= \E[e^{iuS_n/\sqrt{n}}] \\
&= \prod_{i = 1}^n \E[e^{iuX_i/\sqrt{n}}] \\
&= \phi(u/\sqrt{n})^n\\
&= \left(1 - \frac{u^2}{2n} + o\left(\frac{u^2}{n}\right)\right)^n \\
&\to e^{-\tfrac{u^2}{2}}.
\end{align*}
\end{proof}

Q: Measure-preserving map
A: Let $(X,\mathcal{A},\mu)$ be a measure space and $T:X\to X$ measurable. Then $\mu$ is preserved by $T$, or $T$ is $\mu$-preserving, if $\mu\circ T^{-1}=\mu$.

Q: Invariant function
A: A measurable function $f:X\to\R$ is called $T$-invariant if $f\circ T=f$.

Q: Invariant subset
A: A set $A\in\mathcal{A}$ is $T$-invariant if $T^{-1}A=A$.

Q: Invariant subalgebra
A: We write
\[
\mathcal{E}_\Theta = \{A \in \mathcal{E}: A\text{ is $\Theta$-invariant}\}.
\]
Then $\mathcal{E}_\Theta$ is a $\sigma$-algebra and $f: E \to \R$ is invariant iff it is $\mathcal{E}_\Theta$ measurable.

Q: Ergodic
A: $\Theta$ is \emph{ergodic} if $\forall A \in \mathcal{E}_\Theta$:  $\mu(A) = 0$ or $\mu(A^C) = 0$.

Q: Integration using measure-preserving map
A: If $f$ is integrable and $\Theta$ is measure-preserving, then $f \circ \Theta$ is integrable and
\[
\int f \circ \Theta \; \d \mu = \int f \; \d \mu.
\]

Q: Invariant functions: ergodic case
A: If $\Theta$ is ergodic and $f$ is $\Theta$-invariant, then $\exists c$ such that $f = c$ a.e.

Q: Bernoulli shifts
A: Let $(Y_i)$ be $\R$-valued on $(\Omega, \mathcal{F}, \P)$.
Let $E = \R^\N$, $\mathcal{E}=\sigma(\{ \pi_i: i \in\N \})$, $Y = (Y_i) : \Omega \to E$ be the sample path map (for $Y_i$ iid) and $\mu = \P \circ Y^{-1}$ (the product measure).
This $(E, \mathcal{E}, \mu)$ is known as the \term{canonical space}.
$\Theta$ is the (left) \term{shift map}.

Q: Shift map is ergodic
A: The shift map $\Theta$ is an ergodic, measure preserving transformation.
\begin{proof}
Given $A=\prod_i A_i \in \mathcal{E}_\Theta$, we have $\forall k\in\N$:
\[
A=\Theta^{-k}(A)= \{ \omega: (X_{n+k}  \in A_n)\} \in\mathcal{T}_k.
\]
Having checked on a generator, $\mathcal{E}_\Theta\subseteq \mathcal{T}$. From K0-1, $\mathcal{E}_\Theta$ is trivial.
\end{proof}

Q: Maximal ergodic lemma
A: Let $f$ be integrable and
\[
S^* = \sup_{n \geq 0} S_n(f).
\]
Then
\[
\int_{\{S^* > 0\}} f \;\d \mu \geq 0.
\]
\begin{proof}
Let $A_n = \{S_n^* > 0\}$ and
\[
S_n^* = \max_{0 \leq m \leq n} S_m.
\]
For $1 \leq m \leq n$
\[
S_m = f + S_{m - 1} \circ \Theta \leq f + S_n^* \circ \Theta.
\]
Thus
\[
S_n^* = \left\{\begin{alignedat}{2}
 \max_{1 \leq m \leq n} S_m &\leq f + S_n^* \circ \Theta  &&\text{ on } A_n \\
0 &\leq S_n^* \circ \Theta  &&\text{ on } A_n^C.
\end{alignedat}\right.
\]
Integrating
\begin{align*}
\int_E S_n^*\;\d \mu &\leq \int_{A_n} f\;\d \mu + \int_E S_n^* \circ \Theta \;\d \mu\\
&= \int_{A_n} f\;\d \mu + \int_E S_n^* \;\d \mu
\end{align*}
so we know
\[
\int_{A_n} f\;\d \mu \geq 0.
\]
Letting $n \to \infty$, $A_n\nearrow\{S^* > 0\}$, so done by DCT.
\end{proof}

Q: Birkhoff's ergodic theorem
A: Let $(E, \mathcal{E}, \mu)$ be $\sigma$-finite and $f\in L^1$. Then $\exists\bar{f}$ invariant s.t.
\[
\mu(|\bar{f}|) \leq \mu(|f|), \qquad \frac{S_n(f)}{n} \to \bar{f}\text{ a.e.}
\]
\begin{proof}
For $a < b$, let
\[
D = D(a, b) = \left\{x \in E: \liminf\frac{S_n(x)}{n} < a < b < \limsup \frac{S_n(x)}{n}\right\}.
\]
Since $\Q\subseteq\R$ is dense and by $\sigma$-subadditivity, ets $\mu(D(a, b)) = 0$.
$\limsup \frac{S_n}{n}, \liminf \frac{S_n}{n}$ invariant $\implies$ $D$ is invariant. By Maximal Ergodic Theorem
\[
b \mu(D) \leq \int_D f \;\d \mu, \quad (-a) \mu(D) \leq -\int_D f \;\d \mu.
\]
But $a < b$ so $\mu(D) = 0$. Now let $\bar f = \lim S_n(f)/n$ where it exists else 0.
Invariance follows since $\lim S_n(f)/n$ and $\supp\bar f$ are invariant. Finally
\begin{align*}
\mu(|f \circ \Theta^n|) &= \mu(|f|) \qquad \text{($\Theta$ measure-preserving)} \\
\mu(|S_n|) &\leq n \mu(|f|) < \infty \qquad \text{(summing)} \\
\mu(|\bar{f}|) &\leq \mu\left(\liminf_n \left|\frac{S_n}{n}\right|\right)  \qquad\text{(monotonicity)}\\
&\leq \liminf_n \mu\left(\left|\frac{S_n}{n}\right|\right)  \qquad\text{(Fatou)}\\
&\leq \mu(|f|)  \qquad \text{(first inequality)}.
\end{align*}
\end{proof}

Q: Maximal ergodic theorem
A: Let $(E, \mathcal{E}, \mu)$ be $\sigma$-finite, $f\in L^1 $ and $E_\alpha = \{ x\in E: \lim\sup S_n(f)>\alpha \}$. Then
\[
\alpha \mu(E_\alpha) \leq \int_{E_\alpha} f \; \d \mu.
\]
\begin{proof}
For $B\subseteq E_\alpha$ with $\mu(B)<\infty$, by the maximal ergodic lemma
\begin{align*}
g &:= (f - \alpha) \mathbf{1}_B \in L^1 \\
S_n(g) &= S_n(f) - n\alpha > 0  \text{ i.o.} \\
S^*(g) & > 0 \text{ on $B$} \\
0 &\leq \int_{B} f \;\d \mu - \alpha \mu(B) \\
\alpha \mu(B) &\leq \int_{B} f \;\d \mu.
\end{align*}
Partitioning $E$ into countably many such $B$, done.
\end{proof}

Q: von Neumann's ergodic theorem
A: Let $f \in L^p(E, \mathcal{E}, \mu)$ for $p \in [1, \infty)$ and $\mu$ finite. Then $\exists \bar{f} \in L^p$ such that
\[
\frac{S_n(f)}{n} \to \bar{f}\text{ in $L^p$}.
\]
\begin{proof}
From ES4 $\|f \circ \Theta\|_p^p = \|f\|_p^p$, so by Minkowski $ \norm{\frac{S_n}{n}}_p\leq \|f\|_p$.
Given $\varepsilon > 0$, pick $M >0$ such that
\[
g = (f \vee (-M)) \wedge M \implies \|f - g\|_p < \frac{\varepsilon}{3}.
\]
By Birkhoff's theorem $\frac{S_n(g)}{n} \to \bar{g}$ a.e. Since $\abs{\frac{S_n(g)}{n}} \leq M$, by DCT $\exists N\in\N$ such that $\forall n \geq N$:
\[
\norm{\frac{S_n(g)}{n} - \bar{g}}_p < \frac{\varepsilon}{3}.
\]
Then by Fatou
\begin{align*}
\norm{\bar{f} - \bar{g}}_p^p &= \int \liminf_n \abs{\frac{S_n(f - g)}{n}}^p \;\d \mu\\
&\leq \liminf \int \abs{\frac{S_n(f - g)}{n}}^p \;\d\mu\\
&\leq \|f - g\|_p^p.
\end{align*}
Hence $\forall n \geq N$:
\[
\norm{\frac{S_n(f)}{n} - \bar{f}}_p \leq \norm{\frac{S_n(f - g)}{n}}_p + \norm{\frac{S_n(g)}{n} - \bar{g}}_p + \norm{\bar{g} - \bar{f}}_p \leq \varepsilon.
\]
\end{proof}

Q: Strong law of large numbers
A: If $(Y_n)\subseteq L^1$ are iid, then
\[
\frac{S_n}{n} \to \E Y \text{ a.s.}
\]
\begin{proof}
By ergodic theorem with $\Theta$ ergodic
\[
\frac{S_n(Y_1)}{n} \to c\text{ a.e. and in }L^1(E, \mathcal{E}, \mu)
\]
where $(E, \mathcal{E}, \mu)$ is the canonical space
\[
c = \mu(c) = \lim_{n \to \infty} \mu(S_n/n) = \E Y.
\]
\end{proof}

Q: Riesz-Fischer Theorem for $L^\infty$
A: $L^\infty$ is a Banach space.
\begin{proof}
Given $(f_k)\subseteq L^\infty$ Cauchy, $\forall m\in\N, \exists n\in\N, \forall j,k \geq n$:
\[
\|f_j-f_k\|_\infty < \frac{1}{m} \text{ on } N_{j,k,m}^C \tag{$*$}
\]
where $N_{j,k,m}$ is null. Define
\[
N = \bigcup_{j,k,m\in\N} N_{j,k,m}.
\]
Then $N$ is null and $f_k$ is Cauchy on $N^C$. Write $f=\mathbbm{1}_{N^C}\lim f_k$. Letting $k\to\infty$ in $(*)$ gives
\[
\|f_j-f\| _\infty < \frac{1}{m} \text{ on } N^C
\]
Hence $f_j \to f \in L^\infty$.
\end{proof}

Q: Convergence in distribution by smooth functions
A: $\mu_n\wto \mu \iff \forall f\in C_C^\infty(\R^d): \mu_n(f)\to \mu(f)$.
\begin{proof}
Given $f\in C_b(\R), \varepsilon>0$, pick a ball $\bar{B}=\bar{B}(R)$ such that $\mu(\bar{B}(R)^C)<\varepsilon$. Note by $C_C^\infty$ approximation of indicator functions $\mu_n(\bar{B}) \to \mu(\bar{B})$.
By Stone-Weierstrass, $\exists p$ a polynomial such that $\|p-f\|_{\bar{B}(R)}<\varepsilon$. Composing with a $C^\infty$ cutoff function, $\exists g\in C_C^\infty$ such that $\|g-f\|_{\bar{B}(R)}<\varepsilon$. Then for $n$ large enough
\[
|\mu_n(g) - \mu(g)|<\varepsilon, \qquad \|f-g\|_\infty (\mu_n(\bar{B}^C) + \mu(\bar{B}^C) ) < \varepsilon
\]
such that
\begin{align*}
|\mu_n(f)-\mu(f)| & \leq |\mu_n (f-g)| +  |\mu_n(g) - \mu(g)|  +  |\mu (g - f)| \\
& < 3\varepsilon + |\mu_n ((f-g)\mathbbm{1}_{\bar{B}^C})|   + |\mu ((f-g)\mathbbm{1}_{\bar{B}^C})|  \\
&\leq 3\varepsilon +  \|f-g\|_\infty (\mu_n(\bar{B}^C) + \mu(\bar{B}^C) ) \\
&< 4 \varepsilon.
\end{align*}
\end{proof}

Q: Independence for complements
A: $(A_n)$ are MI $\iff$ $(\sigma(A_n))$ are MI.
\begin{proof}
Induction on $|J|$ in
\[
\P\left(\bigcap_I A_i \cap \bigcap_J A_i^C\right) = \prod_I \P(A_i) \prod_J \P(A_i^C).
\]
\end{proof}

Q: Independence from densities
A: $X_1, \cdots X_n$ are MI $\iff$ $f(x_1, \cdots x_n) = f_1(x_1) \cdots f_n(x_n)$.
\begin{proof}
We have that the integral of LHS$-$RHS vanishes over all sets in the generating $\pi$-system.
By Dynkin's Lemma, the integral vanishes over all Borel sets. Hence LHS=RHS.
\end{proof}

Q: Scheff\'e's Lemma
A: Let $(f_n), f\subseteq L^1$ and $f_n\to f$ a.e. If $\|f_n\|_1 \to \|f\|_1$, then $(f_n)\to f_1$ in $L^1$. In particular, convergence of densities implies convergence in distribution.
\begin{proof}
Define $g_n=|f_n| - |f_n-f|$. Then $|g_n|\leq |f|$ and $(g_n)\to |f|$ a.e. so by DCT
\[
\|f_n\|_1 - \|f_n-f\|_1 \to \|f\|_1.
\]
Hence $\|f_n\|_1 \to \|f\|_1 \iff \|f_n-f\|_1 \to 0$.
\end{proof}

Q: Riemann–Lebesgue lemma
A: If $f\in L^1(\R^d)$, then $\hat{f}(u)\to 0$ as $|u|\to\infty$.
\begin{proof}
Given $g\in L^1(\R^d)$, we have
\[
|\hat{f}(u) - \hat{g}(u)| \leq \left|\int |e^{iux}|| f(x)- g(x) | \; \d x \right| = \|f-g\|_1.
\]
Hence ets for a dense subset of $L^1$. For $f=\mathbbm{1}_{[a,b]}$ a step function
\[
\hat{f}(u) = \int_a^b e^{iux} \; \d x = \frac{e^{iub} - e^{iua}}{iu} \to 0.
\]
Similarly for $f\in C_C^\infty$, integrating by parts
\[
|\hat{f}(u)| = \left| \frac{1}{iu} \int_\R f'(x) e^{iux} \; \d x \right| \leq \frac{1}{|u|} \|f'\|_1 \to 0.
\]
\end{proof}

Q: Convergence in distribution vs. weakly
A: $(X_n)\xrightarrow{d}X\iff (\mu_n)\wto \mu$.
\begin{proof}
($\Rightarrow$) We are given convergence on intervals, which implies true for step functions. Hence true for $C_b(K)$ by density of step functions in $L^1$.
($\Leftarrow$) Take $C_b$ approximations of intervals at continuity points of $F$.
\end{proof}