Title: Vectors and Matrices

Q: Triangle inequality
A: For all $z_1, z_2 \in \mathbb{C}$, we have
  \[
    ||z_1| - |z_2||\leq  |z_1 + z_2| \leq |z_1| + |z_2|.
  \]

Q: Show $\exp(z_1+z_2)=\exp(z_1)\exp(z_2)$
A: $\exp(z_1)\exp(z_2) = \exp(z_1 + z_2)$
\begin{proof}
  \begin{align*}
    \exp(z_1)\exp(z_2) &= \sum_{n = 0}^\infty\sum_{m = 0}^\infty \frac{z_1^m}{m!}\frac{z_2^n}{n!}\\
    &= \sum_{r = 0}^\infty\sum_{m = 0}^r \frac{z_1^{r - m}}{(r - m)!}\frac{z_2^m}{m!}\\
    &= \sum_{r = 0}^\infty\frac{1}{r!}\sum_{m = 0}^r \frac{r!}{(r - m)!m!}z_1^{r - m}z_2^m\\
    &= \sum_{r = 0}^\infty\frac{(z_1 + z_2)^r}{r!} \\
& = \exp(z_1 + z_2)
  \end{align*}
\end{proof}
Here we used
\[
    \sum_{n = 0}^\infty\sum_{m = 0}^\infty a_{mn} = \sum_{r = 0}^\infty\sum_{m = 0}^r a_{r - m, m}
  \]
\begin{proof}
  \begin{align*}
    \sum_{n = 0}^\infty\sum_{m = 0}^\infty a_{mn} &= a_{00} + a_{01} + a_{02} + \cdots\\
    &+ a_{10} + a_{11} + a_{12} + \cdots\\
    &+ a_{20} + a_{21} + a_{22} + \cdots\\
    &= (a_{00}) + (a_{10} + a_{01}) + (a_{20} + a_{11} + a_{02}) + \cdots\\
    &= \sum_{r = 0}^\infty\sum_{m = 0}^r a_{r - m, m}
  \end{align*}
\end{proof}

Q: Complex logarithm
A: The \emph{complex logarithm} $w = \log z$ is a solution to $e^\omega = z$. Therefore
\begin{align*}
\log z&=\log(re^{i\theta}) \\
&=\log r + i\theta \\
&=\log|z|+i\arg(z)+2in\pi
\end{align*}
This can be made into a single-valued function if we restrict to $-\pi < \theta \leq \pi$.

Q: Complex power
A: The \emph{complex power} $z^\alpha$ for $z, \alpha\in \mathbb{C}$ is defined as
\begin{align*}
z^\alpha &= e^{\alpha\log z} \\
&=\exp\left(\alpha[\log |z|+i\arg (z)+2in\pi ] \right) \\
&=e^{\alpha\log|z|}e^{i\alpha\theta}e^{2in\pi\alpha}
\end{align*}
This, again, can be multi-valued, as $z^\alpha = e^{\alpha\log|z|}e^{i\alpha\theta}e^{2in\pi\alpha}$ (there are finitely many values if $\alpha\in\mathbb{Q}$, infinitely many otherwise). Nevertheless, we make $z^\alpha$ single-valued by insisting $-\pi < \theta \leq \pi$.

Q: De Moivre's theorem
A: For $n\in\mathbb{Z}$:
\[
    \cos n\theta + i\sin n\theta = (\cos\theta + i\sin\theta)^n.
  \]
\begin{proof}
  First prove for the $n \geq 0$ case by induction. The $n = 0$ case is true since it merely reads $1 = 1$. We then have
  \begin{align*}
    (\cos\theta + i\sin\theta)^{n + 1} &= (\cos\theta + i\sin\theta)^n (\cos\theta + i\sin\theta)\\
    &= (\cos n\theta + i\sin n\theta )(\cos\theta + i\sin\theta)\\
    &= \cos(n+1)\theta + i\sin(n+1)\theta
  \end{align*}
  If $n < 0$, let $m = -n$. Then $m > 0$ and
  \begin{align*}
    (cos\theta + i\sin\theta)^{-m} &= (\cos m\theta + i\sin m\theta)^{-1}\\
    &= \frac{\cos m\theta - i\sin m\theta}{(\cos m\theta + i\sin m\theta)(\cos m\theta - i\sin m\theta)}\\
    &= \frac{\cos (-m\theta) + i\sin (-m\theta)}{\cos^2 m\theta + \sin^2 m\theta}\\
    &= \cos (-m\theta) + i\sin (-m\theta)\\
    &= \cos n\theta + i\sin n\theta
  \end{align*}
\end{proof}
For non-integer powers this gives one of its values. In particular, for $p,q\in\mathbb{Z}$:
\[
\left(\cos\theta+i\sin\theta\right)^{\frac{p}{q}}= \cos\left(\frac{p\theta+2k\pi}{q}\right)+i\sin\left(\frac{p\theta+2k\pi}{q}\right)
\]
where $k=0,1,\ldots q-1$.

Q: Express $\cos n\theta$ in powers of $\cos\theta$ and $\sin n\theta$ in powers of $\sin\theta$
A: $(\cos\theta +i\sin\theta)^n=\cos n\theta +\sin n\theta$ (De Moivre's Theorem)
$(\cos\theta +i\sin\theta)^n=\displaystyle\sum _{k=0}^n {n \choose k} (\cos\theta)^k(i\sin\theta)^{n-k}$ (Binomial expansion)
Therefore:
$\cos n\theta +i\sin n\theta=\displaystyle\sum _{k=0}^n {n \choose k} (\cos\theta)^k(i\sin\theta)^{n-k}$
Equating real and imaginary components gives that $\cos n\theta$ is the sum of the even numbered terms and $\sin n\theta$ is the sum of the odd numbered terms in the binomial expansion respectively.

Q: Express $\cos^n\theta$ as the sum of $\cos$ of larger angles.
A: $(z+\frac{1}{z})^n=(2\cos\theta)^n=2^n\cos^n\theta$ (De Moivre's Theorem)
$(z+\frac{1}{z})^n=\displaystyle\sum _{k=0}^n {n \choose k} z^k\left(\frac{1}{z}\right)^{n-k}$ (Binomial Expansion)
Therefore:
\begin{align*}
2^n\cos^n\theta&=\sum _{k=0}^n {n \choose k} z^k\left(\frac{1}{z}\right)^{n-k}\\
&=\displaystyle\sum _{k=0}^n {n \choose k} z^{2k-n}\\
&={n \choose 0} z^{-n}+{n \choose 1} z^{2-n}\ldots {n \choose k} z^{2(n-1)-n} + {n \choose n} z^{2n-n}\\
&={n \choose 0} z^{-n}+{n \choose 1} z^{2-n}\ldots {n \choose k} z^{n-2} + {n \choose n} z^{n}\\
&={n \choose 0} (z^{n}+z^{-n})+{n \choose 1} (z^{n-2} +z^{-(n-2)}) \ldots
\end{align*}

Q: Line through $z_0 \in \mathbb{C}$ parallel to $w\in \mathbb{C}$
A: \textbf{Parametric form} $z = z_0 + \lambda w$ where $\lambda$ can take any real value.
\textbf{Non-parametric form}
\begin{align*}
\lambda &= \frac{z - z_0}{w} \\
\implies \bar{\lambda} &= \frac{\bar{z} - \bar{z_0}}{\bar{w}}
\end{align*}
But $\lambda\in\mathbb{R}$ so $\lambda=\bar\lambda$. Therefore
\begin{align*}
  \frac{z - z_0}{w} &= \frac{\bar{z} - \bar{z_0}}{\bar{w}}\\
\implies  z\bar w - \bar z w &= z_0 \bar w - \bar z_0 w.
\end{align*}

Q: Properties of vector addition
A: \textsc{Identity} There is a vector $\mathbf{0}$ such that $\mathbf{a} + \mathbf{0} = \mathbf{a}=\mathbf{0}+\mathbf{a}$. \hfill
\textsc{Inverse} For all vectors $\mathbf{a}$, there is a vector $(-\mathbf{a})$ such that $\mathbf{a} + (-\mathbf{a}) = \mathbf{0}=(-\mathbf{a}) +\mathbf{a}$
\textsc{Associativity} $(\mathbf{a} + \mathbf{b}) + \mathbf{c} = \mathbf{a} + (\mathbf{b} + \mathbf{c})$
\textsc{Commutativity}   $\mathbf{a} + \mathbf{b} = \mathbf{b} + \mathbf{a}$

Q: Properties of scalar multiplication of vectors
A: \begin{enumerate}
    \item $\lambda(\mathbf{a + b}) = \lambda\mathbf{a} + \lambda\mathbf{b}$.
    \item $(\lambda + \mu)\mathbf{a} = \lambda\mathbf{a} + \mu\mathbf{a}$.
    \item $\lambda(\mu\mathbf{a}) = \mu(\lambda\mathbf{a}) = (\mu\lambda)\mathbf{a}$.
    \item $1\cdot\mathbf{a = a}$.
  \end{enumerate}

Q: Vector space
A: A \emph{vector space} is a set of vectors $\mathbf{v}\in V$ with closure under scalar multiplication and vector addition.
These vectors form an additive abelian group.

Q: Inner product
A: In a vector space $V$, a \emph{inner product} or \emph{scalar product} is a map $V\times V\to \mathbb{C}$ that satisfies the following axioms. It is written as $\mathbf{x\cdot y}$ or $\langle\mathbf{x\mid y}\rangle$.
  \begin{enumerate}
    \item $\mathbf{x\cdot y = \overline{y\cdot x}}$ \hfill (conjugate symmetry)
    \item $\mathbf{x}\cdot (\lambda\mathbf{y} + \mu\mathbf{z}) = \lambda\mathbf{x\cdot y} + \mu\mathbf{x\cdot z}$ \hfill (linearity in 2nd argument)
    \item $\mathbf{x\cdot x}\geq 0$ with equality iff $\mathbf{x = 0}$\hfill (positive definite)
  \end{enumerate}
Therefore
\begin{align*}
(\lambda\mathbf{y} + \mu\mathbf{z})\cdot\mathbf{x} &=\overline{\mathbf{x}\cdot (\lambda\mathbf{y} + \mu\mathbf{z})}  \\
&= \bar\lambda\mathbf{y}\cdot \mathbf{x} + \bar\mu\mathbf{z}\cdot \mathbf{x}
\end{align*}
so, in particular, the inner product is linear in the second argument for real vector spaces.
 The standard inner product in $\mathbb{C}^n$ is \[
\vec{x\cdot y}=\sum_{i=1}^n \bar x_iy_i.
\]

Q: Cauchy-Schwarz inequality
A: For all $\mathbf{x, y}\in \mathbb{R}$,
  \[
    \mathbf{x\cdot y \leq |x||y|}.
  \]
where $|\vec{x}|$ is the positive square root of $\vec{x}\cdot \vec{x}$.
\begin{proof}
  Consider the expression $|\mathbf{x} - \lambda \mathbf{y}|^2$. From the definition of the real inner product
  \begin{align*}
    (\mathbf{x} - \lambda\mathbf{y})\cdot (\mathbf{x} - \lambda\mathbf{y}) \geq 0\\
    \lambda^2 |\mathbf{y}|^2 - \lambda (2\mathbf{x\cdot y}) + |\mathbf{x}|^2 \geq 0.
  \end{align*}
with equality iff. $\mathbf{x} - \lambda\mathbf{y}=\vec{0}$.
 We can view this as a non-negative quadratic in $\lambda$ which cannot have 2 real roots. Thus the discriminant $\Delta \leq 0$. So
  \begin{align*}
    ((-2\mathbf{x\cdot y}))^2-4|\mathbf{y}|^2|\mathbf{x}|^2 &\leq 0 \\
4(\mathbf{x\cdot y})^2 &\leq 4|\mathbf{y}|^2|\mathbf{x}|^2\\
    (\mathbf{x\cdot y})^2 &\leq |\mathbf{x}|^2|\mathbf{y}|^2\\
    \mathbf{x\cdot y} &\leq \mathbf{|x||y|}
  \end{align*}
with equality iff. the quadratic has one real root i.e. the quadratic is zero so $\mathbf{x} - \lambda\mathbf{y}=\vec{0}$.
\end{proof}

Q: Show that a vector in 2D written wrt some basis is unique
A: \begin{proof}
  Suppose that $\mathbf{r} = \lambda\mathbf{a} + \mu\mathbf{b} = \lambda'\mathbf{a} + \mu'\mathbf{b}$. Take the vector product with $\mathbf{a}$ on both sides to get $(\mu - \mu')\mathbf{a\times b} = 0$. Since $\mathbf{a\times b}\not= 0$, then $\mu=\mu'$. Similarly, $\lambda = \lambda'$.
\end{proof}

Q: Properties of vector product
A: \[
    \mathbf{a\times b} = \mathbf{|a||b|}\sin\theta \hat{\mathbf{n}},
  \]
\begin{enumerate}
    \item $\mathbf{a\times b = -b\times a}$ (as $\hat{n}$ points in the opposite direction by RH rule)
    \item $\mathbf{a\times a = 0}$ ($\theta =0$)
    \item $\mathbf{a\times b = 0}\Rightarrow \mathbf{a} = \lambda\mathbf{b}$ for some $\lambda\in \mathbb{R}$ (i.e. $\theta =0$) or $\vec{a}=\vec{0}$ or $\vec{b}=\vec{0}$.
    \item $\mathbf{a}\times (\lambda \mathbf{b}) = \lambda(\mathbf{a\times b})$ ($|\vec{b}|$ scales by $\lambda$).
\item $\vec{a}\cdot\vec{b}\times\vec{c}= \vec{c}\cdot\vec{b}\times\vec{a}= \vec{b}\cdot\vec{c}\times\vec{a}$ (considering geometric view of the scalar triple product)
  \end{enumerate}
More difficult, we can prove distributivity: $\mathbf{a\times (b + c) = a\times b + a\times c}$.
\begin{proof}
  Let $\mathbf{d = a\times (b + c) - a\times b - a\times c}$. We have
  \begin{align*}
    \mathbf{d\cdot d} &= \mathbf{d\cdot[a\times (b + c)] - d\cdot(a\times b) - d\cdot(a\times c)}\\
    &= \mathbf{(b+c)\cdot(d \times a) - b\cdot(d\times a) - c\cdot(d\times a)}\\
    &= \mathbf{0}
  \end{align*}
  Thus $\mathbf{d = 0}$.
\end{proof}

Q: Forming a basis of $\mathbb{R}^3$
A: If $\mathbf{a}, \mathbf{b}, \mathbf{c}\in\mathbb{R}^3$ are non-coplanar, ie. $\mathbf{a}\cdot(\mathbf{b}\times \mathbf{c})\not= 0$, then they form a basis of $\mathbb{R}^3$.
\begin{proof}
  For any $\mathbf{r}$, write
\begin{align*}
\mathbf{r} &= \lambda\mathbf{a} + \mu\mathbf{b} + \nu\mathbf{c} \\
\therefore\mathbf{r\cdot(b\times c)} &= \lambda \mathbf{a\cdot(b\times c)} + \mu\mathbf{b\cdot (b\times c)} + \nu\mathbf{c\cdot(b\times c)} \\
&= \lambda \mathbf{[a, b, c]} \\
\therefore\lambda &= \frac{\mathbf{[r, b, c]}}{\mathbf{[a,b, c]}}
\end{align*}
The values of $\mu$ and $\nu$ can be found similarly. Thus each $\mathbf{r}$ can be written as a linear combination of $\mathbf{a}, \mathbf{b}$ and $\mathbf{c}$.
  By the formula derived above, it follows that if $\alpha\mathbf{a} + \beta\mathbf{b} + \gamma\mathbf{c} = \mathbf{0}$, then $\alpha = \beta = \gamma = 0$. Thus they are linearly independent.
\end{proof}

Q: Linearly independent vectors
A: A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\cdots \mathbf{v}_m\}$ is \emph{linearly independent} if
  \[
    \sum_{i = 1}^m\lambda_i\mathbf{v}_i = \mathbf{0} \Rightarrow \forall i: \lambda_i = 0.
  \]

Q: Spanning set
A: A set of vectors $\{\mathbf{u}_1, \mathbf{v}_2, \mathbf{v}_3\cdots \mathbf{v}_m\}\subseteq V$ is a \emph{spanning set} of $V$ if
  \[
    \forall \mathbf{x} \in V:\exists \lambda_i: \sum_{i = 1}^n\lambda_i\mathbf{v}_i = \mathbf{x}
  \]

Q: Basis vectors
A: A linearly independent spanning set.

Q: Orthonormal
A: $\{\mathbf{e}_i\}$ is \emph{orthonormal} if \[
\mathbf{e}_i\cdot \mathbf{e}_j=\begin{cases}
0 \textrm{ if } i\neq j \\
1 \textrm{ if } i=j
\end{cases}
\]

Q: Dimension of vector space
A: The \emph{dimension} of a vector space is the number of vectors in its basis.

Q: Vector subspace
A: A \emph{vector subspace} of a vector space $V$ is a subset of $V$ that is also a vector space under the same operations. Both $V$ and $\{\mathbf{0}\}$ are subspaces of $V$. All others are proper subspaces.
A useful criterion is that a subset $U\subseteq V$ is a subspace iff
  \begin{enumerate}
    \item $\mathbf{x, y}\in U \Rightarrow (\mathbf{x + y}) \in U$.
    \item $\mathbf{x}\in U \Rightarrow \lambda\mathbf{x} \in U$ for all scalars $\lambda$.
    \item $\mathbf{0}\in U$.
  \end{enumerate}
  This can be more concisely written as ``$U$ is non-empty and for all $\mathbf{x, y}\in U$, $(\lambda\mathbf{x} + \mu\mathbf{y})\in U$''.

Q: Rules of Einstein summation convention
A: \begin{enumerate}
    \item Suffix appears once in a term: free suffix
    \item Suffix appears twice in a term: dummy suffix and is summed over
    \item Suffix appears three times or more: WRONG!
  \end{enumerate}

Q: Kronecker delta
A: \[
    \delta_{ij} =
    \begin{cases}
      1 & i = j\\
      0 & i\not=j
    \end{cases}.
  \]
  We have
  \[
    \begin{pmatrix}
      \delta_{11} & \delta_{12} & \delta_{13}\\
      \delta_{21} & \delta_{22} & \delta_{23}\\
      \delta_{31} & \delta_{32} & \delta_{33}
    \end{pmatrix} =
    \begin{pmatrix}
      1 & 0 & 0\\
      0 & 1 & 0\\
      0 & 0 & 1
    \end{pmatrix}
    = \mathbf{I}.
  \]
  So $\delta_{ij}$ represents an identity matrix.

Q: Alternating symbol
A: \[
    \varepsilon_{ijk} =
    \begin{cases}
      +1 & \textrm{ if } ijk \text{ is an even permutation}\\
      -1 & \textrm{ if }ijk\text{ is odd permutation}\\
      0 & \text{otherwise (ie. repeated suffices)}
    \end{cases}
  \]

Q: Cross product: suffix notation
A: $(\mathbf{a} \times \mathbf{b})_i = \varepsilon_{ijk}a_jb_k$

Q: Relationship between Kronkecker delta and alternating symbol
A: $\varepsilon_{ijk}\varepsilon_{ipq} = \delta_{jp}\delta_{kq} - \delta_{jq}\delta_{kp}$
\begin{proof}
  \begin{align*}
    \text{RHS} &= \begin{cases}
      +1 &\text{ if } j = p\neq k = q\\
      -1 &\text{ if } j = q \neq k = p\\
      0 &\text{ otherwise}
    \end{cases} \\
&=\text{LHS}
  \end{align*}
\end{proof}

Q: Triple product expansion
A: \[
    \mathbf{a\times (b\times c) = (a\cdot c)b - (a\cdot b)c}
  \]
\begin{proof}
  \begin{align*}
    \mathbf{[a\times(b\times c)]}_i &= \varepsilon_{ijk} a_j(b\times c)_k \\
    &= \varepsilon_{ijk}\varepsilon_{kpq}a_jb_pc_q\\
    &= \varepsilon_{ijk}\varepsilon_{pqk} a_jb_pc_q\\
    &= (\delta_{ip}\delta_{jq}-\delta_{iq}\delta_{jp})a_jb_pc_q\\
    &= a_jb_ic_j - a_jc_ib_j\\
    &= \mathbf{(a\cdot c)}b_i - \mathbf{(a\cdot b)}c_i
  \end{align*}
\end{proof}
\[ \mathbf{(a\times b)\times c = (a\cdot c)b - (b\cdot c)a}
\]
\begin{proof}
  \begin{align*}
   [\mathbf{(a\times b)\times c]}_i &= \varepsilon_{ijk} (a\times b)_j c_k\\
&= \varepsilon_{ijk}\varepsilon_{jpq}a_pb_q c_k\\
&= \varepsilon_{ijk}\varepsilon_{qjp}a_pb_q c_k\\
&= (\delta_{iq}\delta{kp}-\delta_{ip}\delta_{kq})a_pb_q c_k\\
&= a_pb_i c_p-a_ib_qc_q\\
    &= (\vec{a}\cdot \vec{c})b_i - (\vec{b}\cdot \vec{c})a_i
  \end{align*}
\end{proof}
Remember as `outer' minus `inner'.

Q: Scalar quadruple product
A: $\mathbf{(a\times b)\cdot (c\times d) = (a\cdot c)(b\cdot d) - (a\cdot d)(b\cdot c)}$.
\begin{proof}
  \begin{align*}
    \text{LHS} &= (\mathbf{a\times b})_i(\mathbf{c\times d})_i\\
    &= \varepsilon_{ijk}a_jb_k\varepsilon_{ipq} c_pd_q\\
    &= (\delta_{jp}\delta_{kq} - \delta_{jq}\delta_{kp})a_jb_kc_pd_q\\
    &= a_jb_k c_jd_k - a_j b_k c_k d_j\\
    &= \mathbf{(a\cdot c)(b\cdot d) - (a\cdot d)(b\cdot c)} \\
&=\text{RHS}
  \end{align*}
\end{proof}

Q: Spherical cosine rule
A: Consider the unit sphere, center $O$, with $\mathbf{a, b, c}$ on the surface.
Suppose we are living on the surface of the sphere. So the distance from $A$ to $B$ is the arc length on the sphere. We can imagine this to be along the circumference of the circle through $A$ and $B$ with center $O$. So the distance is $\angle AOB$, which we shall denote by $\delta (A, B)$. So $\mathbf{a}\cdot \mathbf{b} = \cos \angle AOB = \cos \delta (A, B)$ and $|\mathbf{a}\times \mathbf{b}| = \sin \delta(A, B)$.
Let $\alpha$ be the angle $\angle BAC$ i.e. the angle between the (normals to the) planes $OAB$ and $OAC$. Then
\begin{align*}
  \cos \alpha &= \mathbf{\frac{(a\times b)\cdot(a\times c)}{|a\times b||a\times c|}}\\
  &= \mathbf{\frac{b\cdot c - (a\cdot b)(a\cdot c)}{|a\times b||a\times c|}} \\
&= \frac{\cos\delta(B,C) - \cos\delta(A,B)\cos\delta(A,C)}{\sin\delta(A,B)\sin\delta(A,C)}
\end{align*}

Q: Spherical equilateral triangle
A: Using the spherical cosine rule,
\begin{align*}
  \cos\alpha &= \frac{\cos\delta - \cos^2\delta}{\sin^2\delta} \\
&= \frac{\cos\delta(1 - \cos\delta)}{(1-\cos^2\delta)} \\
&= \frac{\cos\delta}{1+\cos\delta} \\
&= 1 - \frac{1}{1 + \cos\delta}.
\end{align*}
Since $\cos\delta\leq 1$, we have $\cos\alpha\leq \frac{1}{2}$ and $\alpha \geq 60^\circ$. Equality holds iff $\delta = 0$, ie. the triangle is simply a point. So each angle is greater than $60^\circ$, and the angle sum is greater than $180^\circ$.

Q: Vector equation of a line
A: A line through $\mathbf{a}$ and parallel to $\mathbf{t}$ can be written as
\[
  \mathbf{x} = \mathbf{a} + \lambda\mathbf{t}
\]
where $\lambda$ is an arbitrary scalar, or
  \[
    \mathbf{(x - a)\times t = 0}\text{ or }\mathbf{x\times t = a\times t}.
  \]

Q: Vector equation of a plane
A: The equation of a plane through $b$ with normal $n$ is given by
  \[
   \mathbf{(x - b)\cdot n = 0} \text{ or } \mathbf{x\cdot n = b\cdot n}.
  \]
If $\mathbf{n = \hat n}$ is a unit normal, then $d = \mathbf{x\cdot\hat{n} = b\cdot\hat{n}}$ is the perpendicular distance from the origin to $\Pi$.
Alternatively, if $\mathbf{a, b, c}$ lie in the plane, then the equation of the plane is
\[
  \mathbf{(x - a)\cdot [(b - a)\times (c - a)] = 0}.
\]

Q: Intersection of a line and a plane
A: \begin{align*}
\text{Plane: }\mathbf{x\cdot n} &= \mathbf{b\cdot n} \\
\text{Line: }\mathbf{x\times t} &= \mathbf{a\times t} \\
\mathbf{(x\times t})\times\vec{n} &= \mathbf{(a\times t)}\times\vec{n} \\
\mathbf{(x\cdot n)t - (t\cdot n)x} &=\mathbf{ (a\times t)\times n} \\
\mathbf{(t\cdot n)x }&=\mathbf{ (b\cdot n)t - (a\times t)\times n} \\
\mathbf{x }&= \mathbf{\frac{(b\cdot n)t - (a\times t)\times n}{t\cdot n}} \quad (\mathbf{t\cdot n}\neq 0)
\end{align*}
If $\mathbf{t\cdot n}=0$, then the line is parallel to the plane. So if $\mathbf{a\cdot n} = \mathbf{b\cdot n}$, then the whole line is a solution. Otherwise, there are no solutions.

Q: Shortest distance between two lines.
A: Let $L_1$ be $(\mathbf{x} - \mathbf{a}_1)\times \mathbf{t}_1 = 0$ and $L_2$ be $(\mathbf{x} - \mathbf{a}_2)\times \mathbf{t}_2 = 0$.
      The distance of closest approach $s$ is along a line perpendicular to both $L_1$ and $L_2$, ie. parallel to $\mathbf{t}_1\times \mathbf{t}_2$. The distance $s$ can then be found by projecting $\mathbf{a}_1 - \mathbf{a}_2$ onto $\mathbf{t}_1\times \mathbf{t}_2$. Thus \[s = \left|(\mathbf{a}_1 - \mathbf{a}_2)\cdot\frac{\mathbf{t}_1\times \mathbf{t}_2}{|\mathbf{t}_1\times \mathbf{t}_2|}\right|.\]

Q: Solving vector equations
A: \begin{itemize}
\item To eliminate $\vec x$, take the cross product with $\vec x$.
\item To eliminate $\vec x$ and $\vec y$, take the dot product with $\vec x \times \vec y$.
\item If you have an equation with $\vec x$ and $\vec x \cdot \vec a$, then you can solve for $\vec x \cdot \vec a$ by taking the dot product with $\vec a$.
\end{itemize}

Q: Rotation matrix
A: We rotate by $\theta$ about $\hat {\mathbf{n}}$.
[/latex]
[latex]
We have $\mathbf{x'} = \overrightarrow{OB} + \overrightarrow{BC} + \overrightarrow{CA'}$ where
\begin{align*}
\overrightarrow{OB} &= \mathbf{(x\cdot\hat{n})\hat{n}}\\
|\overrightarrow{BC}| &= |\overrightarrow{BA'}|\cos\theta \\
&= |\overrightarrow{BA}|\cos\theta \\
\implies \overrightarrow{BC} &= \overrightarrow{BA}\cos\theta \quad\\
&= (\overrightarrow{BO} + \overrightarrow{OA})\cos\theta. \\
&= \mathbf{(-(x\cdot\hat{n})\hat{n} + x)}\cos\theta \\
|\overrightarrow{CA'}|&=|\overrightarrow{BA'}|\sin\theta \\
&=|\overrightarrow{BA}|\sin\theta \\
&= |\mathbf{\hat{n}\times x}|\sin\theta \\
\implies\overrightarrow{CA}&=(\hat{\mathbf{n}} \times \mathbf{x})\sin \theta \\
&=-(\hat{\mathbf{x}} \times \mathbf{n})\sin \theta
\end{align*}
Thus $\mathbf{x'} =\vec x \cos\theta+(1 - \cos\theta)(x\cdot\hat{n})\hat{n}-(\hat{\mathbf{x}} \times \mathbf{n})\sin \theta$. In components,
\begin{align*}
x_i' &= x_i\cos\theta + (1 - \cos\theta)n_jx_jn_i - \varepsilon_{ijk}x_jn_k\sin\theta \\
&=R_{ij}x_j \\
\therefore R_{ij} &= \delta_{ij}\cos\theta + (1 - \cos\theta)n_in_j - \varepsilon_{ijk}n_k\sin\theta.
\end{align*}

Q: Reflection matrix
A: We reflect through a plane through $O$ with normal $\hat{\mathbf{n}}$. [/latex]
[latex]
\begin{align*}
\mathbf{x}' &= \mathbf{x} - 2\mathbf{(x\cdot \hat{n})\hat{n}} \\
x_i' &= x_i - 2x_jn_jn_i \\
&=R_{ij}x_i \\
R_{ij} &= \delta_{ij} - 2n_in_j.
\end{align*}

Q: Linear map
A: Let $V, W$ be real (or complex) vector spaces, and $T: V\to W$. Then $T$ is a \emph{linear map} if
  \begin{enumerate}
    \item $T(\mathbf{a + b}) = T(\mathbf{a}) + T(\mathbf{b})$ for all $\mathbf{a, b}\in V$.
    \item $T(\lambda\mathbf{a}) = \lambda T(\mathbf{A})$ for all $\lambda \in \mathbb{R}$ (or $\mathbb{C}$).
  \end{enumerate}
  Equivalently, $T(\lambda\mathbf{a} + \mu\mathbf{b}) = \lambda T(\mathbf{a}) + \mu T(\mathbf{b})$.

Q: Prove that kernels and images of linear maps are subspaces
A: Consider a linear map $f: U\to V$, where $U, V$ are vector spaces. Then $\mathrm{Im}f$ is a subspace of $V$, and $\mathrm{Ker}f$ is a subspace of $U$.
\begin{proof}
  Both are non-empty since $f(\mathbf{0}) = \mathbf{0}$.
  If $\mathbf{x, y}\in $ Im$f$, then $\exists \mathbf{a, b}\in U$ such that $\mathbf{x} = f(\mathbf{a}), \mathbf{y} = f(\mathbf{b})$. Then $\lambda \mathbf{x} + \mu\mathbf {y} = \lambda f(\mathbf{a}) + \mu f(\mathbf{b}) = f(\lambda\mathbf{a} + \mu\mathbf{b})$. Now $\lambda\mathbf{a} + \mu\mathbf{b}\in U$ since $U$ is a vector space, so there is an element in $U$ that maps to $\lambda\mathbf{x}+ \mu\mathbf{y}$. So $\lambda\mathbf{x}+ \mu\mathbf{y}\in \mathrm{Im}f$.
  Suppose $\mathbf{x, y}\in \mathrm{Ker}f$, i.e. $f(\mathbf{x}) = f(\mathbf {y}) = 0$. Then $f(\lambda\mathbf{x} + \mu\mathbf{y}) = \lambda f(\mathbf{x}) + \mu f(\mathbf{y}) = \lambda \mathbf{0} + \mu\mathbf{0} = \mathbf{0}$. So $\lambda\mathbf{x}+ \mu\mathbf{y} \in \mathrm{Ker}f$.
\end{proof}

Q: Rank of linear map
A: The \emph{rank} of a linear map $f: U\to V$, denoted by $r(f)$, is the dimension of the image of $f$.

Q: Nullity of linear map
A: The \emph{nullity} of $f$, denoted $n(f)$ is the dimension of the kernel of $f$.

Q: Rank-nullity theorem
A: For a linear map $f: U \to V$,
  \[
    r(f) + n(f) = \dim (U).
  \]
\begin{proof}
Let $\dim(U) = n$ and $n(f) = m$. If $m = n$, then $f$ is the zero map, and the proof is trivial, since $r(f) = 0 $. Otherwise, $m < n$ as Ker$f$ is a subspace.
  Suppose $\{\mathbf{e}_1, \mathbf{e}_2,\cdots, \mathbf{e}_m\}$ is a basis of $\mathrm{Ker}f$. Extend to a basis of $U$: $\{\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_m, \mathbf{e}_{m+1}, \cdots, \mathbf{e}_n\}$. We are rtp $\{f(\mathbf{e}_{m+1}), f(\mathbf{e}_{m + 2}), \cdots f({\mathbf{e}_n})\}$ is a basis of $\mathrm{Im}f$.
\textbf{Spans} For $\mathbf{y}\in \mathrm{Im}f$:
      \begin{align*}
\mathbf{y} &=f(\mathbf{x}) \\
&= f(\alpha_1\mathbf{e}_1 + \alpha_2\mathbf{e}_2 + \cdots + \alpha_n \mathbf{e}_n) \\
 &= \alpha_1f(\mathbf{e}_1) + \alpha_2f(\mathbf{e}_2) + \cdots + \alpha_m f(\mathbf{e}_m) + \alpha_{m + 1}f(\mathbf{e}_{m + 1}) + \cdots + \alpha_nf(\mathbf{e}_n) \\
&= \alpha_{m + 1} f(\mathbf{e}_{m + 1}) + \cdots + \alpha_n f(\mathbf{e}_n).
      \end{align*}
\textbf{Linearly Independent}
      \begin{align*}
        \mathbf{0}&= \alpha_{m + 1} f(\mathbf{e}_{m + 1}) + \cdots + \alpha_n f(\mathbf{e}_n) \\
 &=f(\alpha_{m + 1}\mathbf{e}_{m + 1} + \cdots + \alpha_n\mathbf{e}_n) \\
\implies\alpha_{m + 1}\mathbf{e}_{m + 1} + \cdots + \alpha_n\mathbf{e}_n&\in \mathrm{Ker}f \\
\therefore\alpha_{m + 1}\mathbf{e}_{m + 1} + \cdots + \alpha_n\mathbf{e}_n &= \alpha_1\mathbf{e_1} + \cdots + \alpha_m\mathbf{e}_m \\
\implies \alpha_1=\alpha_2=\dots =\alpha_m =\dots = \alpha_n&=0
      \end{align*}
\end{proof}

Q: Show that matrices are exactly the linear maps
A: Let $\alpha: \mathbb{R}^n \to \mathbb{R}^m$ be a linear map, and $\mathbf{x}' = \alpha(\mathbf{x})$.
Let $\{\mathbf{e}_i\}$ be a basis of $\mathbb{R}^n$. Then we get
\begin{align*}
\mathbf{x} &= x_j \mathbf{e}_j \\
  \mathbf{x}' &= \alpha(x_j \mathbf{e}_j) = x_j \alpha(\mathbf{e}_j).
\end{align*}
So we get that
\begin{align*}
  x_i' &= [\alpha(\mathbf{e}_j)]_i x_j \\
&=A_{ij}x_j \\
\therefore A_{ij} &= [\alpha(\mathbf{e}_j)]_i
\end{align*}
Where $\mathsf{A}$ is the matrix with columns which are the images of the standard basis vectors under the mapping $\alpha$. \\
Given a matrix
\[
  \{A_{ij}\} =
  \begin{pmatrix}
    A_{11} & \cdots & A_{1n}\\
    \vdots & A_{ij} & \vdots\\
    A_{m1} & \cdots & A_{mn}
  \end{pmatrix}
\]
we can check it is a linear map:
\begin{align*}
[\mathsf{A}(\lambda\vec x+\mu\vec y)]_i&=A_{ij}(\lambda\vec x+\mu\vec y)_j \\
&= A_{ij}(\lambda x_j+\mu y_j) \\
&=\lambda A_{ij} x_j+ \mu A_{ij} y_j \\
&=\lambda (\mathsf{A}\vec{x})_i +\mu (\mathsf{A}\vec{y})_i
\end{align*}

Q: Matrix algebra
A: Consider two linear maps $\alpha, \beta: \mathbb{R}^n\to \mathbb{R}^m$. The sum of $\alpha$ and $\beta$ is defined by
  \begin{align*}
    (\alpha + \beta)(\mathbf{x}) &= \alpha(\mathbf{x}) + \beta(\mathbf{x})\\
    \intertext{In terms of the matrix, we have}
    (A + B)_{ij}x_j &= A_{ij}x_j + B_{ij}x_j,\\
    \intertext{or}
    (A + B)_{ij} &= A_{ij}+B_{ij}.
  \end{align*}
So matrices are added by adding the components.
$(\lambda\alpha)\mathbf{x} = \lambda[\alpha(\mathbf{x})]$. So
\begin{align*}
(\lambda A)_{ij}x_j &= \lambda A_{ij}x_j \\
(\lambda A)_{ij} &= \lambda A_{ij} \\
\end{align*}
Thus scalar multiplication works by scaling every element.
Consider maps $\alpha: \mathbb{R}^\ell \to \mathbb{R}^n$ and $\beta: \mathbb{R}^n\to \mathbb{R}^m$. The composition is $\beta\alpha:\mathbb{R}^\ell\to\mathbb{R}^m$. Take $\mathbf{x}\in \mathbb{R}^\ell\mapsto \vec{x}'=\mathsf{A}\vec{x}\in\mathbb{R}^n\mapsto \mathbf{x}''=\mathsf{B}\vec{x}'\in \mathbb{R}^m$. Then
\begin{align*}
x_i'' &= (B\mathbf{x}')_i  \\
&=B_{ik}x_k' \\
&= B_{ik}(A_{kj}x_j) \\
\implies (BA)_{ij} &= B_{ik}A_{kj}
\end{align*}
So $\mathsf{B}\mathsf{A}$ is the $i$th row of $\mathsf{B}$ dotted with the $j$th column of $\mathsf{A}$.

Q: Hermitian Matrix
A: $\mathsf{A}=\mathsf{A}^\dagger$. $\mathsf{A}$ is a square matrix with a real diagonal.

Q: Hermitian conjugate
A: $A^{\dagger} = (A^T)^*$.

Q: Symmetric matrix
A: A matrix is \emph{symmetric} if $A^T = A$.

Q: Skew-Hermitian matrix
A: A matrix is \emph{skew-Hermitian} if $A^\dagger = -A$. The diagonals are pure imaginary.

Q: Trace of matrix
A: The \emph{trace} of an $n\times n$ matrix $\mathsf{A}$ is the sum of the diagonal. i.e. $\mathsf{A}_{ii}$.

Q: Trace of a product
A: Tr$(BC) =$ Tr$(CB)$
\begin{proof}
  Tr$(BC) = B_{ik}C_{ki} = C_{ki}B_{ik} = (CB)_{kk} = $ Tr$(CB)$
\end{proof}

Q: Transpose of a product
A: $(AB)^T = B^TA^T$
\begin{proof}
$(AB)^T_{ij} = (AB)_{ji} = A_{jk}B_{kj} = B_{ki}A_{jk} $\\$= (B^T)_{ik}(A^T)_{kj} = (B^TA^T)_{ij}$.
\end{proof}

Q: Decomposition of an $n\times n$ matrix
A: Write
\[
  B_{ij} = \underbrace{\frac{1}{2}(B_{ij} + B_{ji})}_{S_{ij}} + \underbrace{\frac{1}{2}(B_{ij} - B_{ji})}_{A_{ij}}.
\]
Then $\mathsf{S}$ is symmetric and $\mathsf{A}$ is antisymmetric. \\
Furthermore , we can decompose $S$ into an isotropic part (a scalar multiple of the identity) plus a trace-less part. Write
\[
  S_{ij} = \underbrace{\frac{1}{n}\mathrm{tr} (S)\delta_{ij}}_{\text{isotropic part}} + \underbrace{(S_{ij} - \frac{1}{n}\mathrm{tr}(S)\delta_{ij})}_{T_{ij}}.
\]
Then $\mathrm{tr}(T) = T_{ii} = S_{ii} - \frac{1}{n}\mathrm{tr}(S)\delta_{ii} = \mathrm{tr}(S) - \frac{1}{n}\mathrm{tr}(S)(n) = 0$ so $\mathsf{T}$ is traceless. \\
Putting all these together,
\[
  B = \frac{1}{n}\mathrm{tr}(B)I + \left\{\frac{1}{2}(B + B^T) - \frac{1}{n}\mathrm{tr}(B)I\right\} + \frac{1}{2}(B - B^T).
\]
In three dimensions, the antisymmetric part $\mathsf{A}$ looks like
\[
  A = \begin{pmatrix}
    0 & \omega_3 & -\omega_2\\
    -\omega_3 & 0 & \omega_1\\
    \omega_2 & -\omega_1 & 0
  \end{pmatrix}
\]
so $A_{ij} = \varepsilon_{ijk}\omega_k$. To solve for $\vec{\omega} $ we multiply by $\varepsilon_{ijl}$:
\begin{align*}
 \varepsilon_{ijk}\varepsilon_{ijl}\omega_k&=\varepsilon_{ijl}A_{ij} \\
2\omega_l&=\varepsilon_{ijl}A_{ij} \\
\omega_l&=\frac{1}{2}\varepsilon_{ijl}A_{ij} \\
&=\frac{1}{2}\varepsilon_{ijl}B_{ij}
\end{align*}
where the last line comes from only the antisymmetric part contributing to the sum.

Q: Orthogonal matrix
A: A real $n\times n$ matrix is \emph{orthogonal} if $A^TA = AA^T = I$ or $A_{ik}A^T_{kj} = \delta_{ij}=A^T_{ik}A_{kj}$ i.e. $A^T = A^{-1}$.\\
This is equivalent to $A_{ik}A_{jk} = \delta_{ij}=A_{ki}A_{kj}$ i.e. the rows/columns are orthonormal vectors.

Q: Unitary matrix
A: A complex $n\times n$ matrix is \emph{unitary} if $U^\dagger U = I=UU^\dagger $ or $A_{ik}U^\dagger_{kj} = \delta_{ij}=U^\dagger_{ik}U_{kj}$ i.e. $U^\dagger = U^{-1}$.\\
This is equivalent to $U_{ik}^*U_{jk} = \delta_{ij}=U_{ki}^*U_{kj}$ i.e. the rows/columns are orthonormal complex vectors.

Q: Determinant
A: Consider a linear map $\alpha: \mathbb{R}^3\to \mathbb{R}^3$. The standard basis $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ is mapped to $\mathbf{e}_1', \mathbf{e}_2', \mathbf{e}_3'$ with $\mathbf{e}_i' = A\mathbf{e}_i$. Thus the unit cube formed by $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ is mapped to the parallelopiped with volume
\begin{align*}
  [\mathbf{e}_1', \mathbf{e}_2', \mathbf{e}_3'] &=(e_1')_i (e_2' \times e_3')_i \\
&= \varepsilon_{ijk}(e_1')_i (e_2')_j (e_3')_k\\
  &= \varepsilon_{ijk} A_{i\ell} \underbrace{(e_1)_\ell}_{\delta_{1\ell}} A_{jm}\underbrace{(e_2)_m}_{\delta_{2m}} A_{kn}\underbrace{(e_3)_n}_{\delta_{3n}}\\
  &= \varepsilon_{ijk} A_{i1}A_{j2}A_{k3}
\end{align*}
This prompts the generalised definition of the determinant
\begin{align*}
    \det (A) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n} \\
 &= \varepsilon_{j_1j_2\cdots j_n}A_{j_11}A_{j_22}\cdots A_{j_nn}.
  \end{align*}
where $\varepsilon$ is the Levi-Civita symbol.

Q: $\det (A^T)$
A: $\det (A) = \det (A^T)$.
\begin{proof}
We reorder the factors using $\rho=\sigma^{-1}\in S_n$ for each $\sigma$:
\begin{align*}
\det(A) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n}  \\
&=\sum_{\sigma\in S_n}\varepsilon(\sigma)A_{\sigma(\rho(1))\rho(1)}A_{\sigma(\rho(2))\rho(2)}\cdots A_{\sigma(\rho(n))\rho(n)} \\
&=\sum_{\rho\in S_n} \varepsilon(\rho) A_{1\rho(1)}A_{2\rho(2)}\cdots A_{n\rho(n)} \\
&=\sum_{\rho\in S_n} \varepsilon(\rho) A_{\rho(1)1}^TA_{\rho(2)2}^T\cdots A_{\rho(n)n}^T \\
&=\det (A^T)
\end{align*}
\end{proof}
Note that this is legitimate as permutations are bijections so are invertible.

Q: Effect of multiplying a row/column by a scalar on the determinant
A: If matrix $B$ is formed by multiplying every element in a single row of $A$ by a scalar $\lambda$, then $\det (B) = \lambda \det (A)$.
\begin{proof}
  \begin{align*}
\det(B) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots \lambda A_{\sigma(i)i}\cdots A_{\sigma(n)n}   \\
&= \sum_{\sigma\in S_n} \varepsilon(\sigma) \lambda A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n} \\
&=  \lambda \sum_{\sigma\in S_n} \varepsilon(\sigma)A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n} \\
&=\lambda \det (A)
\end{align*}
\end{proof}

Q: Determinant of a matrix with 2 rows/columns identical
A: If 2 rows (or 2 columns) of $A$ are identical, the determinant is $0$.
\begin{proof}
  wlog, suppose columns 1 and 2 are the same. Letting $\sigma = \rho(1\; 2)$:
  \begin{align*}
    \det (A) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n} \\
&= \sum_{\rho\in S_n} \varepsilon(\rho)\varepsilon(1,2) A_{\rho(2)1}A_{\rho(1)2}A_{\rho(3)3}\cdots A_{\rho(n)n} \\
    &= \sum_{\rho\in S_n} -\varepsilon(\rho) A_{\rho(2)1}A_{\rho(1)2}A_{\rho(3)3}\cdots A_{\rho(n)n} \\
 &= -\det (A) \\
\implies\det(A) &= 0.
\end{align*}
\end{proof}

Q: Effect of adding multiples of rows/columns to other rows/columns on the determinant
A: If $B$ is formed by adding $\lambda\mathrm{col}_j(A)$ to col$_i(A)$, then $\det (A) = \det (B)$.
\begin{proof}
 \begin{align*}
\det(B) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots (A_{\sigma(i)i}+\lambda A_{\sigma(i)j}) \cdots A_{\sigma(n)n} \\
&= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(i)i} \cdots A_{\sigma(n)n}+\lambda \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(i)j}\cdots A_{\sigma(n)n}   \\
&=\det (A)+\det (A')
\end{align*}
where $A'$ is the matrix formed by making col$_i(A)$ take the values of col$_j(A)$. $A'$ has two columns the same so $\det (A')=0$. Therefore \[
\det(B)=\det (A).
\]
\end{proof}

Q: Determinant of a matrix with LD rows/columns
A: If 2 rows or 2 columns are linearly dependent, then the determinant is zero.
\begin{proof}
  Suppose in $A$, $($column $i) + \lambda($column $j) = 0$. Then we can add $ \lambda($column $j)$ to column $i$ to form a matrix $B$ with $\det (B) =\det (A)$ such that the $i$th column of $B$ is all zeroes. So
\begin{align*}
\det(A)=\det(B) &= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots  A_{\sigma(i)i}\cdots A_{\sigma(n)n}  \\
&= \sum_{\sigma\in S_n} \varepsilon(\sigma) A_{\sigma(1)1}A_{\sigma(2)2}\cdots (0)\cdots A_{\sigma(n)n}  \\
&=0
\end{align*}
\end{proof}

Q: Determinant of a product
A: $\det(AB) = \det(A)\det(B)$.
\begin{proof}
  \begin{align*}
    \det AB &= \sum_\sigma \varepsilon(\sigma)(AB)_{\sigma(1)1}(AB)_{\sigma(2)2}\cdots (AB)_{\sigma(n)}\\
    &= \sum_\sigma \varepsilon(\sigma) \sum_{k_1,k_2,\cdots,k_n} A_{\sigma(1)k_1}B_{k_11}\cdots A_{\sigma(n)k_n}B_{k_nn}\\
    &= \sum_{k_1,\cdots,k_n}B_{k_11}\cdots B_{k_nn}\underbrace{\sum_\sigma \varepsilon(\sigma) A_{\sigma(1)k_1}A_{\sigma(2)k_2}\cdots A_{\sigma(n)k_n}}_{S}
    \intertext{If in $S$, two of $k_1$ and $k_n$ are equal, then $S$ is a determinant of a matrix with two columns the same i.e. $S = 0$. So we only have to consider the sum over distinct $k_i$s. Thus the $k_i$s are are a permutation of $1, \cdots n$, say $k_i =\rho (i)$. Then we can write}
    \det AB &= \sum_\rho B_{\rho(1)1}\cdots B_{\rho(n)n} \sum_\sigma \varepsilon(\sigma) A_{\sigma(1)\rho(1)} \cdots A_{\sigma(n)\rho(n)}\\
    &= \sum_\rho B_{\rho(1)1}\cdots B_{\rho(n)n} (\varepsilon(\rho)\det A)\\
    &= \det A\sum_\rho \varepsilon(\rho) B_{\rho(1)1}\cdots B_{\rho(n)n}\\
    &= \det A\det B.
  \end{align*}
\end{proof}

Q: Determinant of a matrix with some rows/columns swapped
A: If $B_{ij}=A_{i\rho(j)}$, then det$B=\varepsilon(\rho)\det A$.
\begin{proof}
\begin{align*}
\det B&=\sum_\sigma \varepsilon(\sigma) A_{\sigma(1)\rho(1)}\cdots A_{\sigma(n)\rho(n)}  \\
&=\sum_\tau\varepsilon(\tau\rho) A_{\tau(\rho(1))\rho(1)}\cdots A_{\tau(\rho(n))\rho(n)}  \\
&=\varepsilon(\rho)\sum_\tau\varepsilon(\tau) A_{\tau(\rho(1))\rho(1)}\cdots A_{\tau(\rho(n))\rho(n)}  \\
&=\varepsilon(\rho)\sum_\tau\varepsilon(\tau) A_{\tau(1)1}\cdots A_{\tau(n)n} \textrm{ (by reordering factors)}  \\
&=\varepsilon(\rho)\det A  \\
\end{align*}
\end{proof}
Note that writing $\sigma=\tau\rho$ is legitimate because $S_n$ forms a group under composition.

Q: Determinant of an orthogonal matrix
A: If $A$ is orthogonal, $\det A = \pm 1$.
\begin{proof}
  \begin{align*}
    AA^T &= I\\
    \det AA^T &= \det I\\
    \det A\det A^T &= 1\\
    (\det A)^2 &= 1\\
    \det A &= \pm 1
  \end{align*}
\end{proof}
If det$A=1$, then $A$ represents a rotation. If det$A=-1$, then $A$ represents a reflection.

Q: Determinant of a unitary matrix
A: If $U$ is unitary, $|\det U| = 1$ i.e. det$U$ lies on the unit circle in the complex plane.
\begin{proof}
  We have $\det U^\dagger = (\det U^T)^* = \det(U)^*$. Since $UU^\dagger = I$, we have $\det(U)\det(U)^* = 1$.
\end{proof}

Q: Minor
A: The \emph{minor} of the $ij$th element of $A$ is \[
M_{ij} = \det A^{ij}\]
where $A^{ij}$ is the $(m - 1)\times (n - 1)$ matrix in which row $i$ and column $j$ of $A$ have been removed.

Q: Cofactor
A: The \emph{cofactor} of the $ij$th element of $A$ is \[
\Delta_{ij} = (-1)^{i + j}M_{ij}.\]
where $A^{ij}$ is the $(m - 1)\times (n - 1)$ matrix in which row $i$ and column $j$ of $A$ have been removed.

Q: Laplace expansion formula
A: For any particular fixed $i$ (that is, for any particular row/column),
  \[
    \det A = \sum_{j = 1}^{n} A_{ji}\Delta_{ji}.
  \]
\begin{proof}
  \[
    \det A = \sum_{j_i = 1}^nA_{j_ii} \sum_{j_1, \cdots, \overline{j_i}, \cdots j_n}^n \varepsilon_{j_1j_2\cdots j_n} A_{j_11}A_{j_22}\cdots \overline{A_{j_ii}}\cdots A_{j_nn}
  \]
Let $\varepsilon_{j_1j_2\cdots j_n}=\varepsilon(\rho\sigma)$ i.e. let $\rho\sigma$ map $(1, \cdots, n)$ to $(j_1, j_2,\cdots, j_n)$ such that $\sigma$ puts $j_i$ in the $i$th position, i.e. $i\mapsto j_i$ and then $\rho$ orders everything else.
 \[
    \rho =
    \begin{pmatrix}
      1 & \cdots & \cdots & \overline{j_i} & \cdots & n\\
      j_1 & \cdots & \overline{j_i} & \cdots & \cdots & j_n\\
    \end{pmatrix}
  \]
\textbf{Case} $j_i > i$:
  \begin{align*}
    \sigma &=
    \begin{pmatrix}
      1 &\cdots& i & i + 1 & i + 2 & \cdots &j_i - 1&j_i& j_i + 1 & \cdots n\\
      1 & \cdots & j_i & i & i + 1 & \cdots & j_i - 2 & j_i - 1 & j_i + 1 & \cdots n
    \end{pmatrix} \\
&=(j_i \ j_i-1 \dots i+1 \ i) \\
&=(j_i \ j_i-1)(j_i-1 \  j_i -2) \dots (i+2 \ i+1 ) (i+1 \ i)
  \end{align*}
\textbf{Case} $j_i<i$:
\begin{align*}
    \sigma &=
    \begin{pmatrix}
      1 &\cdots& j_i & j_i+1 & \cdots &i - 1& i & i+1 & \cdots n\\
      1 & \cdots & j_i+1 & j_i+2 & \cdots & i  & j_i & i + 1 & \cdots n
    \end{pmatrix} \\
&=(j_i \ j_i+1 \dots i-1 \ i) \\
&=(j_i \ j_i+1)(j_i+1 \  j_i +2) \dots (i-2 \ i-1 ) (i-1 \ i)
  \end{align*}
\textbf{Case} $j_i=i$. $\sigma$ is the identity.
To perform this permutation, $|i - j_i|$ transpositions are made. So $\varepsilon(\sigma) = (-1)^{i - j_i}$.
So $\varepsilon(\rho\sigma) = \varepsilon_{j_1\cdots j_n} = \varepsilon(\rho)\varepsilon(\sigma) = (-1)^{i - j_i} \varepsilon_{j_1\cdots \bar j_i \cdots j_n}$. Hence the original equation becomes
  \begin{align*}
    \det A &= \sum_{j_i = 1}^n A_{j_i i} \sum_{j_1\cdots \bar j_i\cdots j_n}(-1)^{i - j_i} \varepsilon_{j_1\cdots \bar j_i \cdots j_n} A_{j_11}\cdots \overline{A_{j_ii}} \cdots A_{j_nn}\\
    &= \sum_{j_i = 1}^n A_{j_ii} (-1)^{i - j_i}M_{j_ii}\\
    &= \sum_{j_i = 1}^{n} A_{j_ii}\Delta_{j_ii}\\
  \end{align*}
\end{proof}

Q: Inverse of a $2\times 2$ matrix
A: $A\mathbf{x} = \mathbf{d}$ is equivalent to the system of equations
\begin{align*}
  A_{11}x_1 + A_{12}x_2 &= d_1\tag{a}\\
  A_{21}x_1 + A_{22}x_2 &= d_2\tag{b}.
\end{align*}
(a)$\times A_{22} - $(b)$\times A_{12}$ and (a)$\times A_{21} - $(b)$\times A_{11}$ give
\begin{align*}
  (A_{11}A_{22} - A_{12}A_{21})x_1 &= A_{22}d_1 - A_{12}d_2\\
  \underbrace{(A_{11}A_{22} - A_{12}A_{21})}_{\det A}x_2 &= A_{11}d_2 - A_{21}d_1
\end{align*}
Dividing by $\det A$ and writing in matrix form, we have
\[
  \begin{pmatrix}
    x_1\\
    x_2
  \end{pmatrix} = \frac{1}{\det A}
  \begin{pmatrix}
    A_{22} & - A_{12}\\
    -A_{21} & A_{11}
  \end{pmatrix}
  \begin{pmatrix}
    d_1\\
    d_2
  \end{pmatrix}
\]
So $A^{-1}$ exists if $\det A \not= 0$, with
\[
  A^{-1} =\frac{1}{\det A}\begin{pmatrix}A_{22} & - A_{12}\\-A_{21} & A_{11}\end{pmatrix}
\]

Q: Inverse of an $n\times n$ matrix
A: \textbf{Lemma}
  $\sum A_{ik}\Delta_{jk} = \delta_{ij}\det A$.
\begin{proof}
  If $i \not= j$, then consider an $n\times n$ matrix $B$, which is identical to $A$ except the $j$th row is replaced by the $i$th row of $A$. $\Delta_{jk}$ is the same for $A$ and $B$, as it does not depend on the elements in row $j$. Since $B$ has a duplicate row, we know that
  \[
    0 = \det B = \sum_{k = 1}^n B_{jk}\Delta_{jk} = \sum_{k = 1}^n A_{ik}\Delta_{jk}.
  \]
  If $i = j$, then the expression is $\det A$ by the Laplace expansion formula.
\end{proof}
\textbf{Theorem}
  $A^{-1}$ exists iff $\det A \not =0$ and is given by
  \[
    (A^{-1})_{ij} = \frac{\Delta_{ji}}{\det A}.
  \]
\begin{proof}
If  $\det A \neq 0$
  \[
    (A^{-1})_{ik}A_{kj} = \frac{\Delta_{ki}}{\det A} A_{kj} = \frac{\delta_{ij}\det A}{\det A} = \delta_{ij}.
  \]
  So $A^{-1}A = I$. \\
If $\det A = 0$, then it has no inverse, since for any matrix $B$, $\det AB = 0$, and hence $AB$ cannot be the identity.
\end{proof}

Q: Computing the inverse
A: Let $f_n$ be the number of multiplications in finding the determinant of an $n\times n$ square matrix using the Laplace expansion formula. This involves calculating $n$ determinants of $n-1\times n-1$ matrices and then summing $n$ products. Therefore
\begin{align*}
f_n &= nf_{n -1} + n \\
&= n!f_1+n+n(n-1)\ldots +n! \\
&=n!\left(1+\frac{1}{(n-1)!}+\frac{1}{(n-2)!}+\dots +1\right) \\
&\approx (1+e)n!
\end{align*}
So $f_n = O(n!)$.
The inverse is a matrix with $n^2$ elements, each of which is a cofactor i.e. determinant of a $n-1\times n-1$ matrix. So the computing time is $O(n^2 (n - 1)!) = O(n\cdot n!)$.

Q: Gaussian elimination
A: \begin{align*}
  A_{11}x_1 + A_{12}x_2 + \cdots + A_{1n}x_n &= d_1\\
  A_{21}x_1 + A_{22}x_2 + \cdots + A_{2n}x_n &= d_2\\
  \vdots&\\
  A_{m1}x_1 + A_{m2}x_2 + \cdots + A_{mn}x_n &= d_m
\end{align*}
Assume $A_{11}\not=0$ (if not, we can re-order the equations). We can use the first equation to eliminate $x_1$ from the remaining $(m - 1)$ equations. Then use the second equation to eliminate $x_2$ from the remaining $(m - 2)$ equations (if anything goes wrong, just re-order until things work). Repeat.
We are left with
\begin{align*}
  A_{11}x_1 + A_{12}x_2 + A_{13}x_3 + \cdots + A_{1n}x_n &= d_1\\
  A_{22}^{(2)}x_2 + A_{23}^{(2)}x_3 + \cdots + A_{2n}^{(2)}x_n &= d_2\\
  \vdots&\\
  A_{rr}^{(r)}x_r + \cdots + A_{rn}^{(r)}x_n &= d_r\\
  0 &= d_{r + 1}^{(r)}\\
  \vdots&\\
  0 &= d_{m}^{(r)}
\end{align*}
Here $A_{ii}^{(i)} \not=0$ (which we can achieve by re-ordering)
\textbf{Case: overdetermined} $r < m$ and at least one of $d^{(r)}_{r + 1}, \cdots d_m^{(r)} \not= 0$. Then a contradiction is reached. \\
\textbf{Case: determined} If $r = n\leq m$, and all $d_{r + i}^{(r)} = 0$. Then from the $n$th equation, there is a unique solution for $x_n = d_{n}^{(n)}/A_{nn}^{(n)}$, and hence for all $x_i$ by back substitution. \\
\textbf{Case: underdetermined} If $r < n$ and $d_{r + i}^{(r)} = 0$, then $x_{r + 1}, \cdots x_n$ can be freely chosen, and there are infinitely many solutions.
\begin{align*}
  \det A &= (-1)^k
  \begin{vmatrix}
    A_{11} &A_{12}&\cdots& \cdots& \cdots & A_{1n}\\
    0 & A_{22}^{(2)} &\cdots& \cdots & \cdots & A_{2n}^{(n)} \\
    \vdots & \vdots &\ddots & \vdots & \vdots & \vdots\\
    0 & 0 & \cdots & A_{rr}^{(r)} & \cdots & A_{rn}^{(n)}\\
    0 & 0 & \cdots & 0 & 0 & \cdots\\
    \vdots & \vdots & \vdots & \vdots & \vdots & \vdots
  \end{vmatrix} \\
&=\pm \prod_i A_{ii}^{(i)}
\end{align*}
Hence if $r < n$ (and $d_i^{(r)} = 0$ for $i > r$), then $\det A = 0$. If $r = n$, then $\det A = (-1)^k A_{11}A_{22}^{(2)}\cdots A_{nn}^{(n)} \not= 0$.

Q: Prove column rank $=$ row rank
A: \begin{proof}
  Let $r$ be the row rank of an $m\times n$ matrix $A$. Write the biggest set of LI rows as $\mathbf{v}_1^T, \mathbf{v}_2^T, \cdots \mathbf{v}_r^T$ or in component form $\mathbf{v}_k^T = (v_{k1}, v_{k2}, \cdots, v_{kn})$ for $k = 1, 2, \cdots, r$.
  Now denote the $i$th row of $A$ as $\mathbf{r}_i^T = (A_{i1}, A_{i2}, \cdots A_{in})$.
  Note that every row of $A$ can be written as a linear combination of the $\mathbf{v}$'s. Write
  \[
    \mathbf{r}_i^T = \sum_{k = 1}^r C_{ik}\mathbf{v}_{k}^T.
  \]
  For some coefficients $C_{ik}$ with $1 \leq i\leq m$ and $1 \leq k \leq r$.
  Now the elements of $A$ are
  \[
    A_{ij} = (\mathbf{r}_i^T)_j = \sum_{k = 1}^r C_{ik}(\mathbf{v}_k^T)_j,
  \]
  or
  \[
    \begin{pmatrix}
      A_{1j}\\
      A_{2j}\\
      \vdots\\
      A_{mj}
    \end{pmatrix} = \sum_{k = 1}^r {\mathbf{v}_k}_j
    \begin{pmatrix}
      C_{1k}\\
      C_{2k}\\\
      \vdots\\
      C_{mk}
    \end{pmatrix}
  \]
  So every column of $A$ can be written as a linear combination of the $r$ column vectors $\mathbf{c}_k$. Then the column rank of $A \leq r$, the row rank of $A$.
  Apply the same argument to $A^T$ to see that the row rank is $\leq$ the column rank.
\end{proof}

Q: Homogeneous linear equations
A: $A\mathbf{x} = \mathbf{0}\implies \det A=0$ (by product of determinants or multiplying by $A^{-1})$.
\textbf{Geometrical interpretation}
We consider a $3\times 3$ matrix
\[
  A = \begin{pmatrix} \mathbf{r}_1^T\\\mathbf{r}_2^T\\\mathbf{r}_3^T\end{pmatrix}
\]
$A\mathbf{x} = \mathbf{0}$ means that $\mathbf{r}_i\cdot \mathbf{x} = 0$ for all $i$. Each equation $\mathbf{r}_i\cdot \mathbf{x} = \mathbf{0}$ represents a plane through the origin. So the solution is the intersection of the three planes.
There are three possibilities:
\begin{enumerate}
  \item If $\det A =[\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3] \not= 0$, span$\{\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3\} = \mathbb{R}^3$ and thus $r(A) = 3$, $n(A) = 0$ and the kernel is $\{\mathbf{0}\}$. So $\mathbf{x} = \mathbf{0}$ is the unique solution.
  \item If $\det A = 0$, then $\mathrm{dim}(\mathrm{span}\{\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3\}) = 1$ or $2$.
    \begin{enumerate}
      \item If rank $= 2$, wlog assume $\mathbf{r}_1, \mathbf{r}_2$ are LI. So $\mathbf{x}$ lies on the intersection of two planes $\mathbf{x}\cdot \mathbf{r}_1 = 0$ and $\mathbf{x}\cdot \mathbf{r}_2 = 0$, which is the line $\{\mathbf{x}\in \mathbb{R}^3: \mathbf{x} = \lambda \mathbf{r}_1\times \mathbf{r}_2\}$. All such points on this line also satisfy $\mathbf{x}\cdot\mathbf{r}_3 = 0$ since $\mathbf{r}_3$ is a linear combination of $\mathbf{r}_1$ and $\mathbf{r}_2$. The kernel is a line, $n(A) = 1$.
      \item If rank = 1, then $\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3$ are parallel. So $\mathbf{x}\cdot \mathbf{r}_1 = 0 \Rightarrow \mathbf{x}\cdot \mathbf{r}_2 = \mathbf{x}\cdot \mathbf{r}_3 = 0$. So all $\mathbf{x}$ that satisfy $\mathbf{x}\cdot \mathbf{r}_1 = 0$ are in the kernel, and the kernel now is a plane. $n(A) = 2$.
    \end{enumerate}
\end{enumerate}
(We also have the trivial case where $r(A) = 0$, we have the zero mapping and the kernel is $\mathbb{R}^3$)
\textbf{Linear mapping view}
\begin{enumerate}
  \item If $n(A) = 0$, then $A(\mathbf{e}_1), A(\mathbf{e}_2), \cdots, A(\mathbf{e}_n)$ is a LI set, and $r(A) = n$.
  \item If $n(A) > 0$, then the image is not the whole of $\mathbb{R}^n$. Let $\{\mathbf{u}_i\}$ be a basis of the kernel, ie. $A\mathbf{x} = \mathbf{0}\implies\displaystyle \mathbf{x} = \sum_{i = 1}^{n(A)} \lambda_i \mathbf{u}_i$ for some $\lambda_i$. Extend $\{\mathbf{u}_i\}$ to be a basis of $ \mathbb{R}^n$. The vectors $A(\mathbf{u}_i)$ for $i = n(A) + 1, \cdots, n$ form a basis of the image.
\end{enumerate}

Q: General solution of $A\mathbf{x} = \mathbf{d}$
A: Let $A$ be an $m\times n$ matrix and $\mathbf{x}, \mathbf{d}$  $n \times 1$ column vectors.
\textbf{Case 1} $\det(A) \not= 0$. So $A^{-1}$ exists and $n(A) = 0$, $r(A) = n$. Then for any $\mathbf{d}\in \mathbb{R}^n$, a unique solution must exists and it is $\mathbf{x} = A^{-1}\mathbf{d}$.
\textbf{Case 2} $\det(A) = 0$. Then $A^{-1}$ does not exist, and $n(A) > 0$, $r(A) < n$.
      \textbf{Case 2.1} If $\mathbf{d}\not\in \mathrm{Im}A$, then there is no solution
      \textbf{Case 2.2} If $\mathbf{d}\in  \mathrm{Im}A$, then there exists at least one $\mathbf{x}$ such that $A\mathbf{x} = \mathbf{d}$. The general solution of $A\mathbf{x} = \mathbf{d}$ can be written as $\mathbf{x} = \mathbf{x}_0 + \mathbf{y}$, where $\mathbf{x}_0$ is a particular solution (ie. $A\mathbf{x}_0 = \mathbf{d}$), and $\mathbf{y}$ is any vector in $\mathrm{Ker}A$ (ie. $A\mathbf{y} = \mathbf{0}$).  So
        \[
          \mathbf{x} = \mathbf{x}_0 + \sum_{j = 1}^{n(A)} \mu_j \mathbf{u}_j
        \]
        for any $\mu_j$, ie. there are infinitely many solutions.

Q: Characteristic polynomial of a matrix $A$
A: \begin{align*}
0&=\det(A - \lambda I) \\
&= \varepsilon_{j_1j_2\cdots j_n} (A_{j_1 1} - \lambda\delta_{j_11})\cdots (A_{j_n n} - \lambda\delta_{j_nn}) \\
&= c_0 + c_1\lambda + \cdots + c_n\lambda^n
\end{align*}
where
\begin{align*}
c_n &= (-1)^n \\
c_{n - 1} &= (-1)^{n - 1}(A_{11} + A_{22} + \cdots + A_{nn}) \\
&= (-1)^{n - 1}\mathrm{tr}(A) \\
&= (-1)^{n - 1}(\lambda_1 + \lambda_2 + \cdots \lambda_n) \\
c_0 &= p_A(0) \\
& = \det(A) \\
&= \lambda_1\lambda_2\cdots \lambda_n
\end{align*}
So
\begin{align*}
\mathrm{tr}(A) &= \lambda_1 + \lambda_2 + \cdots + \lambda_n \\
 \det (A) &= \lambda_1 \lambda_2 \cdots \lambda_n.
\end{align*}

Q: Algebraic multiplicity of eigenvalue
A: The \emph{algebraic multiplicity} $M(\lambda)$ or $M_\lambda$ of an eigenvalue $\lambda$ is the multiplicity of $\lambda$ in $p_A(x) = 0$. By the fundamental theorem of algebra,
  \[
    \sum_\lambda M(\lambda) = n.
  \]
  If $M(\lambda) > 1$, then the eigenvalue is \emph{degenerate}.

Q: Geometric multiplicity of eigenvalue
A: The \emph{geometric multiplicity} $m(\lambda)$ or $m_\lambda$ of an eigenvalue $\lambda$ is the dimension of the eigenspace.
The \emph{eigenspace}, denoted by $E_\lambda$, is the kernel of the matrix $A - \lambda I$, ie. the set of eigenvectors with eigenvalue $\lambda$.
So the geometric multiplicity is the maximum number of LI eigenvectors with eigenvalue $\lambda$.

Q: Defect of eigenvalue
A: The \emph{defect} $\Delta_\lambda$ of eigenvalue $\lambda$ is
  \[
    \Delta_\lambda = M(\lambda) - m(\lambda).
  \]
  It can be proven that $\Delta_\lambda \geq 0$, ie. the geometric multiplicity is never greater than the algebraic multiplicity.

Q: Number of eigenvectors without degeneracy
A: Suppose $n\times n$ matrix $A$ has \emph{distinct} eigenvalues $\lambda_1, \lambda_2, \cdots, \lambda_n$. Then the corresponding eigenvectors $\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_n$ are linearly independent.
\begin{proof}
  Proof by contradiction: Suppose $\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_n$ are linearly dependent. Then we can find non-zero constants $d_i$ for $i = 1, 2, \cdots, r$, such that
  \[
    d_1\mathbf{x}_1 + d_2\mathbf{x}_2 + \cdots + d_r\mathbf{x}_r = \mathbf{0}.
  \]
  Suppose that this is the shortest non-trivial linear combination that gives $\mathbf{0}$ (we may need to re-order $\mathbf{x}_i$).
Now multiply by $(A - \lambda_1 I)$ to obtain
  \[
    d_1(\lambda_1 - \lambda_1)\mathbf{x}_1 + d_2(\lambda_2 - \lambda_1)\mathbf{x}_2 + \cdots + d_r(\lambda_r - \lambda_1)\mathbf{x}_r = \mathbf{0}.
  \]
  We know that the first term is $\mathbf{0}$, while the others are not (since we assumed $\lambda_i \not= \lambda_j$ for $i\not= j$). So
  \[
    d_2(\lambda_2 - \lambda_1)\mathbf{x}_2 + \cdots + d_r(\lambda_r - \lambda_1)\mathbf{x}_r = \mathbf{0},
  \]
  is a shorter linear combination that gives $\mathbf{0}$. Contradiction.
\end{proof}

Q: Eigenvector Basis
A: If $n\times n$ matrix $A$ has $n$ distinct eigenvalues, and hence has $n$ linearly independent eigenvectors $\mathbf{v}_1, \mathbf{v}_2, \dots \mathbf{v}_n$, then \emph{wrt this eigenvector basis}, $A$ is diagonal.
In this basis, $v_1 = (1, 0, \cdots, 0)$ etc. We know that $A\mathbf{v}_i = \lambda_i\mathbf{v}_i$ (no summation). So the image of the $i$th basis vector is $\lambda_i$ times the $i$th basis. Since the columns of $A$ are simply the images of the basis,
\[
  \begin{pmatrix}
    \lambda_1 & 0 & \cdots & 0\\
    0 & \lambda_2 & \cdots & 0\\
    \vdots & \vdots & \ddots & \vdots\\
    0 & 0 & \cdots & \lambda_n
  \end{pmatrix}
\]

Q: Basis transformation matrix
A: Let $\{\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n\}$ and $\{\tilde{\mathbf{e}}_1, \tilde{\mathbf{e}}_2,\cdots, \tilde{\mathbf{e}}_n\}$ be 2 different bases of $\mathbb{R}^n$ or $\mathbb{C}^n$. To change from the first to the second, you multiply by $P$ where $P$ is the matrix with columns equal to the new basis vectors wrt to the old. The reverse transformation requires the inverse matrix.
\begin{proof}
 We can write
\[
  \tilde{\mathbf{e}_j} = \sum_{i = 1}^n P_{ij}\mathbf{e}_i
\]
ie. $P_{ij}$ is the $i$th component of $\tilde{\mathbf{e}_j}$ wrt the basis $\{\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n\}$.
Matrix $P$ has as its columns the vectors $\tilde{\mathbf{e}_j}$ relative to $\{\mathbf{e}_1, \mathbf{e}_2, \cdots, \mathbf{e}_n\}$. So $P = (\tilde{\mathbf{e}_1}\; \tilde{\mathbf{e}_2}\; \cdots \; \tilde{\mathbf{e}_n})$ and
\[
  P(\mathbf{e}_i) = \tilde{\mathbf{e}_i}
\]
Similarly, we can write
\[
  \mathbf{e}_i = \sum_{k = 1}^nQ_{ki} \tilde{\mathbf{e}_k}
\]
with $Q = (\mathbf{e}_1\; \mathbf{e}_2\;\cdots\;\mathbf{e}_n)$.
Substituting this into the equation for $\tilde{\mathbf{e}_j}$, we have
\begin{align*}
  \tilde{\mathbf{e}_j} &= \sum_{i = 1}^n\left(\sum_{k = 1}^{n} Q_{ki}\tilde{\mathbf{e}_k}\right)P_{ij}\\
  &= \sum_{k = 1}^n \tilde{\mathbf{e}_k} \left(\sum_{i = 1}^n Q_{ki}P_{ij}\right)
\end{align*}
But $\tilde{\mathbf{e}}_1, \tilde{\mathbf{e}}_2,\cdots, \tilde{\mathbf{e}}_n$ are LI so this is only possible if
\[
  \sum_{i = 1}^n Q_{ki}P_{ij} = \delta_{kj},
\]
i.e. $QP = I$, or $Q = P^{-1}$.
\end{proof}

Q: Vector transformation matrix
A: With respect to basis $\left\{\mathbf{e}_i\right\}$, $\mathbf{u} = \sum_{i = 1}^n u_i\mathbf{e}_i$.
With respect to basis $\left\{\tilde{\mathbf{e}_i}\right\}$, $\mathbf{u} = \sum_{i = 1}^n \tilde{u_i}\tilde{\mathbf{e}_i}$. Using the transformation matrix for the basis, we have
\begin{align*}
  \mathbf{u} &= \sum_{j= 1}^n \tilde{u_j} \sum_{i = 1}^{n}P_{ij}\mathbf{e}_i\\
  &= \sum_{i = 1}^n \left(\sum_{j = 1}^n P_{ij}\tilde{u_j}\right) \mathbf{e}_i
\end{align*}
By comparison, we know that
\[
  u_i = \sum_{j = 1}^n P_{ij}\tilde{u_j}
\]
Then
  \[
    \mathbf{u} = P\mathbf{\tilde{u}}\text{ and }\mathbf{\tilde{u}} = P^{-1}\mathbf{u}
  \]

Q: Matrix transformation matrix
A: Let $\mathbf{u}' = A\mathbf{u}$ and $\mathbf{\tilde{u}, \tilde{u}'}$ be the same vectors wrt $\{ \mathbf{e}_i\}$, $\{\tilde{\mathbf{e}_i}\}$. Using the transformation matrix $P$ for vectors:
\begin{align*}
  \mathbf{u}' &= A\mathbf{u}\\
  P\mathbf{\tilde{u}'} &= AP\tilde{\mathbf{u}}\\
  \mathbf{\tilde{u}'} &= P^{-1}AP\mathbf{\tilde{u}}\\
  &= \tilde{A}\tilde{\mathbf{u}}.
\end{align*}
So
  \[
    \tilde{A} = P^{-1}AP.
  \]
More generally, let $A$ represent the linear map $\alpha:\mathbb{C}^m\to\mathbb{C}^n$. Suppose $\mathbb{C}^m$ has a basis $\{\mathbf{e}_i\}$ and $\mathbb{C}^n$ has a basis $\{\mathbf{f}_i\}$. Now change bases to $\{\tilde{\mathbf{e}_i}\}$ and $\{\tilde{\mathbf{f}_i}\}$.
We know that $\mathbf{x} = P\mathbf{\tilde{x}}$ with $P$ being an $m\times m$ matrix, with $\mathbf{x}' = R\tilde{\mathbf{x}}'$ with $R$ being an $n\times n$ matrix.
Combining both of these, we have
\begin{align*}
  R\tilde{\mathbf{x}}' &= AP\tilde{\mathbf{x}}\\
  \tilde{\mathbf{x}}' &= R^{-1}AP\mathbf{\tilde{x}}
\end{align*}
Therefore $\tilde{A} = R^{-1}AP$.

Q: Similar matrices
A: Two $n\times n$ matrices $A$ and $B$ are \emph{similar} if there exists an invertible matrix $P$ such that
  \[
    B = P^{-1}AP,
  \]
  ie. they represent the same map under different bases.
  \begin{enumerate}
    \item Similar matrices have the same determinant.
    \item Similar matrices have the same trace.
    \item Similar matrices have the same characteristic polynomial.
  \end{enumerate}
\begin{proof} \textbf{}
\\
  \begin{enumerate}
    \item
\begin{align*}
        \det B &=\det (P^{-1}AP) \\
&= (\det A) (\det P)^{-1} (\det P) \\
&=\det A
      \end{align*}
    \item
      \begin{align*}
        \mathrm{tr} B &= B_{ii}\\
        &= P_{ij}^{-1} A_{jk} P_{ki}\\
        &= A_{jk} P_{ki}P_{ij}^{-1}\\
        &= A_{jk}(PP^{-1})_{kj}\\
        &= A_{jk}\delta_{kj}\\
        &= A_{jj}\\
        &= \mathrm{tr} A
      \end{align*}
    \item
      \begin{align*}
        p_B(\lambda) &= \det(B - \lambda I)\\
        &= \det(P^{-1}AP - \lambda I)\\
        &= \det(P^{-1}AP - \lambda P^{-1}IP)\\
        &= \det(P^{-1}(A - \lambda I)P)\\
        &= \det(A - \lambda I)\\
        &= p_A(\lambda)
      \end{align*}
  \end{enumerate}
\end{proof}

Q: Diagonalisable matrix
A: An $n\times n$ matrix $A$ is \emph{diagonalisable} if it is similar to a diagonal matrix.
A sufficient condition is that the eigenvalues of $A$ are not degenerate. Then $A$ has $n$ LI eigenvectors $\mathbf{v}_1, \mathbf{v}_2, \cdots \mathbf{v}_n$. We know that $A\mathbf{v}_i = \lambda_i\mathbf{v}_i$ (no summation). So the image of the $i$th basis vector is $\lambda_i$ times the $i$th basis. Since the columns of $A$ are simply the images of the basis vectors, $A$ is similar to
\[
  \begin{pmatrix}
    \lambda_1 & 0 & \cdots & 0\\
    0 & \lambda_2 & \cdots & 0\\
    \vdots & \vdots & \ddots & \vdots\\
    0 & 0 & \cdots & \lambda_n
  \end{pmatrix}
\]
A necessary and sufficient condition is that all of its eigenvalues have $0$ defect, as this is the same as having LI eigenvectors.

Q: Show that eigenvectors are LI
A: Let $\lambda_1, \lambda_2, \cdots, \lambda_r$, with $r \leq n$ be the distinct eigenvalues of $A$. Let $B_1, B_2, \cdots B_r$ be the bases of the eigenspaces $E_{\lambda_1}, E_{\lambda_2}, \cdots, E_{\lambda_r}$ correspondingly. Then the set $\displaystyle B = \bigcup_{i= 1}^r B_i$ is LI.
\begin{proof}
  Write $B_1 = \{\mathbf{x}_1^{(1)}, \mathbf{x}_2^{(1)}, \cdots \mathbf{x}_{m(\lambda_1)}^{(1)}\}$.
  Consider the following general linear combination of all elements in $B$
  \[
    \sum_{i = 1}^r\sum_{j = 1}^{m(\lambda_i)} \alpha_{ij} \mathbf{x}_j^{(i)} = 0.
  \]
Now multiply by the matrix
  \[
    \prod_{k = 1, 2, \cdots, \bar{K}, \cdots, r} (A - \lambda_kI)
  \]
which is the same as multiplying each term by \[
\prod_{k = 1, 2, \cdots, \bar{K}, \cdots, r} (\lambda_i - \lambda_k)
\]
which has a factor of $0$ unless $i=K$. So we obtain
  \[
    \sum_{j = 1}^{m(\lambda_K)}\alpha_{Kj}\left[\prod_{k = 1, 2, \cdots, \bar{K}, \cdots, r}(\lambda_K - \lambda_k)\right]\mathbf{x}_j^{(K)} = 0.
  \]
  Since the $\mathbf{x}^{(K)}_j$ are LI ($B_K$ is a basis), $\alpha_{Kj} = 0$ for all $j$. Since $K$ was arbitrary, all $\alpha_{ij}$ must be zero. So $B$ is LI.
\end{proof}

Q: Canonical form in $2$ dimensions
A: Any $2\times 2$ complex matrix $A$ is similar to exactly one of
  \[
    \begin{pmatrix}
      \lambda_1 & 0\\
      0 & \lambda_2
    \end{pmatrix},
    \begin{pmatrix}
      \lambda & 0\\
      0 & \lambda
    \end{pmatrix},
    \begin{pmatrix}
      \lambda & 1\\
      0 & \lambda
    \end{pmatrix}
  \]
\begin{proof}
  For each case:
  \begin{enumerate}
    \item If $A$ has two distinct eigenvalues, then eigenvectors are linearly independent. Then we can use $P$ formed from eigenvectors as its columns
    \item If $\lambda_1=\lambda_2 = \lambda$ and $\mathrm{dim} E_\lambda = 2$, then write $E_\lambda = \mathrm{span}\{\mathbf{u}, \mathbf{v}\}$, with $\mathbf{u}, \mathbf{v}$ linearly independent. Now use $\{\mathbf{u}, \mathbf{v}\}$ as a new basis of $\mathbb{C}^2$ and $\tilde{A} = P^{-1}AP =
      \begin{pmatrix}
        \lambda & 0\\
        0 & \lambda
      \end{pmatrix} = \lambda I$
      Note that since $P^{-1}AP = \lambda I$, we have $A = P(\lambda I)P^{-1} = \lambda I$. So $A$ is \emph{isotropic}, ie. the same with respect to any basis.
    \item If $\lambda_1 = \lambda_2 = \lambda$ and $\mathrm{dim} (E_\lambda) = 1$, then $E_\lambda = \mathrm{span}\{\mathbf{v}\}$. Now choose basis of $\mathbb{C}^2$ as $\{\mathbf{v}, \mathbf{w}\}$, where $\mathbf{w}\in \mathbb{C}^2\setminus E_\lambda$.
      Let $A\mathbf{w} = \alpha \mathbf{v} + \beta \mathbf{w}$. Hence, wrt $\{\mathbf{v}, \mathbf{w}\}$, $A$ is $\tilde{A} = P^{-1}AP =
      \begin{pmatrix}
        \lambda & \alpha\\
        0 & \beta
      \end{pmatrix}$.
      However, $A$ and $\tilde{A}$ both have eigenvalue $\lambda$ with algebraic multiplicity $2$. So we must have $\beta = \lambda$. To make $\alpha = 1$, let $\mathbf{u} = (\tilde{A} - \lambda I)\mathbf{w} = \tilde A\mathbf{w} - \lambda\mathbf{w}$ so $\tilde A\mathbf{w} = \mathbf{u} + \lambda\mathbf{w}$. We know $\mathbf{u}\not= \mathbf{0}$ since $\mathbf{w}$ is not in the eigenspace. Then
      \[
        (\tilde{A} - \lambda I)\mathbf{u} = (\tilde{A} - \lambda I)^2 \mathbf{w} =
        \begin{pmatrix}
          0 & \alpha\\
          0 & 0
        \end{pmatrix}
        \begin{pmatrix}
          0 & \alpha\\
          0 & 0
        \end{pmatrix}\mathbf{w} = \mathbf{0}.
      \]
      So $\tilde A\mathbf{u} = \lambda\mathbf{u},\tilde A\mathbf{w} = \mathbf{u} + \lambda\mathbf{w}$. Then $A$ wrt $\{\mathbf{u}, \mathbf{w}\}$ this basis is $
      \begin{pmatrix}
        \lambda & 1\\
        0 & \lambda
      \end{pmatrix}$.
      This is a two-stage process: $P$ sends basis to $\{\mathbf{v}, \mathbf{w}\}$, $Q$ sends to basis $\{\mathbf{u}, \mathbf{w}\}$ with $Q^{-1}(P^{-1}AP)Q = (PQ)^{-1}A(PQ)$.
  \end{enumerate}
\end{proof}

Q: Jordan normal form
A: The canonical form, or Jordan normal form, exists for any $n\times n$ matrix $A$. Specifically, there exists a similarity transform such that $A$ is similar to a matrix to $\tilde{A}$ that satisfies the following properties:
  \begin{enumerate}
    \item $\tilde{A}_{\alpha\alpha} = \lambda_\alpha$, ie. the diagonal composes of the eigenvalues.
    \item $\tilde{A}_{\alpha, \alpha + 1} = 0$ or $1$.
    \item $\tilde{A}_{ij} = 0$ otherwise.
  \end{enumerate}

Q: Cayley-Hamilton Theorem
A: Every $n\times n$ complex matrix satisfies its own characteristic equation.
\begin{proof}
  We will only prove for diagonalizable matrices here. So suppose for our matrix $A$, there is some $P$ such that $D = \mathrm{diag}(\lambda_1, \lambda_2, \cdots, \lambda_n) = P^{-1}AP$.
  Note that
  \[
    D^i = (P^{-1}AP)(P^{-1}AP)\cdots(P^{-1}AP) = P^{-1}A^iP.
  \]
  Hence
  \[
    p_D(D) = p_D(P^{-1}AP) = P^{-1}[p_D(A)]P.
  \]
  Since similar matrices have the same characteristic polynomial. So
  \[
    p_A(D) = P^{-1}[p_A(A)]P.
  \]
  However, we also know that $D^i = \mathrm{diag}(\lambda_1^i, \lambda_2^i, \cdots \lambda_n^i)$. So
  \[
    p_A(D) = \mathrm{diag}(p_A(\lambda_1), p_A(\lambda_2), \cdots, p_A(\lambda_n)) = \mathrm{diag}(0, 0, \cdots, 0)
  \]
  since the eigenvalues are roots of $p_A(\lambda) = 0$. So $0 = p_A(D) = P^{-1}p_A(A)P$ and thus $p_A(A) = 0$.
\end{proof}
For $2\times 2$ matrices which are similar to $B=
\begin{pmatrix}
\lambda & 1 \\
0 & \lambda
\end{pmatrix}$,
    we see that the characteristic polynomial $p_B(z) =  (\lambda - z)^2$. Then $p_B(B) = (\lambda I - B)^2 =
    \begin{pmatrix}
      0 & -1\\
      0 & 0
    \end{pmatrix}^2 =
    \begin{pmatrix}
      0 & 0\\
      0 & 0
    \end{pmatrix}$.
So \emph{any} $2\times 2$ matrix satisfies the Cayley-Hamilton theorem.

Q: Calculating inverses using Cayley-Hamilton
A: If $A^{-1}$ exists, then
\begin{align*}
0&=A^{-1} p_A(A) \\
&= A^{-1}(c_0 + c_1A + c_2A^2 + \cdots + c_n A^n) \\
&=c_0 A^{-1} + c_1 + c_2A + \cdots + c_n A^{n - 1} \\
A^{-1} &= \frac{-1}{c_0}(c_1 + c_2 A + \cdots + c_n A^{n -1})
\end{align*}
Since $A^{-1}$ exists, $c_0 = \pm \det A \not= 0$. So we can calculate $A^{-1}$ from positive powers of $A$.

Q: Matrix exponential
A: We can define matrix exponentiation by
    \[
      e^A = I + A + \frac{1}{2!}A^2 + \cdots + \frac{1}{n!}A^n + \cdots.
    \]
    If $A$ is diagonalizable with $P$ with $D = P^{-1}AP = \mathrm{diag}(\lambda_1, \lambda_2, \cdots, \lambda_n)$, then
    \begin{align*}
      P^{-1}e^A P &= P^{-1}IP + P^{-1}AP + \frac{1}{2!}P^{-1}A^2P + \cdots\\
      &= I + D + \frac{1}{2!}D^{2} + \cdots\\
      &= \mathrm{diag}(e^{\lambda_1}, e^{\lambda_2}, \cdots e^{\lambda_n})
    \end{align*}
    So
    \[
      e^A = P[\mathrm{diag}(e^{\lambda_1}, e^{\lambda_2}, \cdots, e^{\lambda_n})]P^{-1}.
    \]

Q: Eigenvalues of a Hermitian matrix
A: The eigenvalues of a Hermitian matrix $H$ are real.
\begin{proof}
  Suppose that $H$ has eigenvalue $\lambda$ with eigenvector $\mathbf{v}\not= 0$. Then
\begin{align*}
H\mathbf{v} &= \lambda\mathbf{v} \\
\mathbf{v}^\dagger H\mathbf{v} &= \lambda \mathbf{v}^\dagger \mathbf{v} \\
(\mathbf{v}^\dagger H\mathbf{v})^\dagger &= (\lambda \mathbf{v}^\dagger \mathbf{v})^\dagger \\
\mathbf{v}^\dagger H^\dagger \mathbf{v} &= \lambda^* \mathbf{v}^\dagger \mathbf{v}\\
 \mathbf{v}^\dagger H \mathbf{v} &=\lambda^* \mathbf{v}^\dagger \mathbf{v}  \\
&=\lambda \mathbf{v}^\dagger \mathbf{v} \\
(\lambda-\lambda^*)\vec{v}^\dagger\vec{v}&=0
\end{align*}
Since $\mathbf{v} \not= 0$, we know that $\mathbf{v}^\dagger \mathbf{v} = \mathbf{v}\cdot \mathbf{v} \not =0$. So $\lambda = \lambda^*$ i.e. $\lambda$ is real.
\end{proof}

Q: Eigenvectors of a Hermitian matrix
A: The eigenvectors of a Hermitian matrix $H$ corresponding to distinct eigenvalues are orthogonal.
\begin{proof}
  Let
  \begin{align*}
    H\mathbf{v}_1 &= \lambda_1\mathbf{v}_1\tag{i}\\
    H\mathbf{v}_2 &= \lambda_2\mathbf{v}_2\tag{ii}.
  \end{align*}
  Pre-multiply (i) by $\mathbf{v}_2^\dagger$ to obtain
  \[
    \mathbf{v}_2^\dagger H\mathbf{v}_1 = \lambda_1 \mathbf{v}_2^\dagger \mathbf{v}_1\tag{iii}.
  \]
  Pre-multiply (ii) by $\mathbf{v}_1^\dagger$ and take the Hermitian conjugate to obtain
  \begin{align*}
(\mathbf{v}_1^\dagger H\mathbf{v}_2)^\dagger &= (\lambda_2\mathbf{v}_1^\dagger\mathbf{v}_2)^\dagger \\
  \mathbf{v}_2^\dagger H^\dagger\mathbf{v}_1 &= \lambda_2 \mathbf{v}_2^\dagger \mathbf{v}_1 \\
    \mathbf{v}_2^\dagger H\mathbf{v}_1 &= \lambda_2 \mathbf{v}_2^\dagger \mathbf{v}_1\tag{iv}.
  \end{align*}
  Equating (iii) and (iv) yields
  \begin{align*}
    \lambda_1 \mathbf{v}_2^\dagger \mathbf{v}_1 &= \lambda_2 \mathbf{v}_2^\dagger \mathbf{v}_1 \\
(\lambda_1-\lambda_2) \mathbf{v}_2^\dagger \mathbf{v}_1&=0
  \end{align*}
  Since $\lambda_1\not= \lambda_2$, we must have $\mathbf{v}_2^\dagger\mathbf{v}_1 = 0$. So their inner product is zero and are orthogonal.
\end{proof}

Q: Gram-Schmidt orthogonalisation
A: Let $\{\mathbf{u_k}\}_{k=1}^n$ be a basis for an inner product space $V$ with inner product $\big<,\big>$. Define $\mathbf{v_1}=\tfrac{\mathbf{u_1}}{\mid\mid\mathbf{u_1}\mid\mid}$ and $\{\mathbf{m_k}\}_{k=2}^n$ recursively by
\begin{equation}
\mathbf{m_k}=\mathbf{u_{k}}
-\sum_{i=1}^{k-1}\big<\mathbf{u_{k}},\vec{v}_i\big>\text{,}
\end{equation}
where $\mathbf{v_k}=\tfrac{\mathbf{m_k}}{\mid\mid\mathbf{m_k}\mid\mid}$ for $2\leq k\leq n$. Then $\{\mathbf{v_k}\}_{k=1}^n$ is an orthonormal basis for $V$.
\begin{proof}
We proceed by induction on $n$. In the case $n=1$, we suppose
$\mathbf{u_1}$ is a basis for the inner product space $V$. Letting $\mathbf{v_1}=\tfrac{\mathbf{u_1}}{\mid\mid\mathbf{u_1}\mid\mid}$, it is clear that $\mathrm{span}\big(\mathbf{v_1}\big)=\mathrm{span}\big(\mathbf{u_1}\big)=V$. Thus $\mathbf{v_1}$ is an orthonormal basis for $V$.
Now let $n\geq 1\in\mathbb{N}$, and suppose the result holds for arbitrary $n$. Then
\begin{align*}
\vec{m_{n+1}}&=\mathbf{u_{n+1}}-\sum\limits_{i=1}^n \langle\mathbf{u_{n+1}},\mathbf{v_i}\rangle\mathbf{v_i}. \\
\intertext{So for $1\leq j\leq n$, using the hypothesis that $\big<\mathbf{v_i},\mathbf{v_j}\big>=\delta_{ij}$:}
\langle\vec{m_{n+1}},\vec{v_j}\rangle &=\langle \vec {u_{n+1}}-\sum_{i=1}^n \langle \vec {u_{n+1}},\vec {v_i}\rangle \vec{v_i}, \vec{v_j} \rangle \\
&=\langle \vec{u_{n+1}},\vec{v_j} \rangle - \langle \sum_{i=1}^n \langle \vec{u_{n+1}},\vec{v_i} \rangle \vec{v_i},\vec{v_j} \rangle  \\
&=\langle\vec{u_{n+1}},\vec{v_j}\rangle -\langle\langle\vec{u_{n+1}},\vec{v_j}\rangle \vec{v_j},\vec{v_j}\rangle  \\
&=\langle\vec{u_{n+1}},\vec{v_j}\rangle -\langle\vec{u_{n+1}},\vec{v_j}\rangle \langle \vec{v_j},\vec{v_j}\rangle  \\
&=\langle\vec{u_{n+1}},\vec{v_j}\rangle -\langle\vec{u_{n+1}},\vec{v_j}\rangle \\
&=0
\end{align*}
Thus $\mathbf{m_{n+1}}$ is orthogonal to $\mathbf{v_j}$ for $1\leq j\leq n$, so we may take $\mathbf{v_{n+1}}=\tfrac{\mathbf{m_{n+1}}}{\mid\mid\mathbf{m_{n+1}}\mid\mid}$ to have $\{\mathbf{v_k}\}_{k=1}^{n+1}$ an orthonormal set of vectors which thus form a basis for $V$.
\end{proof}

Q: Unitary transformation
A: Suppose $U$ is the transformation between one orthonormal basis and a new orthonormal basis $\{\mathbf{u}_1, \mathbf{u}_2, \cdots, \mathbf{u}_n\}$, ie. $\langle \mathbf{u}_i\mid \mathbf{u}_j\rangle = \delta_{ij}$. Then wrt the original basis
\[
  U =
  \begin{pmatrix}
    (\mathbf{u}_1)_1 & (\mathbf{u}_2)_1 & \cdots & (\mathbf{u}_n)_1\\
    (\mathbf{u}_1)_2 & (\mathbf{u}_2)_2 & \cdots & (\mathbf{u}_n)_2\\
    \vdots & \vdots & \ddots & \vdots\\
    (\mathbf{u}_1)_n & (\mathbf{u}_2)_n & \cdots & (\mathbf{u}_n)_n
  \end{pmatrix}
\]
Then
\begin{align*}
  (U^\dagger U)_{ij} &= (U^\dagger)_{ik}U_{kj}\\
  &= U_{ki}^* U_{kj}\\
  &= (\mathbf{u}_i)^*_k(\mathbf{u}_j)_k\\
  &= \langle \mathbf{u}_i \mid \mathbf{u}_j \rangle\\
  &= \delta_{ij}
\end{align*}
So $U$ is a unitary matrix.

Q: Diagonalisability of Hermitian Matrices
A: An $n\times n$ Hermitian matrix has precisely $n$ orthogonal eigenvectors so is diagonalisable.
\begin{proof}
(non-examinable)
\end{proof}

Q: Normal matrices
A: A \emph{normal matrix} is a matrix that commutes with its own Hermitian conjugate, ie.
  \[
    NN^\dagger = N^\dagger N
  \]
It can be shown that:
  \begin{enumerate}
    \item If $\lambda$ is an eigenvalue of $N$, then $\lambda^*$ is an eigenvalue of $N^\dagger$.
    \item The eigenvectors of distinct eigenvalues are orthogonal.
    \item A normal matrix can always be diagonalized with an orthonormal basis of eigenvectors.
  \end{enumerate}
N.B. Hermitian, real symmetric, skew-Hermitian, real anti-symmetric, orthogonal, unitary matrices are all special cases of normal matrices.

Q: Sesquilinear, Hermitian and quadratic forms
A: A \emph{sesquilinear form} is a quantity $F = \mathbf{x}^\dagger A\mathbf{x} = x_i^*A_{ij}x_j$. If $A$ is Hermitian, then $F$ is a \emph{Hermitian form}. If $A$ is real symmetric, then $F$ is a \emph{quadratic form}.
Note that we can always choose the matrix to be symmetric. This is since for any antisymmetric $A$, we have
\begin{align*}
A_{ij}x_i^*x_j &=-A_{ji}x_i^*x_j \\
&=A_{ji}x_i^*x_j \\
\implies A_{ij}x_i^*x_j&= 0\\
\implies \mathbf{x}^\dagger A\mathbf{x} &=\vec 0.
\end{align*}
So we can just take the symmetric part.

Q: Show Hermitian forms are real.
A: \begin{proof}
$\mathbf{x}^\dagger H\mathbf{x}$ is a scalar so
\begin{align*}
(\mathbf{x}^\dagger H\mathbf{x})^* &= (\mathbf{x}^\dagger H\mathbf{x})^\dagger \\
&= \mathbf{x}^\dagger H^\dagger\mathbf{x} \\
&= \mathbf{x}^\dagger H\mathbf{x} \\
\therefore (\mathbf{x}^\dagger H\mathbf{x})^* &= \mathbf{x}^\dagger H\mathbf{x} \\
&\in\mathbb{R}
\end{align*}
\end{proof}

Q: Hermitian forms
A: Any Hermitian matrix can be diagonalized with a unitary transformation so $F(\mathbf{x}) = \mathbf{x}^\dagger H\mathbf{x} = \mathbf{x}^\dagger UDU^\dagger \mathbf{x}$.
Write $\mathbf{x}' = U^\dagger \mathbf{x}$.
So $F = (\mathbf{x}')^\dagger D\mathbf{x}'$, where $D = \mathrm{diag}(\lambda_1,\cdots,\lambda_n)$.
$\mathbf{x}'$ is the vector $\mathbf{x}$ relative to the eigenvector basis so
\[
  F(\mathbf{x}) = \sum_{i = 1}^n \lambda_i |x_i'|^2
\]

Q: Quadric
A: A \emph{quadric} is an $n$-dimensional surface defined by the zero of a real quadratic polynomial, ie.
  \[
    \mathbf{x}^T A\mathbf{x} + \mathbf{b}^T\mathbf{x} + c = 0,
  \]
  where $A$ is a real $n\times n$ matrix, $\mathbf{x}, \mathbf{b}$ are $n$-dimensional column vectors and $c$ is a constant scalar.
Letting $S$ be the symmetric part and letting the antisymmetric part vanish we have
\[
  \mathbf{x}^T S\mathbf{x} + \mathbf{b}^T\mathbf{x} + c = 0
\]
with $S$ symmetric.
Since $S$ is real and symmetric, we can diagonalize it using $S = QDQ^T$ with $D$ diagonal. We write $\mathbf{x}' = Q^T \mathbf{x}$ and $\mathbf{b}' = Q^T \mathbf{b}$. So we have
\[
  (\mathbf{x}')^TD\mathbf{x}' + (\mathbf{b}')^T \mathbf{x}' + c = 0.
\]
If $S$ is invertible, ie. with no zero eigenvalues, then write $\mathbf{x}'' = \mathbf{x}' + \frac{1}{2}D^{-1}\mathbf{b}'$ which shifts the origin to eliminate the linear term $(\mathbf{b}')^T\mathbf{x}'$ and finally drop the prime superfixes to obtain
\begin{align*}
0&=(\vec x -\frac{1}{2}D^{-1}\vec b)^TD((\vec x -\frac{1}{2}D^{-1}\vec b) \\
&=(\vec x -\frac{1}{2}\vec b^T(D^{-1})^T)D((\vec x -\frac{1}{2}D^{-1}\vec b) \\
&=(\vec x -\frac{1}{2}\vec b^T(D^{-1}))D((\vec x -\frac{1}{2}D^{-1}\vec b) \\
&=\mathbf{x}^TD\mathbf{x}-\vec b^T\vec x +\frac{1}{4}\vec b^TD^{-1}\vec b + \vec b^T\vec x + c \\
 \therefore  \mathbf{x}^TD\mathbf{x} &= k.
\end{align*}
where $k=-(c+\frac{1}{4}\vec b^TD^{-1}\vec b)$.
So through two transformations, we have ended up with a simple quadratic form.

Q: Conic sections as quadrics
A: Given a quadric in two dimensions, assuming det$S\neq 0$, we obtain
\begin{align*}
k&=\vec x^T D \vec x \\
&=\lambda_1x_1^2+\lambda_2x_2^2.
\end{align*}
\begin{enumerate}
  \item sgn($\lambda_1)=$sgn$(\lambda_2)$=sgn$(k)$. Then $\lambda_1\lambda_2 > 0$ and we have an ellipse with axes coinciding with eigenvectors of $S$.
  \item $\lambda_1\lambda_2 < 0$: say $\lambda_1 = k/a^2 > 0$, $\lambda_2 = -k/b^2 < 0$. So we obtain
    \[
      \frac{x_1^2}{a^2} - \frac{x_2^2}{b^2} = 1,
    \]
    which is a hyperbola.
  \item $\lambda_1\lambda_2 = 0$: Say $\lambda_2 = 0$, $\lambda_1\not= 0$. Note that in this case, det$S=0$ and we cannot use the standard method of shifting the origin.
    \begin{align*}
\mathbf{x}^T D\mathbf{x} + \mathbf{b}^T\mathbf{x} + c &= 0 \\
      \therefore\lambda_1(x_1')^2 + b_1'x_1' + b_2' x_2' + c &= 0.
    \end{align*}
    We use the coordinate transform (`completing the square') valid for $b_2\neq 0$:
    \begin{align*}
      x_1'' &= x_1' + \frac{b_1'}{2\lambda_1}\\
      x_2'' &= x_2' + \frac{c}{b_2'} - \frac{(b_1')^2}{4\lambda_1b_2'}
    \end{align*}
    to remove the $x_1'$ and constant term. Dropping the primes, we have
    \begin{align*}
0&=\lambda_1(x_1-\frac{b_1}{2\lambda_1})^2+b_1(x_1-\frac{b_1}{2\lambda_1})+b_2(x_2-\frac{c}{b_2}+\frac{b_1^2}{4\lambda_1b_2})+c \\
&=\lambda_1x_1^2-b_1x_1+\frac{b_1^2}{4\lambda_1}+b_1x_1-\frac{b_1^2}{2\lambda_1}+b_2x_2+\frac{b_1^2}{4\lambda_1} \\
&=\lambda_1 x_1^2 + b_2 x_2,
    \end{align*}
    which is a parabola.
 If $b_2' = 0$, we have $\lambda_1(x_1')^2 + b_1' x_1' + c = 0$. If we solve this quadratic for $x_1'$, we obtain 0, 1 or 2 solutions for $x_1$ (and $x_2$ can be any value). So we have 0, 1 or 2 straight lines.
\end{enumerate}

Q: Focus-directrix property
A: The the \emph{foci} of a conic section are $(\pm ae, 0)$ and the \emph{directrices} are $x = \pm a/e$.
  A \emph{conic section} is the set of points whose distance from focus is $e \times$ distance from directrix which is closer to that of focus (unless $e = 1$, where we take the distance to the other directrix).
\textbf{Case} $e < 1$. By definition,
[/latex]
[latex]
     Which is an ellipse with semi-major axis $a$ and semi-minor axis $a\sqrt{1 - e^2}$. (if $e = 0$, then we have a circle)
\textbf{Case} $e > 1$: hyperbola
[/latex]
[latex]
\textbf{Case} $e = 1$: parabola
[/latex]
[latex]
\textbf{Polar coordinates} Let $l/e$ be the distance from the focus to the directrix. So
\begin{align*}
\frac{l}{e}&=|\frac{a}{e}-ae| \\
  l &= a|1 - e^2|
\end{align*}
except for parabolas which have $l=2a$.
Define polar coordinates $(r, \theta)$ centered on a focus. So the focus-directrix property is
\begin{align*}
  r &= e\left(\frac{l}{e} \pm r\cos\theta\right)\\
r(1\pm e\cos\theta) &= l\\
  r &= \frac{l}{1 \pm e\cos\theta}
\end{align*}
where the $\pm$ depends on which focus you use as an origin.
We see that $r\to \infty$ if $\theta \to \cos^{-1}(-1/e)$, which is only possible if $e\geq 1$, ie. hyperbola or parabola. But ellipses have $e < 1$. So $r$ is bounded, as expected.

Q: Conditions for isometric matrix
A: Let $P\in O(n)$. Then the following are equivalent:
  \begin{enumerate}
    \item $P$ is orthogonal
    \item $|P\mathbf{x}| = |\mathbf{x}|$
    \item $(P\mathbf{x})^T(P\mathbf{y}) = \mathbf{x}^T\mathbf{y}$, ie. $(P\mathbf{x})\cdot(P\mathbf{y}) = \mathbf{x}\cdot \mathbf{y}$.
    \item If $(\mathbf{v}_1, \mathbf{v_2}, \cdots, \mathbf{v}_n)$ are orthonormal, so are $(P\mathbf{v}_1, P\mathbf{v}_2, \cdots, P\mathbf{v}_n)$
    \item The columns of $P$ are orthonormal.
  \end{enumerate}
\begin{proof}
  \begin{enumerate}
    \item $\Rightarrow$ (2): $|P\mathbf{x}|^2 = (P\mathbf{x})^T(P\mathbf{x}) = \mathbf{x}^TP^TP\mathbf{x} = \mathbf{x}^T\mathbf{x} = |\mathbf{x}|^2$
    \item $\Rightarrow$ (3):
\begin{align*}
|P(\mathbf{x} + \mathbf{y})|^2 &= |\mathbf{x + y}|^2 \\
|P\mathbf{x} + P\mathbf{y}|^2&=(\mathbf{x}^T + \mathbf{y}^T)(\mathbf{x + y}) \\
 |P\mathbf{x}|^2 + |P\mathbf{y}| + 2(P\mathbf{x})^TP\mathbf{y} &= \mathbf{x}^T\mathbf{x} + \vec y^T\mathbf{y} + \mathbf{y}^T\mathbf{x} + \mathbf{x}^T\mathbf{y} \\
|\mathbf{x}|^2 + |\mathbf{y}|^2 + 2(P\mathbf{x})^TP\mathbf{y} &=  |\mathbf{x}|^2 + |\mathbf{y}|^2 + 2\mathbf{x}^T\mathbf{y} \\
\therefore (P\mathbf{x})^TP\mathbf{y} &= \mathbf{x}^T\mathbf{y}
\end{align*}
    \item $\Rightarrow$ (4): $(P\mathbf{v}_i)^TP\mathbf{v}_j = \mathbf{v}_i^T\mathbf{v}_j = \delta_{ij}$. So $P\mathbf{v}_i$'s are also orthonormal.
    \item $\Rightarrow$ (5): Take the $\mathbf{v}_i$'s to be the standard basis. So the columns of $P$, being $P\mathbf{e}_i$, are orthonormal.
    \item $\Rightarrow$ (1): The columns of $P$ are orthonormal. Then $(PP^T)_{ij} = P_{ik}P_{jk} = (P_i)\cdot (P_j) = \delta_{ij}$. So $PP^T = I$.
  \end{enumerate}
\end{proof}

Q: Lorentz transformations
A: Consider the \emph{Minkowski} 1 + 1 dimension spacetime (ie. 1 space dimension and 1 time dimension)
  The \emph{Minkowski} inner product of 2 vectors $\mathbf{x}$ and $\mathbf{y}$ is
  \[
    \langle \mathbf{x}\mid \mathbf{y}\rangle = \mathbf{x}^TJ\mathbf{y},
  \]
  where
  \[
    J =
    \begin{pmatrix}
      1 & 0\\
      0 & -1
    \end{pmatrix},
  \]
  i.e. $\langle \mathbf{x}\mid \mathbf{y}\rangle = x_1y_1 - x_2y_2$.
  Consider a transformation matrix $M$ which preserves the Minkowski inner product, i.e.
  \[
    \langle \mathbf{x}|\mathbf{y}\rangle = \langle M\mathbf{x} | M\mathbf{y}\rangle
  \]
  for all $\mathbf{x}, \mathbf{y}$. Then $\mathbf{x}^TJ\mathbf{y} = (M\mathbf{x})^TJM\mathbf{y} = \mathbf{x}^T M^TJM\mathbf{y}$. So $J = M^TJM.$
We can show that $M$ takes the form of
\[
  H_\alpha = \begin{pmatrix}
    \cosh \alpha & \sinh \alpha\\
    \sinh \alpha & \cosh \alpha
  \end{pmatrix}\text{ or } K_{\alpha/2} =
  \begin{pmatrix}
    \cosh\alpha & -\sinh\alpha\\
    \sinh\alpha & -\cosh\alpha
  \end{pmatrix}
\]
where $H_\alpha$ is a \emph{hyperbolic rotation}, and $K_{\alpha/2}$ is a \emph{hyperbolic reflection}.
We choose $M_{11} > 0$ to preserve the direction of time.
  A \emph{Lorentz matrix} or a \emph{Lorentz boost} is a matrix in the form
  \[
    B_v = \frac{1}{\sqrt{1 - v^2}}
    \begin{pmatrix}
      1 & v\\
      v & 1
    \end{pmatrix}.
  \]
  Here $|v| < 1$, where we have chosen units in which the speed of light is equal to $1$. We have $B_v = H_{\tanh^{-1}v}$
  The \emph{Lorentz group} is a group of all Lorentz matrices under matrix multiplication.
It is easy to prove that this is a group. For the closure axiom, we have $B_{v_1}B_{v_2} = B_{v_3}$, where
\[
  v_3 = \tanh(\tanh^{-1} v_1 + \tanh^{-1} v_2) = \frac{v_1 + v_2}{1 + v_1v_2}
\]
The set of all $B_v$ is a group of transformations which preserve the Minkowski inner product.