Title: Misc

Q: Imitation learning
A: Solving optimal policy problems through supervised learning from expert actions.
The problem with this is that errors can compound as we shift away from the training distribution

Q: Inner vs Outer Misalignment
A: Alignment research aims to line up three different descriptions of an AI system:
1. Intended goals ('wishes'): “the hypothetical (but hard to articulate) description of an ideal AI system that is fully aligned to the desires of the human operator”;
2. Specified goals (or ‘outer specification’): The goals we actually specify — typically jointly through an objective function and a dataset;
3. Emergent goals (or ‘inner specification’): The goals the AI actually advances.
‘Outer misalignment’ is a mismatch between the intended goals (1) and the specified goals (2),
Whereas ‘inner misalignment’ is a mismatch between the specified goals (2) and the emergent goals (3).

Q: Sample efficiency
A: Number of training samples required for supervised learning to achieve a given level of performance

Q: Batch norm vs. layer norm
A: Batch: (b c w h) -> (c)
Layer: (b s e) -> (b s)
Hence layer norm has no issues with collapsing dimensions and we can evaluate the same way in training and inference.
Batch is used for CV and Layer is used for NLP.

Q: Affine transformation
A: Unlike a purely linear transformation, an affine transformation need not preserve the origin, e.g. a linear layer with a bias

Q: CV
A: Computer Vision

Q: Discrete Fourier Transform
A: Given a tuple of complex numbers \(\left(x_0, x_1, \ldots, x_{N-1}\right)\), it's discrete Fourier transform (DFT) is the sequence \(\left(y_0, y_1, \ldots, y_{N-1}\right)\) defined by:
\[y_k=\sum_{j=0}^{N-1} \omega_N^{j k} x_j\]
where \(\omega_N=e^{-2 \pi i / N}\) is the Nth root of unity.
This can equivalently be written as the following matrix equation:
\[\left[\begin{array}{ccccc} 1 & 1 & 1 & \cdots & 1 \\ 1 & \omega_N & \omega_N^2 & \cdots & \omega_N^{N-1} \\ 1 & \omega_N^2 & \omega_N^4 & \ldots & \omega_N^{2(N-1)} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & \omega_N^{N-1} & \omega_N^{2(N-1)} & \ldots & \omega_N^{(N-1)^2} \end{array}\right]\left[\begin{array}{c} x_0 \\ x_1 \\ \vdots \\ x_{N-1} \end{array}\right]=\left[\begin{array}{c} y_0 \\ y_1 \\ \vdots \\ y_{N-1} \end{array}\right]\]

Q: Inverse Discrete Fourier Transform
A: The inverse transform is given by:
\[x_n=\frac{1}{N} \sum_{k=0}^{N-1} X_k \cdot e^{i \frac{2 \pi}{N} k n}\]
or in matrix form
\[
\mathcal{F}^{-1}\mathbf{x} = \frac{1}{N}\left[\begin{array}{ccccc}
\omega^{-0.0} & \omega^{-0 \cdot 1} & \omega^{-0.2} & \cdots & \omega^{-0 \cdot(N-1)} \\
\omega^{-1 \cdot 0} & \omega^{-1 \cdot 1} & \omega^{-1 \cdot 2} & \cdots & \omega^{-1 \cdot(N-1)} \\
\omega^{-2 \cdot 0} & \omega^{-2 \cdot 1} & \omega^{-2 \cdot 2} & \cdots & \omega^{-2 \cdot(N-1)} \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
\omega^{-(N-1) \cdot 0} & \omega^{-(N-1) \cdot 1} & \omega^{-(N-1) \cdot 2} & \cdots & \omega^{-(N-1) \cdot(N-1)}
\end{array}\right]\left[\begin{array}{c}
x_0 \\
x_1 \\
\vdots \\
x_{N-1}
\end{array}\right]
\]

Q: Continuous Fourier Transform
A: The Fourier Transform is given by
\[
\hat{f}_N(x)=\frac{a_0}{2}+\sum_{n=1}^N a_n \cos n x+\sum_{n=1}^N b_n \sin n x
\]
where
\[
\begin{aligned}
& a_n=\frac{1}{\pi} \int_{-\pi}^\pi f(x) \cos (n x) d x \\
& b_n=\frac{1}{\pi} \int_{-\pi}^\pi f(x) \sin (n x) d x
\end{aligned}
\]

Q: Batch norm vs. layer norm vs. instance norm vs. group norm
A: 1
2
3
4
reduce(img, 'b c h w -> c', 'mean') # batch/power norm
reduce(tokens, 'b s e -> b', 'mean') # layer norm
reduce(img, 'b s h w -> b s', 'mean') # instance norm
reduce(img, 'b (n g) h w -> b n', 'mean', g=group_size) # group norm

Q: Degradation problem
A: More layers can sometimes lead to worse training error

Q: Residual connection
A: Resnets attempt to solve the degradation problem by allowing subnetworks to learn the identity mapping.
The operation \(F + W_s x\) is performed by a shortcut/skip connection and element-wise addition. \(W_s\) is a projection to match dims. We try to learn the residual function \(F\).

Q: Bottleneck structure
A: The bottleneck in a neural network is just a layer with fewer neurons than the layer below or above it. Having such a layer encourages the network to compress feature representations.

Q: Softmax temperature
A: For temperature \(T\), the softmax probabilities are
\[
P_i=\frac{e^{\frac{y_i}{T}}}{\sum_{k=1}^n e^{\frac{y_k}{T}}}
\]
where \(T=0\) is the max-function and \(T=\infty\) corresponds to the uniform distribution.

Q: Nat
A: The natural unit of information is the information content of an event when the probability of that event occurring is 1/e. These are the units in the entropy equation
\[
H = -\sum_i p_i \log p_i.
\]

Q: Polysemanticity
A: Neurons which respond to multiple unrelated inputs

Q: Superposition
A: - when models represent more features than they have dimensions
- allows the model to use fewer neurons, conserving them for more important tasks
- cost is "interference" that requires nonlinear filtering
- this is a property of the geometry of high-dimensional spaces, which only allow for n orthogonal vectors, but exponentially many almost orthogonal vectors

Q: Equivariance
A: Neurons that are slightly transformed versions of the same basic feature, e.g.
- rotated copies
- scaled copies
- flipped copies
- different colors.

Q: Convergent learning
A: Universality (or “convergent learning”) of features is the hypothesis that different neural networks can develop highly correlated neurons

Q: CCS
A: Contrast-Consistent Search: an unsupervised learning method that learns a "probe" which maps normalised hidden states to the unit interval to represent probabilities of truth.
The probe uses a linear projection followed by a sigmoid:
\[
p_{\theta, b}(\tilde{\phi})=\sigma\left(\theta^T \tilde{\phi}+b\right).
\]
The probe is trained using the sum of two loss functions, consistency loss:
\[
L_{\text {consistency }}\left(\theta, b ; q_i\right):=\left[p_{\theta, b}\left(x_i^{+}\right)-\left(1-p_{\theta, b}\left(x_i^{-}\right)\right)\right]^2
\]
and confidence loss:
\[
L_{\text {confidence }}\left(\theta, b ; q_i\right):=\min \left\{p_{\theta, b}\left(x_i^{+}\right), p_{\theta, b}\left(x_i^{-}\right)\right\}^2
\]
Finally, we can break the symmetry between \(x_i^-\) and \(x_i^+\) by checking the truth value of conjunctions \(x\cap x'\) and disjunctions \(x\cup x'\).

Q: KL divergence
A: Kullback-Leibler Diverence is defined as:
\[
D_\text{KL} (P || Q) = \mathbb{E}_{X\sim P} \left[ \log\left(\frac{P(X)}{Q(X)}\right) \right] = \int \log\left(\frac{p(x)}{q(x)}\right) p(x) \ \mathrm{d} x
\]
Intuitions:
- Expected excess surprise \(I_P(x)=-\log P(x)\) of a bad model \(Q\) vs. reality \(P\)
- Expected bits of evidence in favour of \(Q\) over \(P\) using a log-likelihood ratio test
- Expected excess minimum message length \(\operatorname{MML}(P)=-\sum_x P(x) \log_2 P(x)\) for using bad model \(Q\) vs. reality \(P\)
- Expected log-return gambling against the bookie offering fair odds using \(Q\) when you know \(P\).

Q: Gambling return from model differences
A: The Lagrangian for expected log wealth from a unit bet \(\left(b_x\right)_{x\in \chi}\) against a house offering fair odds under model \(Q\) is
\[
L(\lambda ; \mathbf{b})=\sum_x p_x \ln \left(b_x / q_x\right)+\lambda\left(1-\sum_x b_x\right) .
\]
Differentiating wrt \(b_x\) gives that \(p_x=\lambda b_x\) and then summing over \(x\) gives \(\lambda=1\) so your expected log winnings are
\[
\sum_x p_x \times \ln \left(p_x / q_x\right)=D(P \| Q).
\]

Q: Average similarity of random vectors
A: Average squared cosine similarity of two randomly chosen vectors is \(1 / n_\text{dim}\).