Title: TransformerLens

Q: Cache intermediate activations
A: 1 | model.run_with_cache(tokens, remove_batch_dim: bool = False, return_type=Union['logits', 'loss', 'both'], reset_hooks_start = True)

Q: Visualise attention pattern
A: Two slightly different visualisations:
1
2
cv.attention.attention_heads(tokens=gpt2_str_tokens, attention=attention_pattern)
cv.attention.attention_pattern(tokens=gpt2_str_tokens, attention=attention_pattern)

Q: Run hooks
A: A hook method can edit and return an overriden value. One can pick an activation by exact name or boolean callable.
1 | model.run_with_hooks( tokens, fwd_hooks=[(activation_name: Union[str, Callable], hook: Callable[Tensor, HookPoint] )] )
The main difference to pytorch hooks is that they act on activations rather than layers

Q: Clear hooks
A: 1 | model.reset_hooks()

Q: Activation Patching
A: Activation Patching uses
- a clean prompt
  - patch a particular activation with the cached value from a clean run
- a corrupted prompt
  - evaluate the model on this
- metric
  - see how closely the results match according to this.

Q: Folding Layer Norm
A: - Recall layer norm averages over the hidden dimension
- The idea is to fold the linear "centering" part of the layer norm into the weights
- `center_writing_weights` set every weight matrix writing to the residual to have zero mean
- we can fold the scaling and translation weights into the weights of the subsequent layer

Q: circuitsvis
A: Visualisation library usually imported as `cv`.

Q: ActivationCache
A: Indexed by [activation_name, layer]
1 | from transformer_lens import ActivationCache

Q: Downloading models
A: 1
2
3
from transformer_lens.loading_from_pretrained import get_checkpoint_labels
checkpoint_labels, checkpoint_label_type = get_checkpoint_labels(model_name)
model = transformer_lens.HookedTransformer.from_pretrained(model_name, device=device, checkpoint_index: int, checkpoint_value: int)

Q: Hook point
A: - A dummy nn.Module that is the identity function by default
- Added to models in order to more easily cache activations
1 | from transformer_lens.hook_points import HookPoint

Q: Tokenizer methods
A: 1
2
3
4
5
model.to_tokens(str, prepend_bos=True) -> List[int]
model.to_str_tokens(str, prepend_bos=True) -> List[str]
model.to_single_token(str) -> int
model.to_string(t.Tensor[int]) -> Union[str, List[str]] # inverse of to_tokens
model.get_token_position(single_token, tokens, mode="first") #  returns the position of single_token in tokens

Q: Factored Matrix
A: 1
2
3
4
AB_factor = transformer_lens.FactoredMatrix(A, B)
print(AB_factor.norm())
print(AB_factor.eigenvalues)
print(AB_factor.S) # Singular Values

Q: Generating Text
A: 1 | model.generate("(CNN) President Barack Obama caught in embarrassing new scandal\n", max_new_tokens=50, temperature=0.7, prepend_bos=True)

Q: Model utility methods
A: The parent class `HookedRootModule` defines various utility methods, e.g. reset_hooks, run_with_cache and run_with_hooks