every term, defined once

Glossary

Each term is introduced in exactly one week, then linked back from later weeks.

week 01 · Tensors, einops & einsum

Tensor: An n-dimensional array. In this course, tensors are the basic data structure flowing through every model.
Shape: The size of each tensor axis, read like a type signature for the data.
Broadcasting: PyTorch automatically expands compatible smaller tensors across missing dimensions before an operation.
einops: A small library for readable tensor rearrangement, reduction, and repetition.
einsum: Index-notation syntax for dot products, matrix multiplies, reductions, and many tensor contractions.

week 02 · Your first neural network

Weight: A learnable number inside a model, usually stored in a matrix.
Bias: A learnable offset added after a weighted sum.
ReLU: The nonlinearity max(0, x), used to make stacked layers more than one giant linear map.
Logits: Raw model scores before softmax converts them into probabilities.
Residual / skip connection: A direct path that adds an earlier activation to a later one: out = x + f(x).
Softmax: A function that turns a vector of scores into a probability distribution by exponentiating and normalising.
Cross-entropy: The standard classification loss; it measures how little probability the model put on the correct label.
Convolution: A small kernel slid across an image, computing the same local pattern at every position. Translation-equivariant feature detection.

week 03 · How training actually works

Gradient: The local slope of the loss with respect to each parameter.
SGD: Stochastic gradient descent: step parameters opposite the gradient.
Learning rate: The step size an optimizer uses when updating parameters.
Adam: A popular optimizer that adapts per-parameter step sizes using running gradient statistics.
Weight decay: A regularizer that nudges weights toward zero during optimization, discouraging solutions that require large parameter values.

week 04 · Backprop from scratch

Chain rule: The rule that lets backprop multiply local derivatives through a composed computation.
Computational graph: A graph of tensor operations whose reverse traversal computes gradients.
Autograd: Automatic differentiation: software that builds and backpropagates through a computation graph.

week 05 · Build a transformer

Token: An integer chunk of text used as a language model input or output.
Residual stream: The shared per-token vector workspace that transformer layers read from and write to.
Attention: A transformer operation where each token decides which earlier tokens to read from.
Query/key/value: The three attention vectors: what I ask for, what I advertise, and what I copy.
Positional encoding: Information added to a token embedding that tells the model where the token sits in the sequence, since attention alone is order-agnostic.
MLP: The per-token feed-forward block in a transformer: two linear layers with a nonlinearity, where much feature-like and factual information appears.
LayerNorm: A normalisation that rescales each token activation vector to a stable mean and variance before passing it on.
Unembedding: The final matrix that maps a residual-stream vector back to a score (logit) for each token in the vocabulary.

week 06 · Opening the box: induction heads

Mechanistic interpretability: The project of explaining model behavior by identifying the internal algorithms and components that cause it.
TransformerLens: A library that exposes transformer activations and hooks for mechanistic interpretability work.
Induction head: An attention head that implements a "copy what followed this token last time" pattern.
Ablation: Removing or zeroing a component to test whether behavior depends on it.

week 07 · Superposition & toy models

Feature: A human-meaningful property represented somewhere in a model activation.
Superposition: Representing more features than dimensions by packing sparse features into overlapping directions.
Sparsity: The property that most possible features are inactive for any given input.
Polysemantic neuron: A neuron that responds to multiple unrelated features.
Sparse autoencoder (SAE): A model trained to reconstruct activations using a sparse, wider latent representation.

week 08 · SAEs on real models

sae_lens: An open-source library for loading, training, and evaluating sparse autoencoders on model activations.
Neuronpedia: A public browser for inspecting model neurons and SAE features.
Feature dashboard: A report showing examples, stats, and effects for one SAE latent or neuron.

week 09 · The IOI circuit & activation patching

IOI: Indirect Object Identification, a benchmark sentence task used to study a GPT-2 circuit.
Activation patching: Swapping activations between model runs to test which internals causally affect behavior.
Noising vs. denoising: Complementary patching experiments: noising corrupts a clean run to test necessity, while denoising restores part of a corrupted run to test sufficiency.
Logit difference: A metric comparing model scores for the correct and incorrect answer tokens.
Direct logit attribution: Decomposing the final logits into per-component contributions to the residual stream reveals which heads or layers write toward the answer. The result provides correlational evidence.
Name-mover head: An attention head in the IOI circuit that attends to the correct name and copies it into the output logits.
S-inhibition head: An attention head in the IOI circuit that moves the "this name is duplicated" signal to the final position, reducing later heads' attention to the repeated subject name.
Path patching: A refinement of activation patching that isolates a specific sender-to-receiver path while holding the rest of the activation fixed.

week 10 · Linear probes

Linear probe: A simple linear classifier trained on activations to test whether information is linearly readable.
Mass-mean probe: A training-free linear probe whose direction is the difference between the mean activation for each of two labeled classes.
Logistic regression: A linear classifier that learns a weight vector and bias, then maps their score to a class probability with the logistic function.
PCA: Principal component analysis: a way to project high-dimensional activations onto high-variance directions.
Truth direction: A direction in activation space associated with true versus false statements.
Confound: A variable correlated with the label that can create an apparent effect without representing the property the experiment aims to measure.

week 11 · Steering & function vectors

Steering vector: A vector added to an activation to push model behavior in a chosen direction.
Function vector: An activation direction that appears to encode a task such as antonym generation.
Contrastive pair: Two inputs matched except for a target property; subtracting their activations estimates a direction associated with that property.
Activation addition: An intervention that adds a chosen vector to a model activation during a forward pass to steer subsequent behavior.
nnsight: A tracing and intervention library for reading and editing model internals, including remotely hosted models.

week 12 · Capstone: grokking & where to go next

Grokking: A delayed transition where a model moves from memorization to real generalization long after fitting the training set.
Progress measure: A quantity computed during training that tracks the gradual formation of a mechanism, even while headline loss or accuracy is flat.
Fourier basis: A way to represent periodic patterns as sums of sine and cosine waves.
FFT: Fast Fourier transform: an efficient algorithm for computing a signal's discrete Fourier transform and revealing its frequency components.
Restricted ablation: An ablation that removes everything except a hypothesized subspace or mechanism.