Learning to add
A tiny transformer learns (a + b) mod 113 from 30% of all pairs. It gets every training pair right almost at once, by memorising. Much later it suddenly gets pairs it has never seen right too. Drag the timeline to watch what changes inside: the numbers settle onto clocks, and a sum becomes a rotation.
a sum in use
Pick a sum. Most were never in training; dashed ones were.
On each clock, every number is a dot, coloured by its value; its place is where the model keeps that number. The yellow star is the model's state after reading “a + b =”; the ring marks where the right answer belongs on that clock.
eight neurons, over every pair (a across, b down)
Each square is one neuron's firing for all 12,769 pairs. Early it is noise; after the jump it becomes a plaid: a wave across a plus a wave down b, at the same frequency (about 82% of each pattern). The neuron's bend (it can't go below zero) mixes the two waves, and that mixing is what makes the part that depends on a + b (about 14%), the part the readout uses.
What you're seeing. The model has no idea of numbers. It starts with 113 random vectors. To add, it ends up placing the numbers around circles, a different circle for each of a few frequencies (here 8, 21 and 1: on the frequency-8 clock, number n sits at 8n/113 of a turn). Adding becomes turning: a's angle plus b's angle is the answer's angle.
Why several clocks. On one clock, neighbouring answers sit only 3° apart, and each clock is a little blurry: the model's state lands within about 8–15° of the right spot. Read together, the clocks agree on exactly one answer, the way a vernier scale reads finer than its marks.
The surprise in the timeline. The clocks get clean before the model gets better on unseen pairs. The structure that generalises is being built while the model still looks like it is only memorising; the jump in unseen accuracy is the moment it overtakes the memorised lookup. Weight decay keeps squeezing the network until the compact solution wins.
Setup: 1 attention layer (4 heads), 512 ReLU neurons, no layer norm; AdamW, weight decay 1.0, full batch, 9,000 steps, trained on a laptop in under two minutes, following “Progress measures for grokking via mechanistic interpretability” (2023). Every frame is read with the weights of that step. Part of the Neuron Tower work on clankerchurch.com.