Inside the calendar
Nobody told a language model that days repeat. Yet inside Qwen3-4B, the seven weekdays and the twelve months sit on circles, in calendar order, the same trick a tiny model invents when it learns to add on a clock. Slide through the model's depth to watch the ring form. Then see what happens when we try to turn it.
Each small dot is the model's state at the word in one of six sentences (“Today is Friday.”, “See you on Friday!”…); the big dot is their average; the line joins them in calendar order. Shown in the three directions that vary most at that floor. Drag to turn.
how circular, at this floor
1.0 = a perfect ring in calendar order. Shuffled labels reach about by chance.
is the shape the code?
Ask “What day is two days after Friday?” and edit the model's state at “Friday” to look like Saturday. Does the answer move one day? Out of the 5 days it gets right:
■ turning only the visible circle (2 directions) · ■ swapping the whole day (6 directions) · random edits of the same size moved nothing.
The ring is real. At almost every floor the weekdays fall in calendar order around a circle, far beyond what shuffled labels produce (weekdays 0.975 at floor 28, months 0.995 at floor 25; shuffles reach 0.82 and 0.62).
But turning it does nothing. We predicted, before running, that rotating “Friday” one step along the circle would make the model answer one day later. It didn't, at any floor, or at all floors at once. Swapping the day's full pattern, six directions instead of two, does move the answer, 4 times out of 5 at floor 8, less and less deeper in, as the question has already read the day by then.
So the circle is a shadow. The model keeps “which day” in six directions, and the clean ring is the part that fits in two. Pictures of a model's geometry are real, and beautiful, but the variable it actually uses can be bigger than the picture. Compare the tiny model on Learning to add, where the clocks are the whole story.
exp115 on Qwen3-4B, pre-registered (the circle tests passed, the rotation test failed; the six-direction swap was a follow-up). Part of the Neuron Tower work on clankerchurch.com.