Swap one idea for another, inside the model
Every concept a language model reads becomes a state inside it. We measured that state for 56 concepts in Qwen3-4B. Here you can replace one with another while the model is running: it still sees “France” in your sentence, but it thinks Japan, and answers that way.
Measured on every pair of this group with its test question (exp117): yellow = the whole concept, grey = only its visible shape, by floor. The dots on the right are strength at floor 8.
The knowledge comes with it
Swap a concept and you don't just change one answer: the model carries the whole bundle. Asked about Japan while thinking Texas (floor 8):
| asked about Japan | before | after |
|---|---|---|
| capital | Tokyo. | Austin. |
| language | Japanese. | English. |
| currency | Yen. | The U.S. dollar. |
| next to | China, Russia… | Texas is next to Oklahoma, Louisiana, Arkansas, and Missouri. |
| people called | Japanese. | Texan. |
| footwear | Sandal. | Boots. |
| landmark | Tokyo Tower. | The Alamo. |
| who lives there | People of Japanese descent. | People of all backgrounds live in Texas. |
Across 8 swaps of countries for other countries or US states, 86–95% of 11 kinds of fact moved to the new place and almost none stayed with the old one (2 of 87); the rest became blends, most often for “most popular sport”. Swapped for a banana, Japan's people are “Banana people” and it is home to “Monkeys”. exp118, pre-registered; outputs quoted exactly.
What the edit does. For each concept we took the model's internal state at that word, after a given floor, in six plain sentences, and averaged it. To swap X for Y we add (state of Y − state of X) at every place X appears in your prompt, at the floor you pick. Nothing else changes: same words, same weights.
What we found testing it. At floor 8 the whole-concept swap turned the model's answer into the other concept's answer for 80–100% of pairs in all seven groups; random edits of the same size changed nothing. Swapping only the visible shape (the circle of weekdays, the line of numbers) worked 16–37% of the time: the picture is a shadow of the idea. Strength works like a switch, not a dial: below half almost nothing happens, past three-quarters almost everything flips.
Why depth matters. Each question reads its concept at its own floor. A country's capital is looked up between floors 20 and 24; swap later and the answer was already decided. The chart shows where each group stops working.
Qwen3-4B, pre-registered as exp117 (two of three predictions held). Runs on the hosted model; a run takes a few seconds once it's awake. Part of the Neuron Tower; see also Inside the calendar and Learning to add.