Reading the prompt · 15 positions, all at once
Every column is one position. Its thread is the residual stream, the vector the model carries for that token up through 24 layers. Cross-threads are attention, the only way information moves between positions, and only backwards. The soft marks are the MLP rewriting the thread in place. At the top, a distribution over 50,257 tokens; one is chosen and becomes the next thread. The prompt is read in parallel. Generation is one token at a time, and the cloth is the only memory.