I. The Duality of Surprise

When we say that we want to make a language model smarter, we are asking a question about a process, not merely its outcome. A larger model can know more facts, score higher on a benchmark, and produce more impressive answers, yet those outcomes alone do not tell us whether it discovered more reusable structure or simply fit more cases. Intelligence, if the word is going to name something deeper than accumulated skill, should describe how experience changes what the system can learn next: how many cases become governed by the same rule, how far that rule transfers, and how little new evidence is needed before it becomes useful somewhere else.

This is an effective-reductionist question. Intelligence belongs to our description of an agent, while a neural network is implemented as arrays of numbers undergoing physical computation. A list of parameters is too low-level to explain the property, and a benchmark score is too high-level to locate it. We need an intermediate description that is measurable in the mechanism and still says what changed in the learner. For a predictive model, information gives us one such bridge.

A language model repeatedly turns a history \(x_{<t}\) into a distribution over what might come next. If it assigns probability \(q(x_t\mid x_{<t})\) to the token that arrives, an ideal code spends \[ \ell_t=-\log_2 q(x_t\mid x_{<t}) \] bits to record it. A probability of \(1/2\) costs one bit; a probability of \(1/256\) costs eight. Across a sequence, \[ L_q(x_{1:T}) = -\sum_{t=1}^{T}\log_2 q(x_t\mid x_{<t}) = -\log_2 q(x_{1:T}). \] Surprise, predictive loss, and description length are the same bill written in different units. Arithmetic coding can turn the model's probabilities into a bitstream approaching this length, while the lengths of any uniquely decodable code induce a corresponding probability distribution.¹

The equivalence is exact only after we condition on the predictor. The predictor itself is not free. A program with every answer written into its source can assign the data probability one, moving the entire description from the bitstream into the program. A complete account must therefore pay for both parts, \[ L_{\mathrm{total}} = L(\text{predictor}) + L(\text{data}\mid\text{predictor}). \] Memorization moves information between these terms. A regularity produces genuine compression when its cost is paid once and the resulting savings recur across data that were not used to specify it.^{2, 12}

Compression is consequently not a definition of intelligence. A passive compressor need not plan, choose interventions, or distinguish a cause from a correlation. What the accounting gives us is a test for reusable structure. Two networks can make the same predictions on familiar examples while organizing their knowledge very differently. One can preserve every case separately. The other can represent a relation once and spend it again. Their training losses agree until the world asks for a new combination, at which point the difference between stored skill and a more general learner becomes visible.

II. A Rule \(\bmod 7\)

Consider the days of the week. A system can learn the answer to every offset query separately: \[ f_{\mathrm{Mon},+1}\mapsto\mathrm{Tue}, \qquad f_{\mathrm{Mon},+2}\mapsto\mathrm{Wed}, \qquad \ldots \] Continued across seven starting days and seven offsets, this representation is a table of forty-nine answers. Nothing in the table says that its entries are versions of one operation. A different system can assign every day a phase \(k\in\mathbb Z_7\) and learn the same input-output function as \[ f(k,m)=k+m\pmod 7. \] The first program stores a table. The second stores a coordinate system and a rule. Both preserve which day is which, but the second can answer a question such as “three days after Thursday” even when that pair was withheld, provided the other examples were enough to identify the common operation.

Generalization here is not seven concepts collapsing into one point. Such a collapse would destroy the information required to distinguish Monday from Thursday. It is seven arbitrary symbols becoming seven states of one latent variable. The proposed compression is in their relational description: \[ \underbrace{\text{many unrelated transitions}}_{\text{lookup table}} \quad\longrightarrow\quad \underbrace{\text{one generator applied repeatedly}}_{\text{shared rule}}. \] The entropy of the answer need not fall. The length of the program that relates the answers can. This comparison is always relative to a coding language: for a few queries, a table may be cheaper than transmitting coordinates and a generator. The shared rule earns its name only when that fixed cost is amortized across enough offsets, contexts, aliases, or held-out combinations.

A cyclic representation can make this shorter description visible. At checkpoint \(t\), suppose a state \(k\) in context \(c\) has activation \[ h_t(k,c) = b_t + A_t \begin{bmatrix} \cos(2\pi k/7) \\ \sin(2\pi k/7) \end{bmatrix} + r_t(k,c). \] The matrix \(A_t\) embeds a common phase into activation space, while \(r_t\) contains context, surface form, and whatever else the network superposes there. Let \(\widetilde h_t(k,c)=h_t(k,c)-b_t\). A single linear action on the centered state subspace can now implement the successor relation: \[ R_t \widetilde h_t(k,c)\simeq \widetilde h_t(k+1,c), \qquad R_t^7\widetilde h_t(k,c)\simeq \widetilde h_t(k,c). \] Equivalently, the operation on uncentered activations is affine: subtract \(b_t\), apply \(R_t\), then add \(b_t\) again. If the same action works across states, contexts, and unseen combinations, seven cases are behaving as one orbit of one operation.

Weekdays are only the easiest picture. Months and clock positions are other cycles. A two-dimensional periodic grid requires two commuting generators. Binary attributes under XOR form a hypercube. Rotations and reflections form a dihedral system in which the operations no longer commute. Paths, trees, and broken cycles have boundaries or exceptions that prevent the same exact symmetry. If relational compression is a useful account of generalization, the geometry should follow the structure of the problem rather than our preference for circles.

This also explains why low dimensionality is the wrong definition of compression. A cyclic representation may use several Fourier harmonics, \[ h(k) = \sum_q a_q\cos(2\pi qk/7) + b_q\sin(2\pi qk/7), \] and still be exactly equivariant to the same shift. The resulting loop may be rippled and high-rank while the underlying rule remains short. What matters is whether the representation admits a common action, not whether its first two principal components draw a pretty circle.

III. Geometry Is Not Generalization

Circular weekday and month features already appear in trained language models. Engels, Michaud, and collaborators recovered them with SAE clustering in GPT-2 and Mistral 7B, found analogous hidden-state circles in Llama 3 8B, and showed with interventions that circular subspaces affect calendar arithmetic in Mistral and Llama.³ Goodfire later traced corresponding activation and behavior manifolds in Llama 3.1 8B, and found an addition mechanism shared across ordinary arithmetic and cyclic concepts.⁴ These results establish an important possibility: a nonlinear feature can be a computational object inside a trained model.

A circle by itself does not tell us when, or whether, the model learned to generalize. Cyclic geometry can be the passive consequence of cyclic statistics. If the normalized co-occurrence statistics for two states depend mostly on their separation, the resulting matrix is approximately circulant. Circulant matrices have a Fourier eigenbasis, and low-frequency sine and cosine pairs can already trace a circle. Recent work derives this mechanism for calendar concepts and finds the resulting geometry across embedding models and language models.⁵ The shape may therefore record a symmetry in the data before it supports any reusable computation.

Several changes that look identical in a final picture must therefore be separated. A model may first organize observations according to a statistical symmetry, later learn an operation that acts on that organization, and later still stop relying on machinery that handled individual cases. Representational symmetry, operator reuse, held-out behavior, and cleanup can occur at different times. A static activation plot cannot tell us which event we are looking at.

This separation also changes what it means for a concept to be learned. Park and collaborators assigned familiar words to nodes of rings, grids, and other graphs, then supplied a fixed language model with increasingly long traces from those graphs. As context length increased, the graph-alignment energy reached its minimum shortly before rule-following accuracy rose. When a new ordering of weekdays conflicted with the calendar learned during pretraining, both rings remained present in different subspaces.⁶ A concept can acquire a new relational role without erasing everything it previously meant. The relevant object is often a context-conditioned subspace, not a single permanent embedding.

Grokking supplies the training-time analogue. Small transformers trained on modular addition often memorize their observed table long before they generalize to its missing entries. Nanda and collaborators reverse-engineered a Fourier algorithm that gradually forms underneath this apparently sudden behavioral transition, followed by the later removal of memorizing components.⁷ The general lesson is more useful than the circle: a model can improve because the computation supporting its answers has changed, even after its training error has reached zero.

IV. What a Rule Looks Like

What would license the stronger claim that a representation has become a rule? Monday through Sunday occupying nearby points is not enough. Different surface forms and contexts must first become interchangeable as evidence about the same state. The states must then become positions on a structure over which one operation acts, including on examples that did not determine it. In the language of two-part codes, a lookup scheme pays separately for local answers, while a state variable and one transferable operation pay a fixed cost that can be reused.²

This is the sense in which generalization begins to look like intelligence. The answer space has not become smaller. The vocabulary needed to explain its transitions has. A lookup table can answer a familiar question, while a relational representation can answer a new one because what it learned from one case changes how it treats the others. The more general learner discovers which distinctions can be factored away, which must be preserved, and which operation can be spent again on an unseen case.

Geometry gives this act of reduction a possible visible form. If seven clouds of context-dependent activations become seven stable positions on one orbit, and an operator learned on part of that orbit transfers across the rest, the representation is behaving like a variable. A periodic grid should instead admit two commuting directions, an XOR problem a hypercube, and a problem with reflections operations that do not commute. The shape proposes an internal grammar for what transformations are possible. Transfer asks whether that grammar extends beyond the examples used to fit it, while intervention tests whether the computation is sensitive to it.

From far away, language-model training can still look smooth. Mean loss averages over an enormous number of predictions, so many local reorganizations can disappear into one clean scaling curve. Michaud's quanta picture asks whether that smoothness is built from smaller changes in which mechanisms and pieces of knowledge the model has acquired.¹¹ Following geometry through training adds a more precise question: when a capability appears, did a new relational variable form, did an existing one become usable, or did the model merely learn to read out structure that was already there?

V. Measuring Reuse

A picture can rotate or reflect without changing what the representation means, so the first useful measurement concerns relations among states rather than their displayed coordinates. Let \(H_t\in\mathbb R^{n\times d}\) contain centered activation centroids for \(n\) states at checkpoint \(t\), averaged across surface forms and contexts, and let \[ G_t=H_tH_t^\top \] be their Gram matrix. \(G_t\) is unchanged by an orthogonal rotation or reflection of activation space and retains every pairwise inner product in the model's chosen activation metric. It is not invariant to an arbitrary invertible reparameterization, so it removes orientation, not every possible choice of basis.

For a cycle, let \(P\) shift every state by one. If the centroids occupy one common orbit, shifting both arguments should preserve every inner product. We can project the Gram matrix onto the nearest cyclically invariant one, \[ \overline G_t = \frac{1}{n} \sum_{m=0}^{n-1} P^mG_tP^{-m}, \] then summarize the remaining distance as the symmetry defect \[ \Delta_{\mathrm{cyc}}(t) = \frac{\lVert G_t-\overline G_t\rVert_F} {\lVert G_t\rVert_F}. \] computed in the full representation. It becomes small when relative geometry depends only on cyclic separation, without demanding that the first two principal components draw a circle. A low defect is compatible with seven concepts occupying one cyclic system, but it cannot tell us whether the model uses that system.

Statistics alone can produce a ring, and even a regular simplex has abundant symmetry. The stronger question is whether one operation can be learned on some states and contexts, then reused on others. Fit \(R_t(g)\) on one part of the orbit and judge it on held-out states. One application should carry \(\widetilde h_t(k)\) toward \(\widetilde h_t(gk)\), while \(n\) applications should carry each measured state back toward itself. This tests closure on the measured state span, rather than demanding that \(R_t(g)^n\) equal the identity on every unrelated activation direction.

Geometry and operator transfer still do not establish that the predictor itself has become shorter. We can ask a narrower question: whether a predefined relational vocabulary describes held-out activations more economically than either memorizing them or ignoring their state structure. A lookup code \(C_{\mathrm{lookup}}\) transmits a separate centroid for every state. A shared-rule code \(C_{\mathrm{shared}}\) transmits an intercept and a small set of relational components, together with their residual error. An intercept-only code \(C_{\mathrm{null}}\) transmits no state structure at all. Each code pays for its continuous parameters and its unexplained residuals, while the shared code also pays to identify the components it uses. Their comparison gives the usable shared-rule gain \[ G_{\mathrm{usable}}(t) = \min\!\left\{ C_{\mathrm{lookup}}(t), C_{\mathrm{null}}(t) \right\} - C_{\mathrm{shared}}(t). \] A positive value means that this relational code describes the held-out activations more cheaply than either alternative, under the chosen precision and residual model. It does not by itself show that the network's parameters form a shorter program or that the relation drives prediction. Fixing the code in advance, separating the states and surface aliases used for fitting from those used for scoring, and scrambling the relational labels guard against manufacturing a gain from a convenient basis. If exact cyclic closure is built into the shared code, closure is a property of that model class and cannot also be counted as evidence that the network learned it.

An intervention asks whether the proposed action is sufficient to change the model's computation. Replace a centered activation \(\widetilde h_t(k)\) with \(R_t(g)\widetilde h_t(k)\), restore the same intercept, and observe whether the output moves from \(k\) toward \(gk\). If matched random and scrambled transformations do not produce the same movement, the result supports the causal sufficiency of that intervention at that site. It does not establish that the representation is necessary or that it fully mediates the behavior. The operator must also be estimated separately from the examples used to judge the intervention; transporting an activation directly toward a labeled target centroid is useful as an upper bound, but it does not demonstrate a learned, reusable operator.

The evidence is therefore an alignment of held-out behavior, full-space symmetry, usable activation compression, operator transfer, and controlled intervention. Their relative timing tells us what kind of event generalization was. Geometry can precede behavior, behavior can precede cleanup, and a clean manifold may never become causally useful. None of these quantities is a universal intelligence score. Together they form a microscope for one possible act of intelligence: the point at which many cases become one reusable computation.

VI. Rule Shaping

The shortest rule in a dataset does nothing until a learner finds it. Every clean modular-addition table in our experiments was generated by the same compact cycle, yet the models encoded it differently, sometimes used it without making our chosen activation code cheaper, and sometimes lost the behavior after having generalized. The world can be compressible while the learner remains unintelligent with respect to that compression.

This separates four questions that are easy to collapse into one. Held-out behavior asks whether a relation transfers to new cases. Description length asks whether a chosen vocabulary accounts for new activations more economically. Intervention asks whether the proposed operation is causally sufficient to change a computation. Training dynamics asks whether that organization persists. When all four agree, a shape has become strong evidence of a rule. When they disagree, the disagreement tells us which level of the learner changed. The code and exact analysis definitions are available in the experiment repository.

The shallow grokking runs provide the clearest alignment. In the stricter analysis, where states and surface aliases were held out together, a sparse cyclic code beat both lookup and null descriptions in none of 25 folds before generalization and 34 of 35 afterward, with median savings between 3.71 and 6.39 kilobits. Scrambling the cyclic order erased the gain in all 60 folds. In the new three-seed replication, every clean run reached perfect held-out accuracy, the alias-held-out code saved a median 1.72 kilobits at the endpoint, and a canonical step applied before the first transformer block steered the desired successor on 76.3 percent of examples while matched controls remained below 2.2 percent. Here the same relation transfers in behavior, shortens a held-out activation description, and is causally sufficient at an early site.

The broader sweep separates these quantities. Across fifteen clean runs in five model families, every seed crossed 90 percent held-out accuracy at least once. Yet the predefined output-activation code was cheaper in only two families: it saved 1.72 kilobits for the shallow grokking model and 0.57 kilobits for the small transformer, while costing an additional 2.81 kilobits for the micro transformer, 0.43 kilobits for the medium transformer, and 0.15 kilobits for the large transformer. A code is a wager about the right coordinates and residual model. A negative gain rejects that wager; it does not show that the network failed to reuse structure in every possible representation.

Corrupting 15 percent of the table makes the distinction sharper. The canonical rule can explain only about 84.9 percent of the held-out labels because the remaining labels are genuine exceptions. The one-layer grokking model stayed at chance, while the deeper micro and small transformers reached median held-out accuracies of 76.8 and 76.6 percent, and the medium transformer reached 83.5 percent, close to the rule's ceiling. The large transformer broke the apparent trend: one seed reached 82.9 percent, while the other two ended at 1.6 and 2.4 percent. Frequency-matched random tables stayed near chance.

These are model and training presets, not an isolated scaling law. Width and depth change together, and the five families use batch sizes 4096, 4096, 4096, 2048, and 1024. What the comparison establishes is more modest and more interesting: adding parameters did not make the shorter rule inevitable within a fixed number of updates. The large model could recover it through noise, but whether it had done so by step 30,000 depended strongly on the seed and the path taken through optimization.

To distinguish failure from delay, we then continued all three corrupted large runs to step 60,000. They ended at 84.34, 82.51, and 84.06 percent, all near the clean rule's ceiling. This continuation was chosen after seeing the 30,000-step result, so it does not retroactively enter the preregistered spectrum below. It changes the interpretation of that result: the largest preset did not lack access to the rule, but found it on a later and seed-dependent clock. A larger learner can have greater eventual reach while looking worse under a fixed training budget.

At the preregistered endpoint, recovered behavior also did not make the selected activation description cheap. Every corrupted family had a negative alias-held-out code gain. Even so, the canonical early-site intervention steered the desired successor on 50.2 percent of examples in the micro model, 65.9 percent in the small model, and 44.0 percent in the medium model. In the successful large seed it reached 63.2 percent, although the large-seed median stayed near chance because the other two models had not yet generalized. A cyclic action can therefore be causally sufficient even when the output residual stream is not organized in the economical form anticipated by our code. Behavioral reuse, activation economy, and causal reuse are related achievements, but none guarantees the others.

A rule can also be found and then lost. Every clean seed crossed 90 percent held-out accuracy, but several later suffered severe collapses. One medium run reached 100 percent and ended at 3.6 percent. One large run fell from 100 percent to 3.2 percent before recovering to 99.3 percent at the endpoint. Generalization is therefore a state of the training dynamics rather than an irreversible accumulation of knowledge. Discovering a reusable computation does not prevent later optimization from replacing or disrupting it, which is why the faint individual trajectories carry information that a final median hides.

The same limit appears across tasks. A cycle of order 31 generalized reliably across three seeds, ending at a median 99.3 percent. A two-dimensional torus reached a median 92.8 percent, but one seed stopped at 70.4 percent. A dihedral task has a compact group law too, yet its median remained at 1.2 percent, while the random-table control ended at 1.7 percent. The existence of a short description belongs to the problem. Whether gradient descent can discover and use it depends on the learner's inductive bias.

Compression is therefore not one hidden scalar that rises whenever a model gets smarter. Held-out behavior tells us that a rule transferred. Description length tells us that one chosen language made the representation cheaper. Intervention tells us that an operation can alter the computation. Stability tells us whether training preserved the solution. Their agreement licenses the strongest story, while their disagreement prevents us from mistaking a clean picture for a complete mechanism.

The move from seven functions to one does not erase the seven states. It factors their relation away from the individual cases and makes that relation available somewhere new. In this narrow sense, a system becomes smarter when what it learns in one place changes what it can do in another. The world supplies the opportunity for compression; the learner earns it by making the shorter rule reusable.

---

1. Animals give us partial descriptions, though no agreed biological scalar. Comparative test batteries look for shared variation across learning, memory, and reasoning, but evidence for a unitary non-human \(g\) remains weak.⁸ At the mechanistic level, Cantlon and Piantadosi propose that human cognition reflects expanded information-processing capacity shared across memory, attention, and learning.⁹ These are constraints on what might be increasing, not yet a settled meter of intelligence.
2. Effective rank is still useful for measuring cleanup and redundancy, but it is not an order parameter for relational structure. A representation can lose rank without becoming equivariant, or gain redundant Fourier modes while preserving the same exact group action.

References

1. Grégoire Delétang et al., Language Modeling Is Compression, 2023.
2. Ryan Greene, Compression and Intelligence, 2026.
3. Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark, Not All Language Model Features Are Linear, 2024.
4. Daniel Wurgaft et al., Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior; and Sheridan Feucht et al., Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts, 2026.
5. Dhruva Karkada, Daniel J. Korchinski, Andres Nava, Matthieu Wyart, and Yasaman Bahri, Symmetry in language statistics shapes the geometry of model representations, 2026.
6. Core Francisco Park et al., ICLR: In-Context Learning of Representations, 2025.
7. Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt, Progress measures for grokking via mechanistic interpretability, 2023.
8. Marc-Antoine Poirier et al., How General Is Cognitive Ability in Non-Human Animals? A Meta-Analytical and Multi-Level Reanalysis Approach, 2020.
9. Jessica F. Cantlon and Steven T. Piantadosi, Uniquely Human Intelligence Arose from Expanded Information Capacity, 2024.
10. Stella Biderman et al., Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling, 2023.
11. Eric J. Michaud, On neural scaling and the quanta hypothesis, 2026.
12. Jorma Rissanen, Modeling by shortest data description, Automatica 14(5):465–471, 1978.