Every belief a predictor could arrive at about a four-state hidden source, computed from scratch. It is three-dimensional, structured, and — as we found out by checking — not the dimension we first published. Drag to turn it.
This is a primer, not a discovery. It rebuilds a known construction from published equations, extends it from three hidden states to four so the object is genuinely three-dimensional, and then fails to measure its dimension — twice. The failures are the useful part and they are kept on the page.
A hidden process moves between four states and emits a symbol at each step. A predictor watching the symbols never sees the state — it holds a belief, a probability across the four possibilities, and revises it with every symbol that arrives.
Four probabilities summing to one is a point in a tetrahedron. Each corner means certainty about one state; the centre means total ignorance. The cloud is every belief reachable by some sequence of symbols — the set of everything the predictor could come to think.
Nobody designed this shape. It falls out of applying the same update rule over and over, which is why fractals mark recursion: any process that repeatedly transforms its own state leaves this signature.
The dimension of these sets has a theory behind it. Jurgens and Crutchfield give a Ledrappier–Young style result for the statistical complexity dimension of a mixed-state set: d = −h/λ, the entropy rate divided by the Lyapunov exponent (arXiv:2102.10487).
Computed for this process: entropy rate h = 1.382 nats per symbol, Lyapunov exponent λ = −0.268. That gives 5.15 — which exceeds the three dimensions available, and in that framework means the set fills the tetrahedron. Our picture is a sparse sample of something with volume, not a thin fractal sheet.
Two honest caveats, both of which keep this open rather than settled. The same paper warns that this estimate overestimates in high-overlap regimes, and ours is deliberately high-overlap: each state emits its own symbol only 40% of the time. And our own box-counting is biased downward by finite sampling — you cannot fill 300,000 grid cells with 60,000 points. So the truth sits somewhere between a number that was too low because we did not look hard enough, and a formula that is too high because the maps overlap.
What survives is the picture and the structure in it. The visible filaments are real: they show that the measure is wildly non-uniform — some beliefs are reached constantly, others almost never — even if the support turns out to be solid. Concentration, not thinness.
One parameter controls the whole appearance: how ambiguous each state's emissions are. Sharpen them and every belief is pulled hard toward a corner, emptying the middle; blur them and beliefs never leave the centre.
| ambiguity | median distance from centre | points near the centre |
|---|---|---|
0.15 | 1.694 | 0.0% |
0.30 | 1.272 | 8.9% |
0.45 | 0.848 | 23.5% |
0.60 | 0.463 | 72.2% |
At 0.15 the interior is completely empty — the faces curve inward and the
object reads as a hollow pyramid. Try the dial above.
Why this matters beyond the picture: it is corroboration that we built the right object. Ray, Riechers and Shai (Simplex, 9 September 2026) show the same qualitative geometry for compositions of these processes, and state plainly that "Mess3's belief states form a fractal that fills the simplex". Support that fills, structure that lives in the measure rather than the support — which is exactly what the correction above concluded from our own failed measurement, arrived at independently and then found in the source.
Transformers trained on next-token prediction turn out to carry this geometry in their residual stream — linearly, so a later layer reads it with a single matrix multiply (Shai et al., NeurIPS 2024). The published pictures are flat triangles because they use three-state sources. Four states is the same construction with one more dimension, which is what you are turning above.
Two honest limits. This is a toy generator, chosen precisely because theory predicts its shape in advance, so the drawing can be checked. And a real language model's sources have an enormous number of hidden states, so the corresponding object is far beyond any picture — though a projection keeps the dimension even when it loses the arrangement (Marstrand's theorem).
states 4, each preferring to stay (leave probability x = 0.15) symbols 4, each state preferring its own (error probability a = 0.60) update b'(j) ∝ Σᵢ b(i)·T(i,j)·E(j,symbol) points 60,000 iterations, first 2,000 discarded
That is the entire specification. Everything above is what those four lines do when you run them.