28. The state is a function of possible future experiments
On this page · Permalink · Source line 1201
For a history \(h\), define its unnormalized future-behavior column
where \(t\) ranges over every finite test word. Form the infinite history/test matrix
Its rank means the dimension of the span of its columns, equivalently the supremum of the ranks of finite submatrices. The columns are functions, not vectors obtained by physically observing every future. Their finite coordinate realization becomes learnable only when enough experiment probabilities are available.
Proposition 27: Minimal finite behavior realization
If \(\operatorname{rank}\mathscr H=r<\infty\), the vector space \(V=\operatorname{span}\{v_h\}\) has dimension \(r\). There are well-defined linear maps \(B_s:V\to V\), an initial vector \(v_\epsilon\), and a linear mass functional \(\lambda(v)=v(\epsilon)\) satisfying
Every linear realization of \(f\) has dimension at least \(r\). Any two reachable and observable realizations of dimension \(r\) differ by an invertible change of coordinates. For \(f(h)>0\), the normalized state \(v_h/f(h)\) determines every future test probability conditional on \(h\) and the declared future interventions.
Read proof · Recorded Lean scope · Visual guide: Behavior can supply the coordinates · Permalink · Source line 1217
Proof
If \(\sum_hc_hv_h=0\), then for every test \(t\),
because \(st\) is another test. Thus shifting columns preserves all linear relations and defines \(B_s\). The probability formula follows by induction. For any \(d\)-dimensional realization, the history/test matrix factors into test rows and reachable state columns, so its rank is at most \(d\).
For uniqueness, map the reachable vector after each history in one minimal realization to the corresponding vector in the other. If a linear combination vanishes in the first, every future test evaluates to zero on it. Those same evaluations vanish in the second, where observability makes the combination zero. The map is therefore well-defined and bijective, and intertwines every symbol operator and both boundary vectors. Finally \(f(ht)/f(h)\) is exactly the conditional experiment probability. \(\square\)
The coordinate dimension \(r\) includes probability mass. Normalized states lie in the affine hyperplane \(\lambda z=1\), so their affine dimension is at most \(r-1\). A finite hidden-state model of size \(n\) has behavior rank at most \(n\). The theorem itself requires finite behavior rank, not a preferred hidden-state ontology.
Two realized histories with equal normalized future columns are equivalent for every declared future test. This is a predictive equivalence, not equality of their physical pasts or lossless archives.