9. Statistical refinement needs its own theorem
On this page · Permalink · Source line 363
Nested feature vectors are algebraically nested when the smaller vector can be recovered from the larger by a fixed map. Predictive risk is a separate matter.
Proposition 11: Ideal statistical benefit of refinement
Let a target \(Y\) take values in a finite-dimensional Hilbert space with \(\mathbb E\|Y\|^2<\infty\). Let \(\mathcal F\subseteq\mathcal G\) be information sets and define \(m_{\mathcal F}=\mathbb E[Y\mid\mathcal F]\) and \(m_{\mathcal G}=\mathbb E[Y\mid\mathcal G]\). Then
Read proof · Recorded Lean scope · Permalink · Source line 367
Proof
Expand \(Y-m_{\mathcal F}=(Y-m_{\mathcal G})+(m_{\mathcal G}-m_{\mathcal F})\). The cross term has zero conditional expectation given \(\mathcal G\). \(\square\)
Thus finer information cannot worsen the optimal squared-error predictor under a fixed distribution. It can worsen an estimated predictor with finite data, restricted function classes, changed regularization or shifted distributions. The theorem supplies the ideal target for refinement, not a guarantee for every fitting procedure.
An exact archive makes later reanalysis possible without reconstructing discarded observations from a lossy state. It does not eliminate estimation error or distribution shift.