32. Action coverage is a mathematical requirement
On this page · Permalink · Source line 1342
Proposition 31: Missing-intervention nonidentifiability
There are finite controlled stochastic models that agree on every possible observation record generated using only action \(a\), but disagree under another action \(b\). Therefore even infinitely many passive records confined to \(a\) cannot identify the unrestricted controlled model.
Read proof · Recorded Lean scope · Permalink · Source line 1344
Proof
Use one hidden state and two observations. In both models, action \(a\) gives probabilities \((1/2,1/2)\). Under \(b\), let the first model give \((1,0)\) and the second \((0,1)\). Every length-\(k\) record under \(a\) has probability \(2^{-k}\) in both models. Their first observation under \(b\) differs with probability one. \(\square\)
This is a coverage obstruction, not an optimization problem. It remains even when the hidden dimension is known exactly. Richer transforms of the same action-\(a\) observations do not create data about \(b\).
Action selection based on past observations requires a further distinction. Under a known policy \(\pi\), an action/observation word has joint probability
provided the action randomization has no access to hidden information beyond the declared history. The policy factors depend on the observed branch. Conditioning on a selected action sequence generally reweights branches rather than removing these factors. Recovering intervention probabilities from such data requires an appropriate coverage and reweighting argument. The reset experiments in Section 36 avoid that additional inference problem.