48. Conditioning still has an unavoidable cost
On this page · Permalink · Source line 1949
Let \(P,Q\) be probability distributions on a finite record space. A record event \(H\) might be a particular observed prefix. Conditional continuation laws are obtained by restriction to \(H\) and normalization.
Proposition 44: Sharp event-probability dependence of conditioning
If \(\|P-Q\|_{\mathrm{TV}}\leq\delta\), \(p=P(H)>0\) and \(q=Q(H)>0\), then
When \(p>\delta\), positivity of \(q\) follows automatically. The factor \(1/p\) cannot be removed in general.
Read proof · Recorded Lean scope · Permalink · Source line 1953
Proof
Write the subprobability restrictions on \(H\) as \(u,v\), of masses \(p,q\). On the complementary event, the sum of absolute probability differences is at least \(|p-q|\). Therefore
Adding and subtracting \(v/p\) gives
Also \(|p-q|\leq\delta\), establishing the positivity condition. For sharpness, put total mass \(p\) on an event with two outcomes. Let \(P\) assign all of it to the first, and let \(Q\) move mass \(\delta\leq p\) to the second, leaving the complement unchanged. The unconditional total variation is \(\delta\) and the conditional distance is \(\delta/p\). \(\square\)
Combining this proposition with Proposition 43 gives a conditional continuation bound when the evidence event has adequate probability. It does not give a coordinate-state error bound unless test probabilities control that coordinate norm. Validity at every branch and stable inference at every branch remain different properties.