R3 Prior-restricted sufficiency and filtering
On this page · Permalink · Source line 5400
Let \(L\) be a strictly positive finite likelihood and let \({\mathcal B}_L(p)=L\odot p/{\left\langle L,p\right\rangle}\) for a probability vector \(p\). Fix probability vectors \(r^{(a)}\) supported on \(B_a\), and restrict the priors to \(p_i=q_a r_i^{(a)}\) for \(i\in B_a\), with \(q\) a coarse probability vector.
Proposition R3.1: Sufficiency relative to a prior disintegration
Define
For every prior in the declared family,
Read proof · Read statement and proof side by side · Permalink · Source line 5402 · Claim record · Lean coverage · Code · Worked examples
Proof of Proposition R3.1
The evidence in block \(B_a\) is
Apply Theorem E8.2.
Back to statement · Read statement and proof side by side · Permalink · Source line 5417 · Claim record · Lean coverage · Code · Worked examples
Lemma R3.2: Normalization bound
Let \(a,b\in[0,\infty)^m\) be nonzero, and set \(A={\left\lVert a\right\rVert}_1\) and \(B={\left\lVert b\right\rVert}_1\). Then
Read proof · Read statement and proof side by side · Permalink · Source line 5427 · Claim record · Lean coverage · Code · Worked examples
Proof of Lemma R3.2
Using \(A\) in the denominator gives
The second term equals \({\left\lvert B-A\right\rvert}/A\), which is at most \({\left\lVert a-b\right\rVert}_1/A\). Interchanging \(a\) and \(b\) gives the analogous bound with \(B\). Taking the smaller of the two bounds gives the result.
Back to statement · Read statement and proof side by side · Permalink · Source line 5438 · Claim record · Lean coverage · Code · Worked examples
For an arbitrary prior \(p\), put \(q=Cp\) and \(Z(p,L)={\left\langle L,p\right\rangle}\). Define \(\ell_a(p)=\sum_{i\in B_a}L_ip_i/q_a\) when \(q_a>0\); choose any positive value when \(q_a=0\). Only the product \(q_a\ell_a(p)\) enters the formulas.
Theorem R3.3: Exact information loss formula
For every prior,
Let \(\overline L\) be any fixed positive coarse likelihood and set
Then
Read proof · Read statement and proof side by side · Permalink · Source line 5452 · Claim record · Lean coverage · Code · Worked examples
Proof of Theorem R3.3
The packet evidence in block \(a\) is \(q_a\ell_a(p)\), which proves the exact identity. Apply Lemma R3.2 to
Their \(\ell^1\) norms are the two evidences, and \({\left\lVert a-b\right\rVert}_1=d(p\mid L,\overline L)\).
Back to statement · Read statement and proof side by side · Permalink · Source line 5478 · Claim record · Lean coverage · Code · Worked examples
Corollary R3.4: Oscillation bound
Suppose \(\overline L_a\) is any representative likelihood for block \(B_a\), and set
Then
Thus small within block likelihood oscillation gives a small coarse posterior defect whenever the evidence is bounded away from zero.
Read proof · Read statement and proof side by side · Permalink · Source line 5488 · Claim record · Lean coverage · Code · Worked examples
Proof of Corollary R3.4
The conditional average \(\ell_a(p)\) lies in the convex hull of the likelihood values inside \(B_a\). Hence
Multiply by \(q_a\) and sum.
Back to statement · Read statement and proof side by side · Permalink · Source line 5504 · Claim record · Lean coverage · Code · Worked examples
Along the Bayesian flow \(p_i(t)=e^{tg_i}p_i(0)/\sum_j e^{tg_j}p_j(0)\), the following statement holds for every real observable \(h\).
Corollary R3.5: Covariance identity
For any observable \(h\in{\mathbb R}^n\),
In particular,
Read proof · Read statement and proof side by side · Permalink · Source line 5514 · Claim record · Lean coverage · Code · Worked examples
Proof of Corollary R3.5
Differentiate \(\sum_ih_ip_i(t)\) and insert the replicator equation.
Back to statement · Read statement and proof side by side · Permalink · Source line 5529 · Claim record · Lean coverage · Code · Worked examples
For stochastic matrices \(K,\overline K\) and positive likelihoods \(L_y,\overline L_y\), put \(\Phi_y={\mathcal B}_{L_y}\circ K\) and \(\overline\Phi_y={\mathcal B}_{\overline L_y}\circ\overline K\).
Theorem R3.6: Exact coarse hidden Markov filter
Assume
and every likelihood \(L_y\) is constant on each coarse block with block values \(\overline L_y\). Then
for every prior. For any observation sequence \(y_1,\ldots,y_r\), the complete filtering recursion commutes with aggregation:
Read proof · Read statement and proof side by side · Permalink · Source line 5535 · Claim record · Lean coverage · Code · Worked examples
Proof of Theorem R3.6
Exact Markov lumping gives \(CKp=\overline KCp\). The Bayesian sufficiency criterion gives exact commutation of the update. Their composition proves the one step identity. Iterate it over the observation sequence.
Back to statement · Read statement and proof side by side · Permalink · Source line 5552 · Claim record · Lean coverage · Code · Worked examples
Theorem R3.7: One step filter defect
Let \(r=Kp\), \(q=Cr\), and let \(\ell_y(r)\) be the exact coarse likelihood obtained by averaging \(L_y\) inside each block with respect to \(r\). Assume
Then
Read proof · Read statement and proof side by side · Permalink · Source line 5556 · Claim record · Lean coverage · Code · Worked examples
Proof of Theorem R3.7
Insert the intermediate posterior \({\mathcal B}_{\overline L_y}(CKp)\). The first difference is bounded by Theorem R3.3. The second is bounded by Theorem E8.4 applied to the coarse likelihood.
Back to statement · Read statement and proof side by side · Permalink · Source line 5575 · Claim record · Lean coverage · Code · Worked examples