Paper VI: Bayesian transfer and evidence packets

E8 Bayesian transfer and evidence packets

On this page · Permalink · Source line 3327

Let

\[ \Delta_{n-1} = \left\{p\in{\mathbb R}^n: p_i\geq0, \quad \sum_ip_i=1 \right\} \]

be the probability simplex. Let \(\lambda\in[0,\infty)^n\) have positive evidence

\[ Z_\lambda(p)=\sum_i\lambda_ip_i. \]

The Bayesian transfer is

\[ {\mathcal B}_\lambda(p) = \frac{\lambda\odot p}{Z_\lambda(p)}. \]

Permalink · Source line 3329

Proposition E8.1: Scale invariance and composition

For every \(c>0\),

\[ {\mathcal B}_{c\lambda}={\mathcal B}_\lambda. \]

If all relevant evidences are positive, then

\[ {\mathcal B}_\mu({\mathcal B}_\lambda(p)) = {\mathcal B}_{\lambda\odot\mu}(p). \]

Read proof · Read statement and proof side by side · Permalink · Source line 3350 · Claim record · Lean coverage · Code · Worked examples

Proof of Proposition E8.1

A common likelihood scale cancels between the numerator and denominator. In the second identity, the first evidence appears in both the numerator and the denominator of the second update and cancels.

Back to statement · Read statement and proof side by side · Permalink · Source line 3363 · Claim record · Lean coverage · Code · Worked examples

Let \(I=I_1\sqcup\cdots\sqcup I_m\) be a partition. Define packet evidences

\[ Z_\alpha = \sum_{i\in I_\alpha}\lambda_ip_i. \]

For \(Z_\alpha>0\), define

\[ m_\alpha=\frac{Z_\alpha}{\sum_\beta Z_\beta}, \qquad q_{\alpha,i}=\frac{\lambda_ip_i}{Z_\alpha} \quad(i\in I_\alpha). \]

Packets with zero evidence receive posterior mass zero.

Permalink · Source line 3367

Theorem E8.2: Bayesian packet reconstruction

The posterior is the disjoint packet synthesis

\[ {\mathcal B}_\lambda(p) = \sum_{\alpha:Z_\alpha>0} m_\alpha q_\alpha. \]

The local shape \(q_\alpha\) depends only on the prior and likelihood inside packet \(I_\alpha\). Coupling between different packets occurs through the normalization of the packet evidences.

Read proof · Read statement and proof side by side · Permalink · Source line 3382 · Claim record · Lean coverage · Code · Worked examples

Proof of Theorem E8.2

For \(i\in I_\alpha\),

\[ m_\alpha q_{\alpha,i} = \frac{Z_\alpha}{\sum_\beta Z_\beta} \frac{\lambda_ip_i}{Z_\alpha} = \frac{\lambda_ip_i}{Z_\lambda(p)}. \]

The packet supports are disjoint.

Back to statement · Read statement and proof side by side · Permalink · Source line 3393 · Claim record · Lean coverage · Code · Worked examples

Theorem E8.3: Likelihood perturbation

Let \(\lambda,\mu\geq0\) have positive evidences under a fixed prior \(p\). Set

\[ d_p(\lambda,\mu) = \sum_i{\left\lvert\lambda_i-\mu_i\right\rvert}p_i. \]

Then

\[ {\left\lVert{\mathcal B}_\lambda(p)-{\mathcal B}_\mu(p)\right\rVert}_1 \leq \frac{2d_p(\lambda,\mu)} {\max\{Z_\lambda(p),Z_\mu(p)\}}. \]

Read proof · Read statement and proof side by side · Permalink · Source line 3406 · Claim record · Lean coverage · Code · Worked examples

Proof of Theorem E8.3

Add and subtract \(\mu\odot p/Z_\lambda(p)\). This gives

\[ {\left\lVert{\mathcal B}_\lambda(p)-{\mathcal B}_\mu(p)\right\rVert}_1 \leq \frac{2d_p(\lambda,\mu)}{Z_\lambda(p)}. \]

Interchanging \(\lambda\) and \(\mu\) gives the same estimate with \(Z_\mu(p)\). Take the smaller of the two upper bounds.

Back to statement · Read statement and proof side by side · Permalink · Source line 3422 · Claim record · Lean coverage · Code · Worked examples

Theorem E8.4: Stability with respect to the prior

Assume

\[ 0<a\leq\lambda_i\leq b \]

for every \(i\). Then

\[ {\left\lVert{\mathcal B}_\lambda(p)-{\mathcal B}_\lambda(q)\right\rVert}_1 \leq \frac{2b}{a}{\left\lVert p-q\right\rVert}_1 \]

for all priors \(p,q\in\Delta_{n-1}\).

Read proof · Read statement and proof side by side · Permalink · Source line 3432 · Claim record · Lean coverage · Code · Worked examples

Proof of Theorem E8.4

The evidences are at least \(a\). Add and subtract \(\lambda\odot q/Z_\lambda(p)\). The numerator difference is at most \(b{\left\lVert p-q\right\rVert}_1\), and the evidence difference has the same bound. The two terms give the stated constant.

Back to statement · Read statement and proof side by side · Permalink · Source line 3446 · Claim record · Lean coverage · Code · Worked examples

A prior and posterior on two coarse packets. The likelihood is constant in the first packet and varies inside the second, so only the second local conditional distribution changes.

Source and conversion record

Source SHA-256: 736e615131f57daf341b6cb0600761d58950450947c85d433206148cca228333

Unsupported prose commands stop the build. The mathematical expression sequence is checked against the source, with manuscript macros expanded for display.

Conversion records

Read and explore

Proofs and implementation

About this work

Other resources