Paper VI: Bayesian flows, conjugacy, and hidden states

9 Bayesian flows, conjugacy, and hidden states

On this page · Permalink · Source line 1026

Fix a score vector \(g\). The likelihood flow is

\[ \lambda_i(t)=e^{tg_i}, \]

and the Bayesian flow is

\[ \Phi_t(p)_i = \frac{e^{tg_i}p_i}{\sum_je^{tg_j}p_j}. \]

Permalink · Source line 1028

Theorem 9.1: Bayesian nonlinear semigroup

\[ \boxed{ \Phi_{s+t}=\Phi_s\circ\Phi_t. } \]

Its generator is the centered multiplicative field

\[ \boxed{ \frac{d}{dt}p_i(t) = \bigl(g_i-\langle g,p(t)\rangle\bigr)p_i(t). } \]

Read proof · Visual guide: A nonlinear semigroup of odds · Read statement and proof side by side · Permalink · Source line 1039 · Claim record · Lean coverage · Code · Worked examples

Proof of Theorem 9.1

The semigroup law is the likelihood-composition theorem with \(e^{sg}e^{tg}=e^{(s+t)g}\). Differentiating the normalized exponential formula gives

\[ \dot p_i=g_ip_i-p_i\frac{\sum_jg_je^{tg_j}p_j(0)}{\sum_je^{tg_j}p_j(0)}. \]

The quotient is \(\langle g,p(t)\rangle\).

Back to statement · Read statement and proof side by side · Permalink · Source line 1055 · Claim record · Lean coverage · Code · Worked examples

Continuous likelihood accumulation produces a nonlinear semigroup on the probability simplex.

Example 9.2: Beta–Binomial conjugate packet

A \(\operatorname{Beta}(a,b)\) prior and \(s\) successes in \(n\) trials produce

\[ \operatorname{Beta}(a+s,b+n-s). \]

The prior exponents are additive packets: the likelihood increments them by \((s,n-s)\). In the numerical example,

\[ (a,b)=(2,3),\quad (s,n)=(7,10), \]

so the posterior is \(\operatorname{Beta}(9,6)\) with mean \(0.6\).

Visual guide: Conjugacy tracks exponents · Permalink · Source line 1070

A conjugate Bayesian update viewed as exact transfer of exponent packets.

9.1 Hidden Markov filtering

On this page · Permalink · Source line 1089

Let \(K\) be column stochastic and let \(\lambda^{(r)}\) be the likelihood associated with the \(r\)th observation. The filter recursion is

\[ \widehat p_r=Kp_{r-1}, \qquad p_r={\mathcal B}_{\lambda^{(r)}}(\widehat p_r). \]

Permalink · Source line 1091

Proposition 9.3: Prediction-update reconstruction

Every hidden Markov filtering step is the composition of a linear Markov transfer and a normalized nonlinear likelihood transfer. Assume a probability initial vector, a column-stochastic transition matrix, and nonnegative likelihoods with positive evidence at every step. Then every \(p_r\) is a probability vector.

Read proof · Visual guide: Prediction and observation do not commute · Visual guide: One complete coarse filter · Read statement and proof side by side · Permalink · Source line 1098 · Claim record · Lean coverage · Code · Worked examples

Proof of Proposition 9.3

Column stochasticity gives \(\widehat p_r\ge0\) and \({\boldsymbol{1}}^T\widehat p_r=1\). Bayesian normalization then gives \(p_r\ge0\) and \({\boldsymbol{1}}^Tp_r=1\).

Back to statement · Read statement and proof side by side · Permalink · Source line 1102 · Claim record · Lean coverage · Code · Worked examples

A hidden Markov filter alternates a linear prediction semigroup with nonlinear Bayesian normalization.

Source and conversion record

Source SHA-256: 736e615131f57daf341b6cb0600761d58950450947c85d433206148cca228333

Unsupported prose commands stop the build. The mathematical expression sequence is checked against the source, with manuscript macros expanded for display.

Conversion records

Read and explore

Proofs and implementation

About this work

Other resources