10. Bayesian coarsening and the information-loss packet
On this page · Permalink · Source line 382
This section imports a specific existing Komorebi foundation rather than inventing another inference heuristic. Let \(p_i\) be a prior on a finite fine state set and let \(B_a\) be coarse blocks. Define
On zero-mass blocks the value of \(r_a\) is immaterial. Updating finely and then aggregating is exactly updating \(P\) with the effective likelihood \(r(p)\). The effective likelihood generally depends on the fine prior inside each coarse block, information that \(P\) alone does not retain.
Proposition 12: Bayesian closure and its exact defect
Suppose there are at least two coarse blocks and the fine and declared coarse likelihoods are strictly positive. Updating and coarsening commute for every prior if and only if
with one common \(c>0\). For a fixed prior, put \(\delta=r(p)-\bar\ell\), \(\bar b=\mathcal B_{\bar\ell}(P)\) and \(Z_r=r(p)^\top P\). Then
The fixed-prior identity also holds for nonnegative likelihoods when both normalizing evidences are positive.
Read proof · Recorded Lean scope · Permalink · Source line 394
Proof
Constancy up to one global factor gives commutation by summing the fine posterior. Conversely, compare priors supported on one state in each of two distinct coarse blocks. Equality of coarse posterior odds forces the ratio of those states' likelihoods to equal the ratio of the coarse likelihoods. Varying the representatives proves constancy within each block and the common factor across blocks. For the defect, use \(Z_r=\bar\ell^\top P+P^\top\delta\) and subtract the two normalized vectors. \(\square\)
With only one coarse block, both posteriors aggregate to one regardless of the likelihood, so the necessity criterion is false there. This boundary is explicitly retained from the corrected Paper VI statement [K2].
The matrix in the defect formula annihilates \(\bar\ell\), recording the irrelevant common-scale direction. It does not make every likelihood discrepancy irrelevant. The construction in Proposition 9 refines until both the likelihood and transition defects vanish for a specified finite family.
10.1 Parameter inference is not program truth
On this page · Permalink · Source line 418
Within one declared regression model, a normal-inverse-gamma prior gives a standard conditional Bayesian parameter posterior. The software's program weights instead use penalized held-out predictive scores and then may refit coefficients on all available training records. These are finite-candidate predictive weights. Without a corresponding generative derivation, they should not be conflated with the marginal-likelihood posterior over programs under one fixed prior.
Every candidate must be scored on the same target event and base measure. Densities on differently sized coordinate spaces are not directly comparable. An invertible nonlinear coordinate change requires the appropriate Jacobian. A redundant deterministic embedding induces a measure on its consistent subspace, not an independent full-dimensional density on all coefficient entries.
The law of total variance separates within-candidate uncertainty from disagreement of candidate means. It does not supply a probability that the truth lies outside the grammar. An explicit outside-grammar component needs its own prior and predictive distribution. Assigning an arbitrary leftover mass does not solve open-world inference.