Take
and group the first two and last two states. Their coarse laws are \(P=(3/4,1/4)\) and \(Q=(1/2,1/2)\). Define \(D(p\Vert q)=\sum_i p_i\log(p_i/q_i)\) with natural logarithms.
Reading the drawing
The two segments of one bar represent the terms labelled below them. The total is displayed above the bar. The source plate states the positivity assumptions for the reference masses.
Drawing labels and data 3 labels
Read the original labels as selectable text. Tables preserve grid relationships; other short labels are grouped compactly.
\(D(p\Vert q)=\frac14\log2\)
coarse information
\(\frac34\log3-\log2\)
conditional information
\(\frac54\log2-\frac34\log3\)
Evidence note from the source edition
The bar lengths use the displayed exact formulas evaluated for illustration. The split is an identity: coarse information plus posterior-weighted conditional information. It is not a fitted allocation.
The chain rule identifies exactly what aggregation removes
Inside packet A, the conditional laws are \((2/3,1/3)\) and \((1/2,1/2)\); inside B they agree. Hence
Adding the last two lines recovers the first. Every conditional term is nonnegative, so aggregation cannot increase relative entropy. Equality holds precisely when the occupied packets have the same conditional laws, under the stated support convention.
The weighting is by the packet mass \(P_a\) from the first argument of \(D(p\Vert q)\). Replacing it by \(Q_a\) generally breaks the identity. Likewise, forgetting zero-mass conventions or allowing a positive \(p_i\) over a zero \(q_i\) changes which expressions are finite.
Read this as
Coarse relative entropy measures distinguishability that survives aggregation. The conditional term records exactly the distinguishability removed inside the packets.