1 Gaussian quadratic forms
On this page · Permalink · Source line 1
A quadratic statistic may be presented as a matrix, a factor, or a list of spectral coefficients. The representation determines how the calculation is organized. Under the Gaussian model, the spectrum determines the law. This chapter fixes that distinction before any numerical approximation is introduced.
The primary mathematical object is a Gaussian quadratic form.
1.1 Central positive forms
On this page · Permalink · Source line 11
Let
and write \(S=\Sigma^{1/2}\). With \(X=SZ\), \(Z\sim N(0,I)\),
If the nonzero eigenvalues of the positive semidefinite operator
are \(\lambda_1,\ldots,\lambda_r\), then
Here and throughout the central reductions, the variables \(Z_j\) are independent standard normal variables. The numerical tail problem therefore depends only on a positive spectrum, regardless of how that spectrum was generated. If there are no nonzero coefficients, then \(Q=0\) almost surely and \(P(Q\ge q)=1\) for \(q\le0\), while \(P(Q\ge q)=0\) for \(q>0\). This deterministic case is handled directly, without a saddlepoint or an inversion calculation.
Proposition 1.1: Representation equivalence
The same central quadratic-form law may be supplied by any of the following:
a nonnegative spectrum \(\{\lambda_j\}\).
a positive semidefinite matrix \(H\).
a factor \(A\) with \(H=AA^\top\) or \(H=A^\top A\).
a replayable streamed factor
\[ A=[A_1\ \cdots\ A_B]. \]
The nonzero spectra of \(AA^\top\) and \(A^\top A\) are identical, and
Proof
If \(AA^\top u=\lambda u\) with \(\lambda>0\), then \(A^\top u\ne0\) and \(A^\top A(A^\top u)=\lambda A^\top u\). Conversely, an eigenvector \(v\) of \(A^\top A\) with positive eigenvalue maps to the eigenvector \(Av\) of \(AA^\top\). These maps restrict to inverse isomorphisms of the corresponding eigenspaces after scaling, so multiplicities agree. Horizontal concatenation gives \(AA^\top=\sum_b A_bA_b^\top\) by direct matrix multiplication. Orthogonal diagonalization then identifies the associated Gaussian quadratic-form laws.
This proposition is the domain-independent source of the memory-bounded streaming method developed below.
1.2 Central indefinite forms
On this page · Permalink · Source line 76
The positive case is not the natural boundary of the central theory. Let
where \(M\) may be indefinite, and again write \(S=\Sigma^{1/2}\). The symmetric operator
has a real signed spectrum.
Theorem 1.2: Exact central signed reduction
If \(\lambda_1,\ldots,\lambda_r\) are the nonzero eigenvalues of \(H\), then
Proof
Write \(X=SZ\) with \(Z\sim N(0,I)\) and diagonalize
Rotational invariance of the standard normal gives
No positivity assumption on \(M\) is needed for this central reduction.
The sign pattern determines the support. If every \(\lambda_j\ge0\), the lower endpoint is zero. If every \(\lambda_j\le0\), the upper endpoint is zero. If both signs occur, the distribution is unbounded in both directions.
For the right-tail path, the cumulant generating function is
with real domain
Here a missing negative or positive spectral set gives a domain endpoint \(-\infty\) or \(+\infty\), respectively. When \(q>E[Q]=\sum_j\lambda_j\), the saddlepoint is the unique positive solution of
inside this domain whenever a positive spectral component is present. The negative-only case with at least one nonzero coefficient has a positive saddlepoint only for \(E[Q]<q<0\). Indeed, \(\kappa'(\theta)\) increases from \(E[Q]\) to zero as \(\theta\to\infty\). For \(q\ge0\) its survival probability is zero, and no finite positive saddlepoint solves the query.
The packet representation is sign aware: positive and negative eigenvalues are clustered separately, and each same-sign packet stores a center \(\mu\) and centered moments. The local expansion remains
with the same geometric remainder condition
A useful distinction appears in the Fourier tail. Under the saddlepoint tilt,
If \(r\) coefficients are nonzero, then asymptotically
Thus certification difficulty depends strongly on spectral rank, not only on how small the requested probability is.
The signed law has an exact spectral reduction, but its numerical certification requires bounds appropriate to its support and Fourier decay. The finite-endpoint, mixed-sign, and low-rank controls are developed in Chapter 11.
1.3 Noncentral positive forms
On this page · Permalink · Source line 189
Let
Again set \(S=\Sigma^{1/2}\) and diagonalize
Writing
each positive-eigenvalue direction satisfies
Theorem 1.3: Exact noncentral reduction
For the positive eigenvalues \(\lambda_i>0\),
where
and
The noncentral chi-square components are independent. To state the zero-shift condition precisely, put \(T=M^{1/2}S\), \(b=M^{1/2}\mu\), and let \(P_T\) be the orthogonal projector onto \(\operatorname{range}(T)\). Then
Thus \(q_0=0\) exactly when \(M^{1/2}\mu\in\operatorname{range}(M^{1/2}S)\). In particular, \(\Sigma\succ0\) is sufficient. For singular covariance the shift is the squared deterministic component perpendicular to the random range in these weighted coordinates.
Proof
Expand
and rotate by \(V\). Completing the square gives the displayed noncentral components. For a null vector of \(SMS\),
hence
so there is no unresolved stochastic linear term in a zero-eigenvalue direction. For the positive eigenvectors \(v_i\) of \(T^\top T=SMS\), the vectors \(Tv_i/\sqrt{\lambda_i}\) form an orthonormal basis of \(\operatorname{range}(T)\). Their inner products with \(b\) are \(c_i/\sqrt{\lambda_i}\). Hence the subtracted sum equals \(\|P_Tb\|^2\), leaving \(q_0=\|(I-P_T)b\|^2\ge0\). Orthogonal rotation preserves independence of the normal coordinates, so the squared components are independent as stated.
The numerical interface implements this reduction directly. In an isotropic control,
the law collapses to the exact scaled noncentral chi-square distribution
The numerical controls for this reduction appear in Chapter 11. They test a floating analytic-ledger engine and are not assigned the directed-enclosure guarantee of the central positive backend.
1.4 Decomposition axes
On this page · Permalink · Source line 306
The solver separates four logically different representations:
Storage blocks answer where data reside. Structural phases answer how the operator propagates. Krylov/range compression answers which subspace has actually been resolved. Spectral packets answer how the final transform is evaluated. These decompositions should not be conflated.
1.5 A factor, a spectrum, and an exact law
On this page · Permalink · Source line 328
We will follow one small example through the remaining parts of the book. Its entries are exact binary fractions, so the same matrix can be used for symbolic algebra and for a binary64 input without introducing an input-rounding ambiguity. Set
Multiplication gives
The orthogonal matrix
satisfies \(V^\top K V=\operatorname{diag}(1,1,2,2)\). For \(Z\sim N(0,I_4)\),
The spectrum is therefore \((1,1,2,2)\) and \(\mathbb E Q=6\). The six-dimensional feature Gram \(A^\top A\) has these same four nonzero eigenvalues and two additional zeros.
Here the probability law also has a closed form. The densities of \(Y\) and \(2W\) are \(\tfrac12e^{-x/2}\) and \(\tfrac14e^{-x/4}\) on the nonnegative half-line. Their convolution is
Integrating from \(q\) to infinity gives
At \(q=40\) the probability is approximately \(9.079779837\times10^{-5}\). This formula will be used as an independent control, not as a shortcut inside the COS calculation. The next part begins with the factor \(A\) and asks which operations may change its storage or basis without changing this law.