Compact Operators and the Spectral Theorem

A symmetric matrix is diagonalised by an orthonormal basis of eigenvectors with real eigenvalues. The infinite-dimensional statement fails for general operators, which need have no eigenvectors at all, but it survives intact for one class, the compact self-adjoint operators, where the unit ball is squeezed tightly enough that an extremal eigenvector must exist. This post proves the spectral theorem for that class, the engine behind every eigenfunction expansion. The proof combines the compactness of metric spaces with the quadratic-form bound for self-adjoint operators [@rudinFA1991; @reedSimon1980]. Throughout $H$ is a real Hilbert space.

#Compact operators

Definition1

An operator $K:H\to H$ is compact when every bounded sequence $(x_n)$ has a subsequence for which $(Kx_n)$ converges in $H$ .

Equivalently, $K$ carries the unit ball to a set of compact closure. In finite dimensions every operator is compact, by the Bolzano-Weierstrass theorem, but in infinite dimensions the identity is not, since an orthonormal sequence is bounded yet has no convergent subsequence, its terms staying distance $\sqrt 2$ apart. Compactness is the property that recovers the finite-dimensional behaviour, and it is built from the simplest operators by completion.

Proposition2

Every operator of finite rank is compact, and a limit in operator norm of compact operators is compact. Consequently every norm limit of finite-rank operators is compact.

Proof

Let $F$ have finite-dimensional range and $(x_n)$ be bounded, $\norm{x_n}\le C$ . Then $(Fx_n)$ is bounded in the finite-dimensional space $\operatorname{ran}F$ , which is complete and closed in $H$ , so the Bolzano-Weierstrass limit lies in $\operatorname{ran}F\subset H$ and $(Fx_n)$ converges in $H$ ; thus $F$ is compact. Now let $K_m\to K$ in operator norm with each $K_m$ compact, and let $(x_n)$ be bounded by $C$ . Since $K_1$ is compact, extract a subsequence indexed by $S_1$ on which $(K_1 x_n)$ converges; since $K_2$ is compact, extract $S_2\subset S_1$ on which $(K_2 x_n)$ converges, and inductively $S_{m+1}\subset S_m$ on which $(K_{m+1} x_n)$ converges. The compactness of each $K_m$ , not merely of the norm-limit $K$ , supplies each $S_m$ . The diagonal sequence whose $k$ -th index is the $k$ -th element of $S_k$ is eventually a subsequence of every $S_m$ , so $(K_m x_n)$ converges for every fixed $m$ ; call this diagonal sequence $(x_n)$ . For this subsequence,

\norm{Kx_n-Kx_{n'}}\le\norm{(K-K_m)x_n}+\norm{K_m x_n-K_m x_{n'}}+\norm{(K_m-K)x_{n'}} \le 2C\norm{K-K_m}+\norm{K_m x_n-K_m x_{n'}}. \tag{1}

Given $\varepsilon>0$ , fix $m$ with $\norm{K-K_m}<\varepsilon/(4C)$ , then choose $n,n'$ large so the middle term is below $\varepsilon/2$ . Hence $(Kx_n)$ is Cauchy, and converges by completeness, so $K$ is compact.

The proposition identifies the compact operators with the closure of the finite-rank operators. The spectral theorem will exhibit a compact self-adjoint operator as exactly such a limit, with the approximating ranks spanned by eigenvectors.

#The norm is an eigenvalue

For a general operator the supremum defining the norm need not be attained. Compactness forces attainment, and self-adjointness makes the maximiser an eigenvector.

Lemma3

Let $T$ be compact and self-adjoint with $T\neq 0$ . There is a unit vector $e$ and a scalar $\lambda$ with $\abs\lambda=\norm T$ and $Te=\lambda e$ .

Proof

By the quadratic-form identity, $\norm T=\sup_{\norm x=1} \abs{\ip{Tx}{x}}$ , so there are unit vectors $x_n$ with $\abs{\ip{Tx_n}{x_n}}\to\norm T$ . Since $H$ is real, $\ip{Tx}{x}\in\R$ for every $x$ , so each $\ip{Tx_n}{x_n}$ is real, and along a subsequence $\ip{Tx_n}{x_n}\to\pm\norm T=:\lambda$ with $\lambda\in\R$ and $\abs\lambda=\norm T$ . The vectors are approximate eigenvectors, and by symmetry of the real inner product $\ip{\lambda x_n}{Tx_n}+\ip{Tx_n}{\lambda x_n}=2\lambda\ip{Tx_n}{x_n}$ , so

\norm{Tx_n-\lambda x_n}^2=\norm{Tx_n}^2-2\lambda\ip{Tx_n}{x_n}+\lambda^2\le 2\lambda^2-2\lambda \ip{Tx_n}{x_n}\to 0, \tag{2}

using $\norm{Tx_n}\le\norm T=\abs\lambda$ and $\ip{Tx_n}{x_n}\to\lambda$ . By compactness a subsequence has $Tx_n\to z$ , and then $\lambda x_n=Tx_n-(Tx_n-\lambda x_n)\to z$ . Since $\lambda\neq 0$ , $x_n\to e:=z/ \lambda$ , a unit vector as the limit of unit vectors. Continuity of $T$ gives $Te=\lim Tx_n=z=\lambda e$ .

The lemma already contains the induction step of the theorem. It produces the largest eigenvalue and its eigenvector, and the rest of the spectrum is found by removing that direction and repeating.

#The spectral theorem

Theorem4

Let $T$ be a compact self-adjoint operator on $H$ . There is a finite or countably infinite orthonormal system of eigenvectors $(e_n)$ with real eigenvalues $\lambda_n$ satisfying $\abs{\lambda_1}\ge \abs{\lambda_2}\ge\cdots$ , tending to $0$ if infinite, such that

Tx=\sum_n\lambda_n\ip{x}{e_n}e_n\qquad\text{for every }x\in H, \tag{3}

the series converging in $H$ .

Proof

Build the system by exhaustion, as an induction on $n$ . Set $H_0=H$ and $T_0=T$ ; the base case is that $H_0$ is $T$ -invariant. Suppose $H_0,\dots,H_{n-1}$ are $T$ -invariant and $e_1,\dots,e_{n-1}$ are eigenvectors of $T$ . Given the compact self-adjoint $T_{n-1}=T|_{H_{n-1}}$ on the closed subspace $H_{n-1}$ , if $T_{n-1}=0$ stop. Otherwise Lemma 3 gives a unit eigenvector $e_n\in H_{n-1}$ with $T_{n-1}e_n=\lambda_n e_n$ and $\abs{\lambda_n}=\norm{T_{n-1}}$ ; since $H_{n-1}$ is $T$ -invariant, $T_{n-1}e_n=Te_n$ , so $e_n$ is a genuine eigenvector of $T$ . Set $H_n=\{x\in H_{n-1}:x\perp e_n\}=\{e_1,\dots,e_n\}^\perp$ . This $H_n$ is $T$ -invariant, because for $x\in H_n$ and every $k\le n$ self-adjointness gives $\ip{Tx}{e_k}=\ip{x}{Te_k}=\lambda_k\ip{x}{e_k}=0$ , so $Tx\perp e_1,\dots,e_n$ , that is $Tx\in H_n$ , completing the induction. The restriction $T_n=T|_{H_n}$ is again compact and self-adjoint with $\norm{T_n}\le\norm{T_{n-1}}$ , whence $\abs{\lambda_{n+1}}\le\abs{\lambda_n}$ .

If the process stops at stage $N$ , then $T$ vanishes on $H_N=\{e_1,\dots,e_N\}^\perp$ , and writing $x=\sum_{n\le N}\ip{x}{e_n}e_n+r$ with $r\in H_N$ gives $Tx=\sum_{n\le N}\lambda_n\ip{x}{e_n}e_n$ since $Tr=0$ , which is Equation (3) with a finite sum.

If it never stops, the eigenvalues tend to $0$ . Were $\abs{\lambda_n}\ge\delta>0$ for all $n$ , the bounded sequence $(e_n)$ would have $\norm{Te_n-Te_m}^2=\norm{\lambda_n e_n-\lambda_m e_m}^2=\lambda_n^2+ \lambda_m^2\ge 2\delta^2$ for $n\neq m$ by orthonormality, so $(Te_n)$ would have no convergent subsequence, contradicting compactness. Hence $\lambda_n\to 0$ . For the expansion, fix $x$ and set $r_N=x-\sum_{n\le N}\ip{x}{e_n}e_n$ , which lies in $H_N$ with $\norm{r_N}\le\norm x$ by Bessel's inequality. Then

\Big\|Tx-\sum_{n\le N}\lambda_n\ip{x}{e_n}e_n\Big\|=\norm{Tr_N}=\norm{T_N r_N}\le\norm{T_N}\norm{r_N} \le\abs{\lambda_{N+1}}\norm x\to 0, \tag{4}

using $Te_n=\lambda_n e_n$ to pull the finite sum out of $Tx$ and $\norm{T_N}=\abs{\lambda_{N+1}}$ . The partial sums converge to $Tx$ , which is Equation (3).

The eigenvectors with nonzero eigenvalue span the closure of the range of $T$ , and adjoining an orthonormal basis of the kernel $\ker T=\{e_n\}^\perp$ completes them to an orthonormal basis of $H$ when $H$ is separable, the basis in which $T$ is the diagonal operator $e_n\mapsto\lambda_n e_n$ . The decay $\lambda_n\to 0$ is the residue of compactness. It is why the operator is approximated in norm by its finite-rank truncations $\sum_{n\le N}\lambda_n \ip{\cdot}{e_n}e_n$ , with error $\abs{\lambda_{N+1}}$ .

#Positive operators and the square root

The operators arising as covariances are not merely self-adjoint but positive, and positivity pins the sign of the spectrum.

Definition5

A self-adjoint operator $T$ is positive, written $T\ge 0$ , when $\ip{Tx}{x}\ge 0$ for all $x$ .

Corollary6

A compact positive operator has $\lambda_n\ge 0$ for every $n$ , and the expansion Equation (3) has nonnegative coefficients. Its eigenvalues are summable against the squared coordinates of any vector, $\sum_n\lambda_n\ip{x}{e_n}^2=\ip{Tx}{x}\ge 0$ .

Proof

For an eigenvector, $\lambda_n=\ip{Te_n}{e_n}\ge 0$ by positivity. Pairing Equation (3) with $x$ and using continuity of the inner product gives $\ip{Tx}{x}=\sum_n\lambda_n\ip{x}{e_n}^2$ , a sum of nonnegative terms.

This is the exact structure a covariance operator carries, and the spectral theorem applied to it is the Karhunen-Loeve expansion, whose eigenvalues are the variances of the uncorrelated coordinates and whose eigenvectors are the principal directions of the process. When the operator is the integral operator of a continuous kernel, the same decomposition with its eigenfunctions made continuous is Mercer's theorem, the bridge from the abstract spectral theorem to the explicit series a covariance function admits. Among infinite-dimensional operators, the compact self-adjoint ones are uniquely as transparent as a symmetric matrix, and the spectral theorem is why.

Explore connections

see in the atlas

referenced by (6)

cite

@misc{compact-operators,
  author = {Zac Kienzle},
  title  = {Compact Operators and the Spectral Theorem},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/compact-operators}
}

#Compact operators

Definition1

An operator $K:H\to H$ is compact when every bounded sequence $(x_n)$ has a subsequence for which $(Kx_n)$ converges in $H$ .

Proposition2

Every operator of finite rank is compact, and a limit in operator norm of compact operators is compact. Consequently every norm limit of finite-rank operators is compact.

Proof

\norm{Kx_n-Kx_{n'}}\le\norm{(K-K_m)x_n}+\norm{K_m x_n-K_m x_{n'}}+\norm{(K_m-K)x_{n'}} \le 2C\norm{K-K_m}+\norm{K_m x_n-K_m x_{n'}}. \tag{1}

#The norm is an eigenvalue

For a general operator the supremum defining the norm need not be attained. Compactness forces attainment, and self-adjointness makes the maximiser an eigenvector.

Lemma3

Let $T$ be compact and self-adjoint with $T\neq 0$ . There is a unit vector $e$ and a scalar $\lambda$ with $\abs\lambda=\norm T$ and $Te=\lambda e$ .

Proof

\norm{Tx_n-\lambda x_n}^2=\norm{Tx_n}^2-2\lambda\ip{Tx_n}{x_n}+\lambda^2\le 2\lambda^2-2\lambda \ip{Tx_n}{x_n}\to 0, \tag{2}

The lemma already contains the induction step of the theorem. It produces the largest eigenvalue and its eigenvector, and the rest of the spectrum is found by removing that direction and repeating.

#The spectral theorem

Theorem4

Tx=\sum_n\lambda_n\ip{x}{e_n}e_n\qquad\text{for every }x\in H, \tag{3}

the series converging in $H$ .

Proof

\Big\|Tx-\sum_{n\le N}\lambda_n\ip{x}{e_n}e_n\Big\|=\norm{Tr_N}=\norm{T_N r_N}\le\norm{T_N}\norm{r_N} \le\abs{\lambda_{N+1}}\norm x\to 0, \tag{4}

using $Te_n=\lambda_n e_n$ to pull the finite sum out of $Tx$ and $\norm{T_N}=\abs{\lambda_{N+1}}$ . The partial sums converge to $Tx$ , which is Equation (3).

#Positive operators and the square root

The operators arising as covariances are not merely self-adjoint but positive, and positivity pins the sign of the spectrum.

Definition5

A self-adjoint operator $T$ is positive, written $T\ge 0$ , when $\ip{Tx}{x}\ge 0$ for all $x$ .

Corollary6

Proof

Explore connections

see in the atlas

referenced by (6)

cite

@misc{compact-operators,
  author = {Zac Kienzle},
  title  = {Compact Operators and the Spectral Theorem},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/compact-operators}
}