The Karhunen-Loeve Expansion

A stochastic process on an interval is a random function, and a random function is a point in an infinite-dimensional space. The Karhunen-Loeve expansion gives that space its natural coordinates. It writes the process as a series in a fixed orthonormal basis with random coefficients, and the basis it chooses is the eigenbasis of the process's own covariance, the unique basis that makes the coefficients uncorrelated and concentrates the variance into as few terms as possible. It is principal component analysis for functions, and it is the bridge between the spectral theory of operators and the second-order theory of processes [1], [2].

#The covariance operator

Let $(X_t)_{t\in[0,T]}$ be a centered second-order process, $\E[X_t]=0$ and $\E[X_t^2]<\infty$ , with covariance $K(s,t)=\E[X_s X_t]$ continuous on $[0,T]^2$ , which holds when $X$ is mean-square continuous. The covariance defines an integral operator on $L^2[0,T]$ ,

(\mathcal K f)(t)=\int_0^T K(t,s)\,f(s)\,ds. \tag{1}

Proposition1

The operator $\mathcal K$ is self-adjoint, positive, and compact on $L^2[0,T]$ . Its eigenvalues $\lambda_1\ge\lambda_2\ge\cdots\ge 0$ are nonnegative and accumulate only at zero, and its eigenfunctions $(\phi_n)$ form an orthonormal basis of the closure of the range.

Proof

Self-adjointness follows from the symmetry $K(s,t)=K(t,s)$ , since $\ip{\mathcal K f}{g}=\int\int K(t,s)f(s)g(t)\,ds\,dt=\ip{f}{\mathcal K g}$ . Positivity follows from

\ip{\mathcal K f}{f}=\int_0^T\!\!\int_0^T K(t,s)f(s)f(t)\,ds\,dt =\E\Big[\Big(\int_0^T X_t f(t)\,dt\Big)^2\Big]\ge 0, \tag{2}

the kernel being the covariance of the linear functional $\int X_t f(t)\,dt$ . The interchange of the expectation and the double integral is licensed by $\int\!\int\E\lvert X_sX_t\rvert\,\lvert f(s)f(t) \rvert\,ds\,dt\le\big(\int\sqrt{K(t,t)}\,\lvert f(t)\rvert\,dt\big)^2<\infty$ , using Cauchy-Schwarz and the continuity of $K$ on the compact square. Compactness follows because that same continuity makes $K$ square-integrable and $\mathcal K$ a Hilbert-Schmidt operator, hence compact. The spectral theorem for compact self-adjoint operators then supplies a real eigenvalue sequence accumulating only at zero with orthonormal eigenfunctions spanning the closed range, and positivity makes every eigenvalue nonnegative.

The eigenvalue equation is the Fredholm problem $\int_0^T K(t,s)\phi_n(s)\,ds=\lambda_n\phi_n(t)$ , and the pairs $(\lambda_n,\phi_n)$ are the spectral data of the process.

#Mercer's theorem

The spectral data reconstruct the covariance itself, not merely its action on functions.

Theorem2

For a continuous, positive semidefinite covariance $K$ (positivity from Equation (2), where operator positivity $\ip{\mathcal K f}{f}\ge 0$ upgrades to pointwise PSD $\sum_{i,j}c_ic_jK(x_i,x_j)\ge 0$ by taking $f$ as approximate identities concentrated at the $x_i$ with weights $c_i$ and passing to the limit through the continuity of $K$ ), the eigen-expansion

K(s,t)=\sum_{n=1}^\infty\lambda_n\,\phi_n(s)\,\phi_n(t) \tag{3}

converges absolutely and uniformly on $[0,T]^2$ [3].

The proof runs in two steps. On the diagonal, $r_N(t)=K(t,t)-\sum_{n\le N}\lambda_n\phi_n(t)^2$ is continuous, decreasing in $N$ , and tends to $0$ pointwise, since $\sum_n\lambda_n\phi_n(t)^2=K(t,t)$ by the spectral expansion of the positive operator, so Dini's theorem on the compact $[0,T]$ gives $r_N\to 0$ uniformly. Off the diagonal, Cauchy-Schwarz bounds the tail of Equation (3) by $\big|\sum_{n=M}^N\lambda_n\phi_n(s)\phi_n(t)\big|\le\big(\sum_{n\ge M}\lambda_n\phi_n(s)^2\big)^{1/2} \big(\sum_{n\ge M}\lambda_n\phi_n(t)^2\big)^{1/2}\le\sup_u\big(\sum_{n\ge M}\lambda_n\phi_n(u)^2\big)^{1/2} \,\sqrt{K(t,t)}$ , whose first factor tends to $0$ uniformly by the diagonal step while $K(t,t)$ is bounded on the compact diagonal, so the series converges absolutely and uniformly on $[0,T]^2$ . Setting $s=t$ gives the trace identity

\int_0^T K(t,t)\,dt=\sum_{n=1}^\infty\lambda_n=\E\!\int_0^T X_t^2\,dt, \tag{4}

where the exchange $\E\int_0^T X_t^2\,dt=\int_0^T\E[X_t^2]\,dt=\int_0^T K(t,t)\,dt$ is Tonelli, since $(t,\omega)\mapsto X_t(\omega)^2$ is nonnegative and jointly measurable (measurability from mean-square continuity) and $\int_0^T K(t,t)\,dt$ is finite by continuity of $K$ on the compact diagonal. So the eigenvalues sum to the total variance of the process, the energy that the expansion below distributes across its coordinates.

#The expansion

Theorem3

Define the coefficients $\xi_n=\lambda_n^{-1/2}\int_0^T X_t\,\phi_n(t)\,dt$ for $\lambda_n>0$ . Then the $\xi_n$ are centered, uncorrelated, and of unit variance, and

X_t=\sum_{n=1}^\infty\sqrt{\lambda_n}\,\xi_n\,\phi_n(t), \tag{5}

the series converging in mean square uniformly in $t\in[0,T]$ .

Proof

Centering is immediate from $\E[X_t]=0$ . For the second moments,

\E[\xi_m\xi_n]=\frac{1}{\sqrt{\lambda_m\lambda_n}}\int_0^T\!\!\int_0^T K(s,t)\phi_m(s)\phi_n(t)\,ds\,dt =\frac{1}{\sqrt{\lambda_m\lambda_n}}\int_0^T\lambda_n\phi_n(s)\phi_m(s)\,ds=\delta_{mn}, \tag{6}

using the eigen-equation $\int K(s,t)\phi_n(t)\,dt=\lambda_n\phi_n(s)$ and the orthonormality of the eigenfunctions, so the coefficients are uncorrelated with unit variance. Pulling the expectation inside the product of time integrals is the same Fubini bound as Equation (2), since $\int\!\int\E\lvert X_sX_t\rvert\,\lvert\phi_m(s)\phi_n(t)\rvert\,ds\,dt\le\big(\int\sqrt{K(t,t)}\, \lvert\phi_m\rvert\big)\big(\int\sqrt{K(t,t)}\,\lvert\phi_n\rvert\big)<\infty$ by Cauchy-Schwarz and continuity of $K$ on the compact square. Let $S_N(t)=\sum_{n\le N}\sqrt{\lambda_n}\,\xi_n\phi_n(t)$ be the partial sum. Then

\E[(X_t-S_N(t))^2]=K(t,t)-2\E[X_t S_N(t)]+\E[S_N(t)^2]. \tag{7}

The middle term is $\E[X_t S_N(t)]=\sum_{n\le N}\sqrt{\lambda_n}\phi_n(t)\,\E[X_t\xi_n] =\sum_{n\le N}\lambda_n\phi_n(t)^2$ , since $\E[X_t\xi_n]=\lambda_n^{-1/2}\int K(t,s)\phi_n(s)\,ds =\sqrt{\lambda_n}\,\phi_n(t)$ (the interchange of $\E$ with the time integral defining $\xi_n$ is the same Fubini/Cauchy-Schwarz bound as Equation (2), as $\lvert X_t\rvert\in L^2(\Omega)$ and $\phi_n\in L^2[0,T]$ with $K$ continuous on the compact square), and the last term is $\E[S_N(t)^2]=\sum_{n\le N}\lambda_n\phi_n(t)^2$ because the coefficients are uncorrelated of unit variance. Substituting into Equation (7),

\E[(X_t-S_N(t))^2]=K(t,t)-\sum_{n\le N}\lambda_n\phi_n(t)^2, \tag{8}

which is exactly the diagonal residual $r_N(t)$ of the Mercer proof, so the diagonal Dini step of Theorem 2 makes it tend to zero uniformly in $t$ as $N\to\infty$ .

The expansion Equation (5) separates randomness from time. The eigenfunctions $\phi_n$ are deterministic shapes, the modes of the process, and all the randomness lives in the scalar coefficients $\xi_n$ , which are uncorrelated and weighted by $\sqrt{\lambda_n}$ . A process that was a point in an infinite-dimensional space is now a sequence of independent dials, each turning one mode.

#Optimality of the eigenbasis

The eigenbasis is not one good basis among many. It is the best basis for representing the process in finitely many terms.

Theorem4

Among all orthonormal bases $(e_n)$ of $L^2[0,T]$ , the eigenbasis $(\phi_n)$ minimises the mean-square error of every finite truncation. For each $N$ the minimum truncation error is

\min_{(e_n)}\,\E\!\int_0^T\!\Big(X_t-\sum_{n\le N}\langle X,e_n\rangle\,e_n(t)\Big)^{2}dt =\sum_{n>N}\lambda_n, \tag{9}

attained by $e_n=\phi_n$ .

Proof

For any orthonormal basis $(e_n)$ write $\zeta_n=\int_0^T X_t e_n(t)\,dt$ . Since $\E\int_0^T X_t^2\,dt=\int_0^T K(t,t)\,dt<\infty$ by mean-square continuity, $X\in L^2[0,T]$ almost surely, so Parseval holds pathwise, $\int_0^T(X_t-\sum_{n\le N}\zeta_n e_n(t))^2\,dt=\sum_{n>N}\zeta_n^2$ a.s. Taking expectations and using Tonelli on the nonnegative terms gives the truncation error $\E\sum_{n>N}\zeta_n^2=\sum_{n>N}\E[\zeta_n^2]=\sum_{n>N}\ip{\mathcal K e_n}{e_n}$ , since $\E[\zeta_n^2]=\int\int K(s,t)e_n(s)e_n(t)\,ds\,dt=\ip{\mathcal K e_n}{e_n}$ , the swap of expectation and the infinite sum legitimate by monotone convergence. The total $\sum_n\ip{\mathcal K e_n}{e_n}=\sum_n\lambda_n=\int_0^T K(t,t)\,dt$ is finite by Equation (4) and independent of the basis, since $\mathcal K$ is trace class, so minimising the tail is the same as maximising the head $\sum_{n\le N}\ip{\mathcal K e_n}{e_n}$ over orthonormal sets of size $N$ . By the Ky Fan maximum principle for a positive compact self-adjoint operator, the sum of $N$ Rayleigh quotients over an orthonormal set is largest when the set spans the top $N$ eigenvectors, with maximum $\sum_{n\le N}\lambda_n$ . The complementary minimum tail is $\sum_{n>N}\lambda_n$ , attained at $e_n=\phi_n$ , which is Equation (9).

This is the function-space statement of principal component analysis. The eigenvalues, sorted downward, are the variance captured by each mode, and truncating after $N$ terms keeps the most variance any $N$ -dimensional linear representation can keep. A process whose eigenvalues decay fast is nearly finite-dimensional, and the decay rate is the precise sense in which a random function is simple or complex.

#The Gaussian case

When the process is Gaussian the coefficients gain their strongest property.

Proposition5

If $X$ is a centered Gaussian process then the coefficients $\xi_n$ of Equation (5) are independent standard normal random variables.

Proof

Each $\xi_n$ is the mean-square limit of Riemann-sum linear functionals $R(P)=\sum_k X_{t_k}\phi_n(t_k)\Delta t_k$ . These form a Cauchy net in $L^2(\Omega)$ , because $\E[R(P)R(P')]\to\int\!\int K(s,t)\phi_n(s)\phi_n(t)\,ds\,dt$ as the meshes shrink (Riemann sums of the continuous deterministic double integral converge), so $\E[(R(P)-R(P'))^2]\to 0$ ; completeness of $L^2(\Omega)$ then yields the limit $\int_0^T X_t\phi_n(t)\,dt$ , the standard mean-square Riemann integral. Finite linear combinations of the values of a Gaussian process are Gaussian, and a mean-square limit of Gaussian vectors is Gaussian, so any finite collection of the $\xi_n$ is jointly Gaussian and centered. For jointly Gaussian variables zero correlation is independence, and Equation (6) gives $\E[\xi_m\xi_n]=\delta_{mn}$ , so the $\xi_n$ are independent standard normals.

For a Gaussian process the expansion is therefore a genuine series of independent standard normals scaled by $\sqrt{\lambda_n}$ against fixed shapes, which is why simulating such a process reduces to drawing independent normals and summing modes, the basis of spectral simulation and of the polynomial chaos representations of random fields [4].

#Brownian motion

Take standard Brownian motion $W$ on $[0,1]$ , whose covariance is $K(s,t)=\min(s,t)$ . The eigenvalue problem $\int_0^1\min(t,s)\phi(s)\,ds=\lambda\phi(t)$ becomes a differential equation when differentiated. Splitting the integral at $t$ and differentiating once gives $\int_t^1\phi(s)\,ds=\lambda\phi'(t)$ , and differentiating again gives

\lambda\,\phi''(t)=-\phi(t), \tag{10}

with boundary conditions $\phi(0)=0$ from the original equation at $t=0$ and $\phi'(1)=0$ from the once-differentiated equation at $t=1$ . The solutions are $\phi(t)=\sqrt 2\,\sin(t/\sqrt\lambda)$ with $\cos(1/\sqrt\lambda)=0$ , giving the spectral data

\phi_n(t)=\sqrt 2\,\sin\!\Big(\Big(n-\half\Big)\pi t\Big),\qquad \lambda_n=\frac{1}{\big(n-\half\big)^2\pi^2},\qquad n=1,2,\dots, \tag{11}

and the Karhunen-Loeve expansion of Brownian motion,

W_t=\sqrt 2\,\sum_{n=1}^\infty\frac{\sin\!\big((n-\half)\pi t\big)}{(n-\half)\pi}\,\xi_n,\qquad \xi_n\ \text{i.i.d.}\ \mathcal N(0,1). \tag{12}

The eigenvalues decay as $n^{-2}$ , so the low modes dominate, and the first few carry most of the variance.

Mode $n$	$\lambda_n=1/((n-\tfrac12)^2\pi^2)$	Share of total variance
1	$0.4053$	$81.1\%$
2	$0.0450$	$9.0\%$
3	$0.0162$	$3.2\%$
4	$0.0083$	$1.7\%$

The total variance is $\sum_n\lambda_n=\int_0^1 K(t,t)\,dt=\int_0^1 t\,dt=\half$ by the trace identity Equation (4), and the first mode alone holds $\lambda_1/\half=0.811$ of it, so four modes already capture about $95\%$ of the variance of a Brownian path.

#Brownian bridge

The Brownian bridge $B_t=W_t-tW_1$ on $[0,1]$ has covariance $K(s,t)=\min(s,t)-st$ , which vanishes at both ends. The same differentiation gives $\lambda\phi''=-\phi$ now with $\phi(0)=\phi(1)=0$ , the Dirichlet conditions the pinned endpoints impose, so

\phi_n(t)=\sqrt 2\,\sin(n\pi t),\qquad \lambda_n=\frac{1}{n^2\pi^2},\qquad B_t=\sqrt 2\,\sum_{n=1}^\infty\frac{\sin(n\pi t)}{n\pi}\,\xi_n. \tag{13}

The bridge is the pure Fourier sine series of Brownian motion with both ends clamped, and its total variance is $\sum_n 1/(n^2\pi^2)=1/6$ , matching $\int_0^1(t-t^2)\,dt=1/6$ . The two processes differ in their boundary conditions alone, half-integer frequencies and a free right end for the motion, integer frequencies and two clamped ends for the bridge.

The three representations sit side by side.

Basis	Coordinates	Optimal truncation	Coefficients
Fourier	Fixed sines and cosines	No, in general	Correlated, in general
Discrete PCA	Eigenvectors of a covariance matrix	Yes, for vectors	Uncorrelated
Karhunen-Loeve	Eigenfunctions of $\mathcal K$	Yes, for processes	Uncorrelated, unit variance

For Brownian motion and the bridge the Fourier and Karhunen-Loeve bases coincide because the covariance commutes with the second-derivative operator, which is special. For a general process the eigenfunctions are not sinusoids and the Karhunen-Loeve basis is the only one that is both uncorrelated and optimal.

#Numerical illustration

Truncating Equation (12) at a finite order draws a Brownian path from a handful of independent normals, and the optimality result says no other linear basis of the same order reproduces the covariance more faithfully.

import numpy as np
from numpy.random import Generator


def brownian_via_kl(
    n_modes: int, grid: np.ndarray, rng: Generator
) -> np.ndarray:
    """Sample a Brownian path on [0, 1] from a truncated Karhunen-Loeve series.

    Args:
        n_modes: Number of eigenmodes retained in the truncation.
        grid: Increasing times in [0, 1] at which to evaluate the path.
        rng: Seeded generator for reproducibility.

    Returns:
        The sampled path values at the grid times.
    """
    n = np.arange(1, n_modes + 1)
    eigen_root = np.sqrt(2.0) / ((n - 0.5) * np.pi)
    modes = np.sin(np.outer(grid, (n - 0.5) * np.pi))
    coefficients = rng.standard_normal(n_modes)
    return modes @ (eigen_root * coefficients)


def captured_variance(n_modes: int) -> float:
    """Fraction of total Brownian variance held by the first modes.

    Args:
        n_modes: Number of leading eigenmodes.

    Returns:
        The ratio of retained eigenvalue mass to the total one half.
    """
    n = np.arange(1, n_modes + 1)
    eigenvalues = 1.0 / ((n - 0.5) * np.pi) ** 2
    return float(eigenvalues.sum() / 0.5)


rng = np.random.default_rng(0)
grid = np.linspace(0.0, 1.0, 500)
path = brownian_via_kl(n_modes=8, grid=grid, rng=rng)
retained = captured_variance(n_modes=8)

A stochastic process is a random function, and the Karhunen-Loeve expansion is its coordinate system. The covariance operator supplies the axes, Mercer's theorem reconstructs the process from them, the expansion turns the random function into uncorrelated dials, and the optimality theorem proves those axes are the ones that compress it best. Brownian motion is a weighted sine series with random amplitudes, and the same construction handles any second-order process, which is why the expansion is the spectral foundation of simulation, of dimensionality reduction for random fields, and of the principal component analysis of functional data.

[1]

K. Karhunen, “Über lineare Methoden in der Wahrscheinlichkeitsrechnung,” Annales Academiae Scientiarum Fennicae, Series A I, vol. 37, pp. 1–79, 1947.

[2]

M. Loève, Probability Theory II, 4th ed. Springer, 1978.

[3]

J. Mercer, “Functions of positive and negative type, and their connection with the theory of integral equations,” Philosophical Transactions of the Royal Society A, vol. 209, pp. 415–446, 1909.

[4]

R. G. Ghanem and P. D. Spanos, Stochastic Finite Elements: A Spectral Approach. Springer, 1991.

Explore connections

see in the atlas

referenced by (8)

cite

@misc{the-karhunen-loeve-expansion,
  author = {Zac Kienzle},
  title  = {The Karhunen-Loeve Expansion},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/the-karhunen-loeve-expansion}
}

#The covariance operator

(\mathcal K f)(t)=\int_0^T K(t,s)\,f(s)\,ds. \tag{1}

Proposition1

Proof

Self-adjointness follows from the symmetry $K(s,t)=K(t,s)$ , since $\ip{\mathcal K f}{g}=\int\int K(t,s)f(s)g(t)\,ds\,dt=\ip{f}{\mathcal K g}$ . Positivity follows from

\ip{\mathcal K f}{f}=\int_0^T\!\!\int_0^T K(t,s)f(s)f(t)\,ds\,dt =\E\Big[\Big(\int_0^T X_t f(t)\,dt\Big)^2\Big]\ge 0, \tag{2}

The eigenvalue equation is the Fredholm problem $\int_0^T K(t,s)\phi_n(s)\,ds=\lambda_n\phi_n(t)$ , and the pairs $(\lambda_n,\phi_n)$ are the spectral data of the process.

#Mercer's theorem

The spectral data reconstruct the covariance itself, not merely its action on functions.

Theorem2

K(s,t)=\sum_{n=1}^\infty\lambda_n\,\phi_n(s)\,\phi_n(t) \tag{3}

converges absolutely and uniformly on $[0,T]^2$ [3].

\int_0^T K(t,t)\,dt=\sum_{n=1}^\infty\lambda_n=\E\!\int_0^T X_t^2\,dt, \tag{4}

#The expansion

Theorem3

Define the coefficients $\xi_n=\lambda_n^{-1/2}\int_0^T X_t\,\phi_n(t)\,dt$ for $\lambda_n>0$ . Then the $\xi_n$ are centered, uncorrelated, and of unit variance, and

X_t=\sum_{n=1}^\infty\sqrt{\lambda_n}\,\xi_n\,\phi_n(t), \tag{5}

the series converging in mean square uniformly in $t\in[0,T]$ .

Proof

Centering is immediate from $\E[X_t]=0$ . For the second moments,

\E[\xi_m\xi_n]=\frac{1}{\sqrt{\lambda_m\lambda_n}}\int_0^T\!\!\int_0^T K(s,t)\phi_m(s)\phi_n(t)\,ds\,dt =\frac{1}{\sqrt{\lambda_m\lambda_n}}\int_0^T\lambda_n\phi_n(s)\phi_m(s)\,ds=\delta_{mn}, \tag{6}

\E[(X_t-S_N(t))^2]=K(t,t)-2\E[X_t S_N(t)]+\E[S_N(t)^2]. \tag{7}

\E[(X_t-S_N(t))^2]=K(t,t)-\sum_{n\le N}\lambda_n\phi_n(t)^2, \tag{8}

which is exactly the diagonal residual $r_N(t)$ of the Mercer proof, so the diagonal Dini step of Theorem 2 makes it tend to zero uniformly in $t$ as $N\to\infty$ .

#Optimality of the eigenbasis

The eigenbasis is not one good basis among many. It is the best basis for representing the process in finitely many terms.

Theorem4

Among all orthonormal bases $(e_n)$ of $L^2[0,T]$ , the eigenbasis $(\phi_n)$ minimises the mean-square error of every finite truncation. For each $N$ the minimum truncation error is

\min_{(e_n)}\,\E\!\int_0^T\!\Big(X_t-\sum_{n\le N}\langle X,e_n\rangle\,e_n(t)\Big)^{2}dt =\sum_{n>N}\lambda_n, \tag{9}

attained by $e_n=\phi_n$ .

Proof

#The Gaussian case

When the process is Gaussian the coefficients gain their strongest property.

Proposition5

If $X$ is a centered Gaussian process then the coefficients $\xi_n$ of Equation (5) are independent standard normal random variables.

Proof

#Brownian motion

\lambda\,\phi''(t)=-\phi(t), \tag{10}

\phi_n(t)=\sqrt 2\,\sin\!\Big(\Big(n-\half\Big)\pi t\Big),\qquad \lambda_n=\frac{1}{\big(n-\half\big)^2\pi^2},\qquad n=1,2,\dots, \tag{11}

and the Karhunen-Loeve expansion of Brownian motion,

W_t=\sqrt 2\,\sum_{n=1}^\infty\frac{\sin\!\big((n-\half)\pi t\big)}{(n-\half)\pi}\,\xi_n,\qquad \xi_n\ \text{i.i.d.}\ \mathcal N(0,1). \tag{12}

The eigenvalues decay as $n^{-2}$ , so the low modes dominate, and the first few carry most of the variance.

Mode $n$	$\lambda_n=1/((n-\tfrac12)^2\pi^2)$	Share of total variance
1	$0.4053$	$81.1\%$
2	$0.0450$	$9.0\%$
3	$0.0162$	$3.2\%$
4	$0.0083$	$1.7\%$

#Brownian bridge

\phi_n(t)=\sqrt 2\,\sin(n\pi t),\qquad \lambda_n=\frac{1}{n^2\pi^2},\qquad B_t=\sqrt 2\,\sum_{n=1}^\infty\frac{\sin(n\pi t)}{n\pi}\,\xi_n. \tag{13}

The three representations sit side by side.

Basis	Coordinates	Optimal truncation	Coefficients
Fourier	Fixed sines and cosines	No, in general	Correlated, in general
Discrete PCA	Eigenvectors of a covariance matrix	Yes, for vectors	Uncorrelated
Karhunen-Loeve	Eigenfunctions of $\mathcal K$	Yes, for processes	Uncorrelated, unit variance

#Numerical illustration

import numpy as np
from numpy.random import Generator


def brownian_via_kl(
    n_modes: int, grid: np.ndarray, rng: Generator
) -> np.ndarray:
    """Sample a Brownian path on [0, 1] from a truncated Karhunen-Loeve series.

    Args:
        n_modes: Number of eigenmodes retained in the truncation.
        grid: Increasing times in [0, 1] at which to evaluate the path.
        rng: Seeded generator for reproducibility.

    Returns:
        The sampled path values at the grid times.
    """
    n = np.arange(1, n_modes + 1)
    eigen_root = np.sqrt(2.0) / ((n - 0.5) * np.pi)
    modes = np.sin(np.outer(grid, (n - 0.5) * np.pi))
    coefficients = rng.standard_normal(n_modes)
    return modes @ (eigen_root * coefficients)


def captured_variance(n_modes: int) -> float:
    """Fraction of total Brownian variance held by the first modes.

    Args:
        n_modes: Number of leading eigenmodes.

    Returns:
        The ratio of retained eigenvalue mass to the total one half.
    """
    n = np.arange(1, n_modes + 1)
    eigenvalues = 1.0 / ((n - 0.5) * np.pi) ** 2
    return float(eigenvalues.sum() / 0.5)


rng = np.random.default_rng(0)
grid = np.linspace(0.0, 1.0, 500)
path = brownian_via_kl(n_modes=8, grid=grid, rng=rng)
retained = captured_variance(n_modes=8)

[1]

K. Karhunen, “Über lineare Methoden in der Wahrscheinlichkeitsrechnung,” Annales Academiae Scientiarum Fennicae, Series A I, vol. 37, pp. 1–79, 1947.

[2]

M. Loève, Probability Theory II, 4th ed. Springer, 1978.

[3]

[4]

R. G. Ghanem and P. D. Spanos, Stochastic Finite Elements: A Spectral Approach. Springer, 1991.

Explore connections

see in the atlas

referenced by (8)

cite

@misc{the-karhunen-loeve-expansion,
  author = {Zac Kienzle},
  title  = {The Karhunen-Loeve Expansion},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/the-karhunen-loeve-expansion}
}