Fourier methods and symmetry — aniket deshpande

Fourier analysis expresses a function in modes that behave simply under symmetry. On a circle, these modes are the familiar complex exponentials $e^{in\theta}$ . On a sphere, they are spherical harmonics. When an operator respects the relevant symmetry, these modes can also be its eigenfunctions, turning an integral or differential equation into separate equations for individual coefficients.

This is why an eigendecomposition on a symmetric domain can sometimes be understood as a Fourier decomposition. The important condition is that the operator and the integration measure must respect the symmetry. The geometry of the domain alone is insufficient.

Fourier methods also appear in statistical mechanics when we represent a constraint using a Dirac delta function. The underlying mathematics is closely related, but the variable being Fourier-expanded is different. We will develop both applications.

Consider a square-integrable function on the circle, parameterized by $\theta\in[0,2\pi)$ . Its Fourier expansion is

f(\theta)=\sum_{n\in\mathbb Z}f_n e^{in\theta}, \qquad f_n=\int_0^{2\pi}f(\theta)e^{-in\theta}\frac{d\theta}{2\pi}.

Here, $n$ labels the angular frequency. The constant mode $n=0$ describes the average value, while larger $|n|$ describe finer angular variation.

To understand why these particular functions are useful, define the rotation operator

(U_a f)(\theta)=f(\theta-a).

Applying it to a Fourier mode gives

U_a e^{in\theta} = e^{in(\theta-a)} = e^{-ina}e^{in\theta}.

Thus, rotating a Fourier mode only multiplies it by a phase. Each mode is an eigenfunction of every rotation $U_a$ .

Now consider a linear operator that respects rotations:

TU_a=U_aT \qquad\text{for every }a.

This equation says that rotating a function before applying $T$ gives the same result as rotating afterward. Consequently,

U_a(Te^{in\theta}) = T(U_ae^{in\theta}) = e^{-ina}Te^{in\theta}.

The transformed function $Te^{in\theta}$ therefore has the same behavior under every rotation as $e^{in\theta}$ . Since the functions with this behavior form a one-dimensional complex space, we must have

Te^{in\theta}=\lambda_n e^{in\theta}.

The Fourier modes diagonalize $T$ because $T$ respects the rotations that define those modes.

We can see this directly for a convolution operator,

(Tf)(\theta) = \int_0^{2\pi} \kappa(\theta-\phi)f(\phi)\frac{d\phi}{2\pi}.

The kernel depends only on the relative angle. It treats every location on the circle equivalently.

Substituting a Fourier mode and changing variables to $u=\theta-\phi$ , using periodicity, gives

\begin{aligned} (Te_n)(\theta) &= \int_0^{2\pi} \kappa(\theta-\phi)e^{in\phi}\frac{d\phi}{2\pi}\\ &= \int_0^{2\pi} \kappa(u)e^{in(\theta-u)}\frac{du}{2\pi}\\ &= e^{in\theta} \int_0^{2\pi} \kappa(u)e^{-inu}\frac{du}{2\pi}, \end{aligned}

where $e_n(\theta)=e^{in\theta}$ . Hence

\boxed{ \lambda_n = \int_0^{2\pi} \kappa(u)e^{-inu}\frac{du}{2\pi}. }

The operator’s eigenvalues are precisely the Fourier coefficients of its convolution kernel.

For an arbitrary function,

Tf = \sum_{n\in\mathbb Z}\lambda_n f_n e^{in\theta}.

An integral operation has become multiplication of each coefficient by a scalar. Changing $\kappa$ changes the eigenvalues, while the Fourier basis continues to work.

The discrete version is a circulant matrix: a matrix whose entries depend only on the difference between indices, modulo the number of sites. Its eigenvectors form the discrete Fourier basis.

There is already a simple example in the one-dimensional Ising model. At zero external field, the transfer matrix is

\mathsf T = \begin{pmatrix} e^{\beta J}&e^{-\beta J}\\ e^{-\beta J}&e^{\beta J} \end{pmatrix}.

Its normalized eigenvectors are

\mathbf v_+ = \frac1{\sqrt2} \begin{pmatrix}1\\1\end{pmatrix}, \qquad \mathbf v_- = \frac1{\sqrt2} \begin{pmatrix}1\\-1\end{pmatrix},

with eigenvalues

\lambda_+=2\cosh(\beta J), \qquad \lambda_-=2\sinh(\beta J).

These are the Fourier modes of the two-element group $\mathbb Z_2$ . To see this, label the spin states by $s=(-1)^b$ , where $b\in\{0,1\}$ . The two discrete Fourier functions are

e^{2\pi i(0)b/2}=1, \qquad e^{2\pi i(1)b/2}=(-1)^b=s.

One is even under spin flip, and the other is odd. The transfer matrix preserves these symmetry sectors, so it acts on each by multiplication by a scalar. The familiar diagonalization of this $2\times2$ matrix is therefore a small Fourier calculation.

We can now replace the circle by the ordinary unit sphere,

S^2=\{\mathbf x\in\mathbb R^3:\|\mathbf x\|=1\}.

The natural modes are the spherical harmonics $Y_{\ell m}$ , where

\ell=0,1,2,\ldots, \qquad m=-\ell,\ldots,\ell.

They are the angular functions encountered in quantum mechanics. Their relation to ordinary Fourier modes is visible in the eigenvalue equations

-\frac{d^2}{d\theta^2}e^{in\theta} = n^2e^{in\theta}, \qquad -\Delta_{S^2}Y_{\ell m} = \ell(\ell+1)Y_{\ell m}.

Here, $\Delta_{S^2}$ is the Laplacian intrinsic to the sphere. Both bases organize functions by their spatial variation, measured through a Laplacian eigenvalue.

Let $\mu$ be the uniform probability measure,

d\mu=\frac{d\Omega}{4\pi}.

We choose the spherical harmonics to be orthonormal with respect to this measure:

\int_{S^2} Y_{\ell m}(\mathbf x)^* Y_{\ell' m'}(\mathbf x)\,d\mu(\mathbf x) = \delta_{\ell\ell'}\delta_{mm'}.

This convention rescales the usual quantum-mechanics harmonics by $\sqrt{4\pi}$ .

A square-integrable function then has the expansion

f(\mathbf x) = \sum_{\ell=0}^\infty\sum_{m=-\ell}^{\ell} f_{\ell m}Y_{\ell m}(\mathbf x), \qquad f_{\ell m} = \int_{S^2}Y_{\ell m}(\mathbf x)^*f(\mathbf x)\,d\mu(\mathbf x).

Suppose a kernel depends only on the angle between its arguments:

K(\mathbf x,\mathbf x') = \kappa(\mathbf x\cdot\mathbf x').

The associated operator is

(T_Kf)(\mathbf x) = \int_{S^2} \kappa(\mathbf x\cdot\mathbf x')f(\mathbf x')\,d\mu(\mathbf x').

Rotating both inputs preserves their inner product, and rotations preserve $\mu$ . Thus $T_K$ commutes with rotations.

The spherical counterpart of the convolution calculation is the Funk–Hecke theorem:

\boxed{ T_KY_{\ell m}=\lambda_\ell Y_{\ell m}. }

With our probability-measure convention on $S^2$ ,

\boxed{ \lambda_\ell = \frac12\int_{-1}^{1}\kappa(t)P_\ell(t)\,dt, }

where $P_\ell$ is the Legendre polynomial normalized by $P_\ell(1)=1$ . Angular integration has reduced the eigenvalue calculation to a one-dimensional integral.

Every $m$ at the same degree $\ell$ has the same eigenvalue. Rotations generally mix the $2\ell+1$ harmonics within that degree, so an operator respecting all rotations cannot distinguish a preferred orientation among them.

More precisely, each fixed- $\ell$ space is an irreducible rotation representation: it cannot be split into smaller nonzero subspaces preserved by every rotation. Each such representation occurs once in scalar functions on $S^2$ , and Schur’s lemma makes a rotation-invariant operator scalar on it. For more general spaces, symmetry may only reduce an operator to blocks.

For a sufficiently regular kernel, we can consequently write

K(\mathbf x,\mathbf x') = \sum_{\ell,m} \lambda_\ell Y_{\ell m}(\mathbf x) Y_{\ell m}(\mathbf x')^*.

This is the kernel’s spectral decomposition, expressed in a basis supplied by spherical geometry.

The same construction extends to $S^{N-1}\subset\mathbb R^N$ . The low-degree harmonic spaces have a useful interpretation in learning theory:

Degree	Functions spanning the space on the unit sphere	Dimension
$\ell=0$	Constants	$1$
$\ell=1$	Coordinates $x_i$	$N$
$\ell=2$	Traceless quadratics $x_ix_j-\delta_{ij}/N$	$N(N+1)/2-1$

Thus, harmonic degree organizes constant, linear, quadratic, and progressively more complicated angular features. If inputs instead satisfy $\|\mathbf x\|^2=N$ , as is common in statistical-mechanics calculations, the angular variable becomes $\mathbf x\cdot\mathbf x'/N$ .

As a concrete example, take the linear kernel on the unit sphere:

K(\mathbf x,\mathbf x')=\mathbf x\cdot\mathbf x'.

Rotational invariance implies

\mathbb E_\mu[\mathbf x\mathbf x^\top]=cI_N.

Taking the trace gives $Nc=\mathbb E\|\mathbf x\|^2=1$ , hence $c=1/N$ . For a linear function $f(\mathbf x)=\mathbf a\cdot\mathbf x$ ,

\begin{aligned} (T_Kf)(\mathbf x) &= \mathbb E_{\mathbf x'} \big[(\mathbf x\cdot\mathbf x')(\mathbf a\cdot\mathbf x')\big]\\ &= \mathbf x^\top \mathbb E[\mathbf x'\mathbf x'^\top]\mathbf a\\ &= \frac1N f(\mathbf x). \end{aligned}

All linear harmonics therefore have eigenvalue $1/N$ . Moreover,

(T_Kf)(\mathbf x) = \mathbf x\cdot\mathbb E[\mathbf x'f(\mathbf x')]

is linear for every $f$ . Every harmonic degree other than $\ell=1$ consequently has eigenvalue zero.

This distinguishes the population kernel operator from the ordinary input covariance. The covariance $I_N/N$ has no preferred eigenvector: every orthonormal coordinate basis diagonalizes it. The kernel operator acts on an entire function space and identifies which kinds of functions it can represent.

The spectrum also becomes useful for studying learning dynamics. Assume a fixed positive-semidefinite kernel and population gradient flow for squared-error regression:

\partial_t f_t(\mathbf x) = -\int K(\mathbf x,\mathbf x') \big[f_t(\mathbf x')-f_*(\mathbf x')\big]\,d\mu(\mathbf x').

Writing the prediction error as $e_t=f_t-f_*$ , this is

\partial_t e_t=-T_Ke_t.

For a rotationally invariant spherical kernel, expand

e_t(\mathbf x) = \sum_{\ell,m}e_{\ell m}(t)Y_{\ell m}(\mathbf x).

Substitution and orthogonality give

\frac{d}{dt}e_{\ell m}(t) = -\lambda_\ell e_{\ell m}(t),

and therefore

\boxed{ e_{\ell m}(t) = e^{-\lambda_\ell t}e_{\ell m}(0). }

Each harmonic component relaxes independently. A larger eigenvalue means faster learning; a zero eigenvalue means that component of the initial error persists. This mode-wise description also appears in the fixed-kernel limit of neural-network training.

Whether lower-degree features learn faster depends on the eigenvalues of the particular kernel. Symmetry identifies the modes but does not, by itself, order their learning rates.

These statements concern a population operator. With finitely many randomly sampled inputs, the empirical measure generally breaks exact rotational symmetry, and the eigenvectors of the sample kernel matrix need not be exact sampled spherical harmonics. Likewise, when feature learning changes the kernel during training, a single fixed spectral decomposition need not describe the full dynamics.

A second application of Fourier analysis appears when a calculation contains a hard constraint. Consider weights $\mathbf w\in\mathbb R^N$ restricted by

\frac1N\sum_{j=1}^{N}w_j^2=1.

Define their scalar constraint violation by

u(\mathbf w)=1-\frac1N\sum_jw_j^2.

We can impose the constraint inside an integral using

Z = \int_{\mathbb R^N}d^Nw\, \delta\!\big(u(\mathbf w)\big)e^{-\beta H(\mathbf w)}.

The delta restricts the integration to the sphere of radius $\sqrt N$ , with a measure proportional to its surface measure.

On the real line, our Fourier-transform convention is

\widetilde f(k)=\int_{\mathbb R}f(u)e^{-iku}\,du, \qquad f(u)=\int_{\mathbb R}\frac{dk}{2\pi}\widetilde f(k)e^{iku}.

The Fourier transform of the delta distribution is

\widetilde\delta(k) = \int_{\mathbb R}\delta(u)e^{-iku}\,du = 1.

Fourier inversion therefore gives

\boxed{ \delta(u)=\int_{\mathbb R}\frac{dk}{2\pi}e^{iku}. }

This is a distributional identity: it holds after integration against a suitable test function. Intuitively, combining all frequencies produces a spike at zero.

Substituting the spherical constraint,

\begin{aligned} \delta\!\left(1-\frac1N\sum_jw_j^2\right) &= \int_{\mathbb R}\frac{dk}{2\pi} \exp\!\left[ik\left(1-\frac1N\sum_jw_j^2\right)\right]\\ &= \int_{\mathbb R}\frac{dk}{2\pi} e^{ik} \prod_{j=1}^{N}e^{-ik w_j^2/N}. \end{aligned}

For fixed $k$ , a constraint involving every coordinate has become a product of individual coordinate factors.

For example, suppose the remaining Hamiltonian is separable,

H(\mathbf w)=\sum_{j=1}^{N}V(w_j),

with a confining potential $V$ . The constrained integral becomes

Z = \int_{\mathbb R}\frac{dk}{2\pi}e^{ik} \left[ \int_{\mathbb R}dw\, e^{-\beta V(w)-ik w^2/N} \right]^N.

We have replaced an integral over $N$ constrained coordinates by a single-coordinate integral and an auxiliary integral. For quadratic $V$ , the coordinate integral is Gaussian. The power $N$ also prepares the expression for a large- $N$ saddle-point calculation.

JZV uses this construction in his replica notes. His imaginary-axis convention follows by setting $s=ik$ :

\delta(u) = \int_{-i\infty}^{i\infty}\frac{ds}{2\pi i}e^{su}.

It is the same Fourier identity with the factor of $i$ absorbed into the integration variable. Conventional rescalings of $s$ produce the hatted variables appearing in the action.

For example, define the replica overlap

Q^{ab}=\frac1N\sum_{j=1}^{N}w_j^aw_j^b.

To treat $Q^{ab}$ as an integration variable while preserving its definition, insert

1 = \int_{\mathbb R}dQ^{ab}\, \delta\!\left( Q^{ab}-\frac1N\sum_jw_j^aw_j^b \right).

Fourier-representing this delta introduces a conjugate variable, conventionally denoted $\widehat Q^{ab}$ . Its exponential contains a sum over coordinates, allowing the weight integrals to factor across $j$ in the relevant replica calculation. The replicas at a given coordinate can remain coupled.

The field $\widehat Q^{ab}$ acts as a Lagrange multiplier enforcing the overlap definition. Its hat labels an auxiliary conjugate variable; it does not mean a Fourier coefficient of the weight vector.

There is a direct mathematical connection between these constraint representations and the earlier spectral decompositions. A delta distribution is the kernel of the identity operator. On the real line,

f(u)=\int_{\mathbb R}\delta(u-v)f(v)\,dv,

and its Fourier representation is

\delta(u-v) = \int_{\mathbb R}\frac{dk}{2\pi}e^{iku}e^{-ikv}.

Every Fourier mode has eigenvalue one under the identity.

On the sphere, the corresponding completeness relation is

\delta_\mu(\mathbf x,\mathbf x') = \sum_{\ell,m} Y_{\ell m}(\mathbf x)Y_{\ell m}(\mathbf x')^*,

where $\delta_\mu$ is defined by

\int_{S^2} \delta_\mu(\mathbf x,\mathbf x')f(\mathbf x')\,d\mu(\mathbf x') = f(\mathbf x).

Again, all spectral weights are one.

In JZV’s constraint calculation, however, the Fourier variable $k$ is conjugate to the scalar violation $u$ . It does not label angular variation around the weight sphere. This is why the delta-function technique works even when the Hamiltonian has no rotational symmetry: the Fourier expansion takes place in the constraint variable.

The references used here are:

Gilbert Strang, Eigenvectors of Circulant Matrices: Fourier Matrix.
Jean Gallier, Notes on Spherical Harmonics and Linear Representations of Lie Groups.
Francis Bach, Breaking the Curse of Dimensionality with Convex Neural Networks, Appendices C–D.
Arthur Jacot, Franck Gabriel, and Clément Hongler, Neural Tangent Kernel: Convergence and Generalization in Neural Networks.
Jacob Zavatone-Veth, Lecture Notes on the Replica Method for Wishart Matrix Eigenvalues, §5.2.