Chapter 23: Introduction to Random Matrices¶

Prerequisites: Eigenvalues (Ch06) · Probability Theory · Matrix Analysis (Ch14)

Chapter Outline: From Deterministic to Random Matrices → Motivation for Random Matrix Theory (RMT) (Universal Laws of Complex Systems) → Typical Random Matrix Ensembles: Wigner and Wishart Matrices → Core Spectral Distributions: Wigner Semi-circle Law → Marchenko-Pastur (M-P) Law (Limit Spectrum of Sample Covariance) → Edge Distributions: The Tracy-Widom Distribution → The Concept of Universality → Applications: Financial Risk Analysis (Noise Filtering), Nuclear Physics Energy Levels, Wireless Channel Capacity (MIMO), and Compressed Sensing

Extension: Random Matrix Theory studies the spectral structure in the "large dimension, large sample" limit; it proves that when a system is sufficiently complex, local microscopic noise collapses into macroscopically deterministic geometric shapes—the theoretical pillar of modern high-dimensional statistics and data science.

When we do not know the exact values of matrix entries but only their statistical distributions, studying the behavior of their eigenvalues leads us into the realm of Random Matrix Theory (RMT). The fascination of RMT lies in the fact that while individual entries are random, the overall density distribution of eigenvalues follows extremely precise and universal mathematical laws as matrix dimensions tend toward infinity. This chapter introduces this frontier linking probability theory and operator spectral theory.

23.1 Wigner Matrices and the Semi-circle Law¶

Definition 23.1 (Wigner Matrix)

A symmetric matrix $A$ where the upper triangular entries $a_{ij}$ are independent and identically distributed (i.i.d.) random variables with mean 0 and variance $\sigma^2$.

Theorem 23.1 (Wigner Semi-circle Law)

As $n \to \infty$, the spectral density $\rho(\lambda)$ of the normalized Wigner matrix $A/\sqrt{n}$ converges to the semi-circle distribution: $$\rho(\lambda) = \frac{1}{2\pi \sigma^2} \sqrt{4\sigma^2 - \lambda^2}, \quad |\lambda| \le 2\sigma$$ Physical Meaning: The energy level distribution of complex symmetric systems exhibits a perfect geometric arc.

23.2 Wishart Matrices and the M-P Law¶

Definition 23.2 (Wishart Matrix)

$S = \frac{1}{n} X X^T$, where $X$ is a $p \times n$ random matrix. This is typically used to describe high-dimensional sample covariance matrices.

Theorem 23.2 (Marchenko-Pastur Law)

As $p, n \to \infty$ with the ratio $p/n \to \gamma$, the eigenvalue density converges to the M-P distribution. Application: This serves as a mathematical ruler for distinguishing signal from noise. If eigenvalues of real data fall outside the M-P distribution range, it indicates the presence of a real signal.

23.3 Edges and Universality¶

Tracy-Widom Distribution

The fluctuations of the largest eigenvalue $\lambda_{\max}$ do not follow a normal distribution but rather the Tracy-Widom distribution. It determines the "outlier" threshold for components deviating from the main bulk.

Exercises¶

1. [Basics] What is "Universality" in Random Matrix Theory?

Solution

Explanation: Universality refers to the fact that when the matrix dimension $n$ is large, the macroscopic distribution of eigenvalues (like the semi-circle law) and the microscopic spacing properties depend primarily on the symmetry class of the matrix (e.g., real symmetric, complex Hermitian, or symplectic), and are almost independent of the specific distribution of individual entries (e.g., Gaussian, Bernoulli, or Uniform). This is analogous to the Central Limit Theorem in probability.

2. [Calculation] For a Wigner matrix with entry variance $\sigma^2=1$, where are the eigenvalues concentrated according to the semi-circle law?

Solution

Calculation: According to the formula $|\lambda| \le 2\sigma$: Substituting $\sigma = 1$: $|\lambda| \le 2$. Conclusion: Eigenvalues are distributed within the interval $[-2, 2]$.

3. [Wishart] For a $1000 \times 1000$ matrix of pure i.i.d. Gaussian noise, what is the approximate maximum eigenvalue of its sample covariance matrix?

Solution

Applying the M-P Law: 1. Here $p=1000, n=1000 \implies \gamma = 1$. 2. The right edge of the M-P distribution is $\sigma^2(1 + \sqrt{\gamma})^2$. 3. Assuming $\sigma^2=1$: $(1 + \sqrt{1})^2 = 2^2 = 4$. Conclusion: The largest eigenvalue converges to approximately 4. Any eigenvalue significantly larger than 4 represents a non-noise signal component.

4. [Property] Prove that if $A$ is a random symmetric matrix with mean-zero entries, the expectation of $\operatorname{tr}(A)$ is 0.

Solution

Proof: 1. $\operatorname{tr}(A) = \sum a_{ii}$. 2. By linearity of expectation: $E[\operatorname{tr}(A)] = \sum E[a_{ii}]$. 3. Since $E[a_{ii}] = 0$ for all $i$, the sum is 0. Conclusion: The sum of eigenvalues is zero on average.

5. [Density] Why doesn't the eigenvalue histogram look like a perfect semi-circle for small $n$?

Solution

Reasoning: RMT laws are asymptotic laws. At finite dimensions, there are statistical fluctuations and "noise" that prevent the edges from being smooth. Only as $n$ increases does the Law of Large Numbers take effect, causing the histogram to converge to the theoretical curve.

6. [Application] How is RMT used in Finance to filter out spurious correlations?

Solution

Method: 1. Compute the correlation matrix $C$ of stock returns. 2. Plot the eigenvalue spectrum of $C$. 3. Overlay the corresponding M-P distribution curve (assuming pure randomness). 4. Filtering: Eigenvalues falling inside the M-P bulk are treated as unreliable noise. Only eigenvalues significantly larger than the M-P upper bound represent true market trends or sector effects.

7. [Tracy-Widom] How does the fluctuation range of the largest eigenvalue scale with $n$?

Solution

Conclusion: $n^{-2/3}$. This is a profound scaling law in RMT. The largest eigenvalue converges to the boundary faster than the $n^{-1/2}$ of the standard CLT, and the distribution is uniquely asymmetric (thicker tail on the left).

8. [Calculation] If $p/n = 0.25$, find the support interval of the M-P distribution (assume $\sigma^2=1$).

Solution

Steps: 1. $\gamma = 0.25, \sqrt{\gamma} = 0.5$. 2. Left edge: $(1 - \sqrt{\gamma})^2 = (1 - 0.5)^2 = 0.25$. 3. Right edge: $(1 + \sqrt{\gamma})^2 = (1 + 0.5)^2 = 2.25$. Conclusion: The spectrum is supported on $[0.25, 2.25]$. Since $\gamma < 1$, the spectrum does not include 0.

9. [Ensembles] What is the fundamental difference in origin between Wigner and Wishart matrices?

Solution

Comparison: - Wigner Matrix: Symmetric matrix with i.i.d. entries. Often used to describe nuclear energy levels or network connectivity. - Wishart Matrix: Generated from outer products of random vectors ($XX^T$). Used to describe covariance of observed data; its eigenvalues represent the strength of principal components.

10. [Physics] Why do energy levels of heavy nuclei match RMT predictions?

Solution

Explanation: Heavy nuclei contain many interacting hadrons, making the system too complex for analytic solutions. Physicists (like Wigner) hypothesized that the interaction operator (Hamiltonian) could be approximated by a massive random symmetric matrix. RMT successfully predicted "level repulsion"—the phenomenon where energy levels rarely cluster closely—which aligns perfectly with experimental observations.

Chapter Summary¶

Random Matrix Theory reveals the algebraic order behind chaos:

Limits of Determinism: It proves that macroscopic structures can "emerge" from microscopic randomness; the semi-circle and M-P laws establish statistical benchmarks for complex systems.
Boundaries of Noise: By defining the support of spectral densities, RMT provides scientific noise-filtering criteria for signal extraction, image processing, and financial analysis.
Universal Unity: From nuclear levels to radio signals and even the distribution of primes, RMT demonstrates how the spectral theory of linear algebra transcends disciplinary boundaries to become a unifying tool for understanding complexity.