Symmetric matrices and the spectral theorem
Symmetric matrices have the best possible eigenvalue structure: real eigenvalues and an orthonormal basis of eigenvectors. This makes them central to geometry, statistics, and physics.
Symmetric matrices
A matrix is symmetric if \(A^T = A\), so \(a_{ij} = a_{ji}\). Symmetric matrices arise constantly: covariance matrices in statistics, stiffness matrices in engineering, Hessian matrices of second derivatives in optimisation, and \(A^TA\) for any matrix A. Their special properties make them the best-behaved matrices in linear algebra.
Real eigenvalues and orthogonal eigenvectors
Every real symmetric matrix has real eigenvalues, and eigenvectors belonging to different eigenvalues are orthogonal. The proof of orthogonality is short: if \(A\mathbf{u} = \lambda\mathbf{u}\) and \(A\mathbf{v} = \mu\mathbf{v}\), then \(\lambda\,\mathbf{u} \cdot \mathbf{v} = (A\mathbf{u}) \cdot \mathbf{v} = \mathbf{u} \cdot (A\mathbf{v}) = \mu\,\mathbf{u} \cdot \mathbf{v}\), so if \(\lambda \ne \mu\), \(\mathbf{u} \cdot \mathbf{v} = 0\).
The spectral theorem
The spectral theorem states that a real matrix is orthogonally diagonalisable, \(A = QDQ^T\) with Q orthogonal and D diagonal, if and only if it is symmetric. The columns of Q are orthonormal eigenvectors and D holds the eigenvalues. Since \(Q^{-1} = Q^T\), no inverse needs computing.
Spectral decomposition
Writing out \(A = QDQ^T\) gives \(A = \lambda_1\mathbf{q}_1\mathbf{q}_1^T + \dots + \lambda_n\mathbf{q}_n\mathbf{q}_n^T\): a sum of rank-one projection matrices, each weighted by an eigenvalue. A symmetric matrix acts by projecting onto each eigenvector direction and scaling by the eigenvalue, a clear geometric picture.
Quadratic forms
A quadratic form is \(Q(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}\) with A symmetric, such as \(3x^2 + 4xy + 3y^2\). Changing to the eigenvector basis removes the cross terms, the principal axes theorem, turning \(Q\) into \(\lambda_1y_1^2 + \lambda_2y_2^2\). The level curve \(Q = 1\) is then seen to be an ellipse, hyperbola, or other conic aligned with the eigenvectors.
Positive definite matrices
A symmetric matrix is positive definite if \(\mathbf{x}^TA\mathbf{x} > 0\) for every non-zero \(\mathbf{x}\), which holds exactly when all its eigenvalues are positive. Positive definite matrices describe energies, distances, and covariances. In optimisation, a positive definite Hessian at a critical point means a local minimum, the multivariable second-derivative test.
The Cholesky factorisation
Every positive definite matrix factors as \(A = LL^T\) with L lower triangular with positive diagonal, the Cholesky factorisation, named after Andre-Louis Cholesky and published posthumously in 1924. It is about twice as fast as LU and numerically stable, and it is used to solve systems with covariance or stiffness matrices and to generate correlated random numbers in simulations.
Worked example
For \(A = \begin{bmatrix} 3 & 2 \\ 2 & 3 \end{bmatrix}\): trace 6, determinant 5, so eigenvalues 5 and 1, both positive, so A is positive definite. Unit eigenvectors are \(\frac{1}{\sqrt{2}}(1, 1)\) for 5 and \(\frac{1}{\sqrt{2}}(1, -1)\) for 1, which are orthogonal. In those coordinates, \(3x^2 + 4xy + 3y^2 = 5y_1^2 + y_2^2\), so \(Q = 1\) is an ellipse with axes along the lines y = x and y = -x.
Key takeaways
Symmetric matrices (A^T = A) appear as covariances, stiffness matrices, Hessians, and A^T A. They have real eigenvalues and orthogonal eigenvectors for distinct eigenvalues. The spectral theorem: A = Q D Q^T with orthogonal Q exactly when A is symmetric. The spectral decomposition writes A as a sum of eigenvalue-weighted rank-one projections. Quadratic forms x^T A x lose cross terms in the eigenvector basis, the principal axes theorem. Positive definite matrices have all eigenvalues positive and indicate minima in optimisation. The Cholesky factorisation A = L L^T (1924) solves positive definite systems quickly and stably.
Flashcard
A^T = A
Check your understanding
Which property guarantees that a real matrix can be diagonalised by an orthogonal matrix?
