Linear Algebra › Section G · Symmetric matrices and quadratic forms › Topic G.1
Diagonalizing symmetric matrices
Symmetric matrices are the best-behaved square matrices there are: their eigenvalues are always real, and their eigenvectors can always be chosen perpendicular. That makes them easy to diagonalise and is the foundation for quadratic forms, the singular value decomposition and principal component analysis.
Why symmetric matrices deserve their own topic
A matrix is symmetric when \(A^T = A\), so the entry in row \(i\), column \(j\) equals the entry in row \(j\), column \(i\). Only square matrices can be symmetric. They are everywhere: covariance matrices in statistics, stiffness matrices in engineering, Hessian matrices of second partial derivatives, and the product \(A^TA\) for any matrix \(A\), since \((A^TA)^T = A^T(A^T)^T = A^TA\). The topic "Diagonalization" showed that some matrices cannot be diagonalised at all, and that even when \(A = PDP^{-1}\) works, finding \(P^{-1}\) takes effort. For symmetric matrices both problems disappear.
Eigenvectors from different eigenvalues are orthogonal
Suppose \(A\) is symmetric, \(A\mathbf{u} = \lambda\mathbf{u}\) and \(A\mathbf{v} = \mu\mathbf{v}\) with \(\lambda \ne \mu\). Because \(A^T = A\), moving \(A\) across a dot product changes nothing:
So \((\lambda - \mu)(\mathbf{u} \cdot \mathbf{v}) = 0\), and since \(\lambda - \mu \ne 0\), the dot product must be zero. The middle step is the key: \((A\mathbf{u}) \cdot \mathbf{v} = \mathbf{u}^TA^T\mathbf{v}\), which equals \(\mathbf{u}^TA\mathbf{v}\) only because \(A\) is symmetric. For a non-symmetric matrix, eigenvectors are usually not perpendicular.
Orthogonal diagonalisation
A square matrix \(A\) is orthogonally diagonalisable if
where \(D\) is diagonal and \(P\) is an orthogonal matrix, meaning its columns are orthonormal, so \(P^{-1} = P^T\). The inverse costs nothing: you just transpose. Any such matrix is automatically symmetric, because \((PDP^T)^T = (P^T)^TD^TP^T = PDP^T\), using \(D^T = D\). So only symmetric matrices can be orthogonally diagonalised. The remarkable fact, in the next card, is that every symmetric matrix can be.
The spectral theorem
The set of eigenvalues of a matrix is called its spectrum. For an \(n \times n\) symmetric matrix \(A\), the spectral theorem says four things. First, \(A\) has \(n\) real eigenvalues, counting multiplicities. Second, the dimension of each eigenspace equals the multiplicity of its eigenvalue as a root of the characteristic equation, so there is never a shortage of eigenvectors. Third, eigenspaces for different eigenvalues are mutually orthogonal. Fourth, \(A\) is orthogonally diagonalisable. Combined with the previous card: a real square matrix is orthogonally diagonalisable if and only if it is symmetric.
Worked example: a 2 × 2 symmetric matrix
Orthogonally diagonalise \(A\) with rows \((7, 2)\) and \((2, 4)\). 1. The trace is 11 and the determinant is 24, so the eigenvalues solve \(\lambda^2 - 11\lambda + 24 = 0\), giving 8 and 3. 2. For 8, \(A - 8I\) has rows \((-1, 2)\) and \((2, -4)\), so an eigenvector is \((2, 1)\). For 3, \(A - 3I\) has rows \((4, 2)\) and \((2, 1)\), giving \((1, -2)\). 3. Check: \((2, 1) \cdot (1, -2) = 0\), as the theorem promises. 4. Normalise each to length 1 by dividing by \(\sqrt{5}\):
5. Multiplying out \(PDP^T\) returns \(A\).
Worked example: a repeated eigenvalue
Take the \(3 \times 3\) matrix \(A\) with 2 on the diagonal and 1 everywhere else. 1. \(A - I\) is all ones, of rank 1, so the eigenspace for 1 is two-dimensional; the trace, 6, leaves a third eigenvalue \(6 - 1 - 1 = 4\). 2. For 4, \(\mathbf{v}_1 = (1, 1, 1)\). 3. For 1, the eigenspace is the plane \(x_1 + x_2 + x_3 = 0\), with basis \((1, -1, 0)\) and \((1, 0, -1)\). Both are orthogonal to \(\mathbf{v}_1\), but not to each other. 4. Apply Gram-Schmidt inside the eigenspace: \((1, 0, -1) - \tfrac{1}{2}(1, -1, 0) = (\tfrac{1}{2}, \tfrac{1}{2}, -1)\), which we scale to \((1, 1, -2)\). 5. Normalise:
with \(D = \mathrm{diag}(4, 1, 1)\). Check: \(P^TP = I\).
The spectral decomposition
Write \(P\) with columns \(\mathbf{u}_1, \dots, \mathbf{u}_n\) and multiply out \(PDP^T\) column by row:
Each \(\mathbf{u}_j\mathbf{u}_j^T\) is an \(n \times n\) matrix of rank 1, and multiplying it by \(\mathbf{x}\) gives \((\mathbf{u}_j \cdot \mathbf{x})\mathbf{u}_j\), the orthogonal projection of \(\mathbf{x}\) onto the line through \(\mathbf{u}_j\). So a symmetric matrix splits any vector into perpendicular eigenvector components, stretches each by its eigenvalue, and adds them back. In the 2 × 2 example, \(8\mathbf{u}_1\mathbf{u}_1^T\) has rows \((6.4, 3.2)\) and \((3.2, 1.6)\), \(3\mathbf{u}_2\mathbf{u}_2^T\) has rows \((0.6, -1.2)\) and \((-1.2, 2.4)\), and they add to \(A\).
A preview: the matrix AᵀA
For any \(m \times n\) matrix \(A\), the product \(A^TA\) is symmetric, so the spectral theorem applies to it. Its eigenvalues are also never negative. If \(A^TA\mathbf{v} = \lambda\mathbf{v}\) with \(\mathbf{v}\) a unit vector, then
and a squared length cannot be negative. This small fact, together with the orthonormal eigenvectors of \(A^TA\), is exactly what builds the singular value decomposition later in this section, a factorisation that works for every matrix, square or not.
Common mistakes to avoid
First, forgetting to normalise: \((2, 1)\) and \((1, -2)\) are orthogonal, but \(P\) is only orthogonal once each column has length 1. Second, stopping before Gram-Schmidt when an eigenvalue is repeated: two independent eigenvectors in the same eigenspace need not be perpendicular, as \((1, -1, 0)\) and \((1, 0, -1)\) show. Third, using \(P^T\) as the inverse for a matrix whose columns are merely independent. Fourth, assuming a repeated eigenvalue might leave a symmetric matrix undiagonalisable: the spectral theorem rules that out. Fifth, putting the eigenvalues in \(D\) in a different order from their eigenvectors in \(P\).
Key takeaways
A symmetric matrix satisfies Aᵀ = A. Eigenvectors of a symmetric matrix with different eigenvalues are orthogonal. A matrix is orthogonally diagonalisable, A = PDPᵀ with P orthogonal and P⁻¹ = Pᵀ, exactly when it is symmetric. The spectral theorem adds that the eigenvalues are real and that each eigenspace has full dimension. To diagonalise: find the eigenvalues, find a basis of each eigenspace, use Gram-Schmidt inside any repeated eigenspace, normalise, and place the unit eigenvectors in P in the same order as the eigenvalues in D. The spectral decomposition writes A as a sum of eigenvalue-weighted projections onto perpendicular lines.
Flashcards
Flashcard
A square matrix with Aᵀ = A, so aᵢⱼ = aⱼᵢ
Test your understanding
Question 1 of 5
A symmetric matrix A has eigenvectors (3, 1, k) for eigenvalue 5 and (1, 1, 2) for eigenvalue −2. What must k be?
Question 2 of 5
The symmetric matrix with rows (5, 2) and (2, 2) has eigenvalues 6 and 1. Which matrix P orthogonally diagonalises it, with D = diag(6, 1)?
Question 3 of 5
Which statement about a real square matrix is correct?
Question 4 of 5
In orthogonally diagonalising a symmetric 3 × 3 matrix, the eigenvalue 1 has eigenspace basis (1, −1, 0) and (1, 0, −1). What should you do before normalising?
Question 5 of 5
A symmetric matrix has spectral decomposition A = 4u₁u₁ᵀ + 1u₂u₂ᵀ, where u₁ = (1, 0) and u₂ = (0, 1). What is Ax for x = (2, 3)?
