Linear Algebra › Section G · Symmetric matrices and quadratic forms › Topic G.2
Quadratic forms
Energy, variance, squared distance and the curvature of a surface at a critical point are all quadratic expressions in several variables. Writing them with a symmetric matrix lets the spectral theorem strip out the cross terms and reveal their shape at a glance.
What a quadratic form is
A quadratic form on \(\mathbb{R}^n\) is a function \(Q(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}\), where \(A\) is an \(n \times n\) symmetric matrix called the matrix of the form. Every term has total degree exactly 2: squares like \(x_1^2\) and cross terms like \(x_1x_2\), with no linear or constant terms. The simplest example is \(\mathbf{x}^TI\mathbf{x} = \|\mathbf{x}\|^2\). A diagonal matrix gives a form with no cross terms at all, such as \(4x_1^2 - 3x_2^2\) from \(\mathrm{diag}(4, -3)\). The aim of this topic is to turn every quadratic form into one of that tidy diagonal kind.
Reading off the matrix
Multiplying out \(\mathbf{x}^TA\mathbf{x}\) gives \(\sum a_{ii}x_i^2 + \sum_{i<j} 2a_{ij}x_ix_j\), because the cross term \(x_ix_j\) appears twice, once from \(a_{ij}\) and once from \(a_{ji}\). So the rule is: put the coefficient of \(x_i^2\) on the diagonal, and split the coefficient of \(x_ix_j\) equally between positions \((i, j)\) and \((j, i)\). For \(Q = x_1^2 - 3x_2^2 + 4x_3^2 + 6x_1x_2 - 2x_2x_3\):
The 6 becomes 3 and 3, the \(-2\) becomes \(-1\) and \(-1\), and the missing \(x_1x_3\) term gives zeros.
Removing cross terms by a change of variable
Orthogonally diagonalise \(A = PDP^T\), as in "Diagonalizing symmetric matrices", and introduce new variables by \(\mathbf{x} = P\mathbf{y}\), so \(\mathbf{y} = P^T\mathbf{x}\). Then
This is the principal axes theorem: every quadratic form can be turned into one with no cross terms by an orthogonal change of variable. The columns of \(P\), unit eigenvectors of \(A\), are the principal axes. Because \(P\) is orthogonal, \(\|\mathbf{y}\| = \|\mathbf{x}\|\): the new coordinates measure the same point along rotated, perpendicular axes, possibly with one axis reflected.
Worked example: an ellipse in disguise
Remove the cross term from \(Q = 5x_1^2 + 4x_1x_2 + 2x_2^2\). 1. The matrix has rows \((5, 2)\) and \((2, 2)\): trace 7, determinant 6, so the eigenvalues are 6 and 1. 2. Unit eigenvectors: \(\mathbf{u}_1 = (2, 1)/\sqrt{5}\) for 6 and \(\mathbf{u}_2 = (1, -2)/\sqrt{5}\) for 1. 3. With \(\mathbf{y} = P^T\mathbf{x}\), \(Q = 6y_1^2 + y_2^2\). 4. Check at \(\mathbf{x} = (1, 0)\): the original gives 5; and \(\mathbf{y} = (2/\sqrt{5}, 1/\sqrt{5})\) gives \(6 \cdot \tfrac{4}{5} + \tfrac{1}{5} = 5\). 5. The curve \(Q = 6\) becomes \(y_1^2 + y_2^2/6 = 1\): an ellipse with semi-axis 1 along \(\mathbf{u}_1\) and \(\sqrt{6} \approx 2.45\) along \(\mathbf{u}_2\). The larger eigenvalue gives the shorter axis.
The geometry of level curves
In two variables, the curve \(\mathbf{x}^TA\mathbf{x} = c\) with \(c > 0\) becomes \(\lambda_1y_1^2 + \lambda_2y_2^2 = c\) in principal-axis coordinates. If both eigenvalues are positive, it is an ellipse, a circle when they are equal. If they have opposite signs, it is a hyperbola. If both are negative, there are no points at all. If one eigenvalue is zero and the other positive, you get two parallel lines. In every case the axes of symmetry run along the eigenvectors. Viewed as a surface \(z = Q(\mathbf{x})\), positive eigenvalues give a bowl, negative ones a dome, and mixed signs a saddle.
Classifying quadratic forms
A quadratic form \(Q\), and its symmetric matrix, is called positive definite if \(Q(\mathbf{x}) > 0\) for every \(\mathbf{x} \ne \mathbf{0}\); negative definite if \(Q(\mathbf{x}) < 0\) for every \(\mathbf{x} \ne \mathbf{0}\); and indefinite if it takes both positive and negative values. It is positive semidefinite if \(Q(\mathbf{x}) \ge 0\) for all \(\mathbf{x}\), and negative semidefinite if \(Q(\mathbf{x}) \le 0\) for all \(\mathbf{x}\). Since \(Q = \sum \lambda_iy_i^2\), the eigenvalues decide everything: \(Q\) is positive definite exactly when all eigenvalues are positive, negative definite when all are negative, indefinite when there are both signs, and semidefinite when the signs agree but some eigenvalues are zero.
Worked example: classifying two forms
1. Classify \(Q = x_1^2 + 4x_1x_2 + x_2^2\). Its matrix has rows \((1, 2)\) and \((2, 1)\), with eigenvalues 3 and \(-1\), so \(Q\) is indefinite. Check: \(Q(1, 1) = 6\) but \(Q(1, -1) = -2\). Positive coefficients did not make it positive. 2. Classify \(Q = 2x_1^2 + 2x_2^2 + 2x_3^2 - 2x_1x_2 - 2x_2x_3\). The matrix has 2 on the diagonal, \(-1\) next to it, and 0 in the corners. Its eigenvalues are \(2 - \sqrt{2} \approx 0.59\), 2 and \(2 + \sqrt{2} \approx 3.41\), all positive, so \(Q\) is positive definite. 3. A second check: \(Q = x_1^2 + (x_1 - x_2)^2 + (x_2 - x_3)^2 + x_3^2\), a sum of squares that is zero only when \(\mathbf{x} = \mathbf{0}\).
Shortcuts and the Cholesky factorisation
For a 2 × 2 symmetric matrix with rows \((a, b)\) and \((b, c)\), the determinant \(ac - b^2\) is the product of the eigenvalues and \(a + c\) is their sum. So the form is positive definite exactly when \(a > 0\) and \(ac - b^2 > 0\). Every positive definite matrix also factors as \(A = LL^T\), with \(L\) lower triangular and positive on its diagonal. For rows \((4, 2)\) and \((2, 5)\), \(L\) has rows \((2, 0)\) and \((1, 2)\). This Cholesky factorisation, published in 1924 after André-Louis Cholesky's death, needs about half the arithmetic of LU and is the standard way to solve systems with positive definite matrices.
Where definiteness matters
In multivariable calculus, the second-derivative test is a statement about a quadratic form. Near a critical point, a smooth function behaves like its Hessian quadratic form, built from the symmetric matrix of second partial derivatives. If the Hessian is positive definite the point is a local minimum, if negative definite a local maximum, and if indefinite a saddle point. In physics, the kinetic energy of a system of masses is a positive definite quadratic form in the velocities. In statistics, a covariance matrix is always positive semidefinite, because the variance of any weighted sum of the variables is \(\mathbf{w}^TS\mathbf{w} \ge 0\).
Common mistakes to avoid
First, putting the whole cross-term coefficient off the diagonal: \(4x_1x_2\) needs 2 in each off-diagonal place, not 4. Second, judging definiteness from the signs of the entries: the matrix with rows \((1, 2)\) and \((2, 1)\) has only positive entries yet is indefinite. Third, calling a form positive definite when one eigenvalue is zero: \(x_1^2\) on \(\mathbb{R}^2\) is only semidefinite, since \(Q(0, 1) = 0\). Fourth, mixing up the two sets of coordinates: \(\mathbf{y} = P^T\mathbf{x}\), not \(P\mathbf{x}\). Fifth, assuming the larger eigenvalue gives the longer ellipse axis, when it gives the shorter one.
Key takeaways
A quadratic form is Q(x) = xᵀAx with A symmetric: squares go on the diagonal, and each cross-term coefficient is split in half between two mirror positions. The orthogonal change of variable x = Py, with A = PDPᵀ, turns Q into λ₁y₁² + ⋯ + λₙyₙ², the principal axes theorem, with the eigenvectors as principal axes. The eigenvalues classify the form: all positive means positive definite, all negative means negative definite, mixed signs means indefinite, and zeros with otherwise matching signs mean semidefinite. In two variables, a level curve Q = c > 0 is an ellipse when both eigenvalues are positive and a hyperbola when their signs differ, with axes along the eigenvectors. Positive definite matrices have Cholesky factorisations A = LLᵀ and identify minima in the second-derivative test.
Flashcards
Flashcard
Q(x) = xᵀAx with A symmetric
Test your understanding
Question 1 of 5
What is the symmetric matrix of Q = 3x₁² + 8x₁x₂ − x₂²?
Question 2 of 5
A quadratic form has a symmetric matrix with eigenvalues 5, 0 and 2. How is it classified?
Question 3 of 5
Which statement about Q = x₁² + 4x₁x₂ + x₂² is true?
Question 4 of 5
After the change of variable x = Py, a form becomes Q = 9y₁² + 4y₂². What is the curve Q = 36?
Question 5 of 5
For which value of c is the matrix with rows (2, 3) and (3, c) positive definite?
