Linear Algebra › Section H · The geometry of vector spaces › Topic H.4

Hyperplanes

A line divides the plane in two, and a plane divides space in two. Hyperplanes do the same job in any number of dimensions, which makes them the natural tool for describing boundaries, sorting data into classes and proving that two convex sets can be pulled apart.

Dividing space in two

In \(\mathbb{R}^2\) a line such as \(3x_1 - x_2 = 4\) splits the plane into the points where \(3x_1 - x_2 > 4\) and those where \(3x_1 - x_2 < 4\). In \(\mathbb{R}^3\) a plane does the same for space. Both are hyperplanes: flats of dimension \(n - 1\) in \(\mathbb{R}^n\), as defined in "Affine combinations". In four or more dimensions we cannot picture them, but the algebra is identical, and that is what makes hyperplanes useful. A spam filter that scores an email by a weighted sum of hundreds of features and compares it with a threshold is testing which side of a hyperplane the email lies on.

Linear functionals and the implicit description

A linear functional on \(\mathbb{R}^n\) is a linear transformation \(f\) from \(\mathbb{R}^n\) to \(\mathbb{R}\). Every one has the form \(f(\mathbf{x}) = \mathbf{n} \cdot \mathbf{x}\) for a fixed vector \(\mathbf{n}\). For a non-zero functional and any real \(d\), the set

[f : d] = \{\mathbf{x} \in \mathbb{R}^n : f(\mathbf{x}) = d\}

is a hyperplane, and every hyperplane arises this way. The vector \(\mathbf{n}\) is a normal vector: for any two points \(\mathbf{p}, \mathbf{q}\) of the hyperplane, \(\mathbf{n} \cdot (\mathbf{p} - \mathbf{q}) = d - d = 0\), so \(\mathbf{n}\) is perpendicular to every direction within it. The equation is not unique: \([f : d]\) and \([2f : 2d]\) are the same hyperplane.

Half-spaces

A hyperplane \(H = [f : d]\) determines two closed half-spaces,

\{\mathbf{x} : f(\mathbf{x}) \ge d\} \quad \text{and} \quad \{\mathbf{x} : f(\mathbf{x}) \le d\}

and two open half-spaces, using strict inequalities. Each half-space is convex, because \(f\) is linear: if \(f(\mathbf{p}) \ge d\) and \(f(\mathbf{q}) \ge d\), then for \(0 \le t \le 1\),

f((1 - t)\mathbf{p} + t\mathbf{q}) = (1 - t)f(\mathbf{p}) + tf(\mathbf{q}) \ge d

To decide which side a point is on, evaluate \(f\) there and compare with \(d\). The normal \(\mathbf{n}\) points into the side where \(f\) exceeds \(d\). The level sets \([f : d]\) for different \(d\) are parallel hyperplanes, stacked along \(\mathbf{n}\).

Worked example: a plane through three points

Find an implicit description of the plane through \(\mathbf{a} = (1, 2, 0)\), \(\mathbf{b} = (3, 1, 1)\), \(\mathbf{c} = (0, 1, 2)\). 1. Two directions within the plane: \(\mathbf{b} - \mathbf{a} = (2, -1, 1)\) and \(\mathbf{c} - \mathbf{a} = (-1, -1, 2)\). 2. A normal is perpendicular to both. Their cross product is \((-1, -5, -3)\), so take \(\mathbf{n} = (1, 5, 3)\). 3. Then \(d = \mathbf{n} \cdot \mathbf{a} = 1 + 10 + 0 = 11\), and the plane is \(x_1 + 5x_2 + 3x_3 = 11\). 4. Check the other points: \(3 + 5 + 3 = 11\) and \(0 + 5 + 6 = 11\). 5. The origin gives \(0 < 11\), so it lies in the open half-space \(x_1 + 5x_2 + 3x_3 < 11\).

Worked example: a hyperplane in four dimensions

There is no cross product in \(\mathbb{R}^4\), so find the normal as a null space. Take \(\mathbf{p}_1 = (1, 1, 0, 0)\), \(\mathbf{p}_2 = (2, 1, 1, 0)\), \(\mathbf{p}_3 = (1, 2, 0, 1)\), \(\mathbf{p}_4 = (0, 1, 1, 1)\). 1. Differences: \((1, 0, 1, 0)\), \((0, 1, 0, 1)\), \((-1, 0, 1, 1)\). 2. A normal \(\mathbf{n}\) must have zero dot product with each: \(n_1 + n_3 = 0\), \(n_2 + n_4 = 0\), \(-n_1 + n_3 + n_4 = 0\). 3. Substituting \(n_3 = -n_1\) and \(n_4 = -n_2\) into the third gives \(n_2 = -2n_1\); with \(n_1 = 1\), \(\mathbf{n} = (1, -2, -1, 2)\). 4. Then \(d = \mathbf{n} \cdot \mathbf{p}_1 = -1\), so the hyperplane is \(x_1 - 2x_2 - x_3 + 2x_4 = -1\). 5. Check \(\mathbf{p}_4\): \(0 - 2 - 1 + 2 = -1\).

Explicit and implicit descriptions

A hyperplane can be given implicitly, by one equation \(\mathbf{n} \cdot \mathbf{x} = d\), or explicitly, as a flat \(\mathbf{p} + \operatorname{Span}\{\mathbf{w}_1, \dots, \mathbf{w}_{n-1}\}\). To go from implicit to explicit, solve the single equation: for \(x_1 + 5x_2 + 3x_3 = 11\), the free variables \(x_2\) and \(x_3\) give

\mathbf{x} = (11, 0, 0) + x_2(-5, 1, 0) + x_3(-3, 0, 1)

To go from explicit to implicit, find \(\mathbf{n}\) as a non-zero vector orthogonal to every \(\mathbf{w}_i\), then set \(d = \mathbf{n} \cdot \mathbf{p}\), as in the two worked examples. The implicit form is best for testing a point, since one dot product decides it; the explicit form is best for generating points.

Separating hyperplanes

A hyperplane \(H = [f : d]\) separates sets \(A\) and \(B\) if one lies in the closed half-space \(f \le d\) and the other in \(f \ge d\). It separates them strictly if they lie in opposite open half-spaces, so neither touches \(H\). A key theorem: if \(A\) and \(B\) are non-empty, disjoint, convex and compact (closed and bounded), some hyperplane separates them strictly (one compact set and one closed set are in fact enough). In particular, two finite sets of points can be strictly separated exactly when their convex hulls do not meet. Compactness matters. The x-axis and the region \(y \ge 1/x\) with \(x > 0\) are disjoint, closed and convex, but neither is bounded, and no line strictly separates them; only the axis itself separates them, and it touches one set.

Worked example: separating two point sets

A classifier must separate \(A = \{(0, 0), (1, 0), (0, 1)\}\) from \(B = \{(3, 2), (2, 3), (3, 3)\}\). 1. Try \(f(\mathbf{x}) = x_1 + x_2\), since \(B\) lies further along the direction \((1, 1)\). 2. On \(A\), \(f\) takes the values 0, 1 and 1; on \(B\), it takes 5, 5 and 6. 3. Any \(d\) strictly between 1 and 5 works; choosing the midpoint gives \(H = [f : 3]\), the line \(x_1 + x_2 = 3\). 4. Then \(A\) lies in \(f < 3\) and \(B\) in \(f > 3\), so \(H\) separates them strictly. 5. A new point \((2, 0.5)\) gives \(f = 2.5 < 3\), so the classifier assigns it to \(A\)'s class. Support vector machines choose the separating hyperplane with the widest margin.

Common mistakes to avoid

First, confusing the normal vector with a direction in the hyperplane: \(\mathbf{n}\) is perpendicular to the hyperplane, so \((1, 5, 3)\) is not a direction within \(x_1 + 5x_2 + 3x_3 = 11\). Second, forgetting to compute \(d\) from a point, or using a direction vector instead of a point to find it. Third, thinking two equations differing by a scalar describe different hyperplanes; \([f : d]\) and \([cf : cd]\) are the same for any \(c \ne 0\). Fourth, multiplying an inequality by a negative number without reversing it, which swaps the half-spaces. Fifth, quoting the strict separation theorem when neither set is compact.

Key takeaways

A hyperplane in Rⁿ is a flat of dimension n − 1. Implicitly it is [f : d] = {x : f(x) = d} for a non-zero linear functional f(x) = n · x, where n is a normal vector perpendicular to every direction in the hyperplane. It splits Rⁿ into two closed (or open) half-spaces, f(x) ≥ d and f(x) ≤ d, each convex; evaluating f decides which side a point is on. Find n through given points with a cross product in R³, or as a null-space vector orthogonal to the differences in general, then set d = n · p. Disjoint, non-empty, compact convex sets can always be strictly separated by a hyperplane, the principle behind linear classifiers.

Flashcards

Flashcard

Hyperplane in Rⁿ

A flat of dimension n − 1

A line in R², a plane in R³.

Test your understanding

Question 1 of 5

Which equation describes the plane through (1, 0, 0), (0, 2, 0) and (0, 0, 3)?

Question 2 of 5

For the hyperplane 2x₁ − x₂ + 4x₃ = 5, which point lies in the open half-space where 2x₁ − x₂ + 4x₃ > 5?

Question 3 of 5

Which vector is normal to the hyperplane x₁ − 2x₂ − x₃ + 2x₄ = −1 in R⁴?

Question 4 of 5

Two finite sets of points A and B in R³ are given. When can a plane strictly separate them?

Question 5 of 5

A classifier uses f(x) = 3x₁ + x₂ and threshold d = 6, assigning class 1 when f(x) > 6. Which point is assigned to class 1?

← Previous topicNext topic →