Linear Algebra › Section A · Linear equations in linear algebra › Topic A.7
Span and linear independence
Span asks what a set of vectors can build; linear independence asks whether every vector in the set is really needed. Together they decide how many directions a set of vectors truly covers, and both are settled by counting pivots.
Two questions about a set of vectors
Given vectors \(\mathbf{v}_1, \dots, \mathbf{v}_p\), two questions keep returning. First, which vectors can be built from them? That is the span, introduced in Vectors and vector equations. Second, is any vector in the set redundant, meaning it could be built from the others? That is the question of linear independence. A good set of building blocks reaches everything you need and wastes nothing. This topic gives a pivot test for each question and shows how the two fit together.
Span, revisited
Recall that \(\mathrm{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}\) is the set of all linear combinations \(c_1\mathbf{v}_1 + \dots + c_p\mathbf{v}_p\). Put the vectors as the columns of a matrix \(A\). Then \(\mathbf{b}\) is in the span exactly when \(A\mathbf{x} = \mathbf{b}\) is consistent, and the vectors span all of \(\mathbb{R}^n\) exactly when \(A\) has a pivot in every row, as The matrix equation Ax = b showed. So the span question is a question about pivots in rows. Independence, as you will see, is the matching question about pivots in columns.
Linear independence
A set \(\{\mathbf{v}_1, \dots, \mathbf{v}_p\}\) in \(\mathbb{R}^n\) is linearly independent if the vector equation
has only the trivial solution, with every weight zero. The set is linearly dependent if there are weights, not all zero, that make the combination equal \(\mathbf{0}\). Such an equation is called a linear dependence relation. Since the vector equation is the same as \(A\mathbf{x} = \mathbf{0}\) for the matrix \(A\) with these columns, the columns of \(A\) are independent exactly when \(A\mathbf{x} = \mathbf{0}\) has only the trivial solution, which happens exactly when every column of \(A\) is a pivot column.
Worked example: finding a dependence relation
Test \(\mathbf{v}_1 = (1, -1, 2)\), \(\mathbf{v}_2 = (2, 1, 1)\) and \(\mathbf{v}_3 = (4, -1, 5)\). 1. The matrix with these columns has rows \((1, 2, 4)\), \((-1, 1, -1)\) and \((2, 1, 5)\). 2. Row 2 plus row 1 gives \((0, 3, 3)\); row 3 minus 2 times row 1 gives \((0, -3, -3)\); adding these leaves a zero row. 3. Column 3 has no pivot, so \(x_3\) is free and the set is dependent. 4. The reduced rows \((1, 0, 2)\) and \((0, 1, 1)\) give \(x_1 = -2x_3\) and \(x_2 = -x_3\). 5. With \(x_3 = 1\): \(-2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = \mathbf{0}\), so \(\mathbf{v}_3 = 2\mathbf{v}_1 + \mathbf{v}_2\). Check: \(2(1, -1, 2) + (2, 1, 1)\) is \((4, -1, 5)\).
Sets of one or two vectors
A set containing one vector \(\mathbf{v}\) is independent exactly when \(\mathbf{v} \ne \mathbf{0}\), because \(x\mathbf{v} = \mathbf{0}\) forces \(x = 0\) unless \(\mathbf{v}\) is the zero vector. A set of two vectors is dependent exactly when one is a multiple of the other. For example, \((3, -6)\) and \((-1, 2)\) are dependent, since the first is \(-3\) times the second. Geometrically, two vectors are dependent exactly when they lie on one line through the origin. Beware: this multiple test works only for two vectors.
What dependence really means
A set of two or more vectors is linearly dependent exactly when at least one of its vectors is a linear combination of the others. More precisely, if the set is dependent and \(\mathbf{v}_1 \ne \mathbf{0}\), then some \(\mathbf{v}_j\) with \(j > 1\) is a combination of the vectors before it. The theorem does not say that every vector is a combination of the rest. In the set \((1, 0, 0)\), \((2, 0, 0)\), \((0, 1, 0)\), the second vector is twice the first, so the set is dependent, yet \((0, 1, 0)\) is not a combination of the other two. With three or more vectors, no two being multiples is not enough for independence.
Two shortcuts
Some sets are dependent without any calculation. If a set in \(\mathbb{R}^n\) has more vectors than each vector has entries, so \(p > n\), it is dependent: the \(n \times p\) matrix has at most \(n\) pivots, so some column has none and \(A\mathbf{x} = \mathbf{0}\) has a free variable. So any four vectors in \(\mathbb{R}^3\) are dependent. If a set contains the zero vector, it is dependent: give the zero vector weight 1 and the others weight 0. The converses fail: a set of three vectors in \(\mathbb{R}^3\) with no zero vector can still be dependent, as the worked example shows.
Span and independence together
If one vector in a set is a combination of the others, removing it does not change the span: any combination that used it can use its replacement instead. In the worked example, \(\mathrm{Span}\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\} = \mathrm{Span}\{\mathbf{v}_1, \mathbf{v}_2\}\), a plane through the origin. An independent set has no such slack, and every vector in its span can be written as a combination in only one way. In \(\mathbb{R}^3\), three vectors are independent exactly when they do not all lie in one plane through the origin, and then they span all of \(\mathbb{R}^3\).
Worked example: independent and spanning
Do \((1, 0, 1)\), \((1, 1, 0)\) and \((0, 1, 1)\) form an independent set that spans \(\mathbb{R}^3\)? 1. The matrix with these columns has rows \((1, 1, 0)\), \((0, 1, 1)\) and \((1, 0, 1)\). 2. Row 3 minus row 1 gives \((0, -1, 1)\). 3. Row 3 plus row 2 gives \((0, 0, 2)\). 4. The echelon form has a pivot in every column, so the set is independent. 5. It also has a pivot in every row, so the vectors span \(\mathbb{R}^3\). For a square matrix, a pivot in every column and a pivot in every row happen together, a link the topic The invertible matrix theorem develops.
Common mistakes to avoid
First, testing independence by looking for a pivot in every row; that tests spanning, while independence needs a pivot in every column. Second, using the multiple test on three or more vectors. Third, concluding from a dependent set that every vector is a combination of the others. Fourth, thinking that more vectors always means a bigger span; a dependent vector adds nothing. Fifth, deciding that \(p \le n\) vectors in \(\mathbb{R}^n\) must be independent; few vectors can still be dependent.
Key takeaways
A set is linearly independent when only the trivial combination gives 0, and dependent when a non-trivial dependence relation exists. The columns of A are independent exactly when Ax = 0 has only the trivial solution, that is, a pivot in every column; they span Rᵐ exactly when there is a pivot in every row. Dependence means some vector is a combination of the others. More than n vectors in Rⁿ, or any set containing 0, is dependent. Removing a dependent vector leaves the span unchanged.
Flashcards
Flashcard
All linear combinations of v₁, …, vₚ
Test your understanding
Question 1 of 5
Is the set {(2, −4), (−3, 6)} linearly independent?
Question 2 of 5
Four non-zero vectors in R³ are placed as the columns of a 3 × 4 matrix. What can you conclude?
Question 3 of 5
For which value of h are (1, 2, 3), (2, 5, 7) and (1, 1, h) linearly dependent?
Question 4 of 5
A set {v₁, v₂, v₃} is linearly dependent. Which statement must be true?
Question 5 of 5
Row reducing the matrix whose columns are v₁, v₂, v₃ gives pivots in columns 1 and 2 only. What follows?
