Linear Algebra › Section A · Linear equations in linear algebra › Topic A.4

The matrix equation Ax = b

Multiplying a matrix by a vector packs a whole linear system into one short equation. That single idea links three ways of seeing the same problem and tells you when a system can be solved for every right-hand side.

One equation for a whole system

A system of three equations in three unknowns takes three lines to write. As a vector equation it takes one line, but the vectors still have to be listed. The matrix equation \(A\mathbf{x} = \mathbf{b}\) is shorter still, and it treats the coefficient matrix \(A\) as a single object that acts on the unknown vector \(\mathbf{x}\). This notation is used everywhere from circuit analysis to curve fitting, and it leads directly to the idea of a linear transformation later in this section.

Defining the product Ax

Let \(A\) be an \(m \times n\) matrix with columns \(\mathbf{a}_1, \dots, \mathbf{a}_n\), and let \(\mathbf{x}\) be in \(\mathbb{R}^n\). The product \(A\mathbf{x}\) is the linear combination of the columns of \(A\) using the entries of \(\mathbf{x}\) as weights:

A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \dots + x_n\mathbf{a}_n

It is defined only when the number of columns of \(A\) equals the number of entries in \(\mathbf{x}\), and the result lies in \(\mathbb{R}^m\). For example, with

A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & 3 & 4 \end{bmatrix}, \quad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}

the product is

A\mathbf{x} = 2\begin{bmatrix} 1 \\ 0 \end{bmatrix} - \begin{bmatrix} 2 \\ 3 \end{bmatrix} + 3\begin{bmatrix} -1 \\ 4 \end{bmatrix}

which works out to \((-3, 9)\).

The row-vector rule

For hand calculation there is a faster route to the same answer. Entry \(i\) of \(A\mathbf{x}\) is the sum of the products of the entries in row \(i\) of \(A\) with the matching entries of \(\mathbf{x}\). In the example above, row 1 gives \(1(2) + 2(-1) + (-1)(3) = -3\) and row 2 gives \(0(2) + 3(-1) + 4(3) = 9\). The column view explains what the product means; the row view is quicker to compute. The \(n \times n\) identity matrix \(I_n\), with 1s on the main diagonal and 0s elsewhere, satisfies \(I_n\mathbf{x} = \mathbf{x}\) for every \(\mathbf{x}\) in \(\mathbb{R}^n\).

Three views of one problem

If \(A\) has columns \(\mathbf{a}_1, \dots, \mathbf{a}_n\) and \(\mathbf{b}\) is in \(\mathbb{R}^m\), then three things have exactly the same solution set: the matrix equation \(A\mathbf{x} = \mathbf{b}\), the vector equation \(x_1\mathbf{a}_1 + \dots + x_n\mathbf{a}_n = \mathbf{b}\), and the linear system whose augmented matrix is \([\mathbf{a}_1 \ \cdots \ \mathbf{a}_n \ \mathbf{b}]\). You may move freely between them. As a consequence, \(A\mathbf{x} = \mathbf{b}\) has a solution exactly when \(\mathbf{b}\) is a linear combination of the columns of \(A\), that is, when \(\mathbf{b}\) lies in the span of those columns.

When every b has a solution

For an \(m \times n\) matrix \(A\), the following statements are either all true or all false. 1. For each \(\mathbf{b}\) in \(\mathbb{R}^m\), the equation \(A\mathbf{x} = \mathbf{b}\) has a solution. 2. Each \(\mathbf{b}\) in \(\mathbb{R}^m\) is a linear combination of the columns of \(A\). 3. The columns of \(A\) span \(\mathbb{R}^m\). 4. \(A\) has a pivot position in every row. Statement 4 is about the coefficient matrix \(A\), not the augmented matrix. If an echelon form of \(A\) has a zero row, some right-hand side will put a non-zero number at the end of that row, making the system inconsistent. Since there are at most \(n\) pivots, a matrix with more rows than columns can never satisfy statement 4.

Worked example: testing every right-hand side

Let \(A\) have columns \((1, 2, -1)\), \((2, 3, 1)\) and \((3, 5, 0)\). Is \(A\mathbf{x} = \mathbf{b}\) solvable for every \(\mathbf{b} = (b_1, b_2, b_3)\)? 1. Row reduce \([A \ \mathbf{b}]\). Row 2 minus 2 times row 1 gives \((0, -1, -1, b_2 - 2b_1)\), and row 3 plus row 1 gives \((0, 3, 3, b_3 + b_1)\). 2. Row 3 plus 3 times row 2 gives \((0, 0, 0, -5b_1 + 3b_2 + b_3)\). 3. \(A\) has no pivot in row 3, so the answer is no. 4. A solution exists only when \(-5b_1 + 3b_2 + b_3 = 0\). The columns span a plane through the origin, not all of \(\mathbb{R}^3\). Check: each column satisfies the condition, for instance \(-5 + 6 - 1 = 0\).

Linearity of the product

For an \(m \times n\) matrix \(A\), vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^n\), and a scalar \(c\):

A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}
A(c\mathbf{u}) = c(A\mathbf{u})

Both follow from the column definition, because the weights on each column simply add or scale. These two rules are what make \(A\mathbf{x}\) so well behaved. They explain the structure of solution sets in the next topic, and they are the defining property of a linear transformation. For example, if \(A\mathbf{u} = (1, 0)\) and \(A\mathbf{v} = (0, 2)\), then \(A(3\mathbf{u} + \mathbf{v}) = (3, 2)\) without knowing \(A\) at all.

Worked example: a parabola through three points

Find \(y = c_0 + c_1x + c_2x^2\) through \((-1, 6)\), \((1, 2)\) and \((2, 3)\). 1. Each point gives one linear equation in the unknown coefficients, so \(A\mathbf{c} = \mathbf{y}\) with

A = \begin{bmatrix} 1 & -1 & 1 \\ 1 & 1 & 1 \\ 1 & 2 & 4 \end{bmatrix}, \quad \mathbf{y} = \begin{bmatrix} 6 \\ 2 \\ 3 \end{bmatrix}

2. Row 2 minus row 1 gives \(2c_1 = -4\), so \(c_1 = -2\). 3. Then rows 2 and 3 give \(c_0 + c_2 = 4\) and \(c_0 + 4c_2 = 7\), so \(c_2 = 1\) and \(c_0 = 3\). 4. The parabola is \(y = 3 - 2x + x^2\). Check: at \(x = 2\), \(3 - 4 + 4 = 3\).

Sensitive systems

Some systems are ill-conditioned: a tiny change in the data causes a large change in the solution. The system \(x + y = 2\), \(x + 1.001y = 2\) has solution \((2, 0)\). Change the second right-hand side to 2.001 and the solution jumps to \((1, 1)\). The two lines are nearly parallel, so a small shift moves their crossing point a long way. Software reports a condition number that measures this sensitivity. Ill-conditioning belongs to the problem, not to the method, so no clever row reduction removes it; it warns that measured data may not pin the answer down.

Common mistakes to avoid

First, multiplying when the sizes do not match: \(A\mathbf{x}\) needs as many entries in \(\mathbf{x}\) as \(A\) has columns. Second, combining the rows of \(A\) with the weights instead of the columns. Third, checking for a pivot in every row of the augmented matrix \([A \ \mathbf{b}]\) when the theorem is about \(A\) itself. Fourth, expecting the columns of a matrix with more rows than columns to span \(\mathbb{R}^m\); they never can. Fifth, assuming a solution computed from noisy data is reliable without asking how well conditioned the system is.

Key takeaways

Ax is the combination x₁a₁ + … + xₙaₙ of the columns of A, defined when x has as many entries as A has columns; each entry is also a row of A times x. The matrix equation, the vector equation and the augmented system share one solution set, so Ax = b is solvable exactly when b is in the span of the columns. Ax = b has a solution for every b exactly when A has a pivot in every row. A(u + v) = Au + Av and A(cu) = cAu. Ill-conditioned systems magnify small errors in the data.

Flashcards

Flashcard

Ax (column view)

x₁a₁ + x₂a₂ + … + xₙaₙ, a combination of the columns of A

The entries of x are the weights.

Test your understanding

Question 1 of 5

A has rows (2, −1), (0, 3) and (1, 4), and x = (3, 2). What is Ax?

Question 2 of 5

A is a 4 × 3 matrix. Can Ax = b have a solution for every b in R⁴?

Question 3 of 5

Which statement is equivalent to "the columns of the m × n matrix A span Rᵐ"?

Question 4 of 5

Au = (1, 2) and Av = (3, −1). What is A(2u − v)?

Question 5 of 5

Changing one right-hand side of a system from 2 to 2.001 moves its solution from (2, 0) to (1, 1). What does this show?

← Previous topicNext topic →