Matrix equations and applications
Writing a system as Ax = b connects equations, linear combinations, and matrices. This viewpoint, and numerical practice, make systems a powerful modelling tool.
Matrix-vector products
The product \(A\mathbf{x}\) of an m x n matrix A and a vector \(\mathbf{x}\) in \(\mathbb{R}^n\) is the linear combination of A's columns weighted by the entries of \(\mathbf{x}\): \(A\mathbf{x} = x_1\mathbf{a}_1 + \dots + x_n\mathbf{a}_n\). Equivalently, the ith entry is the dot product of row i with \(\mathbf{x}\). Both views are useful.
Three equivalent views
The system, the vector equation \(x_1\mathbf{a}_1 + \dots + x_n\mathbf{a}_n = \mathbf{b}\), and the matrix equation \(A\mathbf{x} = \mathbf{b}\) all have the same solutions. So \(A\mathbf{x} = \mathbf{b}\) is solvable exactly when \(\mathbf{b}\) lies in the span of A's columns, connecting equation-solving to the geometry of spans.
When every b works
For an m x n matrix A, these statements are equivalent: \(A\mathbf{x} = \mathbf{b}\) has a solution for every \(\mathbf{b}\) in \(\mathbb{R}^m\); the columns of A span \(\mathbb{R}^m\); A has a pivot in every row. A 3 x 2 matrix can never satisfy them, since two columns cannot span \(\mathbb{R}^3\).
Linearity of the product
Matrix-vector multiplication is linear: \(A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}\) and \(A(c\mathbf{u}) = cA\mathbf{u}\). This property is why solution sets have the particular-plus-homogeneous structure, and it makes A a linear transformation, as a later unit develops.
Applications
Linear systems model electrical circuits (Kirchhoff's laws give one equation per node or loop), economic input-output models (Wassily Leontief's work, which earned the 1973 Nobel Prize in Economics), structural analysis of trusses, traffic flow, and computer graphics. Fitting a polynomial through given points also leads to a linear system for its coefficients.
Numerical solution
Computers solve systems with millions of unknowns. Partial pivoting, choosing the largest available pivot in each column, reduces rounding errors. Gaussian elimination on an n x n system takes on the order of n^3/3 multiplications. Very large sparse systems, with mostly zero entries, use specialised direct methods or iterative methods such as conjugate gradients.
Conditioning
Some systems are ill-conditioned: tiny changes in the data cause large changes in the solution, as when two equations describe nearly parallel lines. The condition number measures this sensitivity. Ill-conditioning is a property of the problem, not the method, so engineers check condition numbers before trusting computed answers.
Worked example
Find the parabola \(y = a + bx + cx^2\) through (0, 1), (1, 2), and (2, 5). The conditions give \(a = 1\), \(a + b + c = 2\), and \(a + 2b + 4c = 5\). Substituting \(a = 1\): \(b + c = 1\) and \(2b + 4c = 4\). Then \(2(1 - c) + 4c = 4\), so \(c = 1\) and \(b = 0\). The parabola is \(y = 1 + x^2\), which indeed passes through all three points.
Key takeaways
Ax is the combination of A's columns weighted by x, or row-by-x dot products. A system, its vector equation, and Ax = b share solutions, so Ax = b is solvable exactly when b is in the column span. Ax = b is solvable for every b exactly when A has a pivot in every row. Matrix-vector products are linear. Systems model circuits, Leontief input-output economies (1973 Nobel Prize), structures, traffic, graphics, and curve fitting. Computers use partial pivoting, about n^3/3 operations for elimination, and iterative methods for huge sparse systems. Condition numbers measure sensitivity.
Flashcard
Combination of columns weighted by x
Check your understanding
For A = [[1, 2], [3, 4]] and x = (2, -1), what is Ax?
