← Back to Linear Algebra

Vectors and their operations

Linear algebra is the mathematics of vectors and the transformations between them. It underpins physics, computer graphics, data science, and machine learning.

What a vector is

A vector in \(\mathbb{R}^n\) is an ordered list of n real numbers, written as a column such as \(\mathbf{v} = (3, -1, 2)\). Geometrically, vectors in \(\mathbb{R}^2\) and \(\mathbb{R}^3\) are arrows with length and direction, drawn from the origin to a point. In applications, a vector might list a pixel's colour values, a customer's purchases, or the forces on a bridge joint.

Addition and scalar multiplication

Vectors add componentwise: \((1, 2) + (3, -5) = (4, -3)\), which geometrically places arrows head to tail. Multiplying by a scalar c scales every component: \(2(1, -3) = (2, -6)\), stretching the arrow by 2; a negative scalar reverses direction. These two operations obey familiar rules: commutativity, associativity, distributivity, a zero vector, and additive inverses.

Linear combinations

A linear combination of vectors \(\mathbf{v}_1, \dots, \mathbf{v}_k\) is any sum \(c_1\mathbf{v}_1 + \dots + c_k\mathbf{v}_k\) with scalar weights. Linear combinations are the central idea of the subject: asking whether a vector can be built from others, and with which weights, leads to systems of equations, spans, bases, and matrix multiplication.

Length and the dot product

The dot product of \(\mathbf{u}\) and \(\mathbf{v}\) is \(\mathbf{u} \cdot \mathbf{v} = u_1v_1 + \dots + u_nv_n\). The length, or norm, is \(\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}}\); for \((3, 4)\) it is 5. The dot product also measures angle: \(\mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\|\|\mathbf{v}\|\cos\theta\). Vectors are orthogonal (perpendicular) exactly when their dot product is zero.

Unit vectors and distance

A unit vector has length 1; dividing any non-zero vector by its length normalises it: \(\frac{(3, 4)}{5} = (0.6, 0.8)\). The standard unit vectors \(\mathbf{e}_1, \dots, \mathbf{e}_n\) point along the coordinate axes. The distance between points is the length of their difference: \(\|\mathbf{u} - \mathbf{v}\|\), generalising Pythagoras's theorem to any dimension.

Two key inequalities

The Cauchy-Schwarz inequality states \(|\mathbf{u} \cdot \mathbf{v}| \le \|\mathbf{u}\|\|\mathbf{v}\|\), which guarantees that the cosine formula always gives a valid angle. The triangle inequality states \(\|\mathbf{u} + \mathbf{v}\| \le \|\mathbf{u}\| + \|\mathbf{v}\|\): one side of a triangle is never longer than the other two together. Both hold in every dimension.

Vectors in applications

In machine learning, documents, images, and words become high-dimensional vectors, and the cosine of the angle between them measures similarity. In physics, forces and velocities add as vectors. In graphics, positions, directions, and colours are vectors manipulated millions of times per second by GPUs designed for exactly this arithmetic.

Worked example

Let \(\mathbf{u} = (1, 2, 2)\) and \(\mathbf{v} = (2, 0, -1)\). Then \(\mathbf{u} + \mathbf{v} = (3, 2, 1)\) and \(3\mathbf{u} - \mathbf{v} = (1, 6, 7)\). The dot product is \(2 + 0 - 2 = 0\), so they are orthogonal. Their lengths are \(\|\mathbf{u}\| = 3\) and \(\|\mathbf{v}\| = \sqrt{5}\), and since they are perpendicular, \(\|\mathbf{u} + \mathbf{v}\| = \sqrt{9 + 5} = \sqrt{14}\), as Pythagoras predicts.

Key takeaways

Vectors in R^n are ordered lists of numbers, pictured as arrows. They add componentwise and scale by scalars, obeying the usual algebraic rules. Linear combinations, weighted sums of vectors, are the central idea of linear algebra. The dot product gives length, angle, and orthogonality (dot product zero). Unit vectors have length 1; distance is the length of a difference. Cauchy-Schwarz and the triangle inequality hold in every dimension. Vectors represent data, forces, and graphics quantities everywhere.

Flashcard

Vector in R^n

Ordered list of n real numbers

Column.

Check your understanding

What is the dot product of (2, -1, 3) and (4, 5, -1)?

Next lesson →