Calculus › Section D · Applications of derivatives › Topic D.8

Newton's method

Some equations, like cos x = x, have no algebraic solution at all. Newton's method solves them anyway, using one tangent line after another to home in on the answer.

When algebra runs out

Every equation we have solved so far surrendered to factoring or a formula. But try solving \(\cos x = x\): no algebraic manipulation can isolate \(x\) here, and equations like it are common in engineering and physics. The Intermediate Value Theorem from B.4 can prove a solution exists, without finding it. Newton's method finds it, not exactly, but to as many decimal places as you need, using the function and its derivative. The idea is geometric: a smooth curve near a root looks like its tangent line, so keep replacing the curve with its tangent and walk to where each tangent crosses the axis.

Deriving the step

Suppose your current guess for a root of \(f\) is \(x_n\). The tangent line there is \(y = f(x_n) + f'(x_n)(x - x_n)\). Instead of hunting where the curve crosses the axis, ask where this tangent does: set \(y = 0\) and solve for \(x\). The answer is the next guess:

x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}

That is the whole method, one fraction per step. Repeat until the guesses stop changing or \(f(x_n)\) is acceptably small. Two things to notice: you need \(f'\), which is why this is an application of derivatives, and the whole scheme is guess-and-improve, so a sensible first guess matters.

Worked example: computing √2 by hand

Find \(\sqrt{2}\) as the positive root of \(f(x) = x^2 - 2\), with \(f'(x) = 2x\). Start at the reasonable guess \(x_0 = 1.5\). The update gives \(x_1 = 1.5 - \frac{1.5^2 - 2}{3} = 1.5 - \frac{0.25}{3} = \frac{17}{12} \approx 1.4167\). One more: \(f(\frac{17}{12}) = \frac{289}{144} - 2 = \frac{1}{144} \approx 0.00694\) and \(f'(\frac{17}{12}) = \frac{17}{6} \approx 2.8333\), so \(x_2 \approx 1.416667 - 0.002451 = 1.414216\). After two steps the guess is within 0.000003 of \(\sqrt{2} = 1.4142136\ldots\). Each step is one evaluation, one division and one subtraction, cheap enough to run by hand.

Why it converges so fast

When Newton's method works, it works startlingly well: near a simple root (one where \(f'\) is nonzero) the number of correct digits roughly doubles at each step. The √2 run showed it: \(x_0\) had one correct digit, \(x_1\) had three (1.41...), and \(x_2\) had six (1.41421...). The reason is that the tangent model is excellent near a root: the gap between curve and tangent grows like the square of the distance from the point of tangency, controlled by the second derivative. So the new error is roughly a constant times the square of the old one. An error of 0.001 becomes something like 0.000001 in one step. This quadratic convergence is why Newton-type methods are a standard tool inside scientific software.

Worked example: an equation algebra cannot solve

Solve \(\cos x = x\), with \(x\) in radians. There is no formula for the answer, so let \(f(x) = \cos x - x\), with \(f'(x) = -\sin x - 1\), and start at \(x_0 = 1\). Then \(f(1) = \cos 1 - 1 \approx -0.4597\) and \(f'(1) \approx -0.8415 - 1 = -1.8415\), so \(x_1 = 1 - \frac{-0.4597}{-1.8415} \approx 0.7504\). Repeat: \(f(0.7504) \approx -0.0190\) and \(f'(0.7504) \approx -1.6819\), giving \(x_2 \approx 0.7504 - 0.0113 = 0.7391\). The true root is \(0.7390851\ldots\), so after two steps we are within 0.00003 of it. Check by substituting: \(\cos(0.7391) \approx 0.7391\).

When Newton's method fails

The method has honest failure modes, and they all trace to the division by \(f'(x_n)\). If a step lands where the tangent is nearly flat, the quotient explodes and the next guess leaps far away, sometimes never coming back. Some starting guesses make the iterates cycle, bouncing between the same values forever. And a poorly chosen start may converge to a root you did not want, in a function with several. Most of these failures can be avoided: sketch \(f\), pick \(x_0\) close to the intended root and away from level stretches, and check that the guesses are settling down.

A procedure and a stop rule

In practice: choose \(x_0\) near the root (a sign change of \(f\), by the Intermediate Value Theorem, traps one in an interval); compute \(f\) and \(f'\); take the step \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\); repeat until the digits you need stop changing or \(|f(x_n)|\) falls below your tolerance. Compare with bisection, halving a bracketing interval again and again: bisection is guaranteed but slow, only halving the error at each step, while Newton is not guaranteed but converges quadratically. Many practical solvers combine the two, using the bracket for safety and Newton steps for speed.

Common mistakes to avoid

The classic slips: flipping the formula into \(x_n - \frac{f'(x_n)}{f(x_n)}\), which divides by the wrong rate; losing the minus sign, so each step walks away from the tangent intercept instead of towards it; applying the update with \(f'(x_n) = 0\) and dividing by zero; differentiating \(f\) wrongly, a slip that poisons every step; and stopping after one iteration out of optimism, or worse, assuming Newton's answer is exact. It is an approximation scheme: iterate until the values visibly stabilise, then verify by substituting back into \(f\).

Key takeaways

Newton's method trades the curve for its tangent and walks to the tangent's intercept: \(x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\), derived by setting the tangent line to zero. It converges quadratically near simple roots, roughly doubling the correct digits per step, which gave \(\sqrt{2}\) to within 0.000003 in two hand steps and solved \(\cos x = x\), an equation algebra cannot touch. It can fail where the tangent flattens, so sketch first and start close. Paired with the existence guarantee of the Intermediate Value Theorem, it is a practical answer to "solve \(f(x) = 0\)".

Flashcards

Flashcard

Newton iteration

x_next = x − f(x)/f′(x)

From the tangent line's x-intercept.

Test your understanding

Question 1 of 5

What is one Newton step from an approximation x_n?

Question 2 of 5

Using Newton's method on f(x) = x² − 2 with x₀ = 1.5, what is x₁?

Question 3 of 5

Solving cos x = x with f(x) = cos x − x starting from x₀ = 1 (radians) gives x₁ ≈ ?

Question 4 of 5

Newton's method can catastrophically fail a step when…

Question 5 of 5

Geometrically, one Newton step does what?

Standards: Beyond AP

← Previous topicNext topic →