Differential Equations › Section B · First order differential equations › Topic B.10
Runge-Kutta and multistep methods
Euler's method uses one slope per step and pays for it with slow convergence. Sampling the slope more cleverly, or reusing slopes already computed, gives methods that are far more accurate for the same work, and these are the methods real software uses.
Better slopes, not just smaller steps
The topic "Euler's method and error" showed that Euler's method is first order: to gain one more correct digit you need about ten times as many steps. The weakness is that it uses only the slope at the start of each step, even though the slope changes across the step. Higher-order methods fix this in one of two ways. Runge-Kutta methods sample the slope at several points inside each step and combine them; multistep methods reuse slopes from earlier steps. Either way, the error falls much faster than \(h\) as the step shrinks.
Heun's method
Heun's method, also called the improved Euler method, first predicts the end of the step with Euler, then corrects using the average of the starting and predicted slopes:
When \(f\) depends on \(t\) alone, this is exactly the trapezoidal rule from the Calculus topic "Numerical integration: trapezoid and Simpson's rules". Heun's method is second order: its global error is roughly proportional to \(h^2\), so halving the step cuts the error by about a factor of 4. Each step costs two evaluations of \(f\).
The classical Runge-Kutta method
The classical fourth-order Runge-Kutta method, RK4, samples four slopes per step: at the start, twice at the midpoint, and at the end:
The weights 1, 2, 2, 1 are those of Simpson's rule, which is what RK4 reduces to when \(f\) depends on \(t\) alone. Its global error is roughly proportional to \(h^4\): halving \(h\) cuts the error by about 16.
Worked example: one step of each
Take \(y' = 2t - y\), \(y(0) = 1\), whose exact solution gives \(y(0.5) = 3e^{-0.5} - 1 \approx 0.81959\). Use one step of \(h = 0.5\). 1. Euler: \(y_1 = 1 + 0.5(-1) = 0.5\). 2. Heun: \(k_1 = -1\), predicted \(y = 0.5\), \(k_2 = f(0.5, 0.5) = 0.5\), so \(y_1 = 1 + 0.25(-0.5) = 0.875\). 3. RK4: \(k_1 = -1\), \(k_2 = f(0.25, 0.75) = -0.25\), \(k_3 = f(0.25, 0.9375) = -0.4375\), \(k_4 = f(0.5, 0.78125) = 0.21875\). Then
4. Errors: about 0.32, 0.055 and 0.0007. One RK4 step beats the others by a wide margin.
Worked example: equal work
A fair comparison counts evaluations of \(f\), the main cost. Approximate \(y(1) \approx 1.10364\) for the same problem using 8 evaluations each. 1. Euler with \(h = 0.125\), 8 steps: estimate 1.0308, error about 0.073. 2. Heun with \(h = 0.25\), 4 steps of 2 evaluations: estimate 1.1176, error about 0.014. 3. RK4 with \(h = 0.5\), 2 steps of 4 evaluations: estimate 1.10451, error about 0.0009. 4. Conclusion: for the same cost, RK4 is about 80 times more accurate than Euler here, and the advantage grows as more accuracy is demanded. That is why RK4 became the standard fixed-step method for problems without special difficulties.
Checking the order
The same experiment confirms the orders. Running Heun's method on this problem with \(h = 0.5\), 0.25 and 0.125 gives errors at \(t = 1\) of about 0.068, 0.014 and 0.0032: ratios between 4 and 5, close to the factor 4 expected of a second-order method. RK4 with the same steps gives about \(8.7 \times 10^{-4}\), \(4.4 \times 10^{-5}\) and \(2.5 \times 10^{-6}\): ratios of about 20 and 18, approaching the predicted 16 as \(h\) shrinks. Ratios like these are a practical test that a method has been coded correctly, since a programming slip often drops the observed order to 1.
Adaptive step sizes
A fixed step is wasteful: the solution may change rapidly in one place and barely at all elsewhere. Adaptive solvers estimate the error at every step and adjust \(h\) to keep it below a tolerance the user sets. The usual estimate comes from an embedded pair, two Runge-Kutta formulas of different orders that share most of their slope evaluations; their difference measures the error. The Dormand-Prince pair, of orders 5 and 4, is the formula behind MATLAB's ode45 and behind RK45, the default method of SciPy's solve_ivp. You choose tolerances, not step sizes, and the solver takes small steps only where they are needed.
Multistep methods
Instead of new intermediate slopes, a multistep method reuses slopes from previous steps, so each step needs only one new evaluation of \(f\). The two-step Adams-Bashforth method is
where \(f_n = f(t_n, y_n)\). It is second order. Because it needs \(y_1\) before it can start, the first step comes from a one-step method such as RK4. Implicit multistep formulas, the Adams-Moulton methods, are often paired with explicit ones: the explicit formula predicts, the implicit one corrects. These predictor-corrector pairs combine low cost with good accuracy.
Stiff problems and long simulations
Two situations need special methods. Stiff problems, common in chemical kinetics and circuits, combine very fast and very slow decay; explicit methods, RK4 included, must then take tiny steps to stay stable, as Euler did for \(y' = -20y\). Implicit methods such as the backward differentiation formulas (BDF) remain stable with large steps, and solvers offer them separately, such as BDF in solve_ivp. For orbits and molecular dynamics over very long times, ordinary methods slowly drift in energy. Symplectic integrators such as velocity Verlet respect the structure of mechanics and, with a fixed step, typically keep the energy error bounded even over millions of steps.
Common mistakes to avoid
First, evaluating \(k_2\) or \(k_3\) at the wrong time: both belong at \(t_n + \frac{h}{2}\), and \(k_4\) at \(t_n + h\). Second, using \(hk_1\) instead of \(\frac{h}{2}k_1\) inside \(k_2\). Third, getting the weights wrong: they are 1, 2, 2, 1 over 6. Fourth, comparing methods by number of steps rather than by evaluations of \(f\). Fifth, starting a multistep method without a separate first step. Sixth, using an explicit method on a stiff problem and blaming slow progress on the computer rather than the method.
Key takeaways
Higher-order methods sample or reuse slopes to cancel more of the Taylor error. Heun's method averages two slopes, is second order, and reduces to the trapezoidal rule. Classical RK4 combines four slopes with weights 1, 2, 2, 1 over 6, is fourth order, and reduces to Simpson's rule; halving \(h\) cuts its error by about 16. At equal cost RK4 vastly outperforms Euler. Adaptive solvers use embedded pairs such as Dormand-Prince to choose \(h\) automatically. Adams-Bashforth methods reuse past slopes. Stiff problems need implicit methods such as BDF, and long mechanical simulations use symplectic integrators.
Flashcards
Flashcard
yₙ₊₁ = yₙ + (h/2)(k₁ + k₂), with k₂ = f(tₙ + h, yₙ + hk₁)
Test your understanding
Question 1 of 5
Take one step of Heun's method with h = 0.2 for y′ = y, y(0) = 1. What is y₁?
Question 2 of 5
Halving the step size in Heun's method reduces its global error by roughly what factor?
Question 3 of 5
In classical RK4, which expression gives k₂?
Question 4 of 5
How many evaluations of f does one step of classical RK4 use?
Question 5 of 5
A chemical model is stiff. Which kind of method is the usual choice?
