One might hope to improve the accuracy of the numerical solution to a
differential equation by decreasing the step size. This is a correct
strategy, but only up to a point. Then we run into limitations of
machine precision. In the extreme case, consider what happens in the
Herein lies another great advantage of higher order methods. They give high accuracy at larger step sizes, so we can push to greater accuracy, before we are limited by machine precision.
Stability is another important consideration in solving difference equations. Here we have considered only recursion relations. But even with these we run into trouble, if the step size is too large. For example, suppose in the simple ODE (1) is negative and the step size is chosen larger than . Then the solution (2) oscillates wildly, and its magnitude grows explosively-- not at all resembling the true exponentially decaying solution . We say the solution is unstable.
Clearly choosing a step size appropriate to the problem is one way to avoid problems with stability. But some recursion relations can develop instabilities regardless of how small the step size. The Euler and Runge Kutta methods we have described are designed to be stable, at least at a reasonably small step size.