[Pendulums, Time & Motion][Numerical Methods]Jan 4

The Price of Speed: Runge-Kutta vs Euler

How better algorithms let us simulate the future more accurately.

The Price of Speed: Runge-Kutta vs Euler

Act 1: Circa 1890s -> Bringing accuracy to the predictions

The city of Göttingen around this time was the place to be for intellectual experience in Europe—not an exaggeration to say the world [1].

Carl Runge, a man who combined a passion for gymnastics and piano with a dire need to solve the messy equations of solar spectroscopy [2]. In 1895, Runge broke from the "Pure Math" tradition by proposing that one could approximate a path not by solving it exactly, but by "sampling" slopes—a hiker's logic applied to light waves [3]. Despite this new method being genuinely useful, it was frowned upon by intellectuals as "dirty" and "imprecise"—something beneath the beauty of "pure mathematics." Runge's method improved upon Euler's approach by sampling the slope at two points for more accurate time-marching. His work was not ignored; he was later specifically hired at the University of Göttingen to bring this rebellious approach to challenge the "pure mathematics" camp at the University, by Felix Klein [4].

This applied rebellion was furthered in 1900 by the struggling tutor Karl Heun [5], who added a crucial middle step to the calculation, and finally perfected in 1901 by Wilhelm Kutta [6].

Wilhelm Kutta was a mathematician obsessed with glacier physics and wing lift [7]. He likely encountered Runge's 1895 journal article and Heun's improvement, and realized immense potential for further refinement. He utilized Taylor Series expansions [8] to find the precise four-step weighted average that canceled out mathematical error by a lot. You should realize at this point that there were no computers at this stage in the world and these greats were laying solid foundations for future computer algorithms.

✨ Special-Act: Taylor Series—The Foundation of Runge-Kutta

Taylor's series [8] lets us approximate a function by expanding it around a point:

y(t+dt)=y(t)+y′(t)dt+12y′′(t)dt2+16y′′′(t)dt3+⋯y(t + dt) = y(t) + y'(t)dt + \frac{1}{2}y''(t)dt^2 + \frac{1}{6}y'''(t)dt^3 + \cdots

Euler's method uses only the first derivative, while Runge-Kutta methods cleverly sample the slope at multiple points to capture higher-order terms, reducing error and improving accuracy.

Runge and Kutta provided the tools that allowed scientists like Max Planck and Einstein to move from theory to calculation [9]. The method survived the eventual tragic dismantling of the Göttingen community in 1933 (by idiotic Nazis) [10], living on as the foundational algorithm for modern spaceflight, weather prediction, and fluid dynamics [11]—a testament to a time when a physicist and a glacier-measuring mathematician decided that "close enough to the truth" was the highest form of genius

Act 2: The Details—Why Euler Fails and Why RK4 Succeeds

Euler’s method is a first-order approximation. Runge-Kutta (especially RK4) dramatically improves accuracy by sampling the slope at several points within each time step [12].

Equations

Euler:

yn+1=yn+f(yn,tn)⋅dty_{n+1} = y_n + f(y_n, t_n) \cdot dt

Runge-Kutta 4th order (RK4):

k1=f(yn,tn)k2=f(yn+dt2k1,tn+dt2)k3=f(yn+dt2k2,tn+dt2)k4=f(yn+dt k3,tn+dt)yn+1=yn+dt6(k1+2k2+2k3+k4)\begin{align*} k_1 &= f(y_n, t_n) \\ k_2 &= f(y_n + \frac{dt}{2}k_1, t_n + \frac{dt}{2}) \\ k_3 &= f(y_n + \frac{dt}{2}k_2, t_n + \frac{dt}{2}) \\ k_4 &= f(y_n + dt\,k_3, t_n + dt) \\ y_{n+1} &= y_n + \frac{dt}{6}(k_1 + 2k_2 + 2k_3 + k_4) \end{align*}

You can see that Euler's method is faster and simpler, but RK4 is far more accurate. Is this the end? Do we have everything needed for a predictable clockwork universe? The simple answer is no—the story continues, with more demons lurking in long-term predictions and sometimes in initial conditions.

Act 3: Finale - Euler vs RK4 Simulation

Below, we compare Euler and RK4 on the same pendulum. Adjust dt to see how error grows for each method.

Euler vs RK4 Comparison

Red: Euler | Green: RK4

Increase dt to see Euler diverge while RK4 stays stable longer.

Quiz: Integration Methods

Integration Methods Quiz

Why is Runge-Kutta 4th order (RK4) more accurate than Euler's method?
What is the main disadvantage of using a very large time step (dt) with Euler's method?
Which integration method is commonly used in professional simulation software?

This concludes the first series of articles in the anthology "Pendulums, Time and Motion." If you enjoy collections of scientific knowledge and walking through stories of people, discoveries, and interactive simulations, consider supporting the TEA (The Engineering Anthology) project. Get in touch with us for partnerships, sponsorships, or to support our mission.

References

  1. Reid, C. (1970). Hilbert. Springer-Verlag. Chapter on Göttingen's mathematical community. ISBN 978-0-387-04999-1.

  2. Runge, C. & Paschen, F. (1895). "Ueber die Serienspectren der Elemente." Annalen der Physik, 291(7), pp. 293-316. Wiley Online Library

  3. Runge, C. (1895). "Über die numerische Auflösung von Differentialgleichungen." Mathematische Annalen, 46, pp. 167-178. Original paper introducing Runge's method. SpringerLink

  4. Klein, F. & Runge, C. Their collaboration is documented in: Rowe, D.E. (1989). "Klein, Hilbert, and the Göttingen Mathematical Tradition." Osiris, 5, pp. 186-213. JSTOR

  5. Heun, K. (1900). "Neue Methoden zur approximativen Integration der Differentialgleichungen einer unabhängigen Veränderlichen." Zeitschrift für Mathematik und Physik, 45, pp. 23-38. Original paper introducing Heun's method.

  6. Kutta, W. (1901). "Beitrag zur näherungsweisen Integration totaler Differentialgleichungen." Zeitschrift für Mathematik und Physik, 46, pp. 435-453. Original paper presenting the classical RK4 method. Archive.org

  7. Kutta, W. (1902). "Auftriebskräfte in strömenden Flüssigkeiten." Illustrierte Aeronautische Mitteilungen, 6, pp. 133-135. The Kutta condition in aerodynamics.

  8. Taylor, B. (1715). Methodus Incrementorum Directa et Inversa. London. Archive.org

  9. Butcher, J.C. (2008). Numerical Methods for Ordinary Differential Equations, 2nd ed. Wiley. Chapter 2: "Runge-Kutta Methods." ISBN 978-0-470-72335-7.

  10. Siegmund-Schultze, R. (2009). Mathematicians Fleeing from Nazi Germany. Princeton University Press. ISBN 978-0-691-14041-4.

  11. Hairer, E., Nørsett, S.P., & Wanner, G. (1993). Solving Ordinary Differential Equations I: Nonstiff Problems, 2nd ed. Springer-Verlag. The standard reference for RK methods. ISBN 978-3-540-56670-0.

  12. Butcher, J.C. (1987). The Numerical Analysis of Ordinary Differential Equations: Runge-Kutta and General Linear Methods. Wiley. Chapter 3: "Order Conditions." ISBN 978-0-471-91046-0.

← Euler's Method: Simulating the FutureALL CHAPTERS