100% found this document useful (1 vote)

207 views330 pages

Comp Physics With Worked Examples

The document is a comprehensive guide on computational physics authored by Michael Bestehorn, detailing various methods and applications in the field, including examples in FORTRAN and MATLAB. It covers topics such as nonlinear maps, dynamical systems, ordinary and partial differential equations, and Monte Carlo methods, aiming to bridge theoretical physics with practical computational techniques. The book serves as a resource for understanding how to model and solve physical problems using computational approaches.

Uploaded by

Guilherme Matheus

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

100% found this document useful (1 vote)

207 views330 pages

Comp Physics With Worked Examples

Uploaded by

Guilherme Matheus

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 330

Michael Bestehorn

Computational Physics
Also of Interest
Computational Physics. Mit Beispielen in Fortran und Matlab
Michael Bestehorn, 2016
ISBN 978-3-11-037288-5, e-ISBN (PDF) 978-3-11-037303-5,
e-ISBN (EPUB) 978-3-11-037304-2

Biomedical Imaging. Principles of Radiography, Tomography and

Medical Physics
Tim Salditt, Timo Aspelmeier, Sebastian Aeffner, 2017
ISBN 978-3-11-042668-7, e-ISBN (PDF) 978-3-11-042669-4,
e-ISBN (EPUB) 978-3-11-042351-8

Dynamics of Solid Structures. Methods using Integrodifferential

Relations
Georgy Viktorovich Kostin, Vasily V. Saurin, 2017
ISBN 978-3-11-051623-4, e-ISBN (PDF) 978-3-11-051644-9,
e-ISBN (EPUB) 978-3-11-051625-8

Computational Physics of Electric Discharges in Gas Flows

Sergey T. Surzhikov, 2012
ISBN 978-3-11-027033-4, e-ISBN (PDF) 978-3-11-027041-9

Computational Sciences
Ponnadurai Ramasami (Ed.), 2017
ISBN 978-3-11-046536-5, e-ISBN (PDF) 978-3-11-046721-5,
e-ISBN (EPUB) 978-3-11-046548-8
Michael Bestehorn

Computational
Physics

|
With Worked Out Examples in FORTRAN® and MATLAB®
Physics and Astronomy Classiﬁcation Scheme 2010
05.10.Ln, 05.45.-a, 05.65.+b, 07.05.Mh, 44.05.+e, 45.10.-b, 47.11.-j, 47.85.-g

Author
Prof. Dr. Michael Bestehorn
BTU Cottbus-Senftenberg
Dept. of Statistical Physics and Nonlinear Dynamics
Erich-Weinert-Str. 1
03046 Cottbus
Germany
bestehorn@b-tu.de

ISBN 978-3-11-051513-8
e-ISBN (PDF) 978-3-11-051514-5
e-ISBN (EPUB) 978-3-11-051521-3

Library of Congress Control Number: 2018934247

Bibliographic information published by the Deutsche Nationalbibliothek

The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliograﬁe;
detailed bibliographic data are available on the Internet at http://dnb.dnb.de.

© 2018 Walter de Gruyter GmbH, Berlin/Boston

Cover image: Michael Bestehorn
Typesetting: le-tex publishing services GmbH, Leipzig
Printing and binding: CPI books GmbH, Leck
♾ Printed on acid-free paper
Printed in Germany

3.2 Fixed points and instabilities | 43

4 Ordinary differential equations I | 58

4.6.3 Lyapunov exponents | 98

5 Ordinary differential equations II | 117

6 Partial differential equations I, basics | 144

6.3 Alternative discretization methods | 173

7 Partial differential equations II, applications | 193

8 Monte Carlo methods (MC) | 240

A Matrices and systems of linear equations | 270

B Program library | 284

C Solutions of the problems | 290

D README and a short guide to FE-tools | 310

Index | 317
1 Introduction
1.1 Goal, contents, and outline

The present text tries to show, by means of examples coming from different corners
of physics, how physical and mathematical questions can be answered using a com-
puter. Nowadays, theoretical physicists are mainly concerned with two subjects: one
is the devising of models, normally in the form of differential equations, ordinary or
partial, while the other one is their approximate solution, normally with the help of
larger or smaller computers. In a certain way, computational physics can be seen as
the link between theory and experiment. The often involved equations need a numeri-
cal treatment, usually in connection with certain simplifications and approximations.
Once the code is written, the experimental part begins. A computer on which, for in-
stance, a turbulent pipe flow is simulated, can to some extent be regarded as a test
station with easy access to output values such as flow rates, velocities, temperatures,
and also to input parameters like material constants or geometric boundary condi-
tions.
Problems from classical mechanics normally rely on ordinary differential equa-
tions, while those of electrodynamics, quantum mechanics, continuum mechanics,
or hydrodynamics are formulated by partial differential equations, which are often
nonlinear.
This is neither a numeric book, nor a tutorial for a special computer language for
which we refer to the literature [1]. On the other hand, most of the discussed algorithms
are realized in FORTRAN. FORTRAN is an acronym composed of FORmula TRANslator.
This already indicates that it is more appropriate and convenient for solving problems
originating from theoretical physics or applied mathematics, like managing complex
numbers or solving differential equations.
During the past 10 years, the software package MATLAB has enjoyed more and
more popularity in research as well as in teaching [2]. The notion is formed as another
acronym from MATrix LABoratory. Thus MATLAB shows its strengths in the interac-
tive treatment of problems resulting from vector or matrix calculus, but other prob-
lems related to differential equations can also be tackled. Programming is similar to
FORTRAN and for beginners, many things are easier to implement and to handle. In
addition, MATLAB captivates with its comfortable and convenient graphic output fea-
tures.
MATLAB is devised as a standalone system. It runs under Windows and Linux. No
additional text editor, compiler, or software libraries are needed. However, contrary
to FORTRAN, a MATLAB code once written is not compiled to an effective machine
code but rather interpreted line by line. Processes may thus become incredibly slow.
MATLAB may have some advantages in program development and testing, but if in the
end a fast and effective code is desired (which should usually be the case), a compiler

https://doi.org/10.1515/9783110515145-001
2 | 1 Introduction

language is inevitably preferred. Nevertheless we shall give some examples also in

MATLAB, mainly those based on matrix calculation and linear algebra.
In the meantime, highly developed computer algebra systems exist and can be
used to check the formulas derived in the book, but we do not go into this matter.
The motto of the book could read ‘Learning by Doing.’ Abstract formulations are
avoided as much as possible. Sections where theory dominates are followed by an
example or an application.
The chapters of course can be read in sequence, albeit not necessarily. If someone
is interested for instance in partial differential equations, it is possible to start with
Chapter 6 and consult Chapters 3-5 for potential issues. The same is true for the statistic
part, Chapter 8. It should be understandable also without knowledge of the preceding
chapters. Depending on the subject, certain basic knowledge in (theoretical) physics
is required. For Chapters 4, 5 and 7, classical mechanics and a little bit of quantum
mechanics should be at hand, while for Chapter 7, additional basics in hydrodynam-
ics won’t hurt. In Chapter 8 we presume a certain familiarity with the foundations of
statistical mechanics.
In any case, it is recommended to have a computer or notebook running nearby
and, according to the motto, experiment with and check the presented routines im-
mediately while reading.
The topics up to Chapter 4, and to some extent also in Chapter 5, can be categorized
under the notion low-dimensional systems, problems that can be described with only a
few variables. Besides iterative maps, applications from Newton’s classical mechanics
come to mind, for instance, the computations of planetary orbits or the motion of a
pendulum (Figure 1.1). Naturally for this case, ordinary differential equations play a

–2

–2 0 2

Fig. 1.1: Time evolution of low-dimensional systems from Chapter 4. Left: two-planet problem with
gravitational interaction. The orbit of the inner planet (red) is unstable and the planet is thrown out
of the system. Right: deterministic chaos for the driven pendulum, Poincaré section.
1.1 Goal, contents, and outline | 3

0.8

0.6

0.4

0.2

0
0 0.2 0.4 0.6 0.8 1

Fig. 1.2: Numerical solutions of ﬁeld equations, see Chapter 7. Left: time-dependent Schrödinger
equation, probability density, and center of mass of an electron in a constant magnetic ﬁeld.
Right: Hydrodynamic vortex for the driven cavity.

central role. For three dimensions and more (or better: degrees of freedom), chaotic
behavior becomes possible, if not the rule. Large parts of Chapters 2-4 are dedicated
to the so-called deterministic chaos.
Starting with Chapter 5 high-dimensional (or inﬁnite-dimensional) systems come
into the focus, normally described by ﬁeld theories in the form of partial differential
equations. These can be linear as is very often seen in quantum mechanics or electro-
dynamics, but also nonlinear as in hydrodynamics (Figure 1.2) or in diffusion-driven
macroscopic pattern formation (Figure 1.3). Caused by nonlinearities, patterns may
emerge on most diverse spatio-temporal scales, the dynamics becoming chaotic or
turbulent (Figure 1.4).

Fig. 1.3: Macroscopic pattern formation for the

example of a reaction-diffusion system, time
series. Out of a large-scale oscillating solution,
nuclei emerge spontaneously followed by the
generation of small-scaled Turing structures.
The numerical integration of the Brusselator
equations is shown in Chapter 7.
4 | 1 Introduction

Fig. 1.4: Two-dimensional ﬂow ﬁeld showing

turbulence, time series. Due to a vertical tem-
perature gradient, an initially stable layer be-
comes unstable (top). Warmer air (red) rises
in the form of ‘plumes’ in the colder air (blue)
(from Chapter 7).

Fig. 1.5: Monte Carlo simulation of a gas comprised of 1000 particles with hardcore rejection and
long range attraction from Chapter 8. Temperature decreases from left to right, and a phase transi-
tion in a condensed state can be detected.

Another chapter is devoted to Monte Carlo methods. Here, randomness plays a key
role in the determination of equilibrium states. Using Monte Carlo simulations, phase
transitions can be studied without resorting to differential equations (Figure 1.5). The
application on the Ising model of spatially arranged spins is a prominent example.

1.2 The environment required for program development

1.2.1 Operating system

We shall discuss programs mainly written in FORTRAN, but also in MATLAB. MATLAB
is conceived as its own environment, where codes can be written, developed, and
run without leaving the MATLAB application. The operating system of the perform-
ing computer plays a minor role, so Windows can be used as well. On the other hand,
writing and developing FORTRAN codes is quite different. One needs a compiler, a
1.2 The environment required for program development | 5

text editor, and a convenient ﬁle system. For more ambitious applications, software
libraries like LAPACK should be at hand. Also a graphic library is important to have
at least the possibility to plot results in a pleasant and appealing way. If one wants
to have all these possibilities preferably free of charge, the only possibility is to do it
under Linux.
Thus, we assume the reader has a Linux operating system running where GNU
FORTRAN 95 is installed, together with the important libraries. A certain code written
with a text editor, e.g., emacs or xemacs, can be compiled via

$ gfortran program.f

to a binary ﬁle named a.out. That one can be executed with

$ ./a.out

This of course is the most simple case where no further libraries are involved.

1.2.2 Software packages

Besides a compiler, e.g., gfortran from GNU, we shall need external routines to solve
standard problems, mainly from the LAPACK-Library (Linear Algebra PACKage).
LAPACK can be also installed for free via the internet [4]. If the source code resorts to
LAPACK routines, one should use the compiler options -l and -L:

gfortran program.f -L/usr/lib -llapack

The option -L/. . . points to the path where the following library is located on the com-
puter.

1.2.3 Graphics

For graphic output we employ the package PGPLOT [5]. All ﬁgures in this ﬁrst chapter
and most of the book are generated with PGPLOT. Of course the reader can use other
packages if available, but PGPLOT provides all necessary basic tools and is also free of
charge. The compiler options for a source that relies both on LAPACK and GNUPLOT
should look like this:

gfortran program.f -L/usr/lib -llapack -lpgplot

if the PGPLOT-Library is installed in /usr/lib.

6 | 1 Introduction

1.2.4 Program development and a simple script

Program development can be considered as an iterative process. The desired result is

approached slowly and reached, if at all, after N iterations. One iteration step is built
up of the following points:

(1) editing, (2) compiling, (3) executing, (4) go back to (1)

In (1), an ASCII ﬁle is generated using a text editor. This ﬁle is the source and usually
has the extension .f, say, name.f. The editor is invoked, e.g., emacs by

$ emacs name.f

After finishing the first round of editing the source file, compiling and possible linking
with libraries (2) can be achieved by a script file

gfortran -O1 $1.f -o $1 -L/usr/lib -llapack -lpgplot

Let us call the script containing the above line ’make_f95’. Before executing the ﬁle, it
must receive the attribute executable, which is done by

$ chmod u+x make_f95

Running the script with

$ ./make_f95 name

initializes the variable $1 with name. Further options (optimization, etc.) of the
FORTRAN compiler can be found in the built-in manual by

$ man gfortran

The option -o transfers the name in $1 to the created binary executable, instead of
calling it a.out (default).
(3) If the compiler passes without error messages, one ﬁnds in the working directory
a binary ﬁle, here name, that can simply be executed with

$ ./name
1.3 A ﬁrst example – the logistic map | 7

1.3 A ﬁrst example – the logistic map

1.3.1 Map

To inspire the reader a bit, we conclude this chapter with a numerically very simple
example. We study the map

x n+1 = ax n (1 − x n ) , n = 0, 1, 2. . . 0≤a≤4, 0 ≤ x0 ≤ 1 (1.1)

which is known as the logistic map. In this way we can also demonstrate the usage of
the graphic package PGPLOT.
The recursion rule (1.1) can be considered a simple model of the temporal evolu-
tion of a certain population if x n denotes the population density at time (in the year)
n. The term ax n alone leads to an exponential increase if a > 1, otherwise leading to
the extinction of the species. For a > 1, the nonlinearity −ax2n restricts the growth due
to lack of resources (limited foods) and leads to a saturation. The constraint 0 ≤ a ≤ 4
originates from the condition 0 ≤ x n ≤ 1 for all x n . Depending on a, we have one or
two stationary solutions, namely
(1) (2)
xs = 0 for all a, xs = 1 − 1/a for 1<a≤4, (1.2)

which are computed from

(i) (i)
xs = f(xs ) (1.3)
with the abbreviation
f(x) = ax(1 − x) . (1.4)
(2)
However, shown by a linear stability analysis, the nontrivial solution becomes xs
unstable if a > 3. To this end we examine the behavior of an inﬁnitesimal deviation
(2)
|ϵ0 | ≪ 1 of xs
(2)
x n = xs + ϵ n (1.5)
under iteration. Inserting (1.5) into (1.1) yields (Taylor expansion)
(2) (2) (2) (2)
xs + ϵ n+1 = f(xs + ϵ n ) = f(xs ) + f 󸀠 (xs ) ϵ n + O(ϵ2n ) (1.6)

or, considering only linear terms in ϵ n

(2)
ϵ n+1 = f 󸀠 (xs ) ϵ n . (1.7)
(2)
The small deviation ϵ0 will grow during iteration if |f 󸀠 (xs )| > 1, which leads with
(1.4) to the two regions of instability

a<1, a>3.
8 | 1 Introduction

The ﬁrst one is excluded by (1.2). What happens for a > 3? The computer solutions
introduced below show a periodic behavior, (Figure 1.6)

x n = x n−2 = xp1 , x n+1 = x n−1 = xp2

or
xp1 = f(f(xp1 )) , xp2 = f(f(xp2 )) .
Applying again a linear stability analysis shows that the periodic solution becomes
unstable for a > 1 + √6 and a more complex periodic cycle where x n alters now be-
tween four values arises (Problems).

1.3.2 FORTRAN

Numerically we can study the logistic map with the following code

PROGRAM logist
REAL*8 :: a,x,amin,amax ! double precision (8 bytes)
C region for a
amin=2.8; amax=4.
c initialize graphics (pgplot)
CALL INIT(SNGL(amin),SNGL(amax),0.,1.,7.,1.,1)
itmax=200 ! number of iterations for each a
ivor=1000 ! number of hidden iterations before plotting
tiny=1.E-6 ! initial value for x
da=(amax-amin)/1000. ! step width for a
c loop for 1000 a values
DO a=amin,amax,da
C logistic map
x=tiny; ic=0 ! x0
DO it=1,itmax+ivor
x=a*x*(1.-x)
IF(it > ivor) THEN
c after pre-iter. plot a pixel at (a,x) with color ic
ic=MOD(ic,15)+1; CALL PGSCI(ic)
CALL PGPT(1,SNGL(a),SNGL(x),-1)
ENDIF
ENDDO
ENDDO
1.3 A ﬁrst example – the logistic map | 9

1
x

0.8

0.6

0.4

0.2
Fig. 1.6: Logistic map, period doublings, and
a (2)
chaos. The stationary solution xs (bold) be-
0
3 3.5 4 comes unstable for a > 3.

c labels for axes

CALL PGSCI(1); CALL PGSCH(2.5)
CALL PGTEXT(3.5,0.02,'a'); CALL PGTEXT(1.2,0.9,'x')
c steady solution x_{\text{s}}(a)
CALL PGSLW(5) ! in bold
CALL PGMOVE(SNGL(amin),1.-1./amin)
da=(amax-amin)/99.
DO as=amin,amax,da
xs=1.-1./as
CALL PGDRAW(as,xs)
ENDDO
c terminate graphic
CALL PGEND
END

Compiling and executing creates a ﬁgure like Figure 1.6 on the screen.
The subroutine init comprises PGPLOT calls for initializing the graphic screen. It
is described in more detail in Appendix B.
A quick glance at the program shows that the implicit type declarations of
FORTRAN have been utilized. That means it is not necessary to declare variables
in the header of each routine if they are real or integer scalar quantities. Each variable
beginning with a letter from a to h and then from o to z is deﬁned real (single preci-
sion) as default, while all variables beginning with i to n are integers (4 bytes). On
the other hand you can often read in FORTRAN guides (normally written by mathe-
maticians or computer scientists) sentences like we advise strongly against using type
conventions in FORTRAN. Of course the usage of default declarations seems to be a bit
10 | 1 Introduction

sloppy but it saves a lot of typing and going back and forth while writing a program.
Those who do not like this way can put the statement

IMPLICIT NONE

in the ﬁrst line of each routine and switch off the implicit declarations (there are also
compiler options that effect the same). However, each variable has to then be deﬁned
explicitly.

1.3.2.1 MATLAB
The same code can be written in MATLAB. The syntax is very similar to FORTRAN
but easier to write. The graphics calls are different. A code that produces the same
Figure 1.6 is shown here:

clear;
amin=2.8; amax=4.0;
itmax=200; ivor=1000; tiny=1.e-6;y=zeros(1,imax);xx=zeros(1,imax);
da=(amax-amin)/1000;
for a=2.8:da:4.0
x=tiny;
for i=1:itmax+ivor;
x=a*x*(1-x);
if i>ivor
y(i-ivor)=x; xx(i-ivor)=a; % to save CPU time it is essential to
end % store the x-values for each a and
end % then plot them all together
plot(xx,y,'.');
end

However, execution time in MATLAB is normally much longer than in a compiler lan-
guage. To demonstrate this, we remove all graphic output and compare the two codes,
ﬁrst in MATLAB:

clear; t=cputime;
itmax=100000000; ivor=itmax; y=zeros(1,imax);
a=3.95; x=0.01;
for i=1:ivor+itmax
x=a*x*(1-x);
if i>ivor
y(i-ivor)=x;
end
end
cputime-t % prints the CPU time used
Bibliography | 11

then in FORTRAN

REAL*8 :: a,x,y(100000000)
common y ! outwit the FORTRAN optimizer!
itmax=100000000; ivor=itmax
a=3.95; x=0.01
DO it=1,itmax+ivor
x=a*x*(1.-x)
IF(it.gt.ivor) THEN
y(it-ivor)=x
ENDIF
ENDDO
CALL CPU_TIME(s)
WRITE(6,*) s
END

Both codes are running on an HP ProDesk 400 G3 MT, the MATLAB code under
Windows 7 Professional, the FORTRAN code under Linux using GNU-FORTRAN 95.
MATLAB needs about 3 seconds for the 20×106 iterations. If the optimizer in FORTRAN
is turned off (-O0), FORTRAN takes 0.95 seconds. On the first optimization level (-O1),
we get 0.65 seconds, using the second up to the fourth we find 0.6 seconds. Starting
with -O2, it is necessary to outflank the FORTRAN optimizer and to write the y-array
into a dummy COMMON-block. Otherwise gfortran would recognize the computed
quantities inside the iteration loop are not used further and would just skip the loop
completely.

1.3.3 Problems

With pencil and paper:

1. Compute the period two cycle xp1 , xp2 of the logistic map.
2. Show that this cycle becomes unstable if a > 1 + √6.

And to code: Change the code for the logistic map to scale up certain regions in the
x-a plane. The regions could be deﬁned using the mouse cursor.

Bibliography

[1] S. J. Chapman, FORTRAN 95/2003 for Scientists and Engineers, McGraw Hill Book Co. (2007).
[2] Matlab – Mathworks, http://mathworks.com/products/matlab/
[3] Maplesoft – Mathworks, http://www.maplesoft.com/solutions/education/
[4] LAPACK – Linear Algebra PACKage – Netlib, www.netlib.org/lapack/
[5] T. J. Pearson, Guide to the graphics package PGPLOT, Caltech, USA:
http://www.astro.caltech.edu/~tjp/pgplot/old_manual.ps.gz
2 Nonlinear maps
In this chapter we continue with recursion formulas where, starting from an initial
(set of) number(s), all subsequent values of one (ore more) variable(s) follow in a
unique and deterministic way.

2.1 Frenkel–Kotorova model

2.1.1 Classical formulation

As a simple model of a solid body in an externally given, periodic potential V(x) with

V(x) = V(x + 1) (2.1)

we examine a one-dimensional chain of N mass points, each with mass m. The mass
points are connected with chains having the spring rate D = 1 and the equilibrium
length zero (Figure 2.1). They are located at x n and have the momentum p n . The Hamil-
ton function reads:
N
p2n 1
H(x n , p n ) = ∑ [ + V(x n ) + (x n − x n−1 )2 ] . (2.2)
n=1
2m 2

To state the dynamic problem one may write up the canonical equations

∂H ∂H
ṗ n = − , ẋ n = (2.3)
∂x n ∂p n

and have to solve 2N coupled ordinary differential equations.

V(x)

y2 y3 D D

x1 x2 x3 xN x

Fig. 2.1: Sketch of a one-dimensional chain of springs in an external periodic potential.

https://doi.org/10.1515/9783110515145-002
2.1 Frenkel–Kotorova model | 13

2.1.2 Equilibrium solutions

We shall come back in detail to problems like (2.3) in the next chapters. Here we con-
centrate only on steady conﬁgurations of the chain where ṗ n = 0, ẋ n = 0, p n = 0.
From (2.3) follows the equilibrium condition

∂H
=0 (2.4)
∂x n
or
V 󸀠 (x n ) + (x n − x n−1 ) − (x n+1 − x n ) = 0 , (2.5)
where the prime denotes the derivative. We deﬁne the reciprocal density at location x n

y n ≡ x n − x n−1 (2.6)

and obtain from (2.5) and (2.6) the two-dimensional map

y n+1 = y n + V 󸀠 (x n )
(2.7)
x n+1 = x n + y n+1 = x n + y n + V 󸀠 (x n ) .

From a certain initial pair (x0 , y0 ) the whole series (x n , y n ) is uniquely determined.
Of course nothing is said about the stability of these solutions. At this stage we only
know that they are at equilibrium where the net force on each mass point vanishes.
Due to (2.4), the potential energy has an extremum, but this might be a maximum as
well.

2.1.3 The standard map

To go on we must specify V(x). With (2.1) we assume

K
V(x) = cos(2πx) (2.8)
(2π)2

with the control parameter K. Then (2.7) takes the form

K
x n+1 = x n + y n − sin(2πx n )
2π (2.9)
K
y n+1 = y n − sin(2πx n ) .
2π
The recursion relation (2.9) is called Standard map, Chirikov map or Circle map and is
discussed in detail in the literature (e.g., [1]).
14 | 2 Nonlinear maps

The two-dimensional map (2.9) is easy to code:

PROGRAM circle_map

REAL*8 :: x,y,pi2,vor
CHARACTER*1 :: c
INTEGER :: pgcurs

pi2=4.*ASIN(1.D0) ! 2*Pi
amp=1. ! value of K
imax=10000 ! number of iterations for each initial value
CALL INIT(0.,1.,0.,0.5,10.,0.5,1)
ncol=15 ! use 15 colors
DO
k=pgcurs(x0,y0,c) ! mouse query
x=x0; y=y0 ! initial values
ic=mod(ic,ncol)+1; CALL PGSCI(ic)
DO i=1,imax
y=y-amp/pi2*SIN(pi2*x); x=x+y ! circle map
xp=MOD(x,1.); yp=MOD(y,1.) ! plot modulo 1
CALL PGPNTS(1,xp,yp,-1,1)
ENDDO
if(ICHAR(c).eq.32) EXIT
ENDDO
CALL PGEND
END

For each initial value, 10 000 points are plotted. The initial pair is determined using
the mouse. The map is periodic in y → y ± m, x → x ± m󸀠 with integers m, m󸀠 and is
symmetric with respect to the origin. Therefore it is sufficient to take the initial values
from the region
0 ≤ x ≤ 1 , 0 ≤ y ≤ 1/2 .

Two ﬁxed points x n+1 = x n , y n+1 = y n exist. At x, y = 0 we have a center, and x = 1/2,
y = 0 is called saddle point (see Chapter 3). Figure 2.2 shows several series for K = 1.

2.1.4 Problems

1. Examine (2.9) for different K. For K = 0 you will not need your computer.
2. Plot the chain length for ﬁxed x0 as a function of y0 . Try different values for x0 .
2.2 Chaos and Lyapunov exponents | 15

Fig. 2.2: Standard map for K = 1.

2.2 Chaos and Lyapunov exponents

As already known from the discussion of the logistic map in Section 1.3, there exist
regular (periodic or quasiperiodic) and chaotic regimes if the control parameters or, at
least for the Standard map, the initial conditions are changed. Instead of only looking
qualitatively at the patterns of the iteration maps, we shall now derive a more quanti-
tative measure for chaos.

2.2.1 Stability, butterﬂy effect, and chaos

Let us turn again to the logistic map (1.1). Prevenient to the chaotic regime exists a
cascade of period doublings where the relative distance of doubling or bifurcation
points decreases with increasing a. The first few bifurcation points can be computed
analytically, but the effort required increases quickly. Therefore it is desirable to find
a quantity that reflects the stability of a series for any arbitrary a.
Butterfly effect: What do we mean by stability? A descriptive definition of chaos is
given by the development of some tiny disturbances of variables and/or parameters:
if an (infinitesimally) small cause, namely a perturbation of i.e., the initial value

y0 = x0 + ϵ

after a certain time (here: number of iterations) is ampliﬁed to a ﬁnite and measur-
able outcome, a completely different series y n may result. Then the series starting
16 | 2 Nonlinear maps

at x0 is called unstable. A butterfly flaps its wings and perturbs the atmosphere very
slightly. Provided sufficiently unstable weather conditions, this ‘butterfly effect’ may
completely change the further development of the weather after a certain amount of
time.

2.2.2 Lyapunov exponent of the logistic map

Thus we are looking for a criterion that describes the divergence of two initially in-
ﬁnitesimally close series x n and y n if n ≫ 1. Let us start the ﬁrst iteration with x0 .
Then we have for x n
x n = f(f(f(. . .f(x0 ) . . . ))) ≡ f (n) (x0 ) , (2.10)
where f (n) denotes the nth iterate of f . In parallel, a second series y1 , y2 . . .y n is com-
puted for a closely nearby initial value x0 + ϵ, |ϵ| ≪ 1 and reads

y n = f (n) (x0 + ϵ) . (2.11)

To see if the two series are receding from each other, we compute their distance at each
n and ﬁnd with ϵ → 0
󵄨󵄨󵄨 f (n) (x0 + ϵ) − f (n) (x0 ) 󵄨󵄨󵄨 󵄨 󵄨
d n = |x n − y n | = 󵄨󵄨󵄨󵄨ϵ 󵄨󵄨 = lim 󵄨󵄨󵄨󵄨ϵ d f (n) (x)󵄨󵄨󵄨󵄨 .
󵄨 ϵ→0 󵄨󵄨
󵄨 󵄨󵄨 (2.12)
󵄨󵄨 ϵ 󵄨󵄨 󵄨 dx 󵄨 x0
If we assume that this distance increases (unstable) or decreases (stable) exponen-
tially with n,
d n = d0 eλn = |ϵ| eλn , (2.13)
the series is unstable if λ > 0, stable if λ < 0. With (2.12) we obtain from (2.13)
1 󵄨󵄨󵄨󵄨 d (n) 󵄨󵄨󵄨󵄨
λ = lim ln 󵄨󵄨 f (x)󵄨󵄨 , (2.14)
n→∞ n 󵄨󵄨 dx 󵄨󵄨 x0

where λ is called the Lyapunov exponent¹. For the evaluation of (2.14) the chain rule
must be applied
d (n) 󵄨󵄨󵄨󵄨
f (x)󵄨󵄨 = d x f(x0 )d x f(x1 ). . .d x f(x n )
dx 󵄨󵄨 x0
leading ﬁnally to
1 n 󵄨󵄨 d 󵄨󵄨
󵄨 󵄨
λ = lim ∑ ln 󵄨󵄨󵄨 f(x)󵄨󵄨󵄨 . (2.15)
n→∞ n 󵄨󵄨 dx 󵄨󵄨 x=x k
k=0

This relation is easy to code. Of course it is not possible to take inﬁnite n, one should
choose a sufficiently large n, say 1000, to ﬁnd a convergent result. Based on the code
logist from Chapter 1, the program could look like this:

1 Alexander M. Lyapunov, Russian mathematician and physicist, 1857–1918.

2.2 Chaos and Lyapunov exponents | 17

chaos

periodic windows

Fig. 2.3: Lyapunov exponent of the logistic map. The chaotic regime commences at about a ≈ 3.55
and λ > 0. Inside the chaotic regime, periodic windows of different size persist.

PROGRAM lyapunov
REAL*8 :: a,x,flyap
....
DO it=1,itmax+ivor ! iteration loop
x=a*x*(1.-x) ! Logistic map
IF(it.GT.ivor.AND.ABS(x-.5).GT.1.E-30)
* fly=fly+LOG(ABS(a*(1.-2.*x))) ! sum for Lyap. exp.
ENDDO
flyap=flyap/FLOAT(itmax) ! Lyapunov exponent
.... plot flyap as function of a ...

Figure 2.3 shows the Lyapunov exponent over the control parameter a. One ﬁnds neg-
ative λ for the ﬁxed points and for the periodic series, they are stable. But in the chaotic
regions, the Lyapunov exponent turns out to be positive. At the bifurcation points
a = a k , λ vanishes, since the 2k−1 -periodic cycle gets unstable at a k and gives way
to the 2k -periodic solution, which then becomes stable from a > a k through a = a k+1 .

2.2.3 Lyapunov exponents for multidimensional maps

The method can be generalized to N-dimensional maps. Let

x⃗ n+1 = f ⃗(x⃗ n )
with x,⃗ f ⃗ in R N . The distance d n now turns into the length of a vector d⃗ n and instead
of (2.12) one has
d⃗ n = L(n) (x⃗ 0 ) d⃗ 0 , |d⃗ 0 | = ϵ ≪ 0
18 | 2 Nonlinear maps

with the Jacobi matrix

(n) 󵄨󵄨󵄨
(n) ∂f i 󵄨󵄨
L ij (x⃗ 0 ) = 󵄨󵄨
∂x j 󵄨󵄨
󵄨󵄨x⃗ 0

of the nth iterate. With

d⃗ n = d⃗ 0 eλn ≡ d⃗ 0 σ

the eigenvalue problem

(L(n) − σ i ) d⃗ 0 = 0

results. Knowing the N eigenvalues σ i we may ﬁnally compute the N Lyapunov expo-
nents
1
λ i = lim ln σ i . (2.16)
n→∞ n

The Jacobi matrix of the nth iterate can be written as a product of the simple Jacobi
matrices:
L (n) (x⃗ 0 ) = L(x⃗ n ) L(x⃗ n−1 ) . . . L(x⃗ 0 ) . (2.17)

As an example we take the Standard map (2.9). Because of

1 − K cos(2πx) 1
L(x)⃗ = ( ) , x⃗ = (x, y)
−K cos(2πx) 1

we have here the special situation

det L = 1

independent on x.⃗ From (2.17) it follows det L(n) = 1 and as a consequence σ 1 σ 2 = 1.

Since L(n) is a real valued matrix, we may distinguish between the two cases
(i) σ i real valued, one σ k ≥ 1 and λ k ≥ 0,
(ii) σ 1 = σ ∗2 complex valued, |σ i | = 1, and Re λ i = 0,

Case (i) belongs to an unstable series (saddle point, see Chapter 3), case (ii) denotes a
stable one (center).
How can we compute the Lyapunov exponents in practice? To achieve a conver-
gent outcome, one should do at least a few thousand iterations and the product (2.17)
will accordingly include as many matrix multiplications. In the chaotic regime where
the series diverge, an overﬂow error is unavoidable due to the exponential growth with
n. On the other hand, the relation (2.17) is linear in x⃗ 0 and L can be normalized after
each iteration. If we choose the maximum norm
1
L(̄ x⃗ i ) = L(x⃗ i ) with s i = max(|L jk (x⃗ i )|) ,
si jk
2.2 Chaos and Lyapunov exponents | 19

the product
(n)
L̄ (x⃗ 0 ) = L(̄ x⃗ n ) L(̄ x⃗ n−1 ) . . . L(̄ x⃗ 0 ) (2.18)

will not diverge anymore but differs from (2.17) in a possibly rather large factor
n
s = ∏ si .
i=0

The eigenvalues will then differ in the same factor and instead of (2.16) we obtain

n
1 1
λ i = lim (ln σ̄ i + ln s) = lim (ln σ̄ i + ∑ ln s k ) . (2.19)
n→∞ n n→∞ n
k

All we have to do is to sum up ln s k during the iteration which will not lead to an
overﬂow due to the logarithm. The code section computing λ in connection with the
program circle_map could look like this:

REAL*8 :: x,y,bs,s,sp,det,amp,fm1(2,2)
complex xl1,xl2
....
x=x0; y=y0 ! init. values for iteration
fm1=fm(pi2*x,amp) ! Jacobi matrix, defined as function below
bs=MAXVAL(ABS(fm1)) ! maximum norm
fm1=fm1/bs ! normalization
s=LOG(bs) ! norm to s
DO i=1,imax ! iteration loop
y=y-amp/pi2*SIN(pi2*x)
x=x+y
c recursive matrix multiplication
fm1=MATMUL(fm(pi2*x,amp),fm1)
bs=MAXVAL(ABS(fm1))
fm1=fm1/bs
s=s+LOG(bs)
ENDDO
c compute eigenvalues of fm1
det=(fm1(1,1)*fm1(2,2)-fm1(1,2)*fm1(2,1))
sp=fm1(1,1)+fm1(2,2)
xl1=(log(.5*(sp+csqrt(cmplx(sp**2-4.*det,0.))))+s)
xl2=(log(.5*(sp-csqrt(cmplx(sp**2-4.*det,0.))))+s)
.. plot result
20 | 2 Nonlinear maps

c Jacobi matrix
CONTAINS ! makes function fm visible in MAIN
FUNCTION fm(x,amp)
REAL*8 :: fm(2,2),x,amp
fm(1,1)=1.-amp*COS(x)
fm(1,2)=1.
fm(2,1)=-amp*COS(x)
fm(2,2)=1.
END FUNCTION fm
END

If this section is integrated into a double loop over the initial values x0 , y0 , we can
scan the whole plane with a certain resolution. The largest Lyapunov exponent, i.e.,
the maximum of the real parts of xl1, xl2 are then stored in a two-dimensional array and
ﬁnally contour lines or a colored bitmap can be plotted. This can be done using the
routine image, which is based on the PGPLOT routine PGPIXL and explained in detail
in Appendix B. The colors can be initialized using the routine ccircl (Appendix B):

CALL CCIRCL(2,100) ! initializes the color circle for the

! color indices 2..100

Figure 2.4 shows the color-coded largest Lyapunov exponent for three different values
of K with a resolution of 500 × 500. Blue regions correspond to a regular, nonchaotic
series, and all other colors reveal a positive Lyapunov exponent and denote chaotic
behavior. The complete code can be found on the book’s website [2].

2.3 Affine maps and fractals

Next we turn to linear, bijective maps composed of the three operations

translation: q⃗ 󸀠 = q⃗ + a⃗
rotation: q⃗ 󸀠 = L R q⃗
scaling and shear: q⃗ 󸀠 = L S q⃗ .

We restrict ourselves to two dimensions,

q⃗ = (x, y) ,

where L R is the rotation matrix

cos θ − sin θ
LR = ( )
sin θ cos θ
2.3 Affine maps and fractals | 21

0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.5

0.4

0.3 Fig. 2.4: Largest Lya-

punov exponent of
0.2 the Standard map for
K = 0.5 (top), K = 1
0.1 (middle), K = 1.5
(bottom). In the blue
regions, the series are
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 not chaotic.

and L s is the scaling-shear matrix

sx b
LS = ( ) .
0 sy
A compound transformation may read
q⃗ 󸀠 = L R L S q⃗ + a⃗ , (2.20)
in which the different mappings do not commute, i.e., the sequence is important
(in fact rotation and scaling commute if s x = s y , b = 0). If a triangle with area A is
22 | 2 Nonlinear maps

transformed, one ﬁnds with Det(L R ) = 1

A󸀠 = Det(L S ) A = s x s y A .

If the mapping (2.20) is performed recursively,

q⃗ n+1 = L R L S q⃗ n + a⃗ , (2.21)

and s x s y < 1, a self-similar structure is obtained.

2.3.1 Sierpinski triangle

As an application we construct the Sierpinski triangle. With the FORTRAN call

CALL RANDOM_NUMBER(x)

we activate the random number generator. If x is a scalar it contains after the call an
equally distributed random number in the interval

0≤x<1

(for more details on random numbers see Section 8.1.1). A possible construction rule
for the Sierpinski triangle reads [3]:
1. Deﬁne a triangle by its corners (a1 , b 1 ), (a2 , b 2 ), (a3 , b 3 ).
2. Choose an arbitrary (random) point inside the triangle.
3. Choose an arbitrary (random) number i = 1, 2, 3 (equally distributed).
4. Place a point in the middle of (x, y) and (a i , b i ).
5. Go to 3 and take the point computed in 4 as the new point (x, y).

These rules can be formulated as the map

1
q⃗ n+1 = (q⃗ n + a⃗ i ) , i = 1, 2, 3 (random) (2.22)
2
where a⃗ i = (a i , b i ). The elaboration of the code is left to the reader. If we choose the
corners,
a1 = (0, 0) , a2 = (1, 0) , a3 = (0.5, 1) ,

Figure 2.5 should appear after some 1000 iterations.

Obviously a rather regular structure is created despite arbitrary processes playing
an important role in its construction. Also the initial value has no further inﬂuence,
as the iteration converges quickly to the Sierpinski triangle (or gasket), which can be
considered an attractor of the entire plane inside the triangle. The Sierpinski gasket is
2.3 Affine maps and fractals | 23

Fig. 2.5: Sierpinski triangle as a fractal grid.

self-similar, as each arbitrarily chosen subtriangle has the same shape. The plane is
almost empty, and each triangle has inﬁnitely many holes on all length scales (if the
numbers of iterations would be inﬁnite). This property is called a fractal.

2.3.2 About ferns and other plants

At least down to a certain length scale, many objects encountered in nature show a
fractal structure (we refer to the book of B. Mandelbrot, The Fractal Geometry of Na-
ture, [4]). The English mathematician Michael Barnsley proposed in 1985 a map to
construct ferns, and since then these objects have been called Barnsley ferns [5]. Also
here, randomness plays a crucial role. The map reads

q⃗ n+1 = L i q⃗ n + a⃗ i , q⃗ 0 = (0.5, 0) (2.23)

where
0 0 −0.139 0.263
L1 = ( ) , L2 = ( ) ,
0 0.27 0.246 0.224
(2.24)
0.17 −0.215 0.781 0.034
L3 = ( ) , L4 = ( )
0.222 0.176 −0.032 0.739
and
0.5 0.57 0.408 0.1075
a⃗ 1 = ( ), a⃗ 2 = ( ), a⃗ 3 = ( ), a⃗ 4 = ( ).
0 −0.036 0.0893 0.27

The selection of i is random between 1 and 4, not equally distributed but with the
probability
P(i) = (0.02, 0.15, 0.13, 0.7) .
24 | 2 Nonlinear maps

Fig. 2.6: Left: Barnsley fern after (2.24), right: a fractal tree, created with (2.25).

The result after 30.000 iterations is shown in Figure 2.6 on the left. Another iteration
rule creates a tree, as shown in Figure 2.6 on the right. Here we take

0.05 0 0.05 0 0.46 −0.15

L1 = ( ) , L2 = ( ) , L3 = ( ) ,
0 0.6 0 −0.5 0.39 0.38
(2.25)
0.47 −0.15 0.43 0.28 0.42 0.26
L4 = ( ) , L5 = ( ) , L6 = ( ) ,
0.17 0.42 −0.25 0.45 −0.35 0.31

and

0 0 0 0 0 0
a⃗ 1 = ( ) , a⃗ 2 = ( ) , a⃗ 3 = ( ) , a⃗ 4 = ( ) , a⃗ 5 = ( ) , a⃗ 6 = ( ) .
0 1 0.6 1.1 1 0.7

The probability for i from 1 to 6 is now

P(i) = (0.1, 0.1, 0.2, 0.2, 0.2, 0.2) .

2.3.3 Problems

1. Compute the commutator [L R , L S ] using your pencil.

2. Code the Sierpinski gasket, a fern, and a tree after (2.24), (2.25), respectively. Try
different colors for different i. Change the probabilities and ﬁnd out what happens.
2.4 Fractal dimension | 25

Fig. 2.7: Box-counting method. To compute the (fractal) dimension of an object, one counts the
squares needed to cover it completely. If the side length of the squares is halved one needs about
twice as much if the object is a line and has a dimension of 1.

2.4 Fractal dimension

2.4.1 Box-counting

Based on the Euclidean concept of dimension (point = 0, line = 1, plane = 2, etc.) we can
assign a fractal dimension to a fractal object. We shall use the so-called box-counting
method. Let the object whose dimension should be computed lay on the unit square
E, e.g., a line as in the example of Figure 2.7. We divide E into N × N equal squares
with side length L = 1/N. For a given L let M be the number of squares visited by the
object. If L is halved (the number of total squares in E is quadrupled) we should find
about 2M instead of M. Thus we have the relation
M ∼ L−1 (2.26)
if the object was a line. When, on the other hand, the object is an assembly of a certain
number K of points, we shall always find M = K if L is small enough, therefore
M = const. (2.27)
If the object were to finally be two-dimensional, i.e., a part of the plane, M would be
proportional to the number of squares, or
M ∼ L−2 . (2.28)
We can combine all three cases as
M ∼ L−d , (2.29)
where d is the dimension of the object.
26 | 2 Nonlinear maps

2.4.2 Application: Sierpinski triangle

To show how the method works we take the Sierpinski gasket. A marker array ibox is
deﬁned whose element ibox(i,j) is set to one if at least one point on the map falls into
the corresponding square:

PROGRAM fractal_dimension
REAL, DIMENSION(3) :: a,b
INTEGER, DIMENSION(1024,1024) :: ibox

OPEN(2,file='fd.dat') ! open file fd.dat for output

iter=100000 ! number of iterations for the gasket
c define Sierpinski triangle
a=(/0.,1.,0.5/); b=(/0.,0.,1./)
dx=1.; id=1 ! initial side length and number of squares
DO i=1,10 ! ten different grids
id=2*id; dx=dx/2 ! double number of squares in each dimension
ibox=0
x=0.5; y=0.5 ! initial point for Sierpinski iterations
DO n=1,iter ! Sierpinski iterations (see sect.2.3.1)
CALL RANDOM_NUMBER(r)
i=INT(3.*r+1.) ! random choice 1,2,3
x=(x+a(i))/2.; y=(y+b(i))/2.
c find square i,j, set marker in ibox
ix=x/dx+1; iy=y/dx+1; ibox(ix,iy)=1
ENDDO

n=sum(ibox) ! total number of visited squares

c LOG-LOG-plot
xp=LOG(1./dx); yp=LOG(FLOAT(n))
WRITE(2,*) xp,yp ! output on file
ENDDO
END

Thereafter, log(M) is plotted over − log L (log-log scale) and the slope is determined
that corresponds to the fractal dimension d f according to (2.29):
log M
df = − . (2.30)
log L
This time the program does not use graphic output directly but writes the data on a
ﬁle ‘fd.dat’. The ﬁle can then be inspected (Figure 2.8) using gnuplot for instance via
2.5 Neural networks | 27

Fig. 2.8: Number of squares over side length,

double log scale. For the ideal case one ob-
serves a straight line, and its slope corresponds
to the fractal dimension of the covered object.

gnuplot>plot ‘fd.dat’ with lines

Then the slope can be determined at a convenient position where the curve is more
or less straight. If we take, e.g., points 3 and 8 to construct the slope triangle, we ﬁnd
d f ≈ 1.585. The value of the fractal dimension can also be computed analytically
using another construction for the Sierpinski gasket, see [3]. One then obtains

d f = log(3)/ log(2) = 1.5849625. . .

2.4.3 Problem

Determine the fractal dimension of the fern and the tree of Figure 2.6 numerically.

2.5 Neural networks

In about 1990 the discipline neuro-informatics emerged. It comprises certain subjects

from physics, mathematics, chemistry, and medicine, and aims to study and to under-
stand the way the (human) brain works. One approach is the modeling of biological
intelligence such as memory, learning, and logical relations using neural networks.
Here we can only take a short look at this huge and still developing area. We have to
concentrate on two examples, the perceptron and the so-called Kohonen maps. For
further details we refer to the literature [6].
28 | 2 Nonlinear maps

Fig. 2.9: Perceptron with N = 5 input neurons.

Since there is only one processing layer, it is
also called single-layer perceptron (SLP).

2.5.1 Perceptron

The human brain is built of about 80 billion neurons that are linked among each other
with more than 1014 contacts, or synapses. These connections are dynamic and can
be intensiﬁed or attenuated, depending on their usage. This time dependent adaption
process is considered the key to learning. The perceptron is a very simple system re-
alizing this concept. It possesses an input layer of N neurons S1 . . . S N which are not
connected to each other. One processing layer, namely the adjustable weighted con-
nections (synapses) w j , links the input neurons to a single output neuron (Figure 2.9).
Each neuron has two states, active (S j = 1) or passive (S j = −1).
The coupling of the input layer to the output neuron is given by the simple relation

N
S o = sign (∑ w j S j ) (2.31)
j

where
{1 for x ≥ 0
sign(x) = {
−1 for x < 0
{
is the slightly modiﬁed sign function. The strength of the connections is described by
the synaptic weights w j . If w j < 0, the connection is inhibitive; otherwise it is activat-
ing. During the learning process, the values of the w j are changed accordingly.

2.5.1.1 Learning rule

What do we mean by learning process? A neural network is not coded and does not
contain any inherent rules or connections. One offers input/output pairs that are as-
signed to each other (learned). Assume we have M such pairs denoted by
(n)
(S j = x j , S o = y(n) ) , n = 1...M . (2.32)
2.5 Neural networks | 29

(n)
The network learns the assignment x j → y(n) by dynamically changing the synaptic
weights w i . For the optimal case, as many pairs as possible out of (2.32) should fulﬁll
the relation (2.31):
N
! (n)
y(n) = sign (∑ w j x j ) . (2.33)
j

In 1949, D. Hebb² formulated what is nowadays called Hebb’s learning rule or Hebbian
learning, which says that the synaptic weights are adjusted to the activity of the input
and output neurons they do connect. Let ∆w i denote the modiﬁcation of w i at each
learning step, then Hebb’s rule can be written up as

1 (n) (n)
∆w j = y xj . (2.34)
N

If there was only one pair (M = 1) to learn and initially all w j = 0, we ﬁnd after one
step w j = N1 y x j . Inserting this into (2.33) yields

1 N 2
y = sign (y ∑ x ) = sign(y) ,
N j j

which is fulﬁlled for all y = ±1. In practice, the network should learn of course more
than just one pair. Then one has to modify the rule (2.34): each pair changes the
weights only if it is not yet learned correctly (Rosenblatt rule³):

(n) (n)
{ 1 y(n) x j if y(n) (∑Nj w j x j ) ≤ 0
∆w j = { N (2.35)
0 otherwise .
{

In the ideal case when all pairs have been learned correctly, the algorithm stops by
itself. However, as we shall see below, the capability to learn depends on the in-
put/output pairs (2.32), and the complete learning of all pairs is rather an exception.
But let us ﬁrst continue with an example.

2.5.1.2 Prime numbers

We wish to teach our perceptron to distinguish between prime and not prime numbers.
The input of an integer K > 0 is realized by activating input neuron no. K to +1. All
other neurons are in the rest state −1. Obviously, the range of input integers is re-
stricted to K ≤ N. The output neuron should be active (+1) if K is prime.

2 Donald Hebb, Canadian psychologist, 1904–1985.

3 Frank Rosenblatt, American psychologist, 1928–1971.
30 | 2 Nonlinear maps

To gain a clear and ﬂexible code we adjust the three subtasks

1. Generation of the input/output pairs
2. Evaluation
3. Learning step, adjustment of the synaptic weights

to the three subroutines

1. SUBROUTINE inout
2. SUBROUTINE eval
3. SUBROUTINE learn.

The main program is then a simple loop over the repeated learning of integers as long
as a stop-bit (here istop) is set:

PARAMETER (n=100, m=1) ! number of neurons, input(n) and

! output(m) layer
C
INTEGER :: ie(n),ia(m),ial(m)
REAL :: w(n,m)

w=-1. ! initiate all weights with -1

imax=n ! max. number of different input/output pairs
k=1; istop=0
DO WHILE(istop.EQ.0)
CALL inout(ie,ial,n,m,k)
CALL eval(ie,ia,w,n,m)
CALL learn(ie,ia,ial,w,n,m,istop,imax)
wskal=MAXVAL(ABS(w))
w=w/wskal ! scaling of weights
k=MOD(k,imax+1)+1
k1=k1+1
CALL synplo(w,n,m,k1) ! plot of the synaptic weights
ENDDO
write(6,*)'learning steps needed:',k1
END

The array ie contains the value of the input neurons, ia those of the outputs (here only
one). For K, the natural numbers 1, 2, 3, . . . , N are successively offered. The routine
learn sets the stop-bit if imax numbers are recognized correctly one after the other
without changing w. The subroutines read in detail:
2.5 Neural networks | 31

SUBROUTINE inout(ie,ia,n,m,k)
INTEGER :: ie(n),ia(m)
c provides input/output pairs
ie=-1
ie(k)=1 ! input layer
ia(1)=iprim(k) ! output neuron
END

The integer-function iprim(k) returns +1 if k is prime, otherwise −1. We leave the cod-
ing of iprim to the reader.

SUBROUTINE eval(ie,ia,w,n,m)
INTEGER :: ie(n),ia(m)
REAL :: w(n,m)
DO j=1,m
s=SUM(w(1:n,j)*FLOAT(ie))
ia(j)=SIGN(1,FLOOR(s))
ENDDO
END

This is the coding of formula (2.31). The subroutine learn ﬁnally realizes Rosenblatt’s
rule (2.35):

SUBROUTINE learn(ie,ia,ial,w,n,m,istop,imax)
INTEGER :: ie(n),ia(m),ial(m)
REAL :: w(n,m)
c modify weights according to Rosenblatt
istop=0
ll=ll+1
DO j=1,m
IF(ia(j).NE.ial(j)) THEN
ll=0
w(1:n,j)=w(1:n,j)+float(ial(j)*ie)/FLOAT(n)
ENDIF
ENDDO
IF(ll.EQ.imax) istop=1 ! set stop-bit
END

If the computed result ia differs from the one that should be learned, ial, the weights
are adjusted accordingly. The stop-bit istop is set if imax numbers are learned correctly
in a row. In Figure 2.10 we show the learning steps needed as a function of N for dif-
ferent initializations of the weights w i .
32 | 2 Nonlinear maps

1e+07

8e+06
w = random

6e+06 w=1
K1

4e+06

2e+06
w = −1 Fig. 2.10: The number K1 of the learning steps
0 needed depends strongly on the number of
0 200 400 600 800 1000 neurons N of the input layer, but also on the
N initialization of the synaptic weights w i .

2.5.1.3 Logic gates and linear separability

We extend the switching condition (2.31) according to
N
S o = sign (∑ w j S j − ϑ) . (2.36)
j

Now the output neuron becomes active if the sum of its inputs exceeds the threshold
value ϑ, which can be different for each neuron. To indicate the threshold, we write
its value in the respective output neuron. To train the perceptron as a ‘normal’ binary
computer it should at least be able to learn the basic logic functions NOT, AND, OR, and
XOR. Taking an appropriate value for ϑ, this is possible for the ﬁrst three functions,
but not for XOR (Figure 2.11), as already proven by M. Minsky⁴ in the 60s. The truth
table of an XOR gate looks like

S1 S2 So
−1 −1 −1
1 1 −1
−1 1 1
1 −1 1

−1 1/2 1/2 1/2 1/2

0 1/2 −1/2

NOT AND OR

Fig. 2.11: The logic functions NOT, AND, and OR can be realized by an SLP.

4 Marvin Minsky, American artiﬁcial intelligence researcher, 1927–2016.

2.5 Neural networks | 33

Let a1 = −1 (passive) and a2 = 1 (active). Then we must have

w1 a1 + w2 a1 < ϑ
w1 a2 + w2 a2 < ϑ
(2.37)
w1 a1 + w2 a2 > ϑ
w1 a2 + w2 a1 > ϑ .

Subtracting the third from the ﬁrst inequality yields

w2 (a1 − a2 ) < 0 ,

subtracting the fourth from the second leads to

w2 (a2 − a1 ) < 0 ,

which is obviously a contradiction. In N-dimensional vector space, all possible input

patterns of N neurons lay on the corners of an N-dimensional hypercube. Only these
patterns that can be separated by an N − 1-dimensional hyperplane can be distin-
guished by the (single-layer) perceptron, Figure 2.12. If we try to teach our perceptron
the XOR function, the learning process would never come to an end. Problems of this
kind are called not linearly separable.

S2 S2 S2
S1 S1 S1

AND OR XOR
Fig. 2.12: For AND and OR the input states assigned to the same output state can be separated by a
straight line; for XOR this is impossible.

Unfortunately most of the problems are not linearly separable, at least for larger N.
A solution can be the extension to more than one processing layer (multilayer percep-
tron), for which we refer to the literature.

2.5.1.4 Decimal to binary converter

Taking M perceptrons having one output neuron each in parallel, one arrives at a per-
ceptron with M neurons in the output layer, Figure 2.13. The synaptic weights can now
be cast in a rectangular matrix

w ij , i = 1...N , j = 1...M
34 | 2 Nonlinear maps

Fig. 2.13: An SLP having more than one out-

put neuron. Here, N = 5 input neurons are
connected with M = 3 output neurons via the
rectangular weight matrix w ij .

and Hebb’s rule (2.34) turns into

1 (n) (n)
∆w ij = y xi . (2.38)
N j
Such an extended perceptron can not only distinguish between two properties like
prime or not prime, but between 2M states.
As an example, we wish to teach the perceptron how to convert decimals into bi-
nary numbers. During the learning process we offer again integer values K running
from 0 to N. The routine inout creates the input ie according to

ie=-1
IF(k.ne.0) ie(k)=1

The related output is the binary image of K, which can be obtained in FORTRAN using
the BTEST statement:

ie=-1
DO j=0,m-1
IF(BTEST(k,j)) ie(m-j)=1
ENDDO

(the most signiﬁcant bit is ie(1)). The complete listing is provided in [2]. Figure 2.14
shows the synaptic connections for N = 15, M = 4 when ﬁnally all numbers 1. . . 15
have been learned after 80 learning cycles (w ij = 1 initialized). Negative (inhibitive)
connections are drawn dashed.

2.5.2 Self-organized maps: Kohonen’s model

Since there exist no horizontal connections within each layer of the perceptron, the
spatial arrangement of the neurons plays no role. But the brain works differently: sim-
2.5 Neural networks | 35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Fig. 2.14: The SLP as a decimal to binary converter. Inhibiting connections w < 0 are shown dashed.
In the output layer the most signiﬁcant bit is on the left-hand side.

ilar stimuli are processed in nearby regions of the cortex. This constitutes the basic
idea of the self-organized maps developed by the Finnish engineer Teuvo Kohonen
in 1982.

2.5.2.1 The model

We resort to the case where the neurons are located in a plane layer representing a
two-dimensional network. To each neuron we may assign a vector

I
r⃗ = ( )
J

with (I, J) = (0, 0). . .(M − 1, N − 1) (Figure 2.15). Each neuron is connected with an
input layer consisting of L neurons by synapses with the weights

wℓr ⃗ , ℓ = 1...L

(the upper index ℓ denotes the neurons of the input layer). The Kohonen algorithm
modiﬁes Hebb’s learning rule in the following way:
1. Initializing. Take an appropriate initial value for the weights wℓr ⃗ , e.g., randomly
distributed or all zero or ±1.
2. Input signals. The signals or patterns to learn are offered as vℓ , either one by one
or in a random sequence.
3. BMU. Identify the Best Matching Unit (BMU). The BMU is the neuron that has the
smallest (euclidean) distance to the learning signal. If
L L
∑(vℓ − wℓr ⃗ 󸀠 )2 ≤ ∑(vℓ − wℓr ⃗ )2 for all r⃗ , (2.39)
ℓ ℓ

the BMU is located at r ⃗ 󸀠 . Then r ⃗ 󸀠 is called the center of excitation.

36 | 2 Nonlinear maps

J
I Fig. 2.15: 25 neurons arranged in a quadratic processing layer.

4. Dynamics. During the adjustment step all synaptic weights are adapted accord-
ing to
∆wℓr ⃗ = ϵ h(d) (vℓ − wℓr ⃗ ) , with d ≡ |r ⃗ − r ⃗ 󸀠 | (2.40)
for all r.⃗ Except for the new function h(d) this is exactly Hebb’s rule (2.34). The
function h(d) is unimodal and has its maximum at d = 0. It could be for instance
a Gaussian of width σ
h(d) = exp(−d2 /2σ 2 ) . (2.41)
Note that in this step not only the BMU is changed but also the neurons lying in
its neighborhood.
5. Refinement. To reﬁne the spatial structure, σ is decreased after each learning
step.
6. If σ ≥ σ min go to 2.
7. The algorithm stops when only the BMU is changed.

2.5.2.2 Color maps

To be more clear we wish to present an application: a Kohonen map sorting colors. Col-
ors can be classiﬁed by a three-component vector measuring their red, green, and blue
composition, called the RGB value. An RGB value of (1/0/0) denotes red, (0/1/0) green,
(1/1/0) yellow, (0.5,0.5,0.5) gray and (1/1/1) white. Each neuron bears three synaptic val-
ues, namely
(1) (2) (3)
wr⃗ for red , wr⃗ for green , wr⃗ for blue ,

where
(ℓ)
0 ≤ wr⃗ ≤ 1 .

The code that implements the Kohonen algorithm can be found in [2] (the array R is
assigned to w(1) , G to w(2) and B to w(3) using an EQUIVALENCE statement).
Figure 2.16 shows color maps formed by a layer of 15 × 15 neurons after learning
100, 500, and 3000 randomly generated RGB patterns. For a monotonically decreasing
2.5 Neural networks | 37

Fig. 2.16: Evolution of a color map consist-

ing of 15 × 15 neurons applying Koho-
nen’s algorithm. Upper left: initial state;
upper right: after t = 100 learned ran-
dom patterns; bottom left: after t = 500;
bottom right: after t = t max = 3000.

σ we use the relation

σ = σ 0 ⋅ (0.05)t/t max , σ0 = 5 , tmax = 3000 ,

in addition we put ϵ = 0.03. It turns out that the completely random color pattern from
the beginning is eventually organized in such a way that similar colors are assigned
to neurons located close to each other.

2.5.2.3 Traveling Salesman Problem

Next we wish to study a one-dimensional conﬁguration of N neurons in a row. Each
neuron in the chain bears a two-dimensional vector
xi
w⃗ i = ( )
yi

pointing at a certain location in 2D real space, e.g., of a road map. If one passes the
neurons one after the other, the adjoint locations (x i , y i ) constitute a certain path on
the road map. If we offer certain given points (places) to the Kohonen algorithm as
learning inputs, the path will be matched closer and closer to these points.
This leads us to the famous Traveling Salesman Problem (TSP). The task of the
TSP is to ﬁnd the shortest connection between K given places on a road map with the
constraint that each place is only touched once. It turns out that there exist 12 (K − 1)!
different closed paths to fulﬁll this condition. But which is the shortest? Of course one
could try all paths, but the computational effort increases exponentially with K. Even
for a rather small number of places the largest computers would be overcharged.
To apply Kohonen’s method for an approximate solution, we take 30 prescribed
locations, for instance, randomly chosen positions on a map; see the red points in Fig-
ure 2.17. In each learning step we select randomly one of these locations as the input
38 | 2 Nonlinear maps

Fig. 2.17: Two approximate solutions of the TSP computed with Kohonen’s algorithm based on
a chain of 40 neurons (black dots). A certain number of places, here 30 (red dots), should be
connected by the shortest path. Shown is the situation after 20 (top right), 80 (bottom left),
and 2000 (bottom right) learning steps. In the beginning, the chain is initialized on a circle with
radius R = 0.15 (solid lines) and R = 0.5 (dashed). The path length after 2000 steps is 4.176 (solid)
and 4.304 (dashed).

and adjust the synaptic weights according to (2.40). Also in Figure 2.17 we show the
states of the network after 20, 80, and 2000 learning steps for two different initial val-
ues of the weight vectors w⃗ i . The solid lines correspond to initial values on a circle
with radius R = 0.15, the dashed ones with R = 0.5. After 2000 steps all places are
passed by the paths. For the parameters we use ϵ = 0.8 and

σ = σ 0 ⋅ (0.02)t/t max , σ0 = 2 , tmax = 4000 .

Bibliography | 39

Considering the ﬁgure, it is evident that different initial conﬁgurations lead to at least
slightly different solutions. But only one path can be the shortest and Kohonen’s algo-
rithm can solve the TSP only approximately.

2.5.3 Problems

1. Code the routines for the perceptron as a decimal to binary converter.

2. Solve the TSP for K locations and N neurons applying the Kohonen algorithm.

Bibliography

[1] J. Argyris, G. Faust, M. Haase and R. Friedrich, An Exploration of Dynamical Systems and Chaos,
Springer (2015).
[2] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
[3] W. Kinzel and G. Reents, Physics by Computer, Springer (1997).
[4] B. Mandelbrot, The Fractal Geometry of Nature, Freeman and Co. New York (1983).
[5] M. F. Barnsley, Fractals Everywhere, Dover Pub. (2012).
[6] H. Ritter, T. Martinetz and K. Schulten, Neural Computation and Self-Organizing Maps: An Intro-
duction, Addison–Wesley Pub. (1992).
3 Dynamical systems
The foundation of physical theories is normally based on differential equations. In
ﬁeld theories like quantum mechanics, electrodynamics, or hydrodynamics, one is
confronted with (a system of) partial differential equations
F k [r,⃗ t, Ψ i (r,⃗ t), ∂ t Ψ i (r,⃗ t), ∂ tt Ψ i (r,⃗ t), . . . , ∂ x n Ψ i (r,⃗ t), ∂ x n x m Ψ i (r,⃗ t) . . . , ] = 0 , (3.1)
while in Newton’s classical mechanics ordinary differential equations
G k [x i (t), d t x i (t), d2tt x i (t), . . . ] = 0 (3.2)
are the focus of attention. Systems of the form (3.1) can be expanded into a certain
basis for further numerical analysis, according to
N
Ψ i (r,⃗ t) = ∑ a ik (t) ϕ ik (r)⃗ (3.3)
k

and are thereby converted to the form (3.2). We postpone the treatment of partial dif-
ferential equations to Chapters 6 and 7 and commence here with problems of the kind
of (3.2).

3.1 Quasilinear differential equations

Let t be the independent variable, for instance, time. To begin with, we restrict the
treatment to one dependent variable, say x and search for the function x(t). In classi-
cal mechanics this corresponds to a one-dimensional motion. Quasilinear differential
equations are linear in the highest occurring derivative. Then, (3.2) can be written as
x(N) = f (t, x, x(1) , x(2) , . . . .x(N−1) ) (3.4)
with
dk x
x = x(t) , . x(k) ≡
dt k
Equation (3.4) is equivalent to a system of N ﬁrst order ordinary differential equations
(ODEs), obtained by the substitution
x1 = x , x k+1 = x(k) , k = 1...N −1
as
d t x1 = x2
d t x2 = x3
.. (3.5)
.
d t x N−1 = x N
d t x N = f(t, x1 , x2 , . . .x N ) ,
https://doi.org/10.1515/9783110515145-003
3.1 Quasilinear differential equations | 41

or, using vector notation

d x⃗
= f ⃗(t, x)⃗ (3.6)
dt
with f ⃗ = (x2 , x3 , . . .x N , f). It is sufficient to study first order systems as (3.6). Equations
of this kind but for arbitrary f ⃗ constitute a dynamical system with N as the number of
degrees of freedom. The vector x(t) ⃗ lives in an N-dimensional state space (or phase
space).
If we consider the definition of the derivative by the differential quotient
d x⃗ ⃗ + ∆t) − x(t)
x(t ⃗
= lim ,
dt ∆t→0 ∆t
we can write (3.6) for small but finite ∆t approximately (Taylor expansion) in the form
of an N-dimensional map

x(t ⃗ + f ⃗(t, x(t))∆t

⃗ + ∆t) = x(t) ⃗ + O(∆t2 ) , (3.7)

⃗ we can compute x(t

where ∆t is denoted as step size. From each x(t) ⃗ + ∆t) and then
iteratively all the following values

⃗ + m∆t)
x(t

for m integer. It is clear that in addition to the function f ⃗ we need initial values

⃗ 0 ) = x⃗ 0 .
x(t (3.8)

Formally, the dynamical system (3.6) maps the vector x(t ⃗ 0 ) uniquely onto the vector
⃗ 1 ). This can be also expressed by the help of the time-evolution operator U:⃗
x(t

x(t ⃗ 1 , t0 )[x(t
⃗ 1 ) = U(t ⃗ 0 )] . (3.9)

Equations (3.6,3.8) deﬁne an initial value problem, and (3.7) provides a ﬁrst numerical
approximation scheme for its solution, the so-called Euler-forward method or explicit
Euler method.

3.1.1 Example: logistic map and logistic ODE

In context with the logistic map from Chapter 1 we examine the ﬁrst order ODE (logistic
ODE)
dx
= (a − 1)x − ax2 , (3.10)
dt
deﬁning the behavior of x(t) for any given x(0) = x0 (one-dimensional initial value
problem). An exact solution can be found by the separation of variables
a−1
x(t) = , (3.11)
a + c e(1−a)t
42 | 3 Dynamical systems

where the integration constant c = (a − ax0 − 1)/x0 is determined by the initial con-
dition. As t → ∞ one obtains the asymptotic solutions

{0 if a<1
xs = { (3.12)
1 − 1/a if a>1
{
which coincide with the fixed points of the logistic map. How is the ODE (3.10) linked
to the logistic map (1.1)? Following (3.7) we express dx/dt by the differential quotient
dx x(t + ∆t) − x(t)
≈ (3.13)
dt ∆t
which becomes exact for ∆t → 0. Inserting (3.13) on the left-hand side of (3.10) one
finds after a short calculation
1 1
x n+1 = ax n ∆t (1 − (1 − ) − x n ) . (3.14)
a ∆t
Here, x n denotes the x values at the time n∆t, or x n = x(n∆t). If we put now ∆t = 1,
Equation (3.14) becomes equivalent to (1.1). In other words, the discretized form of the
ODE (3.10) corresponds mathematically to the logistic map. This explains the similar
‘temporal’ development up to a < 3, i.e., the asymptotic approach to xs and the correct
asymptotic solutions. What cannot be explained are the further bifurcations of the
solutions of the logistic map for a > 3 and their chaotic behavior for even larger a. All
this is of course not included in the solution (3.11).
How can we explain the much richer dynamic behavior of the discretized form
(3.14)? To get an answer we have to look at the stability of (3.14). A stationary solution
(fixed point) of (3.14) reads xs = 1 − 1/a. If we put

x(t) = xs + u(t)

and linearize with respect to u, we can show that for ∆t = 1 the ﬁxed point xs be-
comes numerically unstable as soon as a > 3. Coming from the logistic ODE, all the
bifurcations through the chaotic sequences originate from a too-large time step during
discretization (see Problems) and vanish with a smaller ∆t. However, then (3.14) is no
longer equivalent to the logistic map.

3.1.2 Problems

With a pencil, show that (3.14) becomes numerically unstable if ∆t > 2/(a − 1).
And to code:
1. Plot the function (3.11) using PGPLOT for different initial conditions and different
values of a.
2. Examine the map (3.14) as a numerical solution of the ODE (3.10) for different time
steps ∆t and different a.
3.2 Fixed points and instabilities | 43

3.2 Fixed points and instabilities

We continue with the N-dimensional dynamical system (3.6) with a given initial con-
dition. For the time being we concentrate on autonomous systems, i.e., the right-hand
sides of (3.15) do not depend explicitly on t:
dx i
= f i (x1 . . .x N ) , i = 1...N . (3.15)
dt

3.2.1 Fixed points

For an autonomous system, ﬁxed points as a stationary solution of (3.15) may exist.
They can be found by solving the algebraic system
(0) (0)
0 = f i (x1 . . .x N ) , i = 1...N . (3.16)
We note that for somewhat involved nonlinear f i , solutions of (3.16) are normally only
accessible numerically, even for small N.

3.2.2 Stability

(0)
If a fixed point x i is known, the behavior of the system in its neighborhood is of inter-
est. How do small deviations evolve? Are they amplified or do they fade away in time?
If the deviations stay small or decay, the fixed point is called stable; if they depart ex-
ponentially, we have an unstable fixed point. If N > 1, the disturbed trajectories may
in addition oscillate around the fixed point. This is called oscillatory or Hopf bifurca-
tion¹.
If one restricts to infinitesimal deviations u i (t) from the fixed point
(0)
x i (t) = x i + u i (t) , (3.17)
one may linearize (3.15) with respect to u. This is called linear stability analysis. Insert-
ing (3.17) into (3.15) yields after linearization the system
N
du i
= ∑ L ij u j (3.18)
dt j

with the Jacobi matrix

󵄨
∂f i 󵄨󵄨󵄨
L ij = 󵄨
∂x j 󵄨󵄨󵄨󵄨x=⃗ x⃗ (0)
. (3.19)

For an autonomous system, the L ij are constant and

u j = q j eλt (3.20)

1 Eberhard Hopf, German-American mathematician, 1902–1983.

44 | 3 Dynamical systems

transforms (3.18) into a linear eigenvalue problem

N
∑ [L ij − λ δ ij ] q j = 0 . (3.21)
j

Due to (3.20), the real parts of the N eigenvalues λ k characterize the stability of the
(0)
fixed point x i .
Let the eigenvalues be ordered according to Re(λ1 ) ≥ Re(λ2 ) ≥ . . . ≥ Re(λ N ).
We distinguish between the four cases:
(i) All λ k have a negative real part: the fixed point is stable.
(ii) Re(λ1 ) > 0, and Im(λ1 ) = 0; the fixed point is monotonically unstable.
(iii) Re(λ1 ) > 0, and Im(λ1 ) ≠ 0; the fixed point is oscillatory (Hopf) unstable.
(iv) Re(λ1 ) = 0, critical point, center if Im(λ1 ) ≠ 0; the fixed point is marginally stable.

(i) is also named node or stable focus if Im(λ1 ) ≠ 0, (ii) is called saddle point, and (iii)
is an unstable focus.

3.2.3 Trajectories

⃗ 0 ) be the location of a particle in the N-dimensional state space at time t0 . The

Let x(t
parametrization x(t) ⃗ corresponds to a line (trajectory) and (3.15) creates a ﬂow. Given
f , the time-evolution is deterministic: how the particle evolves for t > t0 is determined
solely by x(t⃗ 0 ). But then it can be easily seen that crossings of trajectories are excluded.
Taking a potential cross point as x(t ⃗ 0 ) there would be two possible different paths for
t > t0 . But this would contradict the deterministic behavior described by the ﬂow
(3.15). This is formulated as the No-Intersection Theorem:

Two distinct trajectories in N-dimensional state space cannot intersect, nor can a
single trajectory cross itself.

The theorem is valid for arbitrary N.

If f ⃗ is taken as a particle density flow, the source strength
N
∂f i
Q(x)⃗ = ∇ ⋅ f ⃗ = ∑ (3.22)
i
∂x i
can be defined. Knowing Q, we can compute the temporal change of a volume element
⃗
∆V that swims in the flow at x(t):
⃗
d t ∆V(x(t)) ⃗
= ∆V(x(t)) ⃗
Q(x(t)) . (3.23)
Normally Q depends on x⃗ and thus Q(x(t))
⃗ depends on time. For a dissipative system,
one finds ⟨Q⟩ ≤ 0 with a convenient time average ⟨. . . ⟩. Then every volume element
whose corners travel along trajectories shrinks in the course of time and vanishes at
infinite times.
3.2 Fixed points and instabilities | 45

3.2.4 Gradient dynamics

If the circulation (or curl) of f ⃗ vanishes everywhere, a potential function U exists with
∂U(x1 , . . .x N )
fi = − .
∂x i
Then one has
∂2 U
Q(x)⃗ = − ∑
i ∂x2i
and
∂2 U
L ij = − .
∂x j ∂x i
As a consequence, the Jacobi matrix is symmetric and has only real eigenvalues, ex-
cluding the existence of oscillatory instabilities. On the other hand one easily shows
with
N
dU ∂U dx i
=∑ = −|f ⃗|2 ≤ 0
dt i
∂x i dt

that U decays with t monotonically until a ﬁxed point is reached where f ⃗ = 0.

3.2.5 Special case N = 1

The most simple case is N = 1 and reads

d t x = f(x) .

If the solution is bounded, due to the No-Intersection Theorem, every temporal devel-
opment must eventually end in a stable ﬁxed point:

lim x(t) = xs with f(xs ) = 0 and d x f|xs < 0 . (3.24)

t→∞

A potential of
U(x) = − ∫ dx f(x)

always exists, which must have at least one minimum at xs because of (3.24).

3.2.6 Special case N = 2

A direct consequence of the No-Intersection Theorem is the Poincaré–Bendixson The-

orem²:

2 Henri Poincaré, French physicist and mathematician, 1854–1912; Otto Bendixson, Swedish mathe-
matician, 1861–1935.
46 | 3 Dynamical systems

q2 q2 q2

q1 q1 q1

λ2 < λ 1 < 0 λ1 > 0, λ2 < 0 λ 1 > λ2 > 0

q2 q2 q2

q1 q1 q1

Re λi < 0 Re λi = 0 Re λi > 0

Fig. 3.1: Fixed points and their neighborhood in two-dimensional state space, from left to right, top
row: λ i real valued, stable node, saddle point, unstable node. Bottom row: λ i complex valued, stable
focus, center, unstable focus.

If in two-dimensional state space the trajectories are confined to a finite size region
(i.e., do not go to infinity), they either end on a stable fixed point or approach a stable
limit cycle as t → ∞.

A limit cycle is a periodic solution in state space, that is, a closed trajectory x⃗ lc with
x⃗ lc (t) = x⃗ lc (t + T) where T denotes the period.
Near any ﬁxed point, the local behavior of the trajectories is restricted in two di-
mensions to only a few qualitatively different possibilities, shown in Figure 3.1.

3.2.6.1 Example: Van der Pol Oscillator

Van der Pol’s equation reads
d2 x dx
− μ(1 − x2 ) +x=0
dt2 dt
and shows for μ > 0 self-excited nonlinear oscillations as the solution. We shall ex-
amine the system in the form (3.15):
d t x1 = x2
(3.25)
d t x2 = −x1 + μ(1 − x21 ) x2 .
(0) (0)
There exists only one ﬁxed point, x1 = x2 = 0. Its Jacobi matrix turns out to be
0 1
L=( )
−1 μ
3.2 Fixed points and instabilities | 47

and has the eigenvalues

λ12 = μ/2 ± √μ 2 /4 − 1 .
Depending on μ, the ﬁxed point exhibits different characteristics:

μ ≤ −2: stable node λ i ϵ R,

−2 < μ < 0: stable focus λ i ϵ C,
μ = 0: center, crit. point λ i ϵ C,
0 < μ < 2: unstable focus λ i ϵ C,
2 ≤ μ: unstable node λ i ϵ R.

For the source strength introduced in (3.22) one has

Q(x i ) = μ(1 − x21 ) .

Taking polar coordinates x1 = r cos φ, x2 = r sin φ one ﬁnds for r ≫ 1, μ > 0

d t r ≈ −μr3 cos2 φ sin2 φ ≤ 0

and the trajectories remain in a bounded region. For r < 1, it follows from

d t r = μr(1 − r2 cos2 φ) sin2 φ ,

that the region is limited for μ < 0. Without solving the ODEs (3.25) explicitly we obtain
the following scenario:
If μ < 0, all trajectories starting somewhere inside the circle x21 (0) + x22 (0) < 1 end
at the stable fixed point (focus or node) x1 = x2 = 0.
If μ > 0, the fixed point becomes unstable. Since all trajectories remain in a
bounded region in state phase and no other (stable) fixed point exists, the Poincaré–
Bendixson Theorem applies: for t → ∞ all trajectories approach asymptotically a limit
cycle circuiting the unstable fixed point.

3.2.7 Special case N = 3

In the preceding paragraph we saw that in the bounded two-dimensional state space
only regular behavior is possible. All trajectories asymptotically either reach ﬁxed
points or limit cycles. This changes for N = 3. A new object comes into play, the so-
called strange attractor. It can be thought as a certain trajectory having both attractive
and repelling character. The ﬁrst strange attractor was discovered by E. Lorenz³ as a
numerical solution of a system of nonlinear ODEs [1].

3 Edward Lorenz, American mathematician and meteorologist, 1917–2008.

48 | 3 Dynamical systems

3.2.7.1 Example: the Lorenz equations

In the 60s, Lorenz derived a model of three coupled nonlinear ODEs intended for
weather forecasting:
dx1
= −α (x1 − x2 )
dt
dx2
= (δ + 1) x1 − x2 − x1 x3 (3.26)
dt
dx3
= −β x3 + x1 x2 .
dt
Thereby, α, β > 0 denote system parameters and δ is the control parameter or bifur-
cation parameter. The system (3.26) is nowadays called Lorenz equations. It possesses
the three ﬁxed points

(i) x i = 0 , (ii) x1 = x2 = ±√βδ , x3 = δ ,

the latter two existing only if δ ≥ 0. We wish to examine their stability and begin with
the ﬁxed point (i). The Jacobi matrix reads

−α α 0
L = (δ + 1 −1 0) (3.27)
0 0 −β

and has the eigenvalues

1+α 1
λ12 = − ± √(1 + α)2 + 4αδ , λ3 = −β .
2 2

For δ < 0 all λ have a negative real part and the ﬁxed point is a stable node or a stable
focus. At δ ≥ 0 the ﬁxed point turns into a saddle node with one unstable and two
stable directions (or manifolds, that is the eigenvectors of (3.27)). For (ii) (δ > 0), the
Jacobi matrix reads
−α α 0
L=( 1 −1 ∓√βδ) (3.28)
±√βδ ±√βδ −β

with the eigenvalues as roots of the polynomial

P(λ) = λ3 + λ2 (1 + β + α) + λβ(1 + δ + α) + 2βαδ . (3.29)

Since all the coefficients of (3.29) are positive, real eigenvalues may only occur with
λ < 0. Evaluation of (3.29) for certain ﬁxed parameters (α = 10, β = 8/3) shows that
for δ larger than ≈ 0.35 the spectrum has the form λ1 = λ∗2 ϵ C, λ3 < 0. The real parts
3.2 Fixed points and instabilities | 49

Fig. 3.2: The Lorenz attractor for α = 10,

β = 8/3, δ = 24. Shown is the projection of
a trajectory on the x1 -x2 -plane. The two ﬁxed
points (ii) are marked by asterisks.

of the conjugate complex pair become positive if

α(α + β + 3) 451
δ > δc = −1= ≈ 23.7 .
α−β−1 19

Then the two stable focus nodes (ii) turn into unstable saddle foci having a two-
dimensional unstable and a one-dimensional stable manifold. The two ﬁxed points
become oscillatory unstable with the Hopf frequency (δ = δ c )

2αβ(α + 1)
ω c = Im λ1 = √ ≈ 9.62
α−β−1

and ﬁnally the Lorenz attractor emerges, an almost two-dimensional object in state
space; see Figure 3.2.
A peculiarity of (3.26) is its constant source strength

Q L = −α − 1 − β < 0 ,

the Lorenz equations are dissipative in the complete state space. From (3.23) it follows
that the size of each arbitrary volume element ∆V0 exponentially decays in time

∆V(t) = ∆V0 eQ L t .

Every volume element is eventually mapped onto the 2 + ϵ-dimensional attractor.

50 | 3 Dynamical systems

3.3 Hamiltonian systems

We continue with autonomous systems without explicit time-dependence as (3.15).

3.3.1 Hamilton function and canonical equations

In many problems formulated in classical mechanics the forces are derived from a
potential
V(q1 , . . .q N )
where q1 . . . .q N are the (generalized) positions. To achieve a complete description,
the momenta p1 . . .p N need to be known. The Hamilton⁴ function, or for short, the
Hamiltonian
N p2
H(q i , p i ) = ∑ i + V(q i ) (3.30)
i
2m
coincides with the total mechanical energy of the system and is a constant of motion.
The canonical equations are equivalent to Newton’s equations of motion and can be
formulated as [2]
∂H
dt qi =
∂p i
(3.31)
∂H
dt pi = − . i = 1...N .
∂q i
Since we have two sets of variables, each of them with N variables, the motion de-
scribed by (3.31) has 2N degrees of freedom. However, this is the mathematical deﬁ-
nition of a degree of freedom. In mechanics, each pair of conjugated variables q i , p i
corresponds to one degree of freedom, halving their total number. To exclude further
confusion we adopt the notion mechanical degree of freedom for the number of pairs
of variables (here N), degree of freedom for the number of ﬁrst order ODEs in (3.31)
(here 2N).
The system (3.31) has the form of (3.15) but, due to the special form of f ⃗,
N
∂2 H ∂2 H
Q = ∇ ⋅ f⃗ = ∑ [ − ]=0
i
∂q i ∂p i ∂p i ∂q i

applies. The size of volume elements is constant in time, and the dynamics in (3.31) is
called conservative.
If one wishes to know the time dependence of some variable A(q i , p i , t) one must
use the chain rule according to
N N
dA ∂A ∂A dq i ∂A dp i ∂A ∂H ∂A ∂H
− = ∑( + ) = ∑( − ) = {A, H}
dt ∂t i
∂q i dt ∂p i dt i
∂q i ∂p i ∂p i ∂q i

4 William R. Hamilton, Irish mathematician and physicist, 1805–1865.

3.3 Hamiltonian systems | 51

where the abbreviation

N
∂A ∂B ∂A ∂B
{A, B} ≡ ∑ ( − )
i
∂q i ∂p i ∂p i ∂q i
is called the Poisson bracket. Nonexplicitly time-dependent variables (∂A/∂t = 0)
for which the Poisson bracket vanishes are constants of motion (dA/dt = 0). This
of course is especially true for H.

3.3.2 Symplectic integrators

The structure created by the ﬂow (3.31) has symplectic geometry [2]. Normally this
geometry is destroyed when (3.31) is integrated numerically. Then a nonvanishing
source strength is generated and constants of motion, especially the total energy, are
no longer conserved. So-called symplectic methods have been developed to avoid
such problems. Here we can only give a short overview, see also [3]. For the sake of
simplicity we restrict ourselves to a two-dimensional phase space corresponding to
one mechanical degree of freedom, that is H = H(p, q) and
∂H ∂H
dt q = , dt p = − . (3.32)
∂p ∂q
We abbreviate x⃗ = (q, p) and write (3.32) as
⃗ x]⃗
d t x⃗ = H[ (3.33)
with the (in general nonlinear) operator

⃗ x]⃗ = ( ∂H/∂p| x⃗ ) .
H[
− ∂H/∂q| x⃗

A remark on the notation: writing H[ ⃗ x]⃗ indicates that H⃗ acts on x,⃗ and the result of
the operation is a vector of the same dimension. Successive operations of H⃗ are then
denoted by H[ ⃗ H[.
⃗ . . x]⃗ . . . ]. It is convenient to omit the brackets and write H⃗ x⃗ instead
⃗ x].
of H[ ⃗ In the same manner we write products of different operators, e.g.,

A⃗ B⃗ x⃗ ≡ A[⃗ B[⃗ x]]

⃗ .
⃗
As already done in (3.9) we may introduce the time-evolution operator U(t)
x(t) ⃗ x(0)
⃗ = U(t) ⃗

with (an autonomous system assumed)

⃗
U(t) ⃗ = exp(t(T⃗ + V))
= exp(t H) ⃗ , (3.34)
which can be seen by insertion into (3.33). In the last step, we have deﬁned the oper-
ators
∂T/∂p 0
T⃗ = ( ) , V⃗ = ( ) (3.35)
0 −∂V/∂q
52 | 3 Dynamical systems

with
H(p, q) = T(p) + V(q) .
Thus the Hamiltonian should be separable into a momentum-dependent kinetic part
and into a space-dependent potential part. Each part operator

U⃗ T (t) = exp(t T)⃗ , U⃗ V (t) = exp(t V)⃗ (3.36)

generates a time-evolution being trivial in a certain sense. The evolution U⃗ T is that of

a free particle where p =const., q = (p/m)t, whereas U⃗ V describes the ‘motion’ of an
inﬁnitely heavy mass point, where q =const., p = −(dV/dq)t.
The trick to constructing symplectic methods is the conversion of (3.34) into
products of U⃗ T and U⃗ V . But since T⃗ and V⃗ do not commute, we cannot simply
write U⃗ = U⃗ T U⃗ V . The transformation must be done applying the Campbell–Baker-
Hausdorff relation
exp(C) = exp(A) exp(B)
1 1 (3.37)
C = A + B + [A, B] + ([A, [A, B]] + [B, [B, A]]) + . . .
2 12
([A, B] = AB − BA denotes the commutator). If A, B is of O(t), then [A, B] = O(t2 ),
[A, [A, B]] = O(t3 ) and so on. We divide t into many small intervals (time steps) of
⃗
∆t and instead of U(t), ⃗
we look at U(∆t). The formula (3.37) can then be considered
an expansion with respect to ∆t and can be truncated at a certain desired order in ∆t.
After a longer manipulation it can be shown that
⃗
U(t) = U⃗ T (a n ∆t)U⃗ V (b n ∆t)U⃗ T (a n−1 ∆t). . . U⃗ T (a1 ∆t)U⃗ V (b 1 ∆t) + O(∆t n+1 ) (3.38)

where the coefficients a k , b k can be chosen arbitrarily but must fulﬁll the constraints
n n
∑ ak = 1 , ∑ bk = 1 . (3.39)
k k

Since both operators (3.36) are symplectic, this holds for the product (3.38) as well.
Taking certain a k , b k , we may then generate methods having a given order in the time
step, where all of them conserve the symplectic geometry of the Hamiltonian ﬂow.
The relation (3.38) is strongly simpliﬁed if we take

T⃗ T⃗ x⃗ = 0 , V⃗ V⃗ x⃗ = 0

into account. The Taylor expansion stops after the second term
1
U⃗ T = exp(∆t T)⃗ = 1 + ∆t T⃗ + ∆t2 T⃗ T⃗ + . . . = 1 + ∆t T⃗ ,
2
which also holds for U⃗ V . Then we can substitute

U⃗ T = 1 + ∆t T⃗ , U⃗ V = 1 + ∆t V⃗

in (3.38).
3.3 Hamiltonian systems | 53

Let us state the scheme in the lowest order n = 1. Due to (3.39) a1 = b 1 = 1 and
with (3.38) we obtain

x(t ⃗
⃗ + ∆t) = U(∆t) ⃗ = (1 + ∆t T)⃗ (1 + ∆t V)⃗ x(t)
x(t) ⃗ + O(∆t2 ) .

Taking (3.35) and ∂T(p)/∂p = p/m we ﬁnally have

q n+1 p/m qn 0
( ) = (1 + ∆t ( )) [( ) + ∆t ( ∂V )]
p n+1 0 pn − ∂q (q n )

qn 0 (p n − ∆t ∂V
∂q (q n )) /m)
=( ) + ∆t ( ∂V ) + ∆t (
pn − ∂q (q n ) 0

qn p n+1 /m
=( ) + ∆t ( ∂V ) . (3.40)
pn − ∂q (q n )

Thus, the iteration formula for one time step for p and q read
∂V
p n+1 = p n − (q n ) ∆t
∂q (3.41)
p n+1
q n+1 = qn + ∆t .
m
The one-step scheme (3.41) is analogous to the two-step scheme
∆t2 ∂V
q n+1 = 2q n − q n−1 − (q n ) , (3.42)
m ∂q
which can easily be shown by eliminating p n . The scheme (3.42) is also known as the
Verlet algorithm. It can be derived alternatively from Newton’s second law discretizing
the second time derivative by
d2 q q n+1 − 2q n + q n−1
→ .
dt2 ∆t2
A particular scheme of the 3rd order is generated from (3.38), taking

a1 = a2 = 1/2 , b1 = 1 , b2 = 0 ,

and
⃗
U(t) = U⃗ T (∆t/2) U⃗ V (∆t) U⃗ T (∆t/2) + O(∆t3 ) .
Without going into detail we quote the iteration rule
∂V
p n+1/2 = p n − (q n ) ∆t/2
∂q
p n+1/2
q n+1 = qn + ∆t (3.43)
m
∂V
p n+1 = p n+1/2 − (q n+1 ) ∆t/2
∂q
which is similar to the ‘Leapfrog’ version of Verlet’s algorithm where space and mo-
mentum variables are evaluated at ∆t/2 differing times.
54 | 3 Dynamical systems

3.3.2.1 Example: Hénon–Heiles model

To demonstrate the advantage of the symplectic method compared to a simple Euler
method, we shall explore in some detail the Hénon–Heiles model [4]. The model de-
scribes the two-dimensional motion of a star near the center of a disk-shaped galaxy.
Taking a suitable scaling (m = 1), the Hamiltonian reads
1 2 1 2
H(p1 , p2 , q1 , q2 ) = E = p + p + V(q1 , q2 ) (3.44)
2 1 2 2
with
1 2 1 2 1
q + q + q21 q2 − q32 .
V(q1 , q2 ) = (3.45)
2 1 2 2 3
A transformation to polar coordinates immediately shows the 3-fold rotational sym-
metry of V:
1 1
V(r, φ) = r2 ( + r sin 3φ) .
2 3
The motion of the star is bounded close to the origin as long as the total energy (3.44)
1
E < V0 = .
6
For a larger E, the star may leave the triangle (see Figure 3.3), ﬁrst in the directions of
φ = 90°, 210° and 330°.

–1

–2
–2 –1 0 1 2

Fig. 3.3: Potential V(q 1 , q 2 ) of the Hénon–Heiles model. Inside the triangle around the ori-
gin the motion of the star is bounded if its total energy is less than 1/6. Solid lines: V =
1/6, 1/12, 1/24, 1/48, 1/96, dashed: V = 1/3, 2/3, 1.
3.3 Hamiltonian systems | 55

The canonical equations (3.31) read

d t q1 = p1
d t q2 = p2
(3.46)
d t p1 = −q1 − 2q1 q2
d t p2 = −q2 − q21 + q22 .
The iteration (3.41) turns into
p1,n+1 = p1,n − (q1,n + 2q1,n q2,n ) ∆t
p2,n+1 = p2,n − (q2,n + q21,n − q22,n ) ∆t
(3.47)
q1,n+1 = q1,n + p1,n+1 ∆t
q2,n+1 = q2,n + p2,n+1 ∆t .
The Euler algorithm with the same order looks almost identical. One only has to sub-
stitute p i,n+1 with p i,n in the third and fourth equation. However, the results differ
vastly. For the initial conditions
q1 = 0.1 , q2 = 0 , p1 = 0 , p2 = 0.1 , (E = 0.01)
both methods provide nonclosed quasiperiodic orbits around the origin. For the Euler
code, their distance from the center increases at each revolution; after some 100 cycles
the star leaves the bounded region and disappears into inﬁnity. Accordingly the energy
increases constantly, whereas using the symplectic method the star remains in the
bounded region and its energy is constant for many 100 000 cycles, as it should be
(Figure 3.4).

Fig. 3.4: Comparison between the Euler and the symplectic method, both of the same order ∆t 2 . Left
frame: r = √ q 21 + q 22 over t; right frame: difference of the total energies.
56 | 3 Dynamical systems

3.3.3 Poincaré section

For larger values of E and/or for different initial conditions, the trajectories can be-
come more involved, and chaotic regions may appear. The trajectories can be visual-
ized in a comfortable way with the Poincaré section method. To this end, one places an
(N−1)-dimensional hyperplane, the Poincaré plane, somewhere in the N-dimensional
phase space and marks the points of intersection of a trajectory for many revolutions.
Periodic motion is then correlated to single isolated points on the plane, quasiperiodic
behavior to curves, and chaotic regions to fractal objects having a dimension between
one and two on the Poincaré plane.
(n)
Let ξ i , i = 1 . . . N − 1 be the coordinates of the nth intersection with the plane.
(n)
Then from each set ξ i the values for the next intersection point follows uniquely.
Instead of a system of N ODEs, one ﬁnds the relation

ξ ⃗ (n+1) = f ⃗(ξ ⃗ (n) ) (3.48)

which is nothing other than an (N − 1)-dimensional map, called a Poincaré map. How-
ever, computing f ⃗ in (3.48) requires solving the ODEs of the full system.

3.3.3.1 Example: Hénon–Heiles model

For the Poincaré plane we choose q1 = 0 and obtain for (3.48)
ξ1 = q2 (t s ) , ξ2 = p1 (t s ) , ξ3 = p2 (t s )
with t s from q1 (t s ) = 0. Since a conserved quantity exists, namely the total energy E
(3.44), we can reduce the three-dimensional hyperplane further to a two-dimensional
plane, for instance q2 (t s ), p2 (t s ). Since p21 (t s ) ≥ 0, all intersection points on this plane
must be located inside the region
2 3
−g(q2 ) ≤ p2 ≤ g(q2 ) with g(q2 ) = √2E − q22 + q .
3 2
A code for the symplectic integration, plotting of trajectories, and of the Poincaré sec-
tion is provided in [5]. The initial values q2 (0), p2 (0) can be set using the mouse cursor,
p1 (0) for a given E is computed from (3.44).
Figure 3.5 shows some solutions for E = 0.12 and different initial values on the
Poincaré plane (p2 , q2 ) as well as in position space (q1 , q2 ).

Bibliography

[1] E. N. Lorenz, Deterministic nonperiodic ﬂow, J. Atmos. Sci. 20, 130 (1963).
[2] J. V. José and E. J. Saletan, Classical Dynamics, Cambridge Univ. Press (1998).
[3] J. M. Thijssen, Computational Physics, Cambridge Univ. Press (2007).
[4] M. Hénon and C. Heiles, The applicability of the third integral of motion: Some numerical experi-
ments, Astrophys. J. 69 (1964).
[5] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
Bibliography | 57

0.5
0.5

0 0

–0.5
–0.5
–1 –0.5 0 0.5 1 –0.5 0 0.5

0.5

0 0

–0.5
–0.5
–1 –0.5 0 0.5 1 –0.5 0 0.5

0.5
0.5

0 0

–0.5
–0.5
–1 –0.5 0 0.5 1 –0.5 0 0.5

Fig. 3.5: Poincaré plane (left), trajectories in position space (right) for E = 0.12, and several initial
conditions. Dashed: Bounded area deﬁned by the total energy.
4 Ordinary differential equations I,
initial value problems
4.1 Newton’s mechanics

The present chapter mainly deals with problems arising from classical mechanics. We
shall further investigate ordinary differential equations (ODEs) having the form

G k [x i (t), ẋ i (t), ẍ i (t)] = 0 .

The highest derivative occurring is now of the second order, normally in time. From
here on we denote the ﬁrst time derivative of x by x,̇ the second by x.̈

4.1.1 Equations of motion

The second law of Newton provides equations of motion for N mass points at the po-
sitions r ⃗i (t), i = 1. . .N

m i r ̈⃗i = F⃗ i (r1⃗ , . . . r ⃗N , r1̇⃗ , . . . r ̇⃗N , t) (4.1)

with the force F⃗ i acting on the mass point i. The necessary initial conditions are nor-
mally given by positions and velocities at t = t0

r ⃗i (t0 ) = r ⃗i (0) , r ̇⃗i (t0 ) = v⃗ i (0) . (4.2)

Often, the forces do not explicitly depend on time,

F⃗ i = F⃗ i (r ⃗j , r ̇⃗j )

and (4.1) is called an autonomous system. If, on the other hand

F⃗ i = F⃗ i (r ⃗j )

with
∂V(r1⃗ . . . r ⃗N )
F⃗ i (r ⃗j ) = − ,
∂ r i⃗
the system (4.1) is denoted as conservative. In this case, the total energy
1
E= ∑ m i (r ̇⃗i )2 + V(r1⃗ . . . r ⃗N ) (4.3)
2 i

constitutes a constant of motion. For a translational invariant system, a further con-

dition
F⃗ i = F⃗ i (r ⃗j − r ⃗k , r ̇⃗j , t) .
holds for the forces.

https://doi.org/10.1515/9783110515145-004
4.1 Newton’s mechanics | 59

4.1.2 The mathematical pendulum

We commence our elaboration with the equation of motion of the damped one-
dimensional mathematical pendulum:

φ̈ + α φ̇ + Ω20 sin φ = 0 , Ω20 = g/ℓ , (4.4)

realized for instance by a bar pendulum with a massless bar. Here, ℓ is the bar length,
α > 0 the damping rate and φ denotes the displacement angle from the vertical. The
equivalent ﬁrst-order system for the two variables φ(t) and ω(t) reads

φ̇ = ω
(4.5)
ω̇ = −α ω − Ω20 sin φ

with the initial conditions

φ(0) = φ0 , ω(0) = ω0 .

The system (4.5) has two ﬁxed points (rest positions) φ̇ = ω̇ = 0:

(0) (0)
φ0 = 0 , φ1 = π ,
(0) (0)
where φ0 has the properties of a stable focus, φ1 of an unstable saddle node. Mul-
tiplication of (4.4) by φ̇ and integration with respect to t yields

1 2
φ̇ − Ω20 cos φ = E0 − R(t) (4.6)
2
with the integration constant E0 and a monotonically growing function

R(t) = α ∫ dt φ̇ 2 .

The expression on the left-hand side of (4.6) can be identiﬁed with the total mechan-
ical energy E/mℓ2 of the system, R corresponds to the energy dissipated by friction
and transformed to heat:
R(t) = E0 − E(t) .

Since E(t) ≥ −Ω20 and Ṙ ≥ 0, the rest position φ = 0, ω = 0 is asymptotically reached

with t → ∞. During this process the mechanical energy

R(t → ∞) = E0 + Ω20

has been converted to heat and the pendulum has come to rest, independently of its
initial conditions. As long as E > E c with E c = Ω20 , the mass point passes through the
upper unstable rest position (rotation), and if E < E c one observes oscillations around
the lower rest position, also called libration.
60 | 4 Ordinary differential equations I

For the conservative case without damping (α = 0), one ﬁnds

E = E0

and the pendulum, once moving, never comes to rest. Depending on E0 the motion
persists either as oscillation or rotation. For E0 = E c the motion is an inﬁnitely long
approach to the upper rest position. If the initial condition is also chosen with the
upper rest position, the trajectory in phase space (φ-ω-plane) consists of a closed loop
with initial and end points at the very same ﬁxed point, also called a homoclinic orbit
or separatrix.

4.2 Numerical methods of the lowest order

4.2.1 Euler method

Next we try to solve the system (4.5) numerically using the Euler method. Introducing
the discrete variables φ n = φ(n∆t), ω n = ω(n∆t), and the time step ∆t, the iteration
rule reads
φ n+1 = φ n + ω n ∆t
(4.7)
ω n+1 = ω n − (α ω n + Ω20 sin φ n ) ∆t .
Figure 4.1 shows phase space and energy over t for different α and different time steps.
For α = 0 (Figure 4.1 top) we expect a constant energy, which is obviously not the case.
Moreover, all trajectories should be closed (periodic motion) instead of spiraling out-
wards. The simple Euler forward method produces an artificial negative(!) damping,
the energy increases as well as the amplitude. After a certain time the separatrix is
crossed and the oscillations turn into rotations. This behavior is qualitatively inde-
pendent of ∆t. Figure 4.1 shows the damped case in the middle; the result looks more
realistic. The trajectories spiral inwards and the energy reaches the value −Ω20 asymp-
totically. In contrary the behavior shown in Figure 4.1 bottom is again ruled by numer-
ical artifacts. For larger time steps (∆t = 0.15) again a kind of ‘negative damping’ is
obtained and the energy increases again.
Let us draw some preliminary conclusions:
– The results for α = 0 are all wrong (negative numerical ‘damping’).
– For α > 0 we find qualitatively correct results for small ∆t.
– Also for the damped case the energy grows continuously and the trajectories get
off of the stable (!) fixed point if ∆t is chosen too large.

4.2.2 Numerical stability of the Euler method

As we have already remarked in Section 3.1 the time step of the Euler method is
bounded from above by the demand for numerical stability. We start with the general
4.2 Numerical methods of the lowest order | 61

6
2

0
2

–2
0

–2 0 2 0 100 200 300

0.9

2
0.92

0.94
0

0.96

–2 –0.98

–1
–2 0 2 0 100 200 300

2
0

–0.5

–2

–1
–2 0 2 0 100 200 300

Fig. 4.1: Trajectories in phase space (left) and total energy over t for the damped pendulum
(Ω0 = 1), computed using Euler’s method. Top α = 0, ∆t = 0.05, middle: α = 0.1, ∆t = 0.05,
bottom: α = 0.1, ∆t = 0.15. The behaviors shown in the top and the bottom frames do not even
qualitatively correspond to reality but are rather a product of numerical errors and instability.
62 | 4 Ordinary differential equations I

autonomous system
̇⃗ = f ⃗(x(t))
x(t) ⃗ . (4.8)
The Euler method provides the discretization

x(t ⃗ + f ⃗(x(t))∆t
⃗ + ∆t) = x(t) ⃗ + O(∆t2 ) . (4.9)

Let x⃗ (0) be a stable ﬁxed point f ⃗(x⃗ (0) ) = 0. Then we may linearize (4.9) with x(t)
⃗ =
(0)
x⃗ + u(t) ⃗ for small deviations u⃗ around this ﬁxed point:

⃗ + ∆t) = u(t)
u(t ⃗ ∆t
⃗ + L ⋅ u(t) (4.10)

with the Jacobi matrix

󵄨
∂f i 󵄨󵄨󵄨
L ij = 󵄨
∂x j 󵄨󵄨󵄨󵄨x⃗ (0)
. (4.11)

Due to the stability of x⃗ (0) , the eigenvalues of L must all have a nonpositive real part.
We write (4.10) as
u(t ⃗
⃗ + ∆t) = Q Ex ⋅ u(t) (4.12)
with
QEx = 1 + ∆t L .
To reﬂect the behavior close to a stable ﬁxed point, the deviations u⃗ should not grow
while iterating (4.12). Thus the spectral radius of Q Ex must be less or equal to one:

ρ(Q Ex ) ≤ 1 . (4.13)

This condition yields an upper boundary for ∆t.

To demonstrate the method we take again the mathematical pendulum. Lineariz-
ing around the stable ﬁxed point φ = ω = 0 yields the Jacobi matrix

0 1
L=( ) .
−Ω20 −α

For the sake of simplicity we exclude here the overdamped case and assume α < 2Ω0 .
Then the spectral radius of Q Ex turns out to be

ρ = √1 − α ∆t + ∆t2 Ω20

and the stability criterion reads

α
∆t ≤ .
Ω20
This of course explains the strange behavior of the solutions shown in Figure 4.1. The
stability condition at Ω0 = 1 simply reads

∆t ≤ α ,

which is only fulﬁlled for the solutions of the middle row in Figure 4.1. For the un-
damped case α = 0 the Euler method is unstable for arbitrarily small steps ∆t.
4.2 Numerical methods of the lowest order | 63

4.2.3 Implicit and explicit methods

Since the dependent variables at time t+ ∆t occur explicitly on the left-hand side of the
iteration rule, the Euler forward method belongs to the group of explicit integration
schemes. For an implicit method instead of (4.9) one ﬁnds the rule

x(t ⃗ + f ⃗(x(t
⃗ + ∆t) = x(t) ⃗ + ∆t))∆t + O(∆t2 ) . (4.14)

To compute the variables at the next iteration step, one must ﬁrst solve (4.14) for
⃗ + ∆t). This is only possible in a unique way for a linear function f ⃗,
x(t

f ⃗(x)⃗ = A ⋅ x⃗ ,

and one obtains

⃗ + ∆t) = x(t)
(1 − ∆t A) ⋅ x(t ⃗
or
x(t ⃗
⃗ + ∆t) = Q Im ⋅ x(t) (4.15)
with
Q −1
Im
= 1 − ∆t A .
Now the stability is determined from the spectral radius of (1−∆t A)−1 , which normally
yields a much weaker restriction. One ﬁnds for the linearized pendulum
1
ρ(Q Im ) = ,
√1 + α∆t + Ω20 ∆t2

0.4
E

Δt = 0.01
0.2

Δt = 0.03

Δt = 0.1
0
0 50 100 150 200 250 300
t

Fig. 4.2: Harmonic oscillator, implicit ﬁrst-order scheme, α = 0. Although closed trajectories and
a constant energy are expected, the method computes a damped solution. The artiﬁcial numerical
damping goes with ∆t. Despite being stable for arbitrary time steps, the implicit method is still not
very accurate.
64 | 4 Ordinary differential equations I

which is less than one for all ∆t, even if α = 0. Thus the implicit method for the lin-
earized pendulum (that is the harmonic oscillator) is unconditionally stable and there-
fore much more suitable than the explicit scheme. On the other hand, the numerical
truncation error is still of order ∆t2 and this time generates a positive damping, lead-
ing to decaying oscillations in time even if α = 0, Figure 4.2.

4.3 Higher order methods

The Euler method converges slowly and provides inaccurate and partially nonphys-
ical results. Mainly for conservative systems this may lead to a qualitatively wrong
behavior of the solutions. To increase the accuracy, higher order methods have been
developed where the truncation error is of order ∆t3 or even better. In the following we
introduce two higher order methods. Both are explicit and belong to the Runge–Kutta
class.

4.3.1 Heun’s method

We begin with the one-dimensional system

dx
= f(t, x) , (4.16)
dt
which is generalized later to n dimensions. We integrate (4.16) over t
t+∆t t+∆t
dx
∫ dt󸀠 = x(t + ∆t) − x(t) = ∫ dt󸀠 f(t󸀠 , x(t󸀠 )) (4.17)
dt󸀠
t t

and obtain the still exact iteration rule

t+∆t

x(t + ∆t) = x(t) + ∫ dt󸀠 f(t󸀠 , x(t󸀠 )) . (4.18)

If we approximate the integral by the square rule according to

t+∆t

∫ dt󸀠 f(t󸀠 , x) ≈ f(t, x(t)) ∆t ,

the Euler method results. Taking the more accurate trapezoidal rule
t+∆t
∆t
∫ dt󸀠 f(t󸀠 , x) ≈ (f(t, x(t)) + f (t + ∆t, x(t + ∆t))) ,
2
t
4.3 Higher order methods | 65

one obtains for (4.18),

∆t
x(t + ∆t) = x(t) + (f(t, x(t)) + f (t + ∆t, x(t + ∆t))) . (4.19)
2
Since x(t + ∆t) now occurs on the right-hand side as well, this is an implicit (or better a
semi-implicit) algorithm. To convert (4.19) to an explicit scheme one computes x(t+∆t)
on the right-hand side applying the Euler method

x(t + ∆t) = x(t) + f(t, x(t)) ∆t

and ﬁnally ﬁnds

∆t
x(t + ∆t) = x(t) + (f(t, x(t)) + f (t + ∆t, x(t) + f(t, x(t)) ∆t)) . (4.20)
2
This is called Heun’s method, or sometimes the second-order Runge–Kutta method.

4.3.1.1 Accuracy
To which order of ∆t is the truncation error in Heun’s method? We write (4.20) as (for
the sake of simplicity we assume f(x) instead of f(t, x))
∆t
x(t + ∆t) − x(t) = (f(x(t)) + f (x(t) + f(x(t)) ∆t))
2
and expand the left-hand side with respect to ∆t
dx 1 d2 x 2 1 d3 x 3
L.H.S. = ∆t+ ∆t + ∆t + . . .
dt 2 dt2 6 dt3
1 df 1 d2 f df 2
= f∆t + f∆t2 + (f 2 2 + f ( ) ) ∆t3 + . . . ,
2 dx 6 dx dx
the right-hand side with respect to f∆t:
∆t df 1 d2 f 2 2
R.H.S. = (2f + f∆t + f ∆t ) + . . . ,
2 dx 2 dx2

where . . . denote terms of the order ∆t4 . Both sides coincide up to the order ∆t2 , that is,
the truncation error is of the order ∆t3 and thus one order smaller than that of Euler’s
method.

4.3.1.2 Numerical stability

Similar to the Euler method we may formulate one iteration step for Heun’s method
after linearization around a ﬁxed point according to

⃗ + ∆t) = Q H ⋅ x(t)
x(t ⃗

with
1 2 2
Q H = 1 + ∆t L + ∆t L
2
66 | 4 Ordinary differential equations I

and the Jacobi matrix L as in (4.11). Again, numerical stability is given by the restriction
ρ(Q H ) ≤ 1, which eventually leads to
󵄨󵄨 󵄨󵄨
󵄨 1 󵄨
ρ = max 󵄨󵄨󵄨1 + ∆tλ i + ∆t2 λ2i 󵄨󵄨󵄨 ≤ 1 ,
i 󵄨󵄨 2 󵄨󵄨
where λ i are the eigenvalues of L. For the example of the harmonic oscillator with
α 1
λ12 = − ± i√4Ω20 − α 2
2 2
wie obtain the stability limit:
1 2 1 1
−α + α ∆t − αΩ20 ∆t2 + Ω40 ∆t3 = 0 .
2 2 4
This provides an upper time step which is for α > 0, larger than that for Euler’s method
(Figure 4.3, ∆t c ≈ 1.4). However, for the undamped case α = 0, Heun’s method is still
unstable for all ∆t.
Taking the symplectic method from Chapter 3, one iteration step for the harmonic
oscillator can be formulated according to (3.41):

φ φ
( ) = Qs ( )
ω n+1 ω n

with (see Problems)

1 − Ω20 ∆t2 ∆t − α∆t2
Qs = ( ) . (4.21)
−Ω20 ∆t 1 − α∆t
n
er

er
Heu
Eul

Eul

sympl.
n

tta

tta
Heu

Runge–Ku

Runge–Ku
ρ

ρ
sym
pl.

Δt Δt

Fig. 4.3: Spectral radii over ∆t for the harmonic oscillator, Ω0 = 1, left: damped, α = 1/2,
right: undamped, α = 0. Especially for the undamped case the symplectic method discussed in
Chapter 3 is superior to any other, even higher-order method.
4.3 Higher order methods | 67

The stability is given by the spectral radius of Q s . As long as both eigenvalues are
complex one ﬁnds
ρ(Q s ) = √| Det Q s | = √|1 − α∆t| .
For the special (Hamiltonian) case of the undamped oscillator α = 0, one obtains the
pleasant result ρ = 1, and the symplectic method should be stable for ∆t < 2/α. How-
ever, for ∆t ≥ (−α + √α 2 + 4Ω20 )/2Ω20 the complex conjugate pair of eigenvalues of Q s
becomes real valued and the spectral radius quickly exceeds one; compare Figure 4.3.
Despite this restriction and especially for a weak damping (α ≪ Ω), the symplectic
method is much better suited than other methods.

4.3.2 Problems

1. For the Crank–Nicolson method, the right-hand side of the ODEs is written as a
linear combination of explicit and implicit terms. Let y(x) be the solution of
dy(x)
= f(y(x))
dx
̃
and y(x) ̃ 0 ) = y(x0 ),
the numerically obtained approximation. Starting with y(x
the Crank–Nicolson scheme provides
̃ 0 + ∆x) − y(x
y(x ̃ 0) 1
= [f(y(x ̃ 0 + ∆x))] .
̃ 0 )) + f(y(x (4.22)
∆x 2
Show that the numerical error after one time step reads

̃ 0 + ∆x) − y(x0 + ∆x)| = O(∆x3 ) .

|y(x

2. Derive the ampliﬁcation matrix (4.21) for the symplectic method.

4.3.3 Runge–Kutta method

Taking a higher order algorithm increases both step size and accuracy. Even higher
order methods are possible, however, the numerical effort would also increase. For
Heun’s method one has to evaluate f at two different function values for each itera-
tion step. To reduce the algorithm to practice one should always ﬁnd a compromise
between accuracy and effort.
An algorithm that has stood the test of time is the already in 1895 developed
Runge–Kutta method (RK). RK exists in different orders; normally the fourth order
scheme (which we shall abbreviate by RK4) is taken. For the sake of simplicity we
explain the principle at a method of the second order.
Let x = x(t) be the desired solution of the ODE

ẋ = f(t, x(t)) .
68 | 4 Ordinary differential equations I

We expand x at t + ∆t/2

∆t 1 ∆t 2
x(t) = x(t + ∆t/2) − ̇ + ∆t/2) + ( ) x(t
x(t ̈ + ∆t/2) + O(∆t3 )
2 2 2
∆t 1 ∆t 2
x(t + ∆t) = x(t + ∆t/2) + x(t ̈ + ∆t/2) + O(∆t3 )
̇ + ∆t/2) + ( ) x(t
2 2 2
and subtract the two equations

̇ + ∆t/2) + O(∆t3 ) .
x(t + ∆t) = x(t) + ∆t x(t

It results in an iteration scheme of the order of ∆t2 . Similar to Heun’s method one
must know ẋ = f for a time later than t to compute the right-hand side. Again this is
accomplished by the Euler method

̇ + ∆t/2) = f (t + ∆t/2, x(t + ∆t/2)) = f (t + ∆t/2, x(t) + ∆t/2 f(t, x(t))) + O(∆t2 )
x(t

and one iteration step takes the following form:

k 1 = ∆t f(t, x(t))
k 2 = ∆t f(t + ∆t/2, x(t) + k 1 /2) (4.23)
x(t + ∆t) = x(t) + k 2 .

Thus an explicit scheme of order ∆t2 is constructed where f must be evaluated twice at
each step. Although it looks different, the scheme (4.23) is similar to Heun’s method
(4.20) up to the order ∆t3 , which can be shown by expanding (4.23) with respect to
∆t/2 at t and t + ∆t.
In the same manner the fourth order RK scheme is derived:

k 1 = ∆t f(t, x(t))
k 2 = ∆t f(t + ∆t/2, x(t) + k 1 /2)
k 3 = ∆t f(t + ∆t/2, x(t) + k 2 /2) (4.24)
k 4 = ∆t f(t + ∆t, x(t) + k 3 )
1
x(t + ∆t) = x(t) + (k 1 + 2 k 2 + 2 k 3 + k 4 ) ,
6
and is easily coded in FORTRAN95:

SUBROUTINE rk4(x,t,n,dt,eq)
C integrates the ODE system defined in eq from t to t+dt
C x dependent variables x(n)
C t independent variable
C n number of dependent var.
C dt time step
4.3 Higher order methods | 69

C eq the ride-hand sides of the ODEs (subroutine)

REAL, DIMENSION(n) :: x,f1,f2,f3,f4
CALL eq(f1,x,t); f1=dt*f1
CALL eq(f2,x+.5*f1,t+dt/2.); f2=dt*f2
CALL eq(f3,x+.5*f2,t+dt/2.); f3=dt*f3
CALL eq(f4,x+f3,t+dt)
x=x+(f1+2.*f2+2.*f3+f4*dt)/6.
END

The error for RK4 is of the order ∆t5 and the restriction for the step size originates now
from the stability condition (the detailed computation is left for the reader)
󵄨󵄨 1 4 4 󵄨󵄨󵄨󵄨
󵄨 1 1
ρ = max 󵄨󵄨󵄨1 + ∆tλ i + ∆t2 λ2i + ∆t3 λ3i + ∆t λ i 󵄨󵄨 < 1 . (4.25)
i 󵄨󵄨 2 6 24 󵄨󵄨

An evaluation of ρ is also shown in Figure 4.3. As expected, the stability region is much
larger than those of the other schemes having a lower order.

4.3.3.1 Example: mathematical pendulum with RK4

As an application we use RK4 to solve the equations for the nonlinear pendulum with
f ⃗ from (4.5). Note that even for the problematic conservative case α = 0 convergent
results and constant energy are obtained if the time step is not chosen to be too large
(ρ ≈ 1 in (4.25)); see Figure 4.4. A code section computing a trajectory with initial
values φ = 2, ω = 0 could read as

PROGRAM pendulum_rk4
REAL, DIMENSION(2) :: y ! variables phi, omega
...
y=(/2.,0./) ! initial values for phi and omega
t=0.; dt=0.01
DO WHILE(t.LT.tend) ! time iteration loop begins
t=t+dt
CALL rk4(y,t,2,dt,pendulum_deq) ! compute one time step with rk4
...
ENDDO
...

If the subroutine rk4 has been placed in the program library (see Appendix B), only
the subroutine pendulum_deq has to be specified in the file below the main section
(MAIN), here pendulum_rk4. This can be done even inside MAIN using the CONTAINS
statement. Then the parameters, here omega and alpha, are global and also defined in
the subroutine. Moreover, MAIN recognizes pendulum_deq as the name of a subrou-
tine and transfers it accordingly to rk4.
70 | 4 Ordinary differential equations I

0
2
0.2

–0.4
0

–0.6

–2 –0.8

–1
–2 0 2 0 100 200 300

0.5
2

–0.5

–2

–1
–2 0 2 0 100 200 300

Fig. 4.4: The undamped mathematical pendulum (α = 0) with RK4. For a time step too large, numer-
ical friction distorts the result (top, ∆t = 1), for ∆t = 0.1 (bottom) the energy is conserved well for
many periods (Ω0 = 1, T = 2π).

...
CONTAINS
SUBROUTINE pendulum_deq(rhside,y,t)
REAL, DIMENSION(2) :: rhside,y
rhside(1)=y(2)
rhside(2)=-alpha*y(2)-omega**2*SIN(y(1))
END SUBROUTINE pendulum_deq

END PROGRAM pendulum_rk4

The other, more traditional ‘FORTRAN77-like’ way is to avoid the CONTAINS statement
and the sometimes confusing use of global variables by deﬁning pendulum_deq by an
4.3 Higher order methods | 71

EXTERNAL statement in MAIN. The parameter transfer to the subroutine deﬁning the
right-hand sides of the ODEs can then be realized by a COMMON block.

4.3.3.2 RK4 with adaptive step size

Up to here the step size ∆t was constant and, besides stability restrictions, chosen
rather arbitrarily. This is the most easiest way but may not always be the optimal one.
A step width too large renders the results inaccurate, while a step size too small unnec-
essarily increases the number of iterations and therefore the computational effort. An
algorithm adapting the step size during the computations can improve the situation
considerably. On the one hand one can thus prescribe a desired accuracy (or upper
error limit); on the other hand the method can automatically decrease the step size at
times where x⃗ is slowly varying and vice versa.
⃗ and
Using RK4 the truncation error is of O(∆t5 ). Let us start the integration at x(t)
⃗ ⃗
compare two numerically computed x1 (t + ∆t), x2 (t + ∆t), one found with step size ∆t
(one iteration), the other with ∆t/2 and two consecutive iterations. Let d(∆x) be the
euclidean distance
d(∆t) = |x⃗ 1 − x⃗ 2 | ,
then
d(∆t) = |c|∆t5
where c depends somehow on the ﬁfth derivative of x⃗ at time t. If we compute d for
two different step sizes we may eliminate c and ﬁnd

d(∆t1 ) ∆t1 5
=( ) .
d(∆t2 ) ∆t2
This can be solved for ∆t2 :

d(∆t2 ) 1/5
∆t2 = ∆t1 ( ) .
d(∆t1 )
Now we compute one iteration with a certain ﬁxed ∆t1 and require d(∆t2 ) = ϵ as the
desired accuracy. Then the step width to achieve this accuracy reads
1/5
ϵ
∆t2 = ∆t1 ( ) .
d(∆t1 )
In practice the method is more robust if the exponents are taken variably, depending
whether ∆t increases or decreases:
p
ϵ
∆t2 = ∆t1 ( )
d(∆t1 )
and
{1/5 if d<ϵ
p={ .
1/4 if d≥ϵ
{
72 | 4 Ordinary differential equations I

0.3

0.2

Fig. 4.5: The undamped mathematical pendulum

0.1 with time-dependent frequency. Bold: adap-
tive time step, thin: φ(t), numerical solution
(arbitrarily scaled), dashed: Ω0 (t) (arbitrarily
scaled). In the beginning, the initially chosen
∆t = 0.01 rapidly increases. The required accu-
0
0 5 10 racy is ϵ = 10−5 .

We demonstrate the scheme by means of the pendulum with time-dependent fre-

quency:
Ω0 (t) = Ω00 (1 + a sin2 (bt)) ,
the result for Ω00 = 1, a = 5, b = 1/4 is shown in Figure 4.5, and the code is docu-
mented in Appendix B. The time step increases if Ω0 declines and vice versa.

4.3.3.3 Problems
The two coupled rate equations

ṅ 1 = α 1 n1 − α 2 n1 n2
(4.26)
ṅ 2 = −β 1 n2 + β 2 n1 n2 , αi , βi > 0 ,

were proposed by Lotka (1920) and Volterra (1931) to model the interaction between a
prey n1 (t) and a predator n2 (t) population. They are nowadays known as the Lotka–
Volterra model and constitute the most simple predator-prey system.
1. Explain the terms on the right-hand sides of (4.26). Show that performing a proper
scaling of time, n1 and n2 , (4.26) can be cast into the form

ṅ̃ 1 = a ñ 1 − ñ 1 ñ 2
(4.27)
ñ̇ 2 = −ñ 2 + ñ 1 ñ 2 , a>0

2. Compute the ﬁxed points of (4.27) and examine their stability.

3. Show that
W(ñ 1 , ñ 2 ) = ñ 1 + ñ 2 − ln ñ 1 − a ln ñ 2 (4.28)
is a conserved quantity.
4. Compute numerical solutions of (4.27) using RK4 and check the conservation of
(4.28). Plot the trajectories in state space for several a and different initial condi-
tions.
4.4 RK4 applications: celestial mechanics | 73

4.4 RK4 applications: celestial mechanics

The basic subject of classical celestial mechanics is the computation of the trajectories
of point masses (planets or stars) interacting with Newton’s law of gravity. Only the
most simple situation of two bodies in an otherwise empty universe can be solved
exactly. This situation is known as the two-body or Kepler problem. Its solutions are
the well-known ellipses, parabolas, or hyperbolas.

4.4.1 Kepler problem: closed orbits

Two bodies (point masses) with mass m i are located at r i⃗ (t). Between both bodies a
force according to Newton’s law of gravity is assumed. The equations of motion read
r1⃗ − r2⃗
r1̈⃗ = −G m2
|r1⃗ − r2⃗ |3
(4.29)
r2⃗ − r1⃗ m1 ̈
r2⃗̈ = −G m1 = − r1⃗
|r2⃗ − r1⃗ |3 m2
with Newton’s gravity constant G ≈ 6.67 ⋅ 10−11 m3 /(kg s2 ). Introducing relative co-
ordinates η⃗ = r1⃗ − r2⃗ , the six mechanical degrees of freedom can be reduced to three
(this is a consequence of the conservation of the total momentum)
η⃗
η̈⃗ = −G M 3 (4.30)
η
with the total mass M = m1 + m2 . Taking the center of mass as the origin of the coor-
dinate system, one has the relations
m2 m1
r1⃗ = η⃗ , r2⃗ = − η⃗ . (4.31)
M M
Due to the central forces the total angular momentum is conserved and the motion
can be further restricted to a plane, leaving only two mechanical degrees of freedom.
Without loss of generality, we can study the 2D problem with η⃗ = (x, y). After appro-
̃
priate scaling |η| = |η|(GM) 1/3 one obtains the two ODEs of second order without any

further parameters
x
ẍ = − 3
r (4.32)
y
ÿ = − 3
r
√
where r = x + y . The total energy measured in multiples of GM 2/3 reads
2 2

1 2 1
E= (ẋ + ẏ 2 ) − (4.33)
2 r
and represents another constant of motion. Together with the constant angular mo-
mentum
L = x ẏ − y ẋ (4.34)
74 | 4 Ordinary differential equations I

the number of dynamical degrees of freedom in (4.32) is further reduced from four to
two. But this excludes chaotic behavior and one expects at least for bounded solutions
either ﬁxed points or periodic solutions¹. Fixed points with a ﬁnite E do not exist for
the Kepler problem and the periodic solutions correspond to the closed orbits of the
planets.
An obvious solution of (4.32) is a circular orbit with radius R,

x = R cos(ωt) , y = R sin(ωt) .

Inserting this into (4.32) yields

1
ω2 =
, (4.35)
R3
in agreement with Kepler’s third law. Numerical solutions of (4.32) can be achieved
with the help of RK4. The subroutine deﬁning the system (4.32) takes the form

SUBROUTINE kepler_deq(rhside,y,t)
REAL, DIMENSION(4) :: rhside,y
r32=(y(1)**2+y(3)**2)**(3/2.)
rhside(1)=y(2)
rhside(2)=-y(1)/r32
rhside(3)=y(4)
rhside(4)=-y(3)/r32
END

Taking the initial conditions

x(0) = R , y(0) = 0 , ̇
x(0) =0, ̇
y(0) = L/R ,

one ﬁnds a circular orbit for L = L0 = √R with radius R, otherwise ellipses, or for even
larger L, hyperbolas. For L < L0 , the initial point coincides with the aphelion (this is
the largest distance from the planet to the sun), and for L > L0 , with the perihelion of
the ellipse. For E, (4.33) is evaluated as

L2 1
E= − ,
2R2 R
i.e., if L > √2R one has E > 0 and no bounded solutions exist anymore. The two
bodies are free and move on hyperbolas around their common center of mass.
In general there is now friction in the celestial mechanics of mass points. Then
it may become crucial how numerical friction will modify the results. How far is the
energy (4.33) conserved? To shed light on this issue we show in Figure 4.6 the value
of E after 10 000 revolutions as a function of ∆t for a circular orbit with R = 1, L = 1,

1 However, this applies only for the 1/r-potential, see the remarks in Section 4.4.2.
4.4 RK4 applications: celestial mechanics | 75

Fig. 4.6: Deviation of the total energy of a planet revolving on a circular orbit (R = 1, L = 1) from the
exact value E = −1/2 after 10 000 periods, plotted over the time step ∆t. Although only of the order
∆t 2 , the symplectic algorithm is superior to the fourth order Runge–Kutta method for larger ∆t.

ω = 1. One recognizes a very precise energy conservation up to time steps of about

0.05. Note that the orbital period is 2π, and thus for ∆t = 0.05, one circle is resolved
by about 120 time steps only. Nevertheless the total energy after t = 10 000 ⋅ 2π is
about E = −0.50024, a deviation of less than 0.05%.
Taking the symplectic method of the lowest order one time step is easily integrated
by

SUBROUTINE symplect(y,dt)
C symplectic integrator for one time step
REAL, DIMENSION(4) :: y
r32=(y(1)**2+y(3)**2)**(3/2.) ! denominator
y(2)=y(2)-y(1)/r32*dt
y(4)=y(4)-y(3)/r32*dt
y(1)=y(1)+y(2)*dt
y(3)=y(3)+y(4)*dt
END

As it can be seen also from Figure 4.6, the symplectic method is not always preferable.
For a larger time step it conserves the energy better, but for smaller time steps the
accuracy of RK4 is much higher.
76 | 4 Ordinary differential equations I

0.5

–0.5

Fig. 4.7: Perihelion precession for a tiny devi-

–1
ation from the 1/r-potential. Orbits computed
–1 –0.5 0 0.5 1 with RK4 and R = 1, L = 0.7, and ϵ = 0.05.

4.4.2 Quasiperiodic orbits and apsidal precession

For E < 0 all orbits are closed. This is a special property of the 1/r-potential and any
tiny deviation
1
V(r) = − (1+ϵ) , |ϵ| ≪ 1
r
leads to nonclosed, quasiperiodic trajectories whose perihelion turns slowly around
the central mass (Figure 4.7). For ϵ ≠ 0 the only closed orbit is the circle (for a more
detailed discussion see Problems, Section 4.4.5).
Previously we argued that the motion is reduced due to the conservation laws on a
plane with two dynamical degrees of freedom. But how then can crossing trajectories
emerge as depicted in Figure 4.7? Apparently there exist two possible paths after each
crossing point. This is a consequence of the not unique relation ẋ = x(x,
̇ y), ẏ = y(x,
̇ y).
If we eliminate for instance ẏ with the help of (4.34) from the energy law (4.33) we ﬁnd
̇ y) having two different solutions.
a quadratic equation for x(x,

4.4.3 Multiple planets: is our solar system stable?

Things are getting much more involved if the system is extended to more than one
planet circling the central star. Analytic solutions of the equations of motions can be
found only by perturbation theory and for very special cases.
In 1887 the King of Sweden Oscar II asked the question ‘is our solar system sta-
ble?’ and offered a prize of 2500 crowns for the answer. Even nowadays this question
is not deﬁnitely decided. On the one hand, in 1890 Henri Poincaré could ﬁnd a kind
of counter-evidence. He mathematically showed that already for the three-body prob-
lem, i.e., a planet close to a double star, no regular orbits can exist. On the other hand,
the French astronomer Jacques Laskar computed numerically in 1989 the orbits of the
4.4 RK4 applications: celestial mechanics | 77

four inner planets, and in 1994, of all planets in our solar system for the next 25 bil-
lion years and found that the trajectories are ‘weakly chaotic,’ but collisions between
the planets are rather unlikely in the next 200 million years [1, 2]. A further indication
to the instability of certain orbit radii is the distribution of asteroids between Mars
and Jupiter. One observes the so-called Kirkwood gaps occurring at orbit radii having
a period of revolution TA as a rational of the period of Jupiter:
TJ
TA = , n = 2, 7/3, 5/2, 3, . . . . (4.36)
n
Resonances occur along those orbits, which pull out the asteroids located close to
these radii in a short time (at least on the astronomic scale); see also left frame of
Figure 1.1.
The interaction of three celestial bodies can be examined numerically. Based on
(4.29) we consider the extended ODE system

r1⃗ − r2⃗ r1⃗ − r3⃗

r1̈⃗ = −G m2 − G m3
⃗ ⃗
| r1 − r2 | 3 |r1⃗ − r3⃗ |3
r2⃗ − r1⃗ r2⃗ − r3⃗
r2̈⃗ = −G m1 − G m3 (4.37)
|r2⃗ − r1⃗ |3 |r2⃗ − r3⃗ |3
r3⃗ − r1⃗ r3⃗ − r2⃗
r3̈⃗ = −G m1 − G m2 .
|r3⃗ − r1⃗ | 3 |r3⃗ − r2⃗ |3

Again the number of degrees of freedom can be reduced due to the conservation of the
total momentum. Taking the center of mass as the origin of the coordinate system, we
may eliminate for instance r3⃗ :

r3⃗ = −α 1 r1⃗ − α 2 r2⃗ . (4.38)

Here, α i = m i /m3 denote the dimensionless mass ratios. Scaling the time according
to
Gm3 1/2
τ=t( 3 )
ℓ
where ℓ is a length to be determined later, one ﬁnds the dimensionless system

r1⃗ − r2⃗ r1⃗ − r3⃗

r1̈⃗ = −α 2 −
|r1⃗ − r2⃗ |3 |r1⃗ − r3⃗ |3
(4.39)
r2⃗ − r1⃗ r2⃗ − r3⃗
r2̈⃗ = −α 1 −
|r2⃗ − r1⃗ | 3 |r2⃗ − r3⃗ |3

with r3⃗ from (4.38). The energies of the three bodies in units of ℓ/Gm23 read

1 αi αk
Ei = α i (r ̇⃗i )2 + ∑ V ik , V ik = − , (4.40)
2 k=i̸
| r ⃗ i − r ⃗k |
78 | 4 Ordinary differential equations I

which of course are not conserved. The total energy, however,

1 1
E= ∑ α i (r ̇⃗i )2 + ∑ V ik (4.41)
2 i 2 i,k=i̸

constitutes again a constant of motion and can be used as a control value for any nu-
merical integration. In addition, the three components of the total angular momentum

L⃗ = ∑ m i (r ⃗i × r ̇⃗i )
i

must also be conserved. Counting the number of dynamical degrees of freedom in

three dimensions, one ﬁnds 4 × 3 (number of equations) −1 (energy) −3 (momentum)
= 8, more than enough to expect chaotic behavior. We shall however continue the
treatment with the restriction to two spatial dimensions. This of course is a special
case that can be realized if for all bodies the initial condition reads z i (0) = ż i (0) = 0.
The number of degrees of freedom is then reduced to only 4 × 2 − 1 − 1 = 6 but still
enough to leave room for chaos.
The system (4.39) can be integrated by either applying RK4 or using a symplec-
tic algorithm. Depending on the α i and on the initial conditions, various qualitatively
different motions can be obtained. A solution for relatively small α i is depicted in Fig-
ure 4.8. Because of m3 ≫ m1 , m2 , the mass point denoted by i = 3 (blue) plays the role
of the central star. It is revolved by the ‘planets’ with i = 2, 3. Affected by the (weak)
interaction, the orbits are no longer closed and show weakly chaotic behavior.
If the α i are larger, the solutions become very irregular and chaotic. Often one
body can absorb enough energy to escape the system even if the total energy is nega-
tive, E < 0. This mass point detaches from the other two over the course of time and
for t → ∞, a two-body problem remains, showing periodic and closed Kepler orbits
(Figure 4.9).
Both the solutions shown in Figure 4.8 and Figure 4.9 have been computed with
a symplectic method of order ∆t2 , and the changes of total energy and angular mo-
mentum have been recorded. With the time step ∆t = 2 ⋅ 10−6 the maximal relative
deviations 󵄨󵄨 X 󵄨
󵄨 max − Xmin 󵄨󵄨󵄨
∆X = 2 󵄨󵄨󵄨 󵄨
󵄨󵄨 Xmax + Xmin 󵄨󵄨󵄨
in the time interval 0 < t < 2600 are for Figure 4.8

∆E = 4.8 ⋅ 10−7 , ∆L = 3.0 ⋅ 10−12 ,

and for Figure 4.9

∆E = 2.2 ⋅ 10−5 , ∆L = 2.9 ⋅ 10−12 .
The restriction of the motion on a plane strongly increases the probability of collisions
between two bodies. Here of course, a 3D-computation would be more realistic.
4.4 RK4 applications: celestial mechanics | 79

10 –0.0022
–0.0024
–0.0026
1
–0.0028
–0.0030
|ri (t)|

0.1 –0.0032

Ei
–0.0034
–0.0036
0.01
–0.0038
–0.0040
0.001 –0.0042
500 1000 1500 2000 2500 500 1000 1500 2000 2500
t t

Fig. 4.8: The three-body problem and numerical solutions for α 1 = α 2 = 0.001. Left: the distances
|r i | from the origin; right: the energies E 1 , E 2 from (4.40). The two lighter bodies (red, green) circuit
the heavy one (blue) on weakly distorted circular orbits.

1000 0

100 –0.005
10
–0.01
|ri (t)|

1
Ei

–0.015
0.1

0.01 –0.02

0.001 –0.025
500 1000 1500 2000 2500 500 1000 1500 2000 2500
t t

Fig. 4.9: Same as Figure 4.8, but for α 1 = 0.015, α 2 = 0.025. Now the evolution seems much
more chaotic. At t ≈ 100 the at ﬁrst inner planet (green) moves outwards. It continues absorbing
energy from the other two bodies and reaches even larger orbits. Eventually it may escape from the
inﬂuence of the two other masses (E 2 ≈ 0, t > 1400). Then the two remaining bodies form a stable
periodic Kepler system.

4.4.4 The reduced three-body problem

To arrive from (4.37) at the so-called reduced three-body problem, the following ap-
proximations are in order:
– All masses move in the same plane (2D).
– One of the three masses is light enough (a small planet, an asteroid, or a space
ship) not to inﬂuence the motion of the two other ones. Let us assume m1 ≪
m2 , m3 . Then m1 is called the test body (TB).
– The two heavy bodies (named main bodies, e.g., sun and Jupiter or a double star)
move in circles around their common center of mass with the angular frequency ω.
Lengths and time are scaled to their distance and period so that |r2⃗ − r3⃗ | = 1 and
ω = 1.
80 | 4 Ordinary differential equations I

4.4.4.1 The explicitly time-dependent equations of motion

An important system parameter is provided by the mass relation
m3
μ= , 0<μ<1 . (4.42)
m2 + m3

Scaling of time with

Gm3 1/2
τ=t( )
μℓ3
yields after some computations the two circular orbits

cos t cos t
r2⃗ (t) = −μ ( ) , r3⃗ (t) = (1 − μ) ( ) . (4.43)
sin t sin t

For the test body m1 , the equation of motion has the form

r1⃗ − r2⃗ (t) r1⃗ − r3⃗ (t)

r1̈⃗ = −(1 − μ) −μ . (4.44)
|r1⃗ − r2⃗ (t)| 3 |r1⃗ − r3⃗ (t)|3

4.4.4.2 The equations of motion in the corotating system

As shown ﬁrst by Euler, the explicit time dependence of (4.44) can be removed by a
transformation to a coordinate system r ⃗̃ = (x,̃ y)̃ rotating with the two main bodies
according to
x̃ = x cos t + y sin t
ỹ = −x sin t + y cos t .
In a uniformly rotating system, apparent forces due to inertia occur, named Coriolis
and centrifugal forces. One ﬁnds (we drop the tildes):

ẍ = 2 ẏ − ∂ x V
(4.45)
ÿ = −2 ẋ − ∂ y V

with the potential (Jacobi potential):

1−μ μ 1 2
V(x, y) = − − − r (4.46)
r2 r3 2

and
r = √x2 + y2 ,

as well as the distances

r2 = √(x + μ)2 + y2 , r3 = √(x + μ − 1)2 + y2

between the TB and the two main bodies. In the corotating system the main bodies
rest on the x-axis:
r2⃗ = (−μ, 0) , r3⃗ = (1 − μ, 0) ,
4.4 RK4 applications: celestial mechanics | 81

Fig. 4.10: Forbidden regions (gray) for μ = 1/4 and E = −2.0, −1.8, −1.7, −1.55 (from left to right).
For E ≥ −3/2 − μ(μ − 1) ≈ −1.406 the test body has access to the complete plane.

their common center of mass coincides with the origin. As can be shown easily, the
total energy
1
E = (ẋ 2 + ẏ 2 ) + V(x, y) (4.47)
2
is conserved and determined by the initial conditions. Since the kinetic energy cannot
be negative, the TB can only be located in regions where

E − V(x, y) ≥ 0 ,

see Figure 4.10.

4.4.4.3 Lagrangian points and Trojans

The equations (4.45) possess five fixed points (ẋ = ẍ = ẏ = ÿ = 0), the TB ‘rests’ in the
rotating system and relative to the main bodies. The fixed points are called Lagrangian
or libration points. They are computed from the equilibrium conditions

∂x V = ∂y V = 0 .

Three of the Lagrangian points lay on the x-axis (Figure 4.11), their x-values can only
be computed numerically (see Problems). They are unstable for all μ. Of more interest
are the two other ﬁxed points y ≠ 0, the so-called Trojans. They turn out to be

x4,5 = 1/2 − μ , y4,5 = ±√3/2 (4.48)

L3 L1 L2

Fig. 4.11: The ﬁve Lagrangian points for μ = 1/4.

L 4 and L 5 are named Trojans and prove to be stable
L5 for small values of μ.
82 | 4 Ordinary differential equations I

and form an equilateral triangle with the two main bodies with side length one. Lin-
earizing (4.45) around (4.48)

x = x4,5 + x̄ , y = y4,5 + ȳ

yields a system for the deviations

3 3√3
ẍ̄ = 2 ẏ̄ + x̄ ∓ (2μ − 1) ȳ
4 4 (4.49)
3√3 9
ȳ̈ = −2 x̄̇ ∓ (2μ − 1) x̄ + ȳ ,
4 4
and with x,̄ ȳ ∼ exp λt the characteristic polynomial
27
λ4 + λ2 + μ(1 − μ) = 0 (4.50)
4
as the solvability condition. The ﬁxed points x4,5 , y4,5 become unstable if λ2 ϵ C. This
is the case if 1 − 27μ(1 − μ) < 0, or

1 1 23 1 1 23
− √ <μ< + √ .
2 2 27 2 2 27
For all other values of μ the Trojans are stable (purely imaginary λ).
The potential values V1 . . . V5 assigned to the ﬁve Lagrangian points constitute
important energies separating qualitatively different solution behavior. Let us assume
μ < 1/2, i.e., the heavier main body is on the negative x-axis. Then we have (see also
Figure 4.10):
– E ≤ V1 : Depending on initial conditions, the TB circuits one of the two main bod-
ies.
– V1 < E ≤ V2 : The allowed region opens between the main bodies, the TB can
move on orbits enclosing both main bodies.
– V2 < E ≤ V3 : The allowed region opens on the right-hand side and the TB can
separate arbitrarily far from the main bodies (escape-trajectories).
– V3 < E ≤ V4,5 : The allowed region also opens on the left-hand side.
– V4,5 ≤ E: The forbidden region has vanished completely.

4.4.4.4 Regularization and numerical solution

One may try to solve equations (4.45) numerically, applying, for instance, RK4. Since
the orbits can be very irregular, small distances between TB and main bodies become
probable over the course of time, leading to strong interaction forces. This can cause
inaccurate results and restricts the time step drastically to ensure numerical stabil-
ity. To avoid small denominators in (4.45), a regularization is in order. One may dy-
namically scale velocities and time by distance-depending factors leading to a system
without any singularities. We introduce the geometric mean of the distances

s = √r2 r3 .
4.4 RK4 applications: celestial mechanics | 83

If the TB comes close to one of the two main bodies we ﬁnd s ≪ 0. The scaling
1
dτ = dt , u = s2 ẋ , v = s2 ẏ (4.51)
s3
leads after some computations, to the system

d τ x = s2 u
d τ y = s2 v
s2 (4.52)
d τ u = 2s3 v − s4 ∂ x V − Fx
2
s2
d τ v = −2s3 u − s4 ∂ y V − F y
2
with
F x = (r22 + r23 ) ((x + μ)u 2 + yuv) − r22 u 2
(4.53)
F y = (r22 + r23 ) ((x + μ)uv + yv2 ) − r22 uv .
All denominators from (4.45, 4.46) have been removed. However, for large distances
r, s ≫ 1, the regularization (4.51) results in diverging velocities u, v. To get rid of this
difficulty, we switch for s > 1 to the scaling:
1 1
dτ = dt , u= ẋ , v= ẏ
s s
and obtain
dτ x = s u
dτ y = s v
1 1 (4.54)
dτ u = 2 v − ∂x V − 3 Fx
s 2s
1 1
d τ v = −2 u − ∂ y V − 3 F y .
s 2s
The system (4.52,4.54) can be coded using RK4. Even better suited is RK4 with adaptive
time step, see Section 4.3.3.2. The accuracy can also be controlled dynamically taking

ϵ = ϵ0 s .

Then the relative accuracy ϵ/r remains approximately constant. For the complete pro-
gram we refer to the website [3].
Figure 4.12 shows numerically computed orbits for μ = 1/4 and several values
of E. The Lagrangian points are located at

x1 = 0.361 , x2 = 1.266 , x3 = −1.103 , y1,2,3 = 0 , x4,5 = 1/4 , y4,5 = ±√3/2 ,

and the corresponding potential values read

V1 = −1.935 , V2 = −1.781 , V3 = −1.622 , V4,5 = −1.406 .

84 | 4 Ordinary differential equations I

Fig. 4.12: Trajectories for μ = 1/4 and E = −2.0 (top row), E = −1.8 (second row), E = −1.75 (third
and fourth row), E = −1.55 (bottom row).
4.4 RK4 applications: celestial mechanics | 85

For all solutions we fixed ϵ0 = 0.001, leading to a dynamical time step between 10−5
and 10−2 . For E < V1 the orbits are normally periodic or quasiperiodic. If E > V1 , the
trajectories soon become chaotic. For even higher energies most of the orbits leave the
region close to the main bodies and escape to infinity. On the other hand, even here,
bounded orbits exist (last row on the left).
As can be seen in the figure, depending on the initial conditions, one obtains for
the same values of μ and E completely different trajectories. For any given E we may
scan the allowed region for x(t = 0), y(t = 0) and integrate the equations from t = 0 up
to a certain given (long) time t = t e . Then the resulting trajectory is evaluated accord-
ing to its behavior and the initial point x(t = 0), y(t = 0) is colored accordingly. Two
such orbit-type diagrams are shown in Figure 4.13, again for μ = 1/4 and E = −1.8;
E = −1.75. The absolute value of the initial velocity follows from the energy equa-
tion (4.47), its direction chosen perpendicularly² to r1⃗ . At t e = 1000 we distinguish
between
– White: escape trajectory. The trajectory leaves the area r < 4 within t < t e .
– Red: orbit around main body at −μ. Bounded up to t = t e .
– Green: orbit around main body at 1 − μ. Bounded up to t = t e .
– Blue: orbit around both main bodies. Bounded up to t = t e .

The boundaries between different regions look at least for E = −1.75 fractal, a hint
toward chaotic orbits. At this energy value, orbits only enclosing the heavier main
body are rather uncommon.

Fig. 4.13: Scan through the allowed region of initial conditions with a resolution of 400 x 400 dots,
μ = 1/4, E = −1.8 (left), and E = −1.75 (right). The different colors correspond to the different types
of the trajectories (see text).

2 The direction is an additional parameter in the form of an initial value. For other directions, one
obtains other diagrams.
86 | 4 Ordinary differential equations I

4.4.5 Problems

1. Study the system (4.32) for a more general gravity force

x y
ẍ = − , ÿ = − (4.55)
rα rα
for arbitrary α. Show by linearization around a circular orbit with radius r0α = 1/Ω20
and angular frequency Ω0 that such a circle is closed only for certain α. Determine
these α and verify the result by integrating (4.55) numerically.
Hint: transform (4.55) to a rotating frame with angular frequency Ω0 as done in
Section 4.4.4.2.
2. Examine the full three-body problem (4.39) in three room dimensions. Code the
ODE system with the help of a symplectic algorithm. Take as the initial values that
the two orbits are not located in the same plane. Check numerically the stability
of the orbits for several values of α i . Check also if E and L⃗ are conserved.
3. Determine the Lagrangian points L1 to L3 depending on μ numerically.

4.5 Molecular dynamics (MD)

During the past five decades, direct numerical simulations of microscopic many-
particle systems became more and more a focus of interest. In this way material
parameters or even phase transitions can be computed based on ‘first principles.’
However, in statistical physics, ‘many’ means some 1023 particles, a number that will
remain always beyond numerical realization. The first papers in the field [4] clearly
dealt with fewer than 1000 particles, whereas nowadays handling some 100 million
is feasible.

4.5.1 Classical formulation

Most MD systems are based on the laws of classical mechanics. In the simplest case,
one considers N equal particles (mass points) interacting with two-body forces. The
force is often described by the Lennard-Jones potential³
σ 12 σ 6
V(r) = 4ϵ (( ) − ( ) ) (4.56)
r r
where r is the distance of the two interacting particles. The parameters ϵ and σ de-
pend on the material to be simulated and take, e.g., for the inert gas Argon, the values
ϵ/k B = 120 K, σ = 3.4 Å (k B = 1.381 ⋅ 10−23 J/K, Boltzmann’s constant). Again we

3 John Lennard-Jones, English mathematician and physicist, 1894–1954.

4.5 Molecular dynamics (MD) | 87

shall restrict ourselves to a plane problem with two spatial dimensions. Although this
seems not very realistic for a gas or a solid, the codes will be shorter and execute much
faster. The method does not differ from the three-dimensional treatment and we can
leave a 3D extension to the reader.
Applying a scaling to nondimensional variables

r = σ r̃ , t = σ √m/ϵ t ̃ ,

we obtain the 2N dimensionless equations of motion (tildes removed)

N N
ẍ i = 24 ∑ f(r ij ) (x i − x j ) , ÿ i = 24 ∑ f(r ij ) (y i − y j ) (4.57)
j=i̸ j=i̸

where
f(r) = 2r−14 − r−8 (4.58)

and r ij = √(x i − x j )2 + (y i − y j )2 are the two-particle distances. The scaled Lennard-

Jones potential
1 12 1 6
U(r) = 4 (( ) − ( ) ) . (4.59)
r r
possesses a minimum at r0 = 21/6 ≈ 1.12 with U(r0 ) = −1. This would correspond to
the equilibrium conﬁguration if there were only two particles around. The total (inter-
nal) energy scaled by ϵ reads
E = Ek + Ep (4.60)

and must be conserved if no external forces are present. Here we denote the total ki-
netic internal energy by
1 N
Ek = ∑ ( r ̇⃗i ) ,
2
(4.61)
2 i

and the total potential internal energy by

N
Ep = ∑ U(r ij ) . (4.62)
j>i

4.5.2 Boundary conditions

For the majority of MD simulations, periodic boundary conditions are assumed. The
integration domain with periodic length L is thought to be repeated in each space di-
rection inﬁnitely many times. Each particle is not only interacting with every other one
in L but also with the ones in the periodic copies. In this way a much larger system is
simulated. Wall effects, however, cannot be studied since the system is in fact inﬁnite.
88 | 4 Ordinary differential equations I

A much more simple way is to take rigid, impermeable, and perfectly reﬂecting
walls around the integration domain. If while integrating a particle position x i , y i out-
side of the domain turns out, say on the right-hand side x i > L, the particle is to be
reﬂected. Then its new position and x-velocity are simply set to

x i → 2L − x i , ẋ i → −ẋ i .

The same can be done for y and correspondingly for the walls at x = 0, y = 0. The parti-
cles interact with the walls in the form of elastic collisions where energy and absolute
value of momentum are conserved.

4.5.3 Microcanonical and canonical ensemble

The macroscopic state variables of a closed many-particle system are the internal en-
ergy E, the volume V (in 2D V = L2 ), and the number of particles N. All three quantities
are constants of motion and can be adjusted by the system and the initial conditions,
respectively. Using the notions of statistical mechanics, such a system is called a mem-
ber of the microcanonical ensemble. Each member is represented by a point in the
4N-dimensional state space, the Γ-space. When time goes on, these points travel on
trajectories in a 4N − 1-dimensional hyperplane

E(x1 , y1 . . .x N , y N , u 1 , v1 . . .u N , v N ) = const.

where u i = ẋ i , v i = ẏ i denote the particle velocities. We may assign a temperature to

each ensemble member using the equipartition theorem according to
f
kB T = Ek (4.63)
2
where f denotes the number of mechanical degrees of freedom. During the temporal
evolution the temperature will change and reaches its equilibrium value T0 for t → ∞.
Scaling the temperature with ϵ/k B and as in (4.60), the energy with ϵ one obtains with
f = 2N:
T = Ek /N . (4.64)
If one is interested in the thermal behavior of the simulated material, one normally
prescribes the temperature by coupling the system to a heat bath. Then, the internal
energy will adjust itself accordingly and the ensemble is called canonical. There are
different ways to ﬁx the temperature to a desired value. These algorithms are called
thermostats. Due to (4.63) Ek is given through T. Thus, after a certain number of time
steps one can repeatedly multiply the speed of each particle with the factor

√Ts /T,

fulﬁlling (4.63). Here, Ts denotes the prescribed nominal temperature of the heat bath.
4.5 Molecular dynamics (MD) | 89

Another way to realize a thermostat is assuming a particle-speed-dependent fric-

tion in the equations of motion. If we add to the right-hand sides of (4.57) the terms
−γ(T) r ̇⃗i
and take
Ts
γ(T) = γ0 (1 −
) , (4.65)
T
it can be shown that the temperature Ts is reached exponentially in time with
exp(−2γ0 t) (Problems). Each particle is thereby permanently in contact with the
heat bath, releasing energy to the bath if T > Ts , receiving energy for T < Ts .

4.5.4 A symplectic algorithm

Here we shall only brieﬂy describe the algorithm; a detailed and commented listing
can be found in [3]. To integrate (4.57) with (4.65) a symplectic scheme is recom-
mended. Introducing the particle velocities (u, v) = (x,̇ y),
̇ a system of 4N ﬁrst-order
ODEs is obtained having the form:
N
u̇ i = 24 ∑ f(r ij ) (x i − x j ) − γ(T) u i
j=i̸

ẋ i = u i
(4.66)
N
v̇ i = 24 ∑ f(r ij ) (y i − y j ) − γ(T) v i
j=i̸

ẏ i = v i ,
which after time discretization can be brought into the symplectic scheme (see Sec-
tion 3.3.2)
N
(n+1) (n) (n) (n) (n) (n)
ui = ui + [24 ∑ f (r ij ) (x i − x j ) − γ(T) u i ] ∆t
[ j=i̸ ]
(n+1) (n) (n+1)
xi = xi + ui ∆t
(4.67)
N
(n+1) (n) (n) (n) (n) (n)
vi = vi + [24 ∑ f (r ij ) (y i − y j ) − γ(T) v i ] ∆t
[ j=i̸ ]
(n+1) (n) (n+1)
yi = yi + vi ∆t .

4.5.4.1 Optimization
To ﬁnd the forces acting on a certain particle (called reference particle in the follow-
ing) one must sum over all N − 1 pair forces caused by the other particles. This corre-
sponds to N(N − 1) evaluations of the pair potential at each time step. The program
90 | 4 Ordinary differential equations I

can be vastly optimized if only those particles located within a certain distance r m to
the reference particle are taken into account. The Lennard-Jones potential decreases
with 1/r6 , thus it should be sufficient to take say r m = 4r0 . Then U(r m ) ≈ −0.0005 can
safely be neglected against U(r0 ) = −1. To restrict the pair sum on the neighbors laying
in a sphere (or disc in 2D) with radius r m , one must know the ambient particles for the
reference particle, demanding a certain bookkeeping during the dynamical process.
This can be achieved most easily with a two-dimensional array NB. Let the particles
be numbered with k = 1 . . . N (ordinal number). Then the ﬁrst index of NB is assigned
to the ordinal number of the reference particle, say k, while the second one lists the
neighbors. There, NB(k, 1) is equal to the number of neighbors of the reference par-
ticle k. If we have, for instance, NB(k, 1) = 5, particle number k has ﬁve neighbors
inside r m , whose ordinal numbers are stored in NB(k, 2) through NB(k, 6). Since the
particle positions change continuously, the neighborhood array must be updated, tak-
ing an additional computational effort. However, depending on the maximal particle
speed this update is not necessary after each single time step. In practice it proves to
be sufficient to recompute NB after, say, 20–30 steps.

4.5.4.2 Initial conditions

As usual in classical mechanics one needs two complete sets of initial conditions,
namely particle positions and velocities at t = 0. The particles, for instance, can be
ordered on a square grid with width r0 . This of course cannot correspond to an equi-
librium state since next-nearest neighbors also interact (Problems). The velocities can
be chosen close to thermal equilibrium according to a Maxwell–Boltzmann distribu-
tion, applying the Box–Muller method described in more detail in Section 8.1.3.3. On
this behalf, one may take
(0)
ui = √−2T ln(1 − ξ1 ) cos(2πξ2 )
(4.68)
(0)
vi = √−2T ln(1 − ξ1 ) sin(2πξ2 )

with two independent, equally distributed random numbers ξ1 , ξ2 in [0,1).

4.5.5 Evaluation

4.5.5.1 Phases in conﬁguration space

Figure 4.14 shows three snapshots of 1600 particles, each one for a different temper-
ature. For large T, the gaseous state is clearly distinguished. The main difference be-
tween ‘liquid’ and ‘solid’ is, apart from small ﬂuctuations, the constant particle posi-
tion. In Figure 4.15 we plot the distance of two initially neighbored particles over time.
Averaging over all distances one should ﬁnd for the liquid state a behavior as

⟨r2 ⟩ = D t .
4.5 Molecular dynamics (MD) | 91

Fig. 4.14: Conﬁgurations of 1600 particles and different temperatures, each after t = 200.

Fig. 4.15: Distance of two initially neighbored particles as a function of time.

For the solid state, the diffusion constant D goes to zero.

Taking only 1600 particles, a rather large percentage is close to phase separating
walls, and surface effects may play an important role. Surface effects can be avoided by
applying periodic boundary conditions in space, for which we refer to the literature,
e.g., [5].

4.5.5.2 Pair correlation function

In 3D conﬁguration space, the pair correlation function g(r) describes the probability
of ﬁnding a particle at distance r from another particle in a spherical shell with vol-
ume 4πr2 dr. In 2D space, the shell turns into a ring with the area 2πrdr. Let n(r) be
92 | 4 Ordinary differential equations I

Fig. 4.16: Scaled pair correlation function for several temperatures, solid: T = 0.12, dash-dotted:
T = 0.44, dashed: T = 1.25.

the number of particles located on the ring [r, r + ∆r],

N
n(r) = ∑ Θ(r ij − r) Θ(r + ∆r − r ij ) , (4.69)
i,j=i̸

where Θ denotes the step function, and g(r) can be computed as:
1
g(r) 2πr∆r = n(r) . (4.70)
N(N − 1)
To find n(r), one needs a histogram. To this end, the particles n(r) laying on the ring
are counted and stored for several discrete values of r i = i∆r at a certain time. To
avoid fluctuations, one averages (4.69) over a certain time interval (time average =
ensemble average). Figure 4.16 shows the (scaled) pair correlation functions for the
three different temperatures from Figure 4.14. Clearly visible is the transition from a
far order (solid state) to a near order (liquid) and finally to a pure repulsion (gaseous)
if temperature is increased.

4.5.5.3 Speciﬁc heat

The speciﬁc heat of a mechanically closed system (constant volume) is given by

dE
cv = (4.71)
dT
4.5 Molecular dynamics (MD) | 93

Fig. 4.17: Internal energy E p over T , averaged over a time interval ∆t = 100. When close to a phase
transition, E p varies strongly. Simulation with N = 1600.

with the total energy E as (4.60). There are different ways to compute cv from MD sim-
ulations. For a canonical ensemble one ﬁnds
1
cv = Var(E)
kb T2
with the variance Var(E) = ⟨E2 ⟩ − ⟨E⟩2 and ⟨. . . ⟩ as the ensemble or time average.
However, for the computations described above, E is a conserved quantity and thus
Var(E) = 0. If, on the other hand, one substitutes in (4.60) Ek = NT, (4.71) yields
dEp
cv = N +. (4.72)
dT
Ep (T) can be easily gained from an MD simulation. The dependence of Ep on T is
shown in Figure 4.17. The strong increase around T ≈ 0.45 indicates a phase tran-
sition. Indeed, cv should diverge in the thermodynamic limit at a phase transition.
Another method to compute cv was developed by Lebowitz et al. [6]. There, one
needs only the variance of the kinetic energy (4.61) and gets:
N
cv = Var(E k )
. (4.73)
1− NT 2
Figure 4.18 shows cv after (4.73) from the same simulation as used for Figure 4.17. The
data is rather noisy. A maximum can nevertheless be clearly recognized at T ≈ 0.45,
as well as the correct asymptotic behavior of cv .
For small T the equipartition theorem provides E = 2NT (two degrees of freedom
from translation and another two from oscillation around rest position) and therefore
cv = 2N. For a large T one expects ‘almost free’ particles (perfect gas) with cv = N.
94 | 4 Ordinary differential equations I

10000

8000

6000
cv

4000
2N

2000
N

0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
T

Fig. 4.18: Speciﬁc heat from (4.73). For T → 0 one expects c v = 2N (elastic solid), for T → ∞ c v = N
(perfect gas). Simulation with N = 1600.

4.5.6 Problems

1. Thermalization
A method to keep the temperature of a many-particle system constant at Ts is to intro-
duce friction terms into the equation of motion as done in (4.66):

d2tt r ⃗i = F⃗ i − γ(T, Ts ) v⃗ i (4.74)

where
Ts
γ(T, Ts ) = γ0 (1 − ) .
T
For the interaction-free case F⃗ i = 0, derive from (4.74) a ﬁrst-order ODE for T(t). Solve
this equation and show that T approaches Ts exponentially in time with exp(−2γ0 t).

2. 2D-equilibrium configurations
An (infinitely extended) 2D-many-particle system at equilibrium forms a periodic lat-
tice. Consider the two configurations: square and hexagonal (Figure 4.19).
Let the two-particle interaction be given by a Lennard-Jones potential as in (4.59).
Compute the equilibrium distances r S and r H and consider
(i) Only nearest neighbors (blue),
(ii) Also next nearest neighbors (red),
(iii) Also next-next nearest neighbors (green).

Which of the two conﬁgurations is the more stable one?

4.6 Chaos | 95

Fig. 4.19: Two possible periodic conﬁgurations in 2D.

4.6 Chaos

As it already discussed in Chapter 3 but in even more pronounced examples in the two
previous sections of the present chapter, chaotic behavior is more the rule than the
exception if the phase space possesses enough dimensions. Chaotic trajectories may
appear both in conservative (Section 3.3, 4.4.3, 4.4.4) and in dissipative systems (Sec-
tion 3.2.7, 4.5). For Hamiltonian systems, initial conditions normally play a decisive
role. For closely neighbored initial values, qualitatively different behaviors very often
emerge, such as periodic, quasiperiodic, or chaotic; see for instance Figure 4.13. For
a dissipative system, typically the initial conditions play a minor role and are ‘forgot-
ten’ after a certain, by friction effects deﬁned, relaxation time (there are exceptions:
a trajectory starting, e.g., close to a stable ﬁxed point or limit cycle, will stay close
and stationary or periodic behavior will persist. However, this does not exclude more
involved or even chaotic behavior for trajectories starting in another region of state
space).

4.6.1 Harmonically driven pendulum

We continue working with low-dimensional systems (N > 2) having the form

dy i
= f i (y1 , y2 , . . . y N ) , y i (0) = a i , i = 1...N , (4.75)
dt
where friction is included (dissipative systems, no energy conservation). Let us begin
with the harmonically driven nonlinear mathematical pendulum in 2D phase space

φ̈ + α φ̇ + Ω20 sin φ = A cos ωt . (4.76)

96 | 4 Ordinary differential equations I

This is a nonautonomous problem with explicit time dependence. It can be trans-

formed to an autonomous one by introducing an additional dependent variable y1 =
φ, y2 = φ,̇ y3 = ωt having now the form (4.75) and N = 3:

ẏ 1 = y2
ẏ 2 = −α y2 − Ω20 sin y1 + A cos y3 (4.77)
ẏ 3 = ω .

The corresponding RK4-subroutine pendulum_deq reads:

SUBROUTINE pendulum_deq(rhside,y,t)
REAL, DIMENSION(3) :: rhside,y
COMMON /PARAM/ alpha,omega0,a,omega
rhside(1)=y(2)
rhside(2)=-alpha*y(2)-omega0**2*SIN(y(1))+a*COS(y(3))
rhside(3)=omega
END

Clearly the last equation in (4.77) can be immediately integrated and one may just as
well (and with less effort) work with the nonautonomous original system:

rhside(1)=y(2)
rhside(2)=-alpha*y(2)-omega0**2*SIN(y(1))+a*COS(omega*t)

However, the following considerations are valid only for autonomous systems and we
shall resort to (4.77).

4 4

2 2

0 0

–2 –2

–4 –4

–2 0 2 –2 0 2

Fig. 4.20: Chaos for the driven pendulum. Left: phase space, y 2 over y 1 ; right: Poincaré section,
intersection points with the planes y 3 = 3π/2 + 2nπ. A = 1, ω = 0.8, Ω0 = 1, α = 0.1.
4.6 Chaos | 97

Fig. 4.21: Near a separatrix, tiny effects due to the driving force may turn into qualitatively different
behavior. Solid: without external driving; dashed: possible trajectories with external force.

In the left frame of Figure 4.20 a chaotic trajectory is shown in phase space (y1 , y2 ).
The existence of a separatrix of the nondriven system is an important though not nec-
essary ingredient for the emergence of chaos. If the trajectory comes close to the sepa-
ratrix, a suitable phase and amplitude of the driving force can push the phase point to
the other side of the separatrix, and the motion then changes qualitatively, here from
oscillation to rotation (Figure 4.21).

4.6.2 Poincaré section and bifurcation diagrams

To visualize three-dimensional trajectories, certain projections are needed. In this way

Figure 4.20 shows a projection onto the y1 -y2 -plane. Another possibility is to prepare
a stroboscopic snapshot, for instance synchronized with the driving frequency ω in
the case of the pendulum. As already discussed in Section 3.3.3, a Poincaré section
results. In the right frame of Figure 4.20 the y1 -y2 -values are plotted when the driving
force undergoes a zero-crossing with a positive slope, that is for ωt = 3π/2 + 2nπ.
If the Poincaré section for t → ∞ consists of a finite number of separated points, the
trajectory is periodic; a dense line is assigned to quasiperiodicity (cut through a torus)
and filled areas with fractal gaps can be interpreted as chaotic behavior.
A bifurcation diagram is obtained if some control parameter is changed and the
value for a dependent variable is plotted at certain consecutive times over this param-
eter. Figure 4.22 (bottom) shows such a diagram where the value of y1 has been plotted
over the strength of the driving force, again for fixed times ωt = 3π/2 + 2nπ.
98 | 4 Ordinary differential equations I

0.1

–0.1

–0.2

–0.3
0.9 1 1.1 1.2 1.3 1.4
2.5

1.5

0.5

0
0.9 1 1.1 1.2 1.3 1.4

Fig. 4.22: Top: the two nontrivial Lyapunov exponents for the driven pendulum with friction. Param-
eters same as in Figure 4.20. Bottom: bifurcation diagram for the driven pendulum, ordinate: |y 1 | at
ωt = 3π/2 + 2nπ, abscissa: A.

4.6.3 Lyapunov exponents

Let y⃗ (0) (t) be a numerically determined solution of (4.75). We ask for its stability by
performing a linear stability analysis:

⃗ = y⃗ (0) (t) + u(t)

y(t) ⃗ .

Contrary to the linearization around a ﬁxed point, the Jacobi matrix depends now on
time
⃗
d u(t)
= L(t) u(t) ⃗ (4.78)
dt
with 󵄨
∂f i 󵄨󵄨󵄨
L ij (t) = 󵄨
∂y j 󵄨󵄨󵄨󵄨y⃗ (0) (t)
.

Due to the linear character of (4.78) one may assume that for large times, exponential
behavior emerges
⃗
|u(t)| ∼ h(t)e σt , t → ∞
with a bounded h(t). Then the sign of σ
1
σ = lim ⃗
ln |u(t)| (4.79)
t→∞ t
4.6 Chaos | 99

classifies the stability of y⃗ (0) . For σ > 0 arbitrarily small deviations will grow expo-
nentially over the course of time, the trajectory is unstable, and chaotic behavior is
encountered. The σ defined as (4.79) is called the largest Lyapunov exponent.
How can we compute σ? One nearby possibility is to integrate (4.78) numerically
and evaluate (4.79) for large t. However, due to the exponential increase of |u|⃗ for a
positive σ this is not practical since a numerical overflow would soon occur.

4.6.3.1 Determination of the largest Lyapunov exponent

To construct a method that works we begin with the time-evolution operator Q. We
⃗ as
may then formally write u(t)

u(t) ⃗
⃗ = Q(t, 0) u(0) . (4.80)

⃗
As a matter of fact, u(t), ⃗
and therefore σ, depends on u(0). Thus one ﬁnds as many
different Lyapunov exponents as the number of possible linearly independent initial
conditions, namely N, the dimension of the state space. Often it is sufficient to know
the largest σ because it distinguishes between chaotic and regular dynamics. Since

Q(t, t0 ) = Q(t, t1 ) Q(t1 , t0 ) , t > t1 > t0

we may write (4.80) as a product

u(t) ⃗
⃗ = Q(t, t − ∆T) Q(t − ∆T, t − 2∆T) . . .Q(∆T, 0) u(0) .

Introducing the abbreviations

Q k ≡ Q(k∆T, (k − 1)∆T) , u⃗ k ≡ u(k∆T)

⃗ , k = 0, 1, 2. . .

we obtain
u⃗ k = Q k Q k−1 . . .Q1 u⃗ 0 (4.81)
where k = t/∆T. Taking a small enough ∆T avoids an overﬂow at a single step

u⃗ k = Q k u⃗ k−1 .

After each step one may normalize according to

u⃗ k
û k = , d k = |u⃗ k |
dk
and obtains

Q1 û 0 = d0 d1 Q k Q k−1 . . . ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
u⃗ k = d0 Q k Q k−1 . . . Q 2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ Q2 û 1 = d0 d1 . . .d k û k .
= û 1 d1 = û 2 d2

From here one receives

k
|u⃗ k | = ∏ dℓ
ℓ=0
100 | 4 Ordinary differential equations I

d4
d2
û1

d0
û0 û4
û2 û3

d3 y(0) (t)

t = Tv t = Tv + ΔT t = Tv + 2ΔT t = Tv + 3ΔT

Fig. 4.23: Numerical computation of the largest Lyapunov exponent. After a constant time interval
the absolute value of the deviation d k is determined. Then the deviation is normalized to one with-
out changing its direction.

and ﬁnds by substitution in (4.79)

k
1 1 k
σ = lim ln (∏ dℓ ) = lim ∑ ln dℓ , (4.82)
k→∞ k∆T ℓ=0
k→∞ k∆T
ℓ=0

the Lyapunov exponent to the initial condition û 0 .

The following algorithm illustrated in Figure 4.23 has to be implemented:
1. After a certain relaxation time T v compute the reference trajectory from (4.75).
2. Take an arbitrary û 0 , d0 = 1 at time t = T v , and set ℓ = 1.
3. Integrate (4.75), and in the meantime (4.78), over the interval ∆T.
4. Determine dℓ = |u(t⃗ + ℓ∆T)|. Normalize u⃗ to û ℓ and store dℓ .
5. ℓ := ℓ + 1 , t = t + ∆T.
6. If ℓ < k go to 3.
7. Compute σ according to (4.82).

If we denote ln dℓ as a local Lyapunov exponent, (4.82) is the average value of these

local exponents.
The limit k → ∞ has to be substituted by a large k. But what does large mean? We
deﬁne the Lyapunov exponent computed after k iterations:

1 k
σk = ∑ ln dℓ (4.83)
k∆T ℓ=0

and ﬁnd for the next iteration

k
1 k
σ k+1 = ( ∑ ln dℓ + ln d k+1 ) = σ k + ∆σ
(k + 1)∆T ℓ=0 k+1

and with k ≫ 1
1
∆σ = σ k+1 − σ k ≈ ln d k+1 .
k∆T
4.6 Chaos | 101

Obviously, ∆σ ∼ 1/k converges with k → ∞. The sum in (4.82) can be truncated if the
error |∆σ| reaches a ﬁxed lower bound. Alternatively the relative error
󵄨󵄨 ∆σ 󵄨󵄨
󵄨󵄨 󵄨󵄨
󵄨󵄨 󵄨󵄨 < ϵrel
󵄨󵄨 σ k 󵄨󵄨
can be used as a truncation criterion.
An N-dimensional system possesses N Lyapunov exponents, also called Lya-
punov spectra. The above described method provides the largest one. This is easy to
understand if we decompose the deviation u(t) ⃗ for t → ∞ in the base v̂ k , built by the
̂
N different u 0 :
⃗ = c1 v̂ 1 eσ1 t + . . .c N v̂ N eσ N t ,
u(t)
where σ k denotes now the ordered Lyapunov spectrum according to

σ1 ≥ σ2 ⋅ ⋅ ⋅ ≥ σ N .

The constants c k are determined by the initial value u(0).⃗ As time goes on u⃗ will orient
parallel to v̂ 1 , independently from the initial conditions. However, if c1 would exactly
vanish one would expect u⃗ parallel to v̂2 , etc. In this way one could at least in principle
obtain the complete spectrum, what, however, cannot work in practice. Numerically
there is always a tiny component in the direction v⃗ 1 that increases exponentially fast
and dominates the deviations after a certain time ∼ 1/σ 1 . We shall see below how the
complete spectrum can be found anyway. But ﬁrst let us prove an important theorem.

4.6.3.2 Theorem: one Lyapunov exponent vanishes for all trajectories that do not
terminate at a ﬁxed point
We further assume bounded systems with

⃗
|y(t)| ≤ D1 , ̇⃗
|y(t)| = |f ⃗(y(t))|
⃗ ≤ D2 , Di > 0

and show the simple proof of the theorem. Differentiating (4.75) yields
N N
∂f i
ÿ i = ∑ ẏ j = ∑ L ij ẏ j ,
j
∂y j j

from where it follows that with u⃗ = ẏ⃗ the time derivative of each solution of (4.75)
also provides a solution of the linear system (4.78). In particular this is valid for the
reference trajectory y⃗ (0) (t) itself, i.e., the deviation points always tangential to the tra-
jectory (Figure 4.24). This special (marginal) Lyapunov exponent holds with (4.79) as
1 1
σ m = lim ln |ẏ⃗ (0) (t)| ≤ lim ln |D2 | = 0 , (4.84)
t→∞ t t→∞ t

and therefore
σm ≤ 0 .
102 | 4 Ordinary differential equations I

û1
û0 û4
û2
û3
y(0) (t)

Fig. 4.24: A deviation pointing in the direction of the reference trajectory has a vanishing Lyapunov
exponent if the trajectory does not terminate at a ﬁxed point.

Assuming on the other hand σ m < 0, due to

|ẏ⃗ (0) (t)| ∼ eσ m t , t→∞

one arrives at
|ẏ⃗ (0) (t)| = 0 , t→∞
and the trajectory would end at a ﬁxed point. Therefore, if the trajectory does not ter-
minate at a ﬁxed point as assumed, the only possibility remaining is

σm = 0 .

4.6.3.3 Higher order Lyapunov exponents

How can we compute the complete Lyapunov spectrum? To this end we introduce the
Lyapunov exponents of the order p; the quantity deﬁned with (4.79) is then the Lya-
punov exponent of the ﬁrst order corresponding to the average rate of contraction of
a vector (one-dimensional). Accordingly, the exponent of the order p stands for the
mean rate of contraction of the volume of a p-dimensional parallelepiped:
1
σ (p) = lim ln |V p (t)| . (4.85)
t→∞ t

Considering a parallelepiped moving along trajectories and spanned by the base v⃗ k ,

one can show that
σ (p) = σ 1 + σ 2 + . . . σ p
holds. If all N orders of the Lyapunov exponent are known, we can compute the expo-
nents of the ﬁrst order according to

σ 1 = σ (1)
σ 2 = σ (2) − σ (1) (4.86)
(N) (N−1)
σN = σ −σ .

4.6.3.4 Classification
If all first-order exponents are known, the reference trajectory y⃗ (0) (t) can be classified
(Table 4.1). In particular, a positive Lyapunov exponent denotes the divergence of the
4.6 Chaos | 103

Tab. 4.1: Classiﬁcation of trajectories of an autonomous, dissipative system in three-dimensional

state space with the help of the signs of their three Lyapunov exponents.

σ1 σ2 σ3
− − − trajectory terminates at a ﬁxed point
0 − − stable limit cycle (periodic dynamics)
0 0 − stable torus (quasiperiodic dynamics)
+ 0 − strange attractor (chaotic dynamics)

trajectories. If, however, the dynamics is bounded on a ﬁnite region in state space,
contractions must also be present and at least one of the Lyapunov exponents must
be negative.
An N-dimensional volume element V N (t) whose corners move along trajectories
in state space is contracted in the temporal average if the system is dissipative; see
Section 3.2.3. For the sake of simplicity let us ﬁrst assume div f ⃗ = c with c < 0 and c as
constant. From (3.23) follows
V N (t) = V N (0) e ct
and with (4.85),
N
σ (N) = ∑ σ k = c = div f ⃗ = Tr L . (4.87)
k=1

The sum over all Lyapunov exponents corresponds to the divergence of f (or the trace
of the Jacobi matrix L) and is negative for a dissipative system. Hence it follows also
that at least one Lyapunov exponent must be less than zero. Thus, for a three-dimen-
sional state space all possible combinations are listed in Table 4.1. But in general, div f ⃗
is not a constant. Then one can still compute the average and

N t
1
σ (N)
= ∑ σ k = lim ∫ dt div f ⃗ (4.88)
t→∞ t
k=1 0

holds, an expression being also negative for dissipative systems. For the example of
the damped pendulum one obtains
3
σ (3) = ∑ σ k = −α .
k=1

For Hamiltonian systems the equations of motions have canonical form (see Sec-
tion 3.3) and div f ⃗ = 0 results. The sum over all Lyapunov exponents vanishes.

4.6.3.5 Numerical determination of all Lyapunov exponents

Next we present a method for the computation of the complete spectrum σ n and follow
mostly [7]. Starting with a p-dimensional parallelepiped, due to the exponentially dif-
ferent growth rates of the different directions, all the spanning vectors will point soon
104 | 4 Ordinary differential equations I

in more or less one direction, namely in that assigned to the largest Lyapunov expo-
nent. This can be avoided if the spanning vectors are orthogonalized repeatedly after
certain time intervals, applying a Gram–Schmidt procedure for instance.
We demonstrate the algorithm for the three-dimensional state space, N = 3:
1. Select three orthonormal unit vectors û 1 , û 2 , û 3 , (û i ⋅ û j ) = δ ij , t = 0, ℓ = 0.
2. Integrate (4.78) from t through t + ∆T:
w⃗ i = Q(t + ∆T, t) û i
(p)
3. Compute the volumes Vℓ , which are spanned by w⃗ 1 . . . w⃗ p :
(p)
Vℓ = √| det V|
with the help of the p × p matrix
N
(i) (j)
V ij = ∑ w n w n
n
(i)
and w k as the kth component of the vector w⃗ i .
4. Using a Schmidt–Gram method, determine the next orthogonal set û i in such a
way that the first p vectors û i span the subspace defined by w⃗ 1 till w⃗ p :
w⃗ 1
û 1 = ,
|w⃗ 1 |
u⃗ 2
u⃗ 2 = w⃗ 2 − c12 û 1 , û 2 = , c12 = û 1 ⋅ w⃗ 2 ,
|u⃗ 2 |
u⃗ 3
u⃗ 3 = w⃗ 3 − c13 û 1 − c23 û 2 , û 3 = , c13 = û 1 ⋅ w⃗ 3 , c23 = û 2 ⋅ w⃗ 3
|u⃗ 3 |
5. t := t + ∆T, ℓ := ℓ + 1
6. If ℓ < k go to 2.
7. After sufficiently many steps (large k) the Lyapunov exponent of the order p can
be computed:
1 k (p)
σ (p) = ∑ ln Vℓ
k∆T ℓ=1
and from there according to (4.86) finally the whole spectrum σ 1 . . .σ N .

The subroutine dlyap_exp implementing the steps explained above is described in the
Appendix B.

4.6.3.6 Example: the driven pendulum

We demonstrate the method for the driven pendulum (4.77). The Jacobi matrix in (4.78)
reads
0 1 0
L(t) = (−Ω20 cos y1(0) (t) −α −A sin y3(0) (t)) . (4.89)
0 0 0
4.6 Chaos | 105

Due to the last equation (4.77) there are no ﬁxed points and at least one Lyapunov
exponent must vanish. Because of the simple structure of (4.89) we can make some
analytic conclusions. From the third line it follows immediately

u̇ 3 = 0 , u 3 = const.
(0) (0)
For small A we may assume self-consistently |y1 |, |y2 | ∼ A and linearize
(0)
cos y1 ≈ 1 . (4.90)

In particular for u 3 = 0 the two equations for u 1 , u 2 are equivalent to the damped
pendulum having the solutions
α
|(u 1 , u 2 )| ∼ h i (t) exp (− ) , i = 1, 2
2
where h i (t) is an oscillating bounded function. With (4.79) one ﬁnds the two Lyapunov
exponents
σ 2 = σ 3 = −α/2 .
Putting u 3 = 1, the two equations for u 1 , u 2 correspond to the driven harmonic oscil-
lator. The long-time behavior turns out to be

y1 ∼ sin(ωt + β) ,

providing σ 1 = 0. Therefore, for small A the solution y⃗ (0) (t) is a stable limit cycle
(Table 4.1).

–0.05

–0.1

0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8

2.5

1.5

0.5

0
0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8

Fig. 4.25: Lyapunov exponents and bifurcation diagram for the driven pendulum with friction; pa-
rameters as in Figure 4.20.
106 | 4 Ordinary differential equations I

For larger A the linearization (4.90) is invalid, the limit cycle becomes unstable, and
eventually chaotic trajectories emerge. Figure 4.25 shows numerical results for smaller
A, while Figure 4.22 shows those for larger values.

Code. In the code published on the website, instead of the original three-dimensional
autonomous system (4.77), the equivalent two-dimensional nonautonomous system

ẏ 1 = y2
(4.91)
ẏ 2 = −α y2 − Ω20 sin y1 + A cos ωt

is used. The Jacobi matrix then reduces to

0 1
L(t) = ( (0) ) (4.92)
−Ω20 cos y1 (t) −α

and only two Lyapunov exponents exist. The one vanishing for the 3D-system does
not occur (the theorem Section 4.6.3.2 is only valid for autonomous systems).

4.6.3.7 Lyapunov time

For dynamical systems originating from physical problems, the initial conditions are
only known at a ﬁnite accuracy ∆ϵ(0). However, if the dynamics is chaotic, small er-
rors may exponentially grow in time and the largest Lyapunov exponent σ 1 constitutes
a measure for their growth rate (dimension 1/time). If σ 1 is known, a time t∗ can be
estimated after the initial error has exceeded a certain size L, typically the circumfer-
ence of the attractor in state space. Then the initial conditions will no longer play any
role and the albeit fully deterministic theory (the differential equations (4.75)) does
not allow for any forecasting for times beyond t∗ . With
∗
L = ∆ϵ(t∗ ) = ∆ϵ(0) e σ1 t

one ﬁnds
1 L
t∗ = ln ( )
σ1 ∆ϵ(0)
for the so-called Lyapunov time.
Depending on atmospheric conditions, for the weather forecast, t∗ lies in the re-
gion between only a few hours up to a maximum of two weeks. Studying the Centen-
nial Calendar or similar products is therefore proven useless.

4.6.4 Fractal dimension

Initiated by the Euclidean notion of dimension and analogue to the concept already
discussed in Section 2.4.1, we may assign a dimension to an object in state space.
A ﬁxed point has the dimension d = 0, a limit cycle d = 1, a torus d = 2, etc. As it
4.6 Chaos | 107

follows from the No-Intersection theorem, a chaotic attractor cannot be completely ﬁt

into a plane and therefore must have a dimension larger than 2. On the other hand it
normally does not ﬁll the whole state space leading for N = 3 to a fractal dimension

2<d<3.

How can we compute d?

4.6.4.1 Capacity dimension

From the right frame of Figure 4.20 it seems possible to determine the dimension with
the box-counting method introduced in Section 2.4.1. Taking the Poincaré section it is
important to note that the actual dimension of the attractor is larger by one (a limit
cycle corresponds to a ﬁnite number of points on the Poincaré section, which has the
dimension of zero). It is of course also possible to compute d in the N-dimensional
state space by covering the complete space with hypercubes of dimension N and edge
length L. As in Section 2.4.1, one determines d by counting the cubes visited by the
attractor as a function of L:
log M
dK = − .
log L
This magnitude is called capacity dimension.
To demonstrate the method we compute d K of the driven pendulum for two differ-
ent values of A, one in the periodic, the other in the chaotic regime. The three-dimen-
sional phase space in the region

−π ≤ y1 ≤ π , −3.5 ≤ y2 ≤ 3.5 , 0 ≤ y3 ≤ 2π

is thereby covered with n3 cubes, where n runs through the powers of two n =
2, 4, 8, 16 . . . 512. For each n, M is determined and plotted as a function of the edge
length L in Figure 4.26. It turns out that in the chaotic regime the dimension is only
weakly larger than two. Hence, based on the fractal dimension alone one may hardly
distinguish between a chaotic attractor and a quasiperiodic motion.
Finally, Figure 4.27 depicts d K in a wide region of A. But also here, d K only slightly
exceeds values of two whenever chaos emerges.

4.6.4.2 Correlation dimension

Let P be the number of given data points in an N-dimensional state space

y⃗ i = (y1i , y2i , . . . y iN ) , i = 1...P .

The set can be the result of a numerical solution of an ODE but could also originate
from some series of measurements, e.g., temperature depending on time at N different
places, etc.
108 | 4 Ordinary differential equations I

10
A = 1.0
log (M)

5
A = 0.4

1 2 3 4 5 6
−log (L)

Fig. 4.26: Fractal dimension (slope) found with the box-counting method, called capacity dimension.
Driven pendulum, Ω0 = 1, α = 0.1, ω = 0.8, A = 0.4 (dots), A = 1.0 (squares). For A = 0.4 one
obtains d K ≈ 1.04; for A = 1.0 a dimension d K ≈ 2.01.

1.8

1.6

1.4

1.2

1
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4

Fig. 4.27: Fractal dimension of the driven pendulum in the domain 0.4 ≤ A ≤ 1.4, parameters as in
Figure 4.20. The regions where d K ≈ 2 agree well with the chaotic regions of Figure 4.22.

The correlation dimension is computed by determining for each point the number of
neighbor points within the distance ≤ R:

C(R) = number of pairs with |y⃗ i − y⃗ j | ≤ R for all i ≠ j .

This can be formulated by the help of the step-function Θ:

P P
C(R) = ∑ ∑ Θ(R − |y⃗ i − y⃗ j |) .
i=1 j=i+1
4.6 Chaos | 109

16
ln C(R)

12
Fig. 4.28: Correlation dimension (slope) for the
–2 –1 –0 1 driven pendulum, Ω0 = 1, α = 0.1, ω = 0.8.
ln R Chaotic attractor at A = 1, d C ≈ 2.12.

If we have enough points P we should ﬁnd

C(R) ∼ R d

with d the dimension of the object where the data points are located. This can be illus-
trated for N = 3: if the points fill the space uniformly their number is proportional to
the volume of the sphere with radius R resulting in d = 3. If the points lie on a plane
one finds C(R) ∼ R2 and d = 2 and for a line d = 1.
Taking a small enough R, this is the definition of the correlation dimension:

ln C(R)
d C = lim . (4.93)
R→0 ln R
One obtains d C by plotting C over R in a log-log scale as the slope (Figure 4.28). How-
ever, the slope is not constant and as already discussed in Section 2.4.1, the question
is where to evaluate it. For R too small the curve ﬂattens because too few points lay
inside the sphere to do a statistical evaluation. For too large R the slope also decreases
since the attractor has a ﬁnite extension (for the pendulum of the order 2π).

4.6.5 Reconstruction of attractors

One may come to the position to analyze a one-dimensional collection of data, a time
series of a certain magnitude Y(t):

Y0 , Y1 , Y2 , . . .Y P−1 , Y n = Y(n∆t) .

But even without knowing the dimension N of the state space where the system
that produced the series lives, a fractal dimension can be determined. However, for
110 | 4 Ordinary differential equations I

a complex system, N can be quite large. If the data points are completely uncorre-
lated (e.g., the results of a coin toss), the series cannot be embedded at all in a ﬁnite-
dimensional state space and one expects N → ∞.
To reconstruct an attractor, we ﬁrst unfold the series into a space with some given
dimension N 󸀠 , called embedding dimension. To this end we choose a certain time in-
terval ∆T, the delay rate, which is normally much larger than ∆t. Let

K = ∆T/∆t ≫ 1

be an integer. One constructs a sequence of N 󸀠 -dimensional vectors by

y1 (t k ) = Y k
y2 (t k ) = Y k+K
y3 (t k ) = Y k+2K
..
.
y N 󸀠 (t k ) = Y k+(N 󸀠 −1)K

and ﬁnds for

t k = k∆t , k = 0. . .k max , k max = P − 1 − (N 󸀠 − 1)K

all P − (N 󸀠 − 1)K vectors of dimension N 󸀠 that can be used to determine the correlation
dimension as described above.
How large should we take N 󸀠 ? No intersections of the trajectory should be certainly
assured. F. Takens⁴ was able to prove in 1981 the following theorem [8]:
If a deterministic system generates an N-dimensional ﬂow y:⃗

d y⃗
= f ⃗(y)⃗ ,
dt
then
x1 = y i (t) , x2 = y i (t + ∆T) , ... x2N+1 = y i (t + 2N∆T)
represents a continuously differentiable embedding, where y i can be any component
i = 1. . .N of y.⃗
Thus the dimension of an attractor is conserved if

N 󸀠 ≥ 2N + 1 .

In practice one has to increase N 󸀠 step by step until the fractal dimension computed
for consecutive N 󸀠 converges.

4 Floris Takens, Dutch mathematician, 1940–2010.

4.7 ODEs with periodic coefficients | 111

4.7 Differential equations with periodic coefficients

4.7.1 Floquet theorem

The Floquet⁵ theorem is equivalent to Bloch’s theorem in solid state physics and
applies for explicitly time-dependent linear ODEs of the form (4.78), but for a time-
periodic system matrix L(t):
L(t) = L(t + T) .
As already done in (4.80) we may introduce a time-evolution operator C:

⃗
u(T) ⃗
= C(T) u(0) , (4.94)

now developing u⃗ for one period T. This special operator is also called monodromy
matrix. Let w⃗ k be the eigenvectors of C

C w⃗ k = σ k (T) w⃗ k . (4.95)

Applying C two times yields

C C w⃗ k = σ k (T) σ k (T) w⃗ k = σ k (2T) w⃗ k .

But since
(σ k (T)) n = σ k (nT)
one immediately ﬁnds
σ k = exp(λ k T) . (4.96)
The σ k are named Floquet multipliers, while the λ k are the Floquet exponents.
Next we assume that the w⃗ k represent a complete base in N-dimensional state
space. Thus we may expand
N
⃗ = ∑ a k (t) w⃗ k e λ k t .
u(t) (4.97)
k

Applying C yields
N N
⃗ = ∑ a k (t) C w⃗ k eλ k t = ∑ a k (t) σ k w⃗ k eλ k t
C u(t)
k k
N
!
= ∑ a k (t) w⃗ k eλ k (t+T) = u(t
⃗ + T) . (4.98)
k

From (4.97) it follows

N
⃗ + T) = ∑ a k (t + T) w⃗ k e λ k (t+T)
u(t
k

5 Gaston Floquet, French mathematician, 1847–1920.

112 | 4 Ordinary differential equations I

and ﬁnally, comparing with (4.98)

a k (t) = a k (t + T) .

Hence the expansion coefficients in (4.97) are periodic in t with the period T. We have
derived the Floquet theorem that summarizes as:

The solution of
̇⃗ = L(t) u(t)
u(t) ⃗ with L(t) = L(t + T) (4.99)
has the form
N
⃗ = ∑ q⃗ k (t) exp(λ k t) ,
u(t)
k
where q⃗ k are periodic functions

q⃗ k (t) = q⃗ k (t + T) .

The Floquet exponents λ k can be computed from the eigenvalues σ k of the monodromy
matrix C(T) with (4.96)
1 1 1 iα k
λk = ln σ k = ln |σ k |eiα k = ln |σ k | + . (4.100)
T T T T

4.7.2 Stability of limit cycles

We study the stability of a periodic solution of (4.75):

y⃗ (0) (t) = y⃗ (0) (t + T) . (4.101)

Inserting
⃗ = y⃗ (0) (t) + u(t)
y(t) ⃗
and linearizing (4.75) with respect to u⃗ leads to a problem as (4.99) for the small de-
⃗
viations u(t). If one of the Floquet exponents has a positive real part the deviations
will grow exponentially and the limit cycle (4.101) will be unstable. The condition for
stability thus reads
|σ k | ≤ 1 for all k .

4.7.3 Parametric instability: pendulum with an oscillating support

As an application we study the excited pendulum sketched in Figure 4.29. The pivot
oscillates with A sin ωt in the vertical direction, leading to an apparent force that can
be included in (4.4) by the substitution
Aω2
g → g(1 + a sin ωt) , a= .
g
4.7 ODEs with periodic coefficients | 113

A sin ωt

Fig. 4.29: Pendulum driven by a vertically oscillating support.

The system (4.5) for the undamped case (α = 0) then reads

ẏ 1 = y2
ẏ 2 = −Ω20 (1 + a sin y3 ) sin y1 (4.102)
ẏ 3 = ω .

Contrary to (4.77) the time-dependence is now multiplicative and the two ﬁxed points
for the undriven case persist in the (y1 , y2 )-plane. The lower rest position is simply
given by
(0) (0) (0)
y1 = y2 = 0 , y3 = ωt .
Linearizing around the lower rest position leads to

u̇ 1 = u 2
u̇ 2 = −Ω20 (1 + a sin ωt) u 1 (4.103)
u̇ 3 = 0 .

Since u 3 decouples from u 1 , u 2 it is sufficient to further consider the 2D-problem

u̇⃗ = L u⃗ (4.104)

with
u⃗ = (u 1 , u 2 )
and
0 1
L(t) = ( 2 ) . (4.105)
−Ω0 (1 + a sin ωt) 0
The system (4.104) has the form (4.99) with
2π
T= .
ω
To determine the Floquet exponents we must ﬁrst compute the monodromy matrix. To
this end we take two orthogonal initial conditions

u⃗ 1 (0) = (1, 0) , u⃗ 2 (0) = (0, 1)

114 | 4 Ordinary differential equations I

and numerically integrate (4.104) with (4.105) for each u⃗ i until t = T. In this way we
ﬁnd u⃗ i (T). On the other hand, due to (4.94) one obtains

(u⃗ 1 (T), u⃗ 2 (T)) = C ⋅ (⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟

u⃗ 1 (0), u⃗ 2 (0)) = C (4.106)
=1

and the vectors u⃗ i (T) form the columns of C. Now we can compute the eigenvalues σ 12
according to:
1 1
σ 12 = Tr C ± √(Tr C)2 − 4 Det C (4.107)
2 2
and from there with (4.100) also the Floquet exponents. The relation (4.88) or (4.87)
holds also for the sum of all the Floquet exponents and from Tr L = 0 it follows

λ1 + λ2 = 0 . (4.108)

There are two possibilities for the roots of (4.107):

1. Both σ k are real valued and larger than zero, so the two λ k are also real and one
of them is larger than zero due to (4.108). The ﬁxed point y⃗ (0) is unstable (saddle
node).
2. The σ k as well as the λ k form a complex-conjugate pair. Then because of (4.108)
λ1 = −λ2 = −λ∗1 and
λ12 = ±iα , α ϵ R .

The ﬁxed point y⃗ (0) is stable (center). The loci in parameter space (a, ω) where two
real-valued solutions of (4.107) turn into a complex-conjugate pair are found from

(Tr C)2 − 4 Det C = 0 (4.109)

and separate the stable from the unstable regions.

4.7.4 Mathieu equation

The two ﬁrst-order ODEs (4.104) with (4.105) are equivalent to one ODE of the second
order, the so-called Mathieu⁶ equation:

ü + Ω20 (1 + a sin ωt)u = 0 (4.110)

where u = u 1 . In the literature, one often ﬁnds the form

̃ = 0,
ü + (p + 2b sin 2t)u (4.111)

which can be obtained from (4.110) by the scaling

ω
t̃ = t
2

6 Emile Mathieu, French mathematician, 1835–1890.

4.7 ODEs with periodic coefficients | 115

10 4

8
3

6
2
b

b
4

1
2

0 0
–5 0 5 10 15 20 –4 –3 –2 –1 0
p p

Fig. 4.30: Left: stability chart for the Mathieu equation (4.111). Black: λ1 = λ2 = 0, stability limits,
red: λ k real valued, unstable; blue: λ k = ±iα, stable. Right: blow-up of the dashed area for negative
p: the upper rest position is stable in the small wedge between the black lines.

and
4Ω20 2Ω20
p=
, b = a. (4.112)
ω2 ω2
As described above we determine the stability regions of the lower ﬁxed point by com-
puting the monodromy matrix for certain parameters p, b:

C = C(p, b) .

Then the zero lines of the function

f(p, b) = (Tr C(p, b))2 − 4 Det C(p, b)

are plotted in a parameter plane; Figure 4.30, left frame, shows the result. At the res-
onances
p = n2 , n = 1, 2, . . . (4.113)

arbitrarily small amplitudes b are sufficient to destabilize the pendulum, so the oscil-
lations grow exponentially. Inserting p = n2 into (4.112), one ﬁnds for the resonances
the ratio
Ω0 n
=
ω 2
between the eigenfrequency of the pendulum and the driving frequency. Interestingly
enough one obtains even for a negative p a tiny region with imaginary Floquet ex-
ponents; see Figure 4.30, right frame. But negative p corresponds to a linearization
around the unstable, upper ﬁxed point of the pendulum, y1 = π. If b is chosen appro-
priately, the pendulum can be kept stable in its top position.
116 | 4 Ordinary differential equations I

4.7.5 Problems

1. Linearize (4.102) around the upper, unstable rest position

(0) (0) (0)
y1 = π , y2 = 0 , y3 = ωt .

Derive an equation of the form (4.111) for the deviations y1 = π + u(t). What is the
difference from (4.111)?
Show with the equation

u(t) = u 0 cos(t + α) e λt ,

that the pendulum may be stabilized in the upper position. Neglect higher har-
monics and use the approximation
1 1
sin 2t cos(t + α) = ( sin(t − α) + sin(3t + α)) ≈ sin(t − α) .
2 2
Which condition holds for λ to stabilize the upper rest position?
2. Examine the damped Mathieu equation

ü + α u̇ + Ω20 (1 + a sin ωt)u = 0

numerically. Plot the stability regions in the q-b-plane for given α > 0. What
changes qualitatively in Figure 4.30?

Bibliography

[1] J. Laskar, Large-scale chaos in the solar system, Astron. Astrophys. 287, L9 (1994).
[2] J. Laskar and M. Gastineau, Existence of collisional trajectories of Mercury, Mars and Venus with
the Earth, Nature 459, 7248 (2009).
[3] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
[4] A. Rahman, Correlations in the Motion of Atoms in Liquid Argon, Phys. Rev. 136, A405 (1964).
[5] J. M. Thijssen, Computational Physics, Cambridge Univ. Press (2007).
[6] J. L. Lebowitz, J. K. Percus and L. Verlet, Ensemble Dependence of Fluctuations with Application
to Machine Computations, Phys. Rev. 153, 250 (1967).
[7] J. Argyris, G. Faust, M. Haase and R. Friedrich, An Exploration of Dynamical Systems and Chaos,
Springer (2015).
[8] F. Takens, Dynamical Systems and Turbulence, Lect. Notes in Math. Vol. 898 (1981).
5 Ordinary differential equations II,
boundary value problems
5.1 Preliminary remarks

5.1.1 Boundary conditions

We continue considering systems of N ordinary differential equations of the ﬁrst order

⃗
d y(x)
= f ⃗(y(x),
⃗ x) . (5.1)
dx
Contrary to initial value problems, for boundary value problems one prescribes con-
ditions at two (or more) different points, say x = a, b:
⃗
A y(a) ⃗
+ B y(b) = c⃗ . (5.2)
Normally, a and b coincide with the boundaries of x and one wants to ﬁnd solutions
⃗ in the region between
for y(x)
a≤x≤b.
The boundary conditions written in the form (5.2) linearly depend on y.⃗ However, non-
linear boundary conditions may also occur, having the more general form
⃗
g i (y(a), ⃗
y(b)) =0, i = 1...N
where the functions g i depend on 2N variables. In practice, boundary conditions are
often separable:
⃗
A1 y(a) = c⃗ 1 , B1 y(b)
⃗ = c⃗2 . (5.3)
In order not to have an over- or under-determined problem, N linearly independent
conditions must appear in (5.3).
The initial value problem (4.2) from Chapter 4 is included in the form (5.3) with
A1 = 1 , B1 = 0 , c⃗1 = (r ⃗i (0) , v⃗ i (0) ) .
We shall restrict ourselves on linear, separable boundary conditions (5.3). Well-
posed initial value problems normally have a unique solution. On the other hand, even
linear boundary value problems may have multiple solutions or no solution at all. This
becomes evident at a simple example. Consider the ODE of second order:
y󸀠󸀠 + y = 0 .
For the boundary conditions y(0) = 0, y(π/2) = 1 there is just one solution
y(x) = sin x ,
while for y(0) = 0, y(π) = 0 we have inﬁnitely many
y(x) = A sin x
with arbitrary A, and for y(0) = 0, y(π) = 1 there exists no solution at all.

https://doi.org/10.1515/9783110515145-005
118 | 5 Ordinary differential equations II

5.1.2 Example: ballistic ﬂight

(0) (0)
A mass point starts at (x, y) = 0 with a certain given velocity v⃗ (0) = (v x , v y ) in a
constant gravity ﬁeld and lands after a certain time t = T at x(T) = L, y(T) = 0. The
equations of motion read
ẍ = −α ẋ + β y
(5.4)
ÿ = −α ẏ − g ,
where speed-dependent friction α as well as a linearly, height-increasing horizontal
wind force (shear ﬂow, β) are included. If α = β = 0, the trajectory is a parabola

(0) 1 2 (0)
y(t) = v y t − gt , x(t) = v x t
2
or
(0)
vy g
y(x) = (0)
x− x2
vx (0) 2
2 (v x )

with
(0) (0) (0)
2v y 2v x v y
T= , L= . (5.5)
g g
(0) (0)
̇
Given the initial conditions x(0) = 0, y(0) = 0, x(0) ̇
= v x , y(0) = v y , the solution
x(t), y(t) as well as the ﬂight time T and the landing point L follow uniquely. This
of course is a classical initial value problem. How can we formulate it as a boundary
value problem? We seek a solution of (5.4) fulﬁlling the boundary conditions

x(0) = y(0) = 0 , x(T) = L , y(T) = 0

for a given T (parameter). Hence the mass point should arrive after a given ﬂight time
(0) (0)
T at the given place x = L. From (5.5) we ﬁnd v y = g T/2 and v x = L/T or

1 L
y(t) = g t (T − t) , x(t) = t. (5.6)
2 T
But what to do if (5.4) looks a bit more involved (nonlinear wind or friction forces,
(0) (0)
etc.) and can be only solved numerically? One could iteratively try values for v x , v y
in such a way that the trajectory ended after t = T in (L, 0). This is the idea behind the
so-called shooting method to which we shall come back in more detail in Section 5.5.

5.2 Finite differences

As already done for initial value problems one can express the derivatives by the help
of differential quotients resulting in an algebraic system, the discretized equations.
5.2 Finite differences | 119

5.2.1 Discretization

We demonstrate the method with the example of the ballistic ﬂight in Section 5.1.2.
At ﬁrst we divide the axis 0 ≤ t ≤ T with equidistant mesh points

t i = i∆t , i = 0...n , ∆t = T/n .

Then the derivatives are substituted by

x i+1 − x i−1 x i+1 − 2x i + x i−1
ẋ i = , ẍ i = , (5.7)
2∆t ∆t2
and accordingly for y. Here, x i , y i stands for x(t i ), y(t i ). From (5.4) the two linear sys-
tems result:
n−1
∑ A ij y j = a i (5.8)
i=1
n−1
∑ A ij x j = β y i + b i . (5.9)
i=1

Here A ij denotes the tridiagonal matrix

2 1 α 1 α
A ii = − , A i,i+1 = + , A i,i−1 = − (5.10)
∆t2 ∆t 2 2∆t ∆t 2 2∆t
and
a i = −g .
The boundary conditions read

x0 = y0 = 0 , xn = L , yn = 0

and have to be incorporated into the system (5.8, 5.9). Since A1,0 = 0, the points on the
left side are already included in the ﬁrst equations (i = 1). On the right side (i = n − 1)
the last equation of (5.9) yields

A n−1,n−1 x n−1 + A n−1,n L = β y n−1 + b n−1 .

Because A n−1,n does not exist (A is an (n − 1) × (n − 1) matrix), the additional inho-

mogeneity must be considered in b n−1 :
1 α
b n−1 = − ( + ) L, b i = 0 for i = 1 . . . n − 2 .
∆t2 2∆t
Numerically we can solve the equations (5.8,5.9) one after the other applying a LAPACK
routine, e.g., SGTSV [1]:
120 | 5 Ordinary differential equations II

PARAMETER (n=10) ! number of mesh points

REAL, DIMENSION(n) :: x,y,dl,du,di
g=9.81
tend=1. ! T, right side boundary
xl=1. ! length L=1
....
C matrix A (tridiagonal) to solve y eqs.
dl=1./dt**2-alpha/2./dt ! sub-diagonal
du=1./dt**2+alpha/2./dt ! super-diagonal
di=-2./dt**2 ! diagonal
y=-g ! inhomogeneity
c LAPACK call for y eqs.
CALL SGTSV(n,1,dl,di,du,y,n,info)
C matrix A (tridiagonal) to solve x eqs.
dl=1./dt**2-alpha/2./dt ! sub
du=1./dt**2+alpha/2./dt ! super
di=-2./dt**2 ! diagonal
x=beta*y ! inhomogeneity
x(n)=x(n)-xl*du(n) ! boundary cond. at x=L
c LAPACK call for y eqs.
CALL sgtsv(n,1,dl,di,du,x,n,info)
... output, plotting, etc...

The results for only 10 mesh points (in practice one would use much more) is shown
in Figure 5.1 for several values of α and β. A, however, more involved analytical solu-
tion can be even found for the case α, β ≠ 0 (see Problems), which is also shown in
the ﬁgure. The ﬁnite difference scheme provides a quite exact result even for a small
number of mesh points. Due to the approximations in (5.7) the discretization error is
∼ ∆t or ∼ 1/n.
The central task concerning this problem is the inversion of matrices, thus an im-
plementation using MATLAB seems to be nearby. For demonstration we list the pro-
gram including a graphical evaluation:

clear;
n=100; tend=1; g=9.81; xl=1;
alpha=[0.5,3,5.5]; beta=[-5,-2.5,0];
dt=tend/(n-1); dt2=dt^2; k=0;
for i=1:3
for j=1:3
a=(1/dt2-alpha(j)/2/dt)*ones(n,1); ! matrix elements
b=(1/dt2+alpha(j)/2/dt)*ones(n,1);
m=spdiags([b,a],[-1,1],n,n)-2/dt2*speye(n);
5.2 Finite differences | 121

1
1
1

0.5
0.5 0.5

α = 0.5 β = –5.0 α = 3.0 β = –5.0 α = 5.5 β = –5.0

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1

1
1
1

0.5
0.5 0.5

α = 0.5 β = –2.5 α = 3.0 β = –2.5 α = 5.5 β = –2.5

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1

1
1
1

0.5
0.5 0.5

α = 0.5 β = –0.0 α = 3.0 β = –0.0 α = 5.5 β = –0.0

0 0 0
0 0.5 1 0 0.5 1 0 0.5 1

Fig. 5.1: The ballistic ﬂight, numerical solutions of (5.4) for 10 mesh points (circles) compared to the
exact solution (solid), L = 1 m, T = 1 s.

inh(1:n)=-g; ! inhomogeneity
y=inh/m; ! inversion y
inh=beta(i)*y;
inh(n)=inh(n)-xl*m(n,n-1);
x=inh/m; ! inversion x
k=k+1;
subplot(3,3,k); plot(x,y); ! plot
end
end

A ﬁgure similar to Figure 5.1 will show up.

122 | 5 Ordinary differential equations II

5.2.2 Example: Schrödinger equation

One of the standard problems of quantum mechanics is solving the Schrödinger equa-
tion [2]
ℏ2 ∂
[− ∆ + U(r,⃗ t)] Ψ(r,⃗ t) = iℏ Ψ( r, t) (5.11)
2m ∂t
for a certain given potential U(r,⃗ t). This is a partial differential equation that can be
transformed into an ODE for the special case of only one spatial dimension, say x, and
a time-independent potential U = U(x). Doing the separation

Ẽ
Ψ(x, t) = Φ(x) exp (−i t) ,
ℏ

one arrives at the time-independent or stationary (one-dimensional) Schrödinger

equation
− Φ󸀠󸀠 (x) + V(x) Φ(x) = E Φ(x) (5.12)
with the abbreviations
2m 2m ̃
V(x) = U(x) , E= E.
ℏ2 ℏ2
The ODE (5.12) has to be completed by boundary conditions for the wave function Φ,
which depends on the problem under consideration. Contrary to (5.4), equation (5.12)
constitutes a homogeneous boundary value problem, however, with variable coeffi-
cients. We can also interpret (5.12) as a linear eigenvalue problem

̂ n = En φn
Hφ

where E n denote the eigenvalues and φ n (x) the eigenfunctions of the differential op-
erator (Hamiltonian)
d2
Ĥ = − 2 + V(x) .
dx
Discretizing (5.12) leads to a homogeneous algebraic eigenvalue problem of the
form:
∑ H ij Φ j = E Φ i (5.13)
j

with the tridiagonal matrix

2 1
H ii = + V(x i ) , H i,i−1 = H i,i+1 = − (5.14)
∆x2 ∆x2
and Φ i = Φ(x i ).
5.2 Finite differences | 123

5.2.2.1 Stark effect

As an application we wish to examine a particle in a one-dimensional quantum well
with length L. If the walls are inﬁnitely high the probability density outside the well
must be zero. From the continuity of Φ it follows for the boundary conditions

Φ(0) = Φ(L) = 0 . (5.15)

If an external electric ﬁeld is applied, the potential in (5.12) reads

V(x) = V0 ⋅ (x − L/2) . (5.16)

If V0 = 0 the exact solutions
2 kπ k2 π2
Φ k (x) = √ sin x, Ek = , k = 1, 2, . . .
L L L2
are known. Hence, Ĥ has countably inﬁnite eigenfunctions, all with different eigen-
values. From ﬁrst-order perturbation theory [2] it follows
L
(1) 2 kπ
Ek = ∫ dx V(x) sin 2 x,
L L
0

which vanishes for all k due to symmetry reasons. Thus, the change of the spectrum
is at least of the second order, ∼ V02 and named the quadratic Stark effect.
For a direct numerical solution one has to ﬁnd the eigenvalues and eigenvectors
of the problem (5.13-5.15) with
L
V(x i ) = V0 ⋅ (i∆x − L/2) , i = 1...n , ∆x = .
n+1
Here, i = 0 corresponds to the left, i = n + 1 to the right wall and H has the form of an
n × n tridiagonal matrix. Figure 5.2 depicts the probability densities |Φ(x)|2 for the ﬁrst
three states and several values of V0 . It turns out that for increasing V0 the probability
densities are shifted more and more to the left where the potential is minimal.
To compute the eigenvalue problem the LAPACK routine SSTEQR (real-valued,
symmetric tridiagonal matrix) can be used:

...
xl=1. ! length L=1 of the well
dx=xl/FLOAT(n+1)
DO i=1,n
dl(i)=-1./dx**2 ! matrix elements
di(i)=2./dx**2 + v0*(FLOAT(i)*dx-xl/2)
ENDDO
CALL SSTEQR('i',n,di,dl,z,n,work,info)
c .. eigenvalues in di, eigenvectors in z
c .. z(1:n,k) belongs to di(k)
.. output, plot, etc
124 | 5 Ordinary differential equations II

Fig. 5.2: Numerical solutions of the stationary Schrödinger equation with (5.16). Shown in
each frame are the ﬁrst three states (probability densities |Φ|2 ) (solid, thin, dashed) for
V 0 = 0, 300, 1000, 5000 (from left to right), L = 1, n = 1000.

100

–50
0 200 400 600 800 1000

Fig. 5.3: The ﬁrst three eigenvalues over V 0 for L = 1.

The ﬁrst three eigenvalues as a function of V0 are shown in Figure 5.3. All energies de-
crease for large V0 . Then also the higher states are localized in the region of a negative
energy (left).
Since this code again is mainly based on matrix computation we present here the
complete MATLAB code producing Figure 5.2:
5.2 Finite differences | 125

clear;
n=1000; v0=[0,300,1000,5000];
dx=1/(n+1); dx2=dx^2;
x=-.5+dx:dx:0.5-dx;
a=-1/dx2*ones(n,1);
for k=1:4
b=2/dx2*ones(n,1)+v0(k)*x';
m=spdiags([a,b,a],[-1,0,1],n,n);
[v,e]=eigs(m,3,'sr');
subplot(1,4,k);
plot(x,v(1:n,1).^2,x,v(1:n,2).^2,x,v(1:n,3).^2);
end

5.2.2.2 The harmonic oscillator

The stationary Schrödinger equation of the harmonic oscillator with eigenfrequency
ω0 reads
ℏ2 d2 1
[− + mω20 x2 ] Φ(x) = Ẽ Φ(x) ,
2m dx2 2
or with the scaling of (5.12)

− Φ󸀠󸀠 (x) + Ω20 x2 Φ(x) = E Φ(x) (5.17)

with
Ω0 = ω0 m/ℏ .
The problem belongs to the few ones in quantum mechanics that can be solved exactly.
One ﬁnds the equidistant eigenvalues, the energy levels¹

Eex
k = Ω 0 (1 + 2k) , k = 0, 1, . . .

and for the eigenfunctions, the Hermitian polynomials. Anyway we shall describe ﬁrst
a numerical solution and will later generalize the quadratic potential to a nonlinear
spring. A new complication in solving (5.17) arises from the asymptotic boundary con-
ditions
lim Φ(x) = 0 ,
x→±∞

saying that the x-region is in fact inﬁnite. In practice one can choose the x-interval
still ﬁnite but large enough to ensure an almost vanishing wave function at its borders
x = ±L/2, leading to the boundary conditions

Φ(L/2) = Φ(−L/2) = 0 . (5.18)

1 Doing the rescaling, we of course arrive at the more familiar relation E k = ℏω0 ( 12 + k).
126 | 5 Ordinary differential equations II

0.8

0.6

0.4

0.2

0
–15 –10 –5 0 5 10 15

Fig. 5.4: The ﬁrst three eigenfunctions |Φ|2 together with the 50th for the harmonic oscillator,
L = 30, Ω0 = 1, 1000 mesh points.

Hence we are again in the position to solve the problem of an inﬁnitely high poten-
tial well, but now with an additional quadratic potential inside. We can resort to the
same program written for the Stark effect and must only adjust the potential. The ap-
propriate value for L can be found by trial and error. Figure 5.4 depicts the ﬁrst three
probability densities as well as the 50th. One should respect that the spatial exten-
sion of the wave functions increase with k, see Section 5.2.2.4 below. To compute the
higher states correctly L should be large enough. Table 5.1 notes the computed energy
values together with the exact ones. The relative errors stay well below one percent.
The number of mesh points has been chosen with n = 1000 for the whole interval.

Tab. 5.1: Energy levels of the harmonic oscillator; ﬁnite difference method compared to the exact
values.

state k E k (numerical) E kex = 1 + 2k error |E k − E kex |/E k

0 0.999717832 1 −2.82168388E-04
1 2.99929714 3 −2.34285995E-04
2 4.99920225 5 −1.59549716E-04
3 6.99843979 7 −2.22887305E-04
4 8.99729633 9 −3.00407410E-04
5 10.9976883 11 −2.10155136E-04
6 12.9952221 13 −3.67531407E-04
7 14.9935150 15 −4.32332366E-04
8 16.9906712 17 −5.48755401E-04
9 18.9886627 19 −5.96698956E-04
... ... ... ...
49 96.7354813 97 −2.72699725E-03
5.2 Finite differences | 127

300

p=4

200 p=3
Ek

100 p=2

p = 3/2

0
0 20 40 60 80 100
k

Fig. 5.5: The ﬁrst 100 energy levels for different potentials (5.19).

5.2.2.3 Anharmonic oscillator

Now we can examine other potentials. Figure 5.5 shows the ﬁrst 100 levels for poten-
tials having the form
V(x) = Ω20 |x|p (5.19)
for Ω0 = 1 and p = 3/2, 2, 3, 4. The larger the p, the stronger the increase of the ener-
gies, and in the classical picture, the spring becomes more rigid for larger amplitudes.
A deviation for p = 3/2 becomes visible at k ≈ 70. Here, the wave functions are too
wide for the given area L = 30 and the approximate boundary conditions (5.18) distort
the result.

5.2.2.4 Lateral extension of wave function

The lateral width of the wave function of the quantum mechanical oscillator can be
simply estimated by writing (5.17) as

Φ󸀠󸀠 = γ(x) Φ

with γ = Ω20 |x|p − E. If γ > 0, Φ decreases monotonically. Thus, γ = 0, or

1/p
E
xL = ( )
Ω20
gives an estimate of the wave function’s extension x L belonging to a certain energy
level E. For the harmonic oscillator (p = 2) one obtains

1 + 2k
xL = √ ,
Ω0
in very good agreement with Figure 5.4.
128 | 5 Ordinary differential equations II

5.3 Weighted residual methods

5.3.1 Weight and base functions

The keynote of the weighted residual methods (WRMs) is to solve ODEs of the form

̂ x )y(x) = b(x)
L(d (5.20)

approximately in the domain X by the ansatz

̃ 1 , a2 , . . . a N , x) ,
y = y(a (5.21)

where ỹ is called a test function. The N free parameters a k are determined in a way
that the residuum (the remainder)

R(x) = L̂ ỹ − b ,

projected on a certain set of problem speciﬁc weight functions w k (x) vanishes:

R k = ∫ dx R(x) w k (x) = 0 , k = 1...M . (5.22)

If the number of the weight functions M is equal to the number of parameters N, the a k
can be determined from the N equations (5.22).
In particular we wish to examine in this section linear differential operators L̂ and
in a k linear test functions (5.21) of the form
N
̃
y(x) = ∑ a i φ i (x) (5.23)
i=1

with linearly independent base functions φ i (x). Then (5.22) turns into the linear sys-
tem
N
∑ L ij a j = b i (5.24)
j=1

with the matrix elements

̂ x )φ j (x) ,
L ij = ∫ dx w i (x)L(d b i = ∫ dx w i (x) b(x) .
X X

For a nonlinear operator L,̂ instead of (5.24), a nonlinear algebraic system for the set
a k would result, which can be only solved iteratively.
Depending on the weight functions, one distinguishes between different WRMs.
We mention the most important ones:
5.3 Weighted residual methods | 129

1. Subdomain method. The domain X is divided into M subdomains D k which may

overlap. One chooses w k (x) equal to one if x is in D k , otherwise zero:

{1 if x ϵ Dk
w k (x) = {
0 else
{
In this way one can enlarge the accuracy (smaller and more subdomains) of the
method at certain places where the expected solution varies strongly, while in
other ‘quieter’ regions larger subdomains can be appropriate.
2. The collocation method can be considered as a special case of 1. Let each D k
shrink to a point
w k (x) = δ(x − x k )
where δ(x) denotes Dirac’s delta-function. Due to (5.22) one has

R(x k ) = 0

and
̂ x )φ j (x)󵄨󵄨󵄨󵄨
L ij = L(d b i = b(x i ) .
󵄨 x=x ,
i

The x k are called collocation points.

3. Least-squares method. Instead of requiring (5.22) one can also minimize the
mean squared residuum
S = ∫ dx R2 (x) (5.25)
X

and determine a i from

∂S
=0. (5.26)
∂a i
From (5.25) one obtains
∂S ∂R
= 2 ∫ dx R(x) =0.
∂a i ∂a i
X

For the least-squares method no weight functions are needed.

4. Galerkin method. Here, weight and base functions are identical:

w k (x) = φ k (x) , k = 1...N .

Due to (5.22) the residuum is orthogonal to the subspace spanned by the base func-
tions. If one further increases N, this subspace is getting more and more complete
leaving less and less space for the residuum. Thus, R(x) must vanish for N → ∞.
130 | 5 Ordinary differential equations II

5.3.2 Example: Stark effect

We wish to compute the ground state and the ﬁrst excited state of the stationary
Schrödinger equation

− Φ󸀠󸀠 (x) + V0 ⋅ (x − 1/2) Φ(x) = E Φ(x) , Φ(0) = Φ(1) = 0 , (5.27)

corresponding to the Stark effect in an inﬁnitely high potential well with L = 1. Let the
test function be a polynomial of the third degree:

̃
Φ(x) = a0 + a1 x + a2 x2 + a3 x3 .

If Φ̃ fulﬁlls the boundary conditions it follows that

a0 = 0 , a3 = −a1 − a2 ,

or
Φ̃ = a1 φ1 + a2 φ2
with the two linearly independent base functions

φ1 = x − x3 , φ2 = x2 − x3 .

Contrary to (5.20), the problem (5.27) is homogeneous and instead of (5.24) we obtain
a generalized linear eigenvalue problem of the form
2
∑ (L ij − E M ij ) a j = 0 . (5.28)
j=1

The matrix elements L ij , M ij thereby depend on the method. The two eigenvalues E
are then determined from the solvability condition

Det (L ij − E M ij ) = 0 . (5.29)

1. Subdomain method. Deﬁning the two subdomains D1 : 0 ≤ x ≤ 1/2, D2 : 1/2 <

x ≤ 1 yields
1
E0,1 = 30 ∓ √32 400 + 5V02 .
10
2. The collocation method with x1 = 1/4, x2 = 3/4 provides
64 1
E0,1 = ∓ √16 384 + 9V02 .
3 12
3. For the least-squares method we obtain from (5.26) two homogeneous algebraic
equations where E occurs quadratically. Evaluating the solvability condition
yields a polynomial of the 4th degree in E. Therefore this method seems not to be
adequate in determining the eigenvalues E.
5.3 Weighted residual methods | 131

collocation
E

subdomain ﬁnite diff.

20
Galerkin

2000 40 60 80 100
V0

Fig. 5.6: Stark effect, the methods by comparison, energy of the ground state, and the ﬁrst excited
state over V 0 .

Tab. 5.2: The ﬁrst two energy levels for V 0 = 0 for the three different methods.

exact subdomain collocation Galerkin

E0 π ≈ 9.87
2
12.0 10.7 10.0
E1 4π 2 ≈ 39.5 48.0 32.0 46.0

4. Galerkin method. With w k = φ k we eventually ﬁnd

V02
E0,1 = 26 ∓ √256 + .
28

Of course the results depend strongly on the applied method. Also note that only two
base functions have been used and more exact results cannot be expected from the
first. Figure 5.6 compares the outcome of the three methods with that of the finite differ-
ence algorithm from Section 5.2.2.1. In Table 5.2 we list the values for the undisturbed
problem V0 = 0.
All three methods provide a quadratic dependence on V0 as well as the correct
sign for the curvature of E. Not surprisingly, the first excited state inaccurately takes
a test function with only two free parameters. On the other hand, the energy of the
ground state agrees amazingly well at least for the Galerkin method. For the two
other methods, additional freedom lies in the choice of the subdomains or collocation
points, respectively.
132 | 5 Ordinary differential equations II

5.4 Nonlinear boundary value problems

So far we have examined only linear problems of the form (5.20). In this section we
wish to extend the treatment to certain nonlinear systems
̂ nx )y(x) + g(y, d n−1
L(d x y, . . . ) = b(x) (5.30)

where L̂ denotes a linear differential operator and comprises the highest occurring
derivative. The nonlinear function g is at least bilinear in y and/or its derivatives. In-
dependently from the applied algorithm, one arrives after discretization at a nonlinear
system of algebraic equations.

5.4.1 Nonlinear systems

There is no general receipt for solving nonlinear algebraic systems. This can be seen
already at a system of only two equations. Let us seek a solution of

f(x, y) = 0
(5.31)
g(x, y) = 0

with some nonlinear functions f, g of the two variables x, y. Graphically, we can plot
the zero lines of f and g and determine their intersections, as in Figure 5.7. But the two
functions are completely independent on each other. The zeros of each function can

Fig. 5.7: The zero lines of two functions f(x, y) = 0 (solid) and g(x, y)=0 (dashed). The intersections
correspond to the solutions of (5.31). As we can see, many solutions may exist, among them also
‘almost’ solutions and double zeros. A numerical solution found iteratively will surely depend on the
initial value x (0) , y (0) of any iterative scheme.
5.4 Nonlinear boundary value problems | 133

consist of an arbitrary number of discontiguous curves. Already for two equations,

zero, one, many, or even infinitely many solutions can exist. Without any further in-
formation a solution of (5.31) can be very arbitrary and sometimes hard to find. This
additional information can be the approximate position(s) of the zero(s) to be com-
puted. With other words, one needs an initial value x(0) , y(0) from where an iterative
refinement then could yield the desired solution of (5.31). We shall come back to this
question with some applications in Section 5.4.3.2 and 5.4.4.

5.4.2 Newton–Raphson

The Newton or Newton–Raphson method is well-known for ﬁnding zeros of a function

f(x) = 0 (5.32)

iteratively. Let x(0) be an (initial) value not too far from the desired zero x k . In its neigh-
borhood, the Taylor expansion

f(x(0) + δx) = f(x(0) ) + d x f(x(0) ) δx + O(δx2 ) (5.33)

is therefore valid. The values for f(x(0) ) and d x f at x(0) are given with f(x). From the
requirement
f(x(0) + δx) = 0
we can determine δx:
f(x(0) )
δx = − + O(δx2 ) (5.34)
d x f(x(0) )
and from there the zero
x k ≈ x(0) + δx . (5.35)
Because of the neglect of higher order terms in (5.33) the zero will not be reached ex-
actly and (5.35) provides only an approximate value x̃ k . However, x̃ k will be closer to
the exact zero than the initial value x(0) . Substituting x̃ k in formula (5.34) will yield an
even closer value and so on. Thus, an iteration rule
f(x(i) )
x(i+1) = x(i) − (5.36)
d x f(x(i) )
is obtained, where the sequence x(i) converges against the zero x k . For a stopping cri-
terion one may take
|δx(i) | = |x(i+1) − x(i) | < ϵ
with given accuracy ϵ. If (5.32) possesses many solutions the result will depend on the
initial value x(0) .
The Newton–Raphson method can be generalized straightforwardly to systems
with N equations:
f i (x1 , x2 , . . . x N ) = 0 i = 1 . . . N . (5.37)
134 | 5 Ordinary differential equations II

Instead of (5.33) one ﬁnds

N
∂f i
f i (x⃗ (0) + δ x)⃗ = f i (x⃗ (0) ) + ∑ δx j + . . . (5.38)
j=1
∂x j

and from there

N
∑ α ij δx j = −f i (5.39)
j=1

with the Jacobian

∂f i
α ij = . (5.40)
∂x j
To determine δx i one has to solve at each iteration step the inhomogeneous system
(5.39). From δx i one obtains along (5.35)

x⃗ (i+1) = x⃗ (i) + δ x⃗ (i) .

5.4.3 Example: the nonlinear Schrödinger equation

We study the nonlinear stationary Schrödinger equation (NSE) in one dimension

− Φ󸀠󸀠 + γ |Φ|2 Φ = EΦ (5.41)

that can serve as a model of a charged particle in a self-generated charge density cloud,
having the potential
V = γ |Φ2 | .
Taking a real-valued Φ, we obtain

Φ󸀠󸀠 + EΦ − γ Φ3 = 0 . (5.42)

Applying the ﬁnite difference method of Section 5.2.2 leads to the algebraic system
N
f i (Φ1 , Φ2 . . . Φ N ) = ∑ L ij Φ j − γ Φ3i = 0 (5.43)
j

with the tridiagonal matrix

2 1
L ii = − +E, L i,i+1 = L i,i−1 = . (5.44)
∆x2 ∆x2
For the matrix α in (5.40) one obtains with (5.43)

∂f i
α ij = = L ij − 3 δ ij γ Φ2i
∂Φ j

with the Kronecker symbol δ ij .

5.4 Nonlinear boundary value problems | 135

5.4.3.1 Exact solutions

We saw above that it is important to have a certain idea of the sought solutions, provid-
ing reasonable initial values for the iteration procedure. For the NSE two exact non-
trivial solutions are known that we shall derive brieﬂy.
Multiplying (5.42) by Φ󸀠 and integrating yields
γ 4
(Φ󸀠 )2 = C − EΦ2 + Φ (5.45)
2
with a constant C. A second integration can be performed by the separation of vari-
ables resulting for an arbitrary C in elliptic integrals. For certain C one may distinguish
two physically relevant cases:
1. E > 0, γ > 0, C = E2 /2γ, repelling potential, free solution:

E E
Φ(x) = ±√ tanh (√ (x − x0 )) . (5.46)
γ 2

2. E < 0, γ < 0, C = 0, attractive potential, bounded self-focusing solution:

2E 1
Φ(x) = ±√ . (5.47)
γ cosh (√−E (x − x0 ))

The solution (5.46) corresponds to a front or a kink at x = x0 connecting the two

asymptotic solutions Ψ(x → ±∞) = ±√E/γ, (5.47) to a localized wave function around
x = x0 , Figure 5.8.

Φ Φ

x x

Fig. 5.8: Left: front for E, γ > 0, right: localized wave for E, γ < 0. Both are exact solutions of the NSE.

5.4.3.2 Numerical implementation

Both solutions (5.46,5.47) are infinitely extended along the x-axis and fulfill asymptotic
boundary conditions at x → ±∞. Again we use the trick from above and apply these
boundary conditions at x = ±L/2, taking a large L.
Let us begin with the first case, E, γ > 0. For the front solution, the derivatives
have to vanish at infinity, requiring

d x Φ|x=−L/2 = 0 , → Φ0 = Φ1 , d x Φ|x=L/2 = 0 , → Φ N+1 = Φ N . (5.48)

136 | 5 Ordinary differential equations II

0.5
0.2
Φ

Φ
0 0

–0.2
–0.5

x x

Fig. 5.9: Two numerically found solutions of the NSE. Left: front four E = 1/2, γ = 1, right: another
solution for E = 1, γ = 1. The left frame corresponds to the exact solution (5.46).

This can be incorporated into the differential matrix (5.44) by modifying the first and
the last diagonal element according to
2 1 1
L11 = L NN = − +E+ 2 =− 2 +E.
∆x2 ∆x ∆x
As the initial value for Φ we can take a straight line Φ = x/L. Depending on parameters
one obtains different solutions where the exact one (5.46) is also present (Figure 5.9).
For the second case, E, γ < 0, the wave function has to vanish at infinity. Thus
we set
Φ|x=−L/2 = 0 → Φ0 = 0 , Φ|x=L/2 = 0 → Φ N+1 = 0 . (5.49)
These boundary conditions are already included in (5.44). To obtain a convergent iter-
ation one should again take an initial condition close to the expected solution, e.g.,
π
Φ = cos ( x) .
L
Again the computed solutions depend strongly on parameters. For γ = −1, E = −1
the algorithm converges to the state (5.47); for γ = −1, E = −2 one obtains numeri-
cal rubbish that, however, also fulfills the stopping criterion (Figure 5.10). As always,
‘prudence is the better part of valor!’

5.4.4 Example: a moonshot

In J. Verne’s² famous novel [3] a space ship is shot toward the moon by a giant cannon.
Before the development of smoothly launching rockets this was the only at least hy-
pothetical possibility for leaving the gravity ﬁeld of the earth. Albeit not realizable, at

2 Jules Verne, French novelist, 1828–1905.

5.4 Nonlinear boundary value problems | 137

20
1
10

0 0
Φ

Φ
–10
–1
–20
0 2 4 6 8 10 0 2 4 6 8 10
x x

Fig. 5.10: Two solutions in the parameter region of bounded states. Left: localized wave for E = −1,
γ = −1, right: numerical artifact for E = −2, γ = −1. For all computations 100 mesh points have been
used.

least with surviving passengers, this idea is perfectly appropriate to be formulated as a

boundary value problem as done in Section 5.1.2 for the ballistic ﬂight. Assuming that
the ﬂight path of the space ship lies in the earth-moon plane and the moon revolves
in a circular orbit, the system earth-moon-space ship can be considered as a reduced
three-body problem and the equations derived in Section 4.4.4 can be applied. For the
mass parameter μ one has

Mmoon
μ= ≈ 0.012 .
Mearth + Mmoon

After discretization of the time derivatives by ﬁnite differences, (4.45) turns into a non-
linear algebraic system
2n
∑ M ij q j = p j (q1 . . . q2n ) (5.50)
i

with the difference matrix (band matrix, width =6)

−2 0 1 −∆t 0 ..
0 −2 ∆t 1 0 ..
( 1 ∆t −2 0 1 −∆t 0 .. )
( )
(−∆t 1 0 −2 ∆t 1 0 .. )
1 (
(
)
)
M= 2( ...... ) ,
∆t ( )
( ...... )
( )
( ...... )
.. 0 1 ∆t −2 0
( .. 0 −∆t 1 0 −2)

the solution vector

q⃗ = (x1 , y1 , . . . x n , y n ),
138 | 5 Ordinary differential equations II

and the nonlinear inhomogeneities

1−μ μ
p2i−1 = − (x i + μ) − (x i + μ − 1) + x i
r32 (x i , y i ) r33 (x i , y i )
(5.51)
1−μ μ
p2i = − yi − yi + yi , i = 1...n
r32 (x i , y i ) r33 (x i , y i )

(n is the number of mesh points; for the other notations see Section 4.4.4.2). Differing
from Jules Verne our space ship should ‘land’ on the moon after a given ﬂight time
t = T. This leads to the Dirichlet boundary conditions:

x(0) = x0 = xs , y(0) = y0 = y s , x(T) = x n+1 = x t , y(T) = y n+1 = y t

with the start vector (xs , y s ), prescribed somewhere on the surface of the earth and the
target coordinates (x t , y t ) somewhere on the moon.
Being nonlinear, the system (5.50) can only be tackled iteratively. To improve con-
vergence it is advantageous to add on the right-hand side of (5.50) a damping term
having the form s (q⃗ k+1 − q⃗ k ). Solving for q⃗ k+1 then yields
−1
q⃗ k+1
= (M − s 1) (p(⃗ q⃗ k ) − s q⃗ k ) . (5.52)

Essentially we have to solve successively inhomogeneous systems of equations. We

list the MATLAB Code:

clear;
n=10000; tend=[.2,0.75,2.5] % flight times (scaling as in sect.4.4.4)
mu=1./82; earth_radius=6300/380000; moon_radius=1600/380000;
xs=-mu+earth_radius; ys=0; xt=1-mu-moon_radius; yt=0; % Dirichlet
% conditions
s=100; % convergence factor

for kk=1:3 % three flight times

dt=tend(kk)/(n-1); dt2=dt^2;
q(1:2*n)=0;
for i=1:2:2*n-1 % define sparse matrix
a(i)=dt; b(i+1)=dt;
end
a(2*n)=0; delta=1;
m=spdiags([-b',ones(2*n,1),a',(-2-s*dt2)*ones(2*n,1),b',
ones(2*n,1),-a'],[-3,-2,-1,0,1,2,3],2*n,2*n);

while delta>0.1 % iteration loop

for i=1:2:2*n-1 % right-hand side
i1=i+1;
5.4 Nonlinear boundary value problems | 139

r13=sqrt((q(i)+mu)^2+q(i1)^2)^3;
r23=sqrt((q(i)+mu-1)^2+q(i1)^2)^3;
p(i)=(-(1-mu)/r13*(q(i)+mu)-mu/r23*(q(i)+mu-1)+q(i)*(1-s))*dt2;
p(i1)=(-(1-mu)/r13*q(i1)-mu/r23*q(i1)+q(i1)*(1-s))*dt2;
end
% boundary conditions as additional inhomogeneity.
p(1)=p(1)-ys*dt-xs; p(2)=p(2)+xs*dt-ys;
p(2*n)=p(2*n)-xt*dt-yt; p(2*n-1)=p(2*n-1)+yt*dt-xt;
q1=q;
q=p/m;
delta=norm(q-q1); % stopping criterion
end
yof=(kk-1)*3;
subplot(3,3,1+yof); plot(q(1:2:2*n),q(2:2:2*n)); % output
subplot(3,3,2+yof); plot(1:n,q(1:2:2*n));
subplot(3,3,3+yof); plot(1:n,q(2:2:2*n));
end

Taking the L2 norm, the deviation from one step to the next reads

δ k = |q⃗ k − q⃗ k−1
|.

If δ k falls below a certain given value, the iteration is stopped and the results are
plotted (Figure 5.11.). Apart from initial and target conditions, the ﬂight paths depend
strongly on the given ﬂight time T.

0 0
0.8
–0.02 0.6 –0.02
0.4
–0.04 0.2 –0.04

0.2 0.4 0.6 0.8 5 10 15 20 5 10 15 20

0 0
0.8
–0.1 0.6 –0.1
0.4
–0.2 0.2 –0.2
0.2 0.4 0.6 0.8 20 40 60 20 40 60
0 0
1

–0.5 0.5 –0.5

–1 0 –1
0 0.5 1 50 100 150 200 250 50 100 150 200 250

Fig. 5.11: Trajectories of the space ship (left), x (middle), and y (right) coordinates over time for three
different ﬂight times T = 20 h, 75 h, 250 h (from top to bottom).
140 | 5 Ordinary differential equations II

Going back to dimensional variables one finds for the three flight times 20 h, 75 h, and
250 h, the touch-down speeds v(T) = √ẋ 2 (T) + ẏ 2 (T) of 18 000 km/h, 7000 km/h, and
5500 km/h, respectively. At launch the speed for all three cases is roughly 40 000 km/h.
Jules Verne was right assuming that his pilots could not have been injured by the
pressure wave of the detonation of the canon because they would have exceeded the
speed of sound in the first moment. However, the devastating acceleration would
have mashed them anyway beforehand.

5.5 Shooting

5.5.1 The method

The main idea of the shooting method is to ﬁrst neglect the boundary conditions on
one side, say on the right, and to convert the system under consideration to an initial
value problem. Then the solution is not unique and one varies iteratively the initial
values (aiming) until the boundary conditions on the right are met (meaning the shot
is on target).
Let us explain the method in more detail with a simple example. We are looking
for a solution y(x) of a second-order ODE in 0 ≤ x ≤ 1:

y󸀠󸀠 = f(y, y󸀠 , x)

or of the equivalent problem

y󸀠1 = y2 , y󸀠2 = f(y1 , y2 , x) (5.53)

with the boundary condition

y(0) = a , y(1) = b . (5.54)

One integrates (5.53) for instance with RK4 numerically from x = 0 through x = 1
taking the initial conditions

y1 (0) = a , y2 (0) = s

and different values for s (Figure 5.12). For each s one ﬁnds on the right-hand boundary
some
y R (s) = y1 (x = 1, s) .
If one is able to get a value for s leading to y R = b, this solution corresponds to a
solution of (5.53) fulﬁlling (5.54). Thus one has to seek the zero(s) of the function

f(s) = y R (s) − b ,
5.5 Shooting | 141

f(s2)
s3
b
Fig. 5.12: Shooting method. Three different
0 1 x
trials that fulﬁll the left side boundary condi-
tions only.

by applying for instance the Newton–Raphson method from Section 5.4.2:

f(s)
s(i+1) = s(i) − .
f 󸀠 (s)
However, f 󸀠 is in general not known and one has to approximate the derivative by
taking the differential quotient
f(s + ∆s) − f(s)
f 󸀠 (s) ≈ .
∆s
Hence, for each iteration step, one needs two different numerical solutions for s and
a nearby s + ∆s of (5.53).

5.5.2 Example: free fall with quadratic friction

A parachutist exits a nonmoving helicopter at the height of 1000 m at t = 0. At which

height y f should he open his parachute if he wants to touch down at t = 60 seconds
on the ground y = 0? Assume the ﬂight takes place without friction if the parachute
is closed. For the open parachute, assume friction proportional to the square of the
speed.
The task can be formulated as a boundary value problem according to

ÿ = −f(y, y,̇ y f ) ẏ − g (5.55)

with the boundary conditions

y(t = 0) = 1000 m , ̇ = 0) = 0 ,
y(t y(t = 60s) = 0 . (5.56)

The function f allows for friction:

{0 if y > y f
f ={ .
α |y|̇ if y ≤ y f
{
142 | 5 Ordinary differential equations II

Tab. 5.3: Height y after one minute ﬂight time and touch-down velocity v for different values of y f .
The optimal y f is between 450 and 550 m.

y f [m] 10 30 250 450 550 750

y(t = 60 s) [m] −472 −451 −246 −63 28 207
v(y = 0) [m/s] −50 −12 −10 −10

1000

500

0
0 10 20 30 40 50 60

Fig. 5.13: Solution y(t) over t for the correct value y f = 518 s.

Since we have a second order ODE and there are three conditions to fulﬁll, the problem
looks over-determined at ﬁrst glance. However, an additional degree of freedom comes
with the choice of y f . Taking the initial conditions (5.56), the equations (5.55) are inte-
grated with RK4 until t = 60s for different values of y f . The results for α = 0.1/m and
g = 10 m/s2 are shown in Table 5.3.
The sought value is about 500 m, which is now determined more exactly by a
Newton–Raphson method. Taking an accuracy of 1 meter, the iteration converges
within four steps at y f = 518 m. The solution for this value is shown in Figure 5.13.

5.5.3 Systems of equations

Next we wish to extend the method to the general case of a system of N ﬁrst-order
ODEs
dy i
= f i (y1 , y2 . . . y N , x) , i = 1 . . . N . (5.57)
dx
Again we seek a solution in a ≤ x ≤ b. Assuming linear and separable boundary
conditions we may have on the left-hand boundary n1 conditions

⃗
A y(a) = a⃗ , (5.58)

and on the right-hand side

⃗
B y(b) = b⃗ (5.59)
Bibliography | 143

n2 = N − n1 conditions to fulﬁll. The rectangular matrices A and B have n1 × N and

n2 × N elements, respectively. Using (5.58) we can uniquely determine n1 of the y i (a)
as an initial condition. The still undetermined y i can be confined to an n2 -component
vector V.⃗ By numerically integrating (5.57) we obtain on the right-hand boundary
y Ri (V)⃗ = y i (x = b, V)⃗ ,
which should fulfill the n2 right side boundary conditions (5.59). Thus, one has to find
the simultaneous zeros of the functions
f ⃗(V)⃗ = B y⃗ R (V)⃗ − b⃗ . (5.60)
The n2 equations (5.60) determine the n2 components of V.⃗ Applying the Newton–
Raphson method we obtain an iteration algorithm according to
α δ V⃗ (i) = −f ⃗(V⃗ (i) )
and
V⃗ (i+1) = V⃗ (i) + δ V⃗ (i)
with the n2 × n2 matrix
∂f k
α kℓ =
.
∂Vℓ
Since f is known only numerically, the derivatives with respect to Vℓ can only be found
numerically as well. Approximating the derivatives with the differential quotients
∂f k f k (V1 , . . . Vℓ + ∆Vℓ , . . . V n2 ) − f k (V1 , . . . Vℓ . . . V n2 )
≈ ,
∂Vℓ ∆Vℓ
the system (5.57) must be integrated n2 + 1 times at each iteration step.

5.6 Problems

1. Find an analytical solution of (5.4) for the general case α, β ≠ 0.

2. Verify the solutions (5.46) and (5.47). What is different for other (arbitrary) values
of C?
3. Solve the problem from Section 5.1.2 (ballistic ﬂight) with the help of the shooting
method. Also examine the inﬂuence of nonlinear friction having the form
1/2 ẋ
F⃗ f = −α (ẋ 2 + ẏ 2 ) ( ) .
ẏ

Bibliography

[1] LAPACK–Linear Algebra PACKage–Netlib, www.netlib.org/lapack/, February 8, 2018.

[2] See text books on quantum mechanics, e.g., D. J. Griffiths, Introduction to Quantum Mechanics,
Cambridge Univ. Press (2016).
[3] Jules Verne, From the Earth to the Moon, Bantam Classics (1993).
6 Partial differential equations I, basics
If we are interested in studying and modeling physical problems originating from con-
tinuum physics, we are rapidly confronted with partial differential equations (PDEs).
‘Continuum physics’ in a wider sense deals with time-dependent objects filling the
whole (real) space, like flow fields, mass or charge densities, velocity or displacement
fields, electromagnetic fields, or quantum mechanical wave functions, to name but
a few. In most physical models, however not in all, first and/or second order deriva-
tives in both the time and space of the field functions may appear. The PDEs can be
classified by a number of characteristics and are grouped into elliptic, parabolic, and
hyperbolic equations.

6.1 Classiﬁcation

We commence with the mathematical standard classiﬁcation based on characteris-

tics and study the consequential speciﬁcations for a numerical implementation and
boundary conditions. A characteristic, or characteristic curve, is a certain curve in
the space of independent variables along which the PDE becomes an ordinary differ-
ential equation. If the characteristic is known, this ODE can be solved and a special
solution of the PDE is thereby found. If one of the variables is the time, the character-
istic deﬁnes curves along which information from the initial conditions is spread in
space.

6.1.1 PDEs of the ﬁrst order

We restrict ourselves for the time being to one dependent variable, say u. Let us start
with only two independent variables, x and t. The most general (quasilinear) PDE of
the first order then reads
∂u ∂u
A(u, x, t) + B(u, x, t) = C(u, x, t) . (6.1)
∂x ∂t
If A and/or B depend additionally on the first derivatives of u, (6.1) is a nonlinear PDE.
In contrast, a linear first-order PDE has the general form,

̃ ∂u ̃ ∂u ̃ ̃
A(x, t) + B(x, t) = C(x, t) u + D(x, t) . (6.2)
∂x ∂t

https://doi.org/10.1515/9783110515145-006
6.1 Classiﬁcation | 145

With the help of (6.1), the variation of u (the total differential) can be written as
∂u ∂u C ∂u B
du = dx + dt = dx + [dt − dx]
∂x ∂t A ∂t A
C ∂u A
= dt + [dx − dt] . (6.3)
B ∂x B
If we choose the path in the tx-plane in such a way that
dx A
= , (6.4)
dt B
both square brackets vanish and
C C
du = dt = dx (6.5)
B A
remains along the characteristic curve xc (t) that is given as a solution of the ODE (6.4).
For the special case C = 0 (homogeneous PDE) one obviously obtains du = 0 and
thereby, u(xc (t), t) = const.

6.1.1.1 Example: convection equation

To exemplify the idea, we wish to study the convection equation

∂ t u + v(x)∂ x u = 0 (6.6)

with a space-dependent velocity as a special case of (6.1) with A = v(x), B = 1, C = 0.

Let
v(x) = α (xs − x) . (6.7)
Integrating (6.4) one ﬁnds the characteristics (Figure 6.1) as a family of curves along
which u is not changing (C = 0):

xc (t) = xs (1 − e−α(t−t 0) )

with arbitrary but constant t0 (integration constant).

Equation (6.6) could serve as a model for the spatial evolution and spreading of
a certain concentration, i.e., of some pollutants, transported by a given air ﬂow (6.7).
Due to the special form of v(x), the pollutants accumulate in the course of time near
x = xs .

u = const. Fig. 6.1: Characteristics of the convection equa-

tion (6.6) with v(x) as (6.7). Along the character-
istics, initial information is transported in the
t positive t-direction.
146 | 6 Partial differential equations I, basics

(a) (b) (c)

Fig. 6.2: As the convection equation, the Burgers equation yields as a solution for u > 0 traveling
waves to the right. Since the velocity is proportional to the amplitude, wave crests move faster and
catch up to wave troughs (b), eventually leading to a breaking wave (c). In (c), u can no longer be
expressed as a function of x.

6.1.1.2 Example: Burgers equation

As a prototype for a quasilinear PDE we shall examine the one-dimensional Burgers
equation
∂t u + u ∂x u = 0 . (6.8)

The nonlinearity steepens the wave front more and more and ﬁnally causes a singu-
larity at which the wave would break, Figure 6.2. However, at this point (6.8) loses its
validity.
A special solution of Burgers equation is linear in x and reads
αx
u(x, t) = − . (6.9)
1 − αt
For α > 0, a singularity exists at t∗ = 1/α, corresponding to an inﬁnite slope and
wave breaking. This can also be recognized looking at the characteristics. From (6.4)
we obtain
dxc α xc
=u=− ,
dt 1 − αt
and once integrated,
xc (t) = x0 ⋅ (1 − αt)

with the constant x0 = xc (0). All characteristics meet at the singularity t = t∗ , x = 0

(Figure 6.3).

Fig. 6.3: Burgers equation; characteristics of the

special solution (6.9). The intersection of the
characteristics is linked with a singularity of u.
6.1 Classiﬁcation | 147

6.1.2 PDEs of the second order

We discuss again the case of two independent variables, now x, y. The general form
of a second order PDE reads
∂2 u ∂2 u ∂2 u ∂u ∂u
A + B + C +D +E +F=0 (6.10)
∂x2 ∂x∂y ∂y2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
∂x ∂y
≡H

with A . . . F as functions of x, y and possibly of u. As we shall see in a moment, only

the coefficients of the second derivatives A, B, C determine the characteristics. We try
the same procedure as in Section 6.1.1: what are the curves along which the variation
of u is described by an ODE as in (6.5)? Let the curves sought after be expressed as

y = y c (x) (6.11)

with their local slope m(x, y c ) = dy c /dx at x, y c . Introducing the abbreviations

P = ∂x u , Q = ∂y u

one obtains the total differentials

du = P dx + Q dy (6.12)

and
dP = (∂2xx u) dx + (∂2xy u) dy , dQ = (∂2xy u) dx + (∂2yy u) dy . (6.13)
Using (6.13), the second derivatives along (6.11) can be expressed as
dP dQ 1 2
∂2xx u = − m ∂2xy u , ∂2yy u = − ∂ u. (6.14)
dx dy m xy

Inserting (6.14) into (6.10) yields

dP dQ
− ∂2xy u [A m2 − B m + C] + (A + H) m + C =0. (6.15)
dx dx
If we choose the local slope of the characteristics as

1 B √ B2
m1,2 = ( ± − AC) , (6.16)
A 2 4

the square bracket of (6.15) vanishes and the variation of P and Q along that curve can
be computed from
m i A dP i + C dQ i = −H m i dx i . (6.17)
P und Q (and with (6.12) also u) follow then from their previous values P i , Q i , u (see
Figure 6.4) as
P = P i + dP i , Q = Q i + dQ i . (6.18)
148 | 6 Partial differential equations I, basics

P, Q

yc(1) yc(2)

P2, Q2 Fig. 6.4: From a given P i , Q i one can determine

P1, Q1
P, Q with the help of (6.17, 6.18).

From the six equations (6.17, 6.18), the six unknowns P, Q, dP i , dQ i can be uniquely
determined. Knowing m i from (6.16), the characteristics can be found solving the ODEs
(i)
dy c (i)
= m i (y c , x) . (6.19)
dx
For a linear PDE, (6.16) and therewith y c (x) are independent from the solution u.

6.1.2.1 Discriminant
Are there always two different families of characteristic curves? Obviously this can be
only the case if the two roots (6.16) are real-valued and do not coincide. For a further
classiﬁcation we consider the discriminant
B2
D= − AC (6.20)
4
and distinguish the cases
– D > 0 : m1 ≠ m2 , both real-valued, two characteristics, hyperbolic PDE
– D = 0 : m1 = m2 , real-valued, one characteristic, parabolic PDE
– D < 0 : m1 = m∗2 , complex-valued, no characteristics, elliptic PDE

In Table 6.1 we list several PDEs and their allocation according to the three classes
above, now for three spatial dimensions.

Tab. 6.1: Several important second order PDEs from different areas of physics and their classiﬁca-
tion with respect to characteristics in three space dimensions, ∆ = ∂2xx + ∂2yy + ∂2zz . The eigenvalues
of the coefficient matrix A (6.21) are denoted by λ i .

hyperbolic parabolic elliptic

wave equation diffusion equation Poisson equation

∂2tt u − c 2 ∆u = 0 ∂ t u − κ∆u = 0 ∆u = −ρ/ϵ0
λ1 = 1, λ2,3,4 = −c 2 λ1 = 0, λ2,3,4 = −κ λ1,2,3 = 1
D = c6 > 0 D=0 D = −1
Navier–Stokes equation stationary Schrödinger equation
ρ ddtv⃗ = η∆ v⃗ − ∇P −∆u + Vu = Eu
λ1 = 0, λ2,3,4 = −η λ1,2,3 = −1
D=0 D = −1
6.1 Classiﬁcation | 149

6.1.2.2 N independent variables

Next, we extend the concept to N independent variables x i , i = 1 . . . N and write the
general form as
N N
∂2 u ∂u
∑ A ij + ∑ Bi +Cu = 0. (6.21)
i,j=1
∂x i ∂x j i=1 ∂x i

The coefficient matrix A ij is symmetric and has only real eigenvalues whose signs al-
low for a classiﬁcation according to Section 6.1.2.1. At ﬁrst we take N = 2. Using the
notation from (6.10), the matrix A reads

A B/2
A=( )
B/2 C

and has the eigenvalues

1
λ1,2 = (Tr A ± √(Tr A)2 + 4D)
2
and
D = − Det A .
With λ1 λ2 = DetA we obtain according to Section 6.1.2.1
– D > 0: λ1 λ2 < 0, hyperbolic,
– D = 0: λ1 λ2 = 0, parabolic,
– D < 0: λ1 λ2 > 0, elliptic.

This can be extended straightforwardly to N variables: Let λ i be the spectrum of the

N × N coefficient matrix A. The PDE (6.21) is
– hyperbolic if all λ i ≠ 0 and all λ i but for one have the same signs (saddle node),
– parabolic if one arbitrary λ i = 0,
– elliptic if all λ i ≠ 0 and all λ i have the same signs (node).

6.1.3 Boundary and initial conditions

As for ordinary differential equations additional information for a unique solution of a

PDE is needed to arrive at a well-posed problem, again in the form of boundary and/or
initial conditions. But here the type of the PDE plays an essential role. If we take for
example the convection equation with v = const. > 0 and search for a solution in the
domain 0 ≤ x ≤ L, we may prescribe u = u 0 (x) at t = 0 (initial condition). But in the
course of time, u moves along the characteristic u(x, t) = u 0 (x − vt), excluding some
prescribed boundary conditions on the right-hand side x = L. For PDEs of the second
order, boundary value problems on a closed boundary are well-posed only for elliptic
equations where no characteristics exist.
150 | 6 Partial differential equations I, basics

R
n̂

u(x, y)

Fig. 6.5: Typical boundary value problem for an

elliptic second order PDE. On a closed boundary
R, u or the gradient of u is prescribed and n̂
denotes the unit vector normal to the boundary
x (here for two dimensions).

6.1.3.1 Elliptic PDEs

One distinguishes between problems where u is given along a boundary R (Dirichlet
condition) and those where the derivative normal to the boundary is ﬁxed (Neumann
condition), Figure 6.5. The most general form is a combination of both, called the
Robin condition:
α u + β n̂ ⋅ ∇u = γ , r⃗ ϵ R . (6.22)

It includes the two former cases Dirichlet (β = 0) and Neumann (α = 0).

As an example consider the stationary heat equation in two dimensions (Laplace
equation)
∂2 T ∂2 T
+ =0. (6.23)
∂x2 ∂y2
If we ﬁx the temperature along R (Dirichlet)

T(x, y) = T R (x, y) , x, y ϵ R ,

the solution is unique and corresponds to the stationary temperature proﬁle of a heat
conducting plate. If, on the other hand, the heat current through the boundary (λ =
heat conductivity)
−λ n̂ ⋅ ∇T = j R , x, y ϵ R

is given, j R must be compatible with (6.23). This becomes clear by integrating (6.23)
over the whole area

1
∫∫ dxdy ∆T = ∫∫ dxdy div(∇T) = ∮ n̂ ⋅ ∇T dr = − ∮ j R dr = 0 . (6.24)
λ
F(R) F(R) R R
6.1 Classiﬁcation | 151

n̂

dr
TR

j = jR n̂
T(x,y) T (x,y)

Dirichlet
Neumann

Fig. 6.6: Specifying the temperature (left) on, or the heat current (right) through the boundary deter-
mines the temperature inside the domain as a solution of the stationary heat equation.

The amount of heat ﬂowing in the domain must be the same as that ﬂowing out. Oth-
erwise a stationary temperature on the plate is not feasible, Figure 6.6.

6.1.3.2 Parabolic and hyperbolic PDEs

As it turned out in Section 6.1.2.2 while diagonalizing the coefficient matrix, for
parabolic as well as for hyperbolic PDEs, a distinguished direction exists, namely that
belonging to the vanishing eigenvalue or that with the different sign, respectively. In
physical problems this variable corresponds very often to time. Since properties of the
solution travel along characteristics, it is not possible to pose boundary conditions
for this variable. One can still prescribe, say, u at t = t0 . If the PDE is of the second
order one may additionally ﬁx the initial derivative ∂ t u at t0 , obtaining in this way an
initial value problem (Cauchy problem).
Take as an example the time-dependent, (2+1)-dimensional¹ heat equation

∂T ∂2 T ∂2 T
= κ( 2 + ), (6.25)
∂t ∂x ∂y2

which is parabolic. In addition to the boundary conditions discussed in Section 6.1.3.1

we need an initial condition of the form

T(x, y, t = t0 ) = T0 (x, y) .

Note that for Neumann conditions the compatibility restriction (6.24) is now obsolete.
If the total heat ﬂux through the boundaries is not zero, there will be no stationary

1 With (n + 1)-dimensional, we denote a system or an equation in n spatial dimensions plus one for
time.
152 | 6 Partial differential equations I, basics

solution and, depending on the sign of the ﬂux, the plate is heating up or cooling
down forever.
The distinction between initial and boundary values is also important for hyper-
bolic equations. If we take for instance the (1+1)-dimensional wave equation

∂2 u ∂2 u
2
= c2 2 , (6.26)
∂t ∂x
there exist unique solutions in 0 ≤ x ≤ L, t ≥ 0 for given boundary conditions

u(0, t) = u ℓ , u(L, t) = u r (6.27)

and additional initial conditions

u(x, 0) = u 0 (x) , ∂ t u(x, t)|t=0 = v0 (x) . (6.28)

Table 6.2 lists several combinations of second order PDEs and boundary/initial values.

Tab. 6.2: Only certain combinations of equation types and boundary conditions constitute well-
posed problems. With ‘Cauchy’ we denote here an initial value problem of the form (6.28). For the
deﬁnition of ‘open’ and ‘closed’; see Figure 6.7.

elliptic hyperbolic parabolic

Dirichlet open under-determined over-determined well-posed

closed well-posed over-determined over-determined
Neumann open under-determined under-determined well-posed
closed well-posed over-determined over-determined
Cauchy open nonphysical well-posed over-determined
closed over-determined over-determined over-determined

Fig. 6.7: Open and closed boundaries.

6.2 Finite differences | 153

6.2 Finite differences

As for ODEs, the procedure for a numerical integration of PDEs consists in discretizing
the different independent variables and ﬁnally in an iterative or, in rare cases, a direct
solution of a normally large algebraic system of equations. The discretization can be
realized by different methods which should be suited to the problem at hand. We wish
to introduce here the most important ones and shall start with the ﬁnite-difference
method (FD).

6.2.1 Discretization

For the moment we concentrate on the case of two independent variables. One reduces
the sought solution u(x, y) on discrete mesh points or grid points

u ij = u(x i , y j ) , x i = x0 + i∆x , y j = y0 + j∆y , i = 0...I , j = 0...J ,

which are assumed to lie on a (not necessarily) regular grid in the integration domain

x0 ≤ x ≤ x1 , y0 ≤ y ≤ y1 .

The quantities
∆x = (x1 − x0 )/I , ∆y = (y1 − y0 )/J
are denoted as step sizes. In its most simple form the FD method obviously will work
only for rectangular geometries. Extensions are possible; one could choose space-
dependent step sizes, which could account for a better resolution in domains where
u is strongly varying.
Derivatives with respect to space are approximated by differential quotients,
which are available in different orders of ∆x and ∆y and in several symmetries. We list
the discretization for the ﬁrst four derivatives²:

−1 1
󵄨
du 󵄨󵄨󵄨 1
󵄨 = (u i+1,j − u i−1,j ) + O(∆x2 ) , (6.29)
dx 󵄨󵄨󵄨x i ,y j 2∆x

1 −2 1
󵄨
d2 u 󵄨󵄨󵄨 1
󵄨󵄨 = (u i+1,j − 2u i,j + u i−1,j ) + O(∆x2 ) , (6.30)
dx2 󵄨󵄨󵄨x i ,y j ∆x2

2 The formulas can be derived most easily by a Taylor expansion of u around x i .

154 | 6 Partial differential equations I, basics

−1 2 −2 1
󵄨
d3 u 󵄨󵄨󵄨 1
󵄨󵄨 = (u i+2,j − 2u i+1,j + 2u i−1,j − u i−2,j ) + O(∆x2 ) , (6.31)
dx3 󵄨󵄨󵄨x i ,y j 2∆x3

1 –4 6 −4 1
󵄨
d4 u 󵄨󵄨󵄨 1
󵄨󵄨 = (u i+2,j − 4u i+1,j + 6u i,j − 4u i−1,j + u i−2,j ) + O(∆x2 ) . (6.32)
dx 󵄨󵄨󵄨x i ,y j ∆x4
4

For elliptic PDEs and if no direction is further distinguished by means of geometrical

restraints, one normally uses square grids with ∆x = ∆y. For this case (2D) we state the
four most important formulas for the discretized Laplace operator (Figure 6.8) which,
except for (d), are all of the same order in ∆x, but have different symmetries with re-
spect to the rectangular grid geometry:
(a) 1
∆2 u|x i ,y j = (u i+1,j + u i−1,j + u i,j+1 + u i,j−1 − 4u i,j ) + O(∆x2 )
∆x2
(b) 1
= (u i+1,j+1 + u i−1,j+1 + u i+1,j−1 + u i−1,j−1 − 4u i,j ) + O(∆x2 )
4∆x2
(c) 1
= (u i+1,j+1 + u i+1,j + u i+1,j−1 + u i,j+1
3∆x2
+ u i,j−1 + u i−1,j+1 + u i−1,j + u i−1,j−1 − 8u i,j ) + O(∆x2 )
(d) 1
= (u i+1,j+1 + u i+1,j−1 + u i−1,j+1 + u i−1,j−1
6∆x2
+ 4 (u i+1,j + u i,j+1 + u i,j−1 + u i−1,j ) − 20u i,j ) + O(∆x4 ) . (6.33)

Conﬁguration (a) is most common and (b) normally leads to numerical oscillations
since even grid points (i + j even) are decoupled from odd points. (b) is therefore not
suited to diffusion equations. The nine-points formulas (c) and (d) are linear combina-
tions of (a) and (b). The symmetry of (c) comes closest to the (continuous) rotational
symmetry of ∆, while (d) minimizes the truncation error to the order ∆x4 .
The biharmonic operator is built from fourth order derivatives also including the
mixed one. Here we state only the 13-points formula of lowest order (Figure 6.9):
󵄨 ∂4 u ∂4 u ∂4 u
∆22 u 󵄨󵄨󵄨󵄨x ,y = ( 4 + 2 2 2 + 4 )
i j ∂x ∂x ∂y ∂y x i ,y j

1
= (20u i,j − 8 (u i+1,j + u i−1,j + u i,j+1 + u i,j−1 )
∆x4
+ 2 (u i+1,j+1 + u i−1,j+1 + u i+1,j−1 + u i−1,j−1)
+ u i+2,j + u i−2,j + u i,j+2 + u i,j−2) + O(∆x2 ) . (6.34)
6.2 Finite differences | 155

1 1 1

1 −4 1 −4

1 1 1

(a) (b)

1 1 1 1 4 1

1 −8 1 4 −20 4

1 1 1 1 4 1

Fig. 6.8: Several discretizations of the Laplace operator in two dimensions.

2 −8 2

1 −8 20 −8 1

2 −8 2

Fig. 6.9: The biharmonic operator in two dimensions.

156 | 6 Partial differential equations I, basics

6.2.2 Elliptic PDEs, example: Poisson equation

The standard problem of electrostatics is the computation of the electric field for a
given charge distribution and boundary conditions. Thus one has to find solutions for
the Poisson equation [2]
∆ u(x, y) = −4π ρ(x, y) (6.35)
with ρ as the charge density and u as the electrostatic potential. From the latter, the
electric field directly follows
E⃗ = −∇u .
We shall treat the problem again in two dimensions. If u is prescribed along the bound-
aries (Dirichlet condition) the problem is well-posed and a unique solution exists.

u = ub(x)
b

u=0 Δu = 0 u=0

0 u=0 a Fig. 6.10: A separable problem.

For the special case ρ = 0 and a rectangular geometry as shown in Figure 6.10, the
problem is separable for certain boundary conditions. If, for instance,

u(0, y) = u(a, y) = 0 , u(x, 0) = 0 , u(x, b) = u b (x) ,

the separation ansatz

∞
nπ nπ
u(x, y) = ∑ A n sin ( x) sinh ( y) (6.36)
n=1
a a

is successful. Note that the boundary conditions at x = 0, a and at y = 0 are already

implemented. The coefficients A n are determined to fulﬁll the boundary conditions at
y = b:
∞
nπ nπb
∑ A n sin ( x) sinh ( ) = u b (x) .
n=1
a a
Thus we ﬁnd
a
2 nπ
An = ∫ dx u b (x) sin ( x) .
a sinh(nπb/a) a
0

Depending on u b the sum in (6.36) will converge more or less quickly.

6.2 Finite differences | 157

For more involved boundary conditions or geometries, or if ρ = ρ(x, y) ≠ 0, a

separation will normally no longer work. Taking the FD scheme from Figure 6.8 (a),
the PDE (6.35) is transformed into an inhomogeneous system of equations

4u i,j − u i,j+1 − u i,j−1 − u i+1,j − u i−1,j = 4πh2 ρ i,j , (6.37)

where ∆x = ∆y = h, I = a/h, J = b/h, and ρ i,j = ρ(x i , y j ). The walls are located at the
grid lines i = 0, I, and j = 0, J. Since the boundary values are prescribed, the system
(6.37) has to be solved only in the bulk for i = 1 . . . I − 1 , j = 1 . . . J − 1, resulting
in N = (I − 1) × (J − 1) equations. If we combine the node values u ij and ρ ij in vector
notation, e.g., using the assignment

ũ k = u ij , ρ̃ k = ρ ij , k = i + (j − 1)(I − 1) ,

we can write (6.37) in the form

N
∑ L kk󸀠 ũ k󸀠 = ρ̃ k , (6.38)
k󸀠 =1

where L is an N × N sparse matrix with only ﬁve nonzero elements in each line.

6.2.2.1 Implementation of the boundary conditions

The boundary values appear in the rows or columns next to the walls as additional
inhomogeneities. Take for instance the equation j = 1 from (6.37):

4u i,1 − u i,2 − u i+1,1 − u i−1,1 = 4πh2 ρ i,1 + u i,0 . (6.39)

Here we have already written the boundary term u i,0 on the right-hand side. Thus one
may substitute
ρ 󸀠i,1 = ρ i,1 + u i,0 /(4πh2 )
in (6.37). The other boundaries are treated in the same manner.
If Neumann or mixed (Robin) conditions are posed, u ij has to be computed also
on the boundaries, increasing the number of equations and variables to (I +1)×(J +1).
To compute the derivatives at the walls an additional ‘virtual’ line outside the area is
appropriate whose node values must be computed from the boundary conditions.
To make it more clear we consider as an example a given Neumann condition

∂ y u|y=0 = f(x)

on the lower wall at y = 0. Taking the discretization (6.29) the virtual line

u i,−1 = u i,1 − 2h f i (6.40)

is established and can be added as in (6.39) to the corresponding inhomogeneity.

158 | 6 Partial differential equations I, basics

6.2.2.2 Iterative methods

The matrix L in (6.38) contains many zeros. However, their nonzero elements can be
located far from the diagonal. Also N can become very large, especially in three dimen-
sions, and a direct solution by inverting L may cause problems. Iterative methods can
(n)
help. A sequence of approximations u ij is constructed converging preferably quickly
to the desired solution. The most simple iteration rule is obtained by rearranging (6.37)
according to
(n+1) 1 (n) (n) (n) (n)
u i,j = (u i,j+1 + u i,j−1 + u i+1,j + u i−1,j ) + πh2 ρ i,j . (6.41)
4
This is the Jacobi method which, however, already converges not very quickly in two
dimensions. As a stopping condition, an upper bound for the total difference
󵄨󵄨 (n) (n−1) 󵄨󵄨
d(n) = ∑ 󵄨󵄨󵄨u ij − u ij 󵄨󵄨󵄨 < ϵ (6.42)
󵄨 󵄨
ij

can be used.
As an improvement of the Jacobi method, faster convergence is provided by the
(n+1)
Gauss–Seidel method where on the right-hand side of (6.41) values of u ij are used
if they have been already computed in the same loop:
(n+1) 1 (n) (n+1) (n) (n+1)
u i,j = (u i,j+1 + u i,j−1 + u i+1,j + u i−1,j ) + πh2 ρ i,j . (6.43)
4
But note that the scheme (6.43) breaks the symmetries x → −x, y → −y inherent in the
Laplace-Operator (and also in (6.41)) and artificial results with predominant direction
may emerge. This shortage can be compensated running the loops in (6.43) over i and
j alternately back and forwards. For a backwards loop the iteration is modified to
(n+1) 1 (n+1) (n) (n+1) (n)
u i,j = (u i,j+1 + u i,j−1 + u i+1,j + u i−1,j ) + πh2 ρ i,j .
4
Also combinations like i forwards and j backwards should occur.
Convergence can be accelerated further by so-called successive over-relaxation.
One first writes
(n+1) (n)
u i,j = u i,j + ωR i,j
with the relaxation factor ω and R ij depending on the method, e.g., taking Gauss–
Seidel
1 (n) (n+1) (n) (n+1) (n)
R i,j = (u i,j+1 + u i,j−1 + u i+1,j + u i−1,j ) + πh2 ρ i,j − u i,j .
4
If ω = 1, the Gauss–Seidel method is recovered; for ω > 1 one obtains over-relaxation.
Figure 6.11 shows the number of necessary iterations for the Poisson equation
(6.35) with Dirichlet conditions u = 0 at the boundaries and a quadratic geometry with
50 × 50 grid points. As a stopping criterion according to (6.42), we choose ϵ = 10−4 ,
and the charges are given as

ρ 20,20 = −1 , ρ 40,30 = +1 . (6.44)

6.2 Finite differences | 159

Fig. 6.11: Convergence for different ω. The Jacobi method diverges as soon as ω > 1.

Fig. 6.12: Approximate solutions of (6.35) for ϵ ≈ 10−4 , FD, 50 × 50 grid. Left: u b = 0 and charges as
(6.44), right: ρ = 0 and inhomogeneous boundary condition u b (x) = sin πx + sin 2πx + sin 3πx.

Figure 6.12 illustrates the result (left frame), together with a run for a charge free area
and an inhomogeneous Dirichlet condition (right). The simple Jacobi method shows a
rather slow convergence and becomes unstable for over-relaxation. The Gauss–Seidel
method converges much faster up to ω ≈ 1.9 but then also diverges.
Even fewer iteration steps for the same accuracy are needed for the ADI method,
an acronym for ‘Alternating Direction Implicit.’ One direction, say y, and the diagonal
terms in (6.37) are treated implicitly,
(n+1) (n+1) (n+1) (n) (n)
4u i,j − u i,j+1 − u i,j−1 = u i+1,j + u i−1,j + 4πh2 ρ i,j , (6.45)

and can be written in matrix form

(n+1) (n)
L v⃗ i = a⃗ i + Q v⃗ i (6.46)

by the assignment

v⃗ i = (u i,1 , u i,2 . . . u i,J ) , a⃗ i = 4πh2 (ρ i,1 , ρ i,2 . . . ρ i,J )

160 | 6 Partial differential equations I, basics

with the two tridiagonal matrices

0 1 0 0 ... 4 −1 0 0 ...
Q = (1 0 1 0 . . .) , L = (−1 4 −1 0 . . .) . (6.47)
0 1 0 1 ... 0 −1 4 −1 ...

With the help of the Thomas algorithm (Appendix A.3.5) L is inverted effectively,
(n+1) (n)
v⃗ i = L −1 [a⃗ i + Q v⃗ i ] , (6.48)

completing the ﬁrst half step of the iteration. In the second half, the other direction is
treated implicitly:
(n+2) (n+2) (n+2) (n+1) (n+1)
4u i,j − u i+1,j − u i−1,j = u i,j+1 + u i,j−1 + 4πh2 ρ i,j . (6.49)

Rearranging the u ij according to

w⃗ j = (u 1,j , u 2,j . . . u I,j ) , a⃗ 󸀠j = 4πh2 (ρ 1,j , ρ 2,j . . . ρ I,j ) (6.50)

(6.49) takes the same form as (6.46):

(n+2) (n+1)
L w⃗ j = a⃗ 󸀠j + Q w⃗ j .

Inversion of L as in (6.48) completes the ﬁrst step. In addition to the fact that more
elements are treated implicitly, the method has the advantage of a symmetric handling
of x and y. It can be easily extended to three dimensions adding a further step per
iteration.
A complete step is shown in the following code section:

PARAMETER (idim=..., jdim=...) ! define grid size

REAL, DIMENSION (idim,jdim) :: u,rho
REAL, DIMENSION (max(idim,jdim)) :: a,b,c,d
.....
v=4.*dx**2*Pi
a=-1.; c=-1. ! upper, lower diagonals
epsilon=.... ! accuracy wanted

DO ! iteration loop
s=0.
DO i=2,idim-1 ! 1st half step, y implicitly
DO j=2,jdim-1
d(j-1)=v*rho(i,j)+u(i+1,j)+u(i-1,j)
ENDDO
b=4.
CALL tridag(a,b,c,d,jdim-2) ! Thomas algorithm
6.2 Finite differences | 161

DO j=2,jdim-1
ua=u(i,j)
u(i,j)=d(j-1)
s=s+ABS(u(i,j)-ua)
ENDDO
ENDDO
DO j=2,jdim-1 ! 2nd half step, x implicitly
DO i=2,idim-1
d(i-1)=v*rho(i,j)+u(i,j+1)+u(i,j-1)
ENDDO
b=4.
CALL tridag(a,b,c,d,idim-2)
DO i=2,idim-1
ua=u(i,j)
u(i,j)=d(i-1)
s=s+ABS(ua-u(i,j))
ENDDO
ENDDO
IF(s<epsilon) EXIT ! stop criterion
ENDDO
...

6.2.2.3 Cyclic reduction

We conclude the section on Poisson solvers by brieﬂy explaining a direct method. Tak-
ing the vectors deﬁned in (6.50) we may write (6.37) as (we omit the prime at a)⃗

w⃗ j−1 − L w⃗ j + w⃗ j+1 = −a⃗ j , j = 1...J −1 (6.51)

with L given in (6.47). We evaluate (6.51) for three consecutive mesh points:
w⃗ j−2 − L w⃗ j−1 + w⃗ j = −a⃗ j−1
L (w⃗ j−1 − L w⃗ j + w⃗ j+1 ) = −L a⃗ j
w⃗ j − L w⃗ j+1 + w⃗ j+2 = −a⃗ j+1 .
Adding the three equations eliminates w⃗ j±1 and yields
(1)
w⃗ j−2 − L(1) w⃗ j + w⃗ j+2 = −a⃗ j , j = 2, 4, . . . J − 2 (6.52)

with
L(1) = 2 ⋅ 1 − L ⋅ L , a⃗ (1) = a⃗ j−1 + 2L a⃗ j + a⃗ j+1 .
With (6.52) we have halved the numbers of equations to solve. Now we continue ap-
plying the same steps until we arrive at the boundaries where the last equation (6.52)
reads
(k)
L(k) w⃗ (J+1)/2 = a⃗ (J+1)/2 + w⃗ 0 + w⃗ J+1 , (6.53)
162 | 6 Partial differential equations I, basics

with w⃗ 0 and w⃗ J+1 given from the boundary conditions. The method will work only if
the number of grid points in the y-direction is chosen with J = 2k − 1. In (6.53), L(k)
denotes an I × I matrix that is dense and constructed recursively according to

L(k) = 2 ⋅ 1 − L(k−1) ⋅ L(k−1) .

Once w⃗ (J+1)/2 is known, the intermediate lines

w⃗ ℓ , ℓ = (J + 1)/2 ± (J + 1)/4

can be computed, and so on until all w⃗ j are determined.

6.2.3 Parabolic PDEs, example: heat equation

As an example, we examine the heat equation (6.25), at ﬁrst in one dimension

∂T ∂2 T
=κ 2 (6.54)
∂t ∂x
that for instance describes the temporal evolution of the temperature distribution
along an (inﬁnitely) thin rod having the thermal diffusivity κ and the length L. Bound-
ary conditions are needed at its ends, e.g., Dirichlet,

T(x = 0, t) = T a , T(x = L, t) = T b ,

as well as an initial condition

T(x, t = 0) = T0 (x) .

For constant T a , T b , the ansatz

x
T(x, t) = T a + (T b − T a ) + ϑ(x, t)
L
transforms the inhomogeneous boundary value problem to a homogeneous one

∂ϑ ∂2 ϑ
=κ 2
∂t ∂x
with
ϑ(x = 0, t) = ϑ(x = L, t) = 0 .
Decomposition of ϑ into suitable trigonometric functions yields the analytic solution
in the form of an inﬁnite series
x ∞
T(x, t) = T a + (T b − T a ) + ∑ g n (t) sin k n x (6.55)
L n=1

where
g n (t) = A n e−k n κt ,
2
k n = nπ/L .
6.2 Finite differences | 163

Finally A n is determined from the initial condition T = T0 :

L
2 2
A n = ∫ dx T0 (x) sin k n x − (T a − (−1)n T b ) .
L nπ
0

For t → ∞ we ﬁnd g n → 0 and the temperature asymptotically approaches the sta-

tionary solution
x
Ts (x) = T a + (T b − T a ) .
L

6.2.3.1 Iterative method

Instead of iterating a stationary, normally elliptic, problem, the iteration is now iden-
tical to marching forward in time. Thus, time has to be discretized too which is again
achieved by approximating the time derivative with the corresponding differential
quotient. At the lowest order of the time step ∆t we may substitute
∂T T(x, t + ∆t) − T(x, t)
= . (6.56)
∂t ∆t
From here on we use the abbreviations for space-time dependent ﬁeld variables
(n) (n)
Φi ≡ Φ(i∆x, n∆t) , Φ ij ≡ Φ(i∆x, j∆y, n∆t) (6.57)

in one and in two dimensions, respectively. Starting from the initial condition
(0)
Ti = T0 (x i )

the iteration scheme gained by discretization of (6.54)

(j+1) (j) ∆t (j) (j) (j)

Ti = Ti + κ (T i+1 − 2T i + T i−1 ) (6.58)
∆x2
(n) (j+1)
allows for the successive computation of all node values T i for n > 1. Having T i
already isolated on the left-hand side, (6.58) is called an explicit scheme in time, some-
times also FTCS scheme, as an acronym for ‘Forward Time Centered Space’. The corre-
sponding implicit (backward) formula is obtained by considering the right-hand side
of (6.58) at time (j + 1)∆t:

(j+1) (j) ∆t (j+1) (j+1) (j+1)

Ti = Ti + κ (T i+1 − 2T i + T i−1 ) . (6.59)
∆x2
Here we have to solve a linear system of equations at each time step, which seems to be,
compared to (6.58), needlessly complicated. However, implicit (or partially implicit)
methods are very common in practice and have large advantages due to their much
better numerical stability properties. We shall come back to this point in more detail
in Section 6.2.3.3.
164 | 6 Partial differential equations I, basics

6.2.3.2 Example: temperature in a room with a stove and a window

As an application we wish to compute the temperature field in a room with a length
and height of 3 meters. In the room, a stove with a given temperature TH is placed
below the window. Walls, floor, and ceiling are assumed to be perfect thermal insu-
lators where homogeneous Neumann boundary conditions ∂ n T = 0 apply. Along the
open window the temperature is fixed to the outdoor temperature TE (Dirichlet); see
Figure 6.13.
Except for the mesh points coinciding with the window and the heating, the sys-
tem (6.58) is iterated, but now in two dimensions:

(n+1) (n) ∆t (n) (n) (n) (n) (n)

T i,j = T i,j + κ (T i,j+1 + T i+1,j − 4T i,j + T i−1,j + T i,j−1 ) . (6.60)
∆x2
To complete the computations for the boundary nodes, virtual points outside the do-
main are again needed that can be determined from the Neumann conditions. One
ﬁnds for instance along the right-hand wall T I+1,j = T I−1,j , and so on. The nodes along
Dirichlet boundaries (at the stove and the window) are just ﬁxed at the appropriate
values.

T
=0
y

open window

T=TE
T T
=0 =0
x x

heating

T=TH

T
=0
y

Fig. 6.13: Sketch of the system: room with window and heating. FD-grid, dashed: virtual points.
Along window and heating we take Dirichlet conditions; along walls, Neumann conditions apply.
6.2 Finite differences | 165

Fig. 6.14: Time series, temperature distribution after 0.1, 1, 20 days. Stove and window marked in
green, TE = 0°C, TH = 70°C. The extremely long relaxation time is a consequence of the total disre-
gard of any convection.

Figure 6.14 shows a time series computed on a 70 × 70 grid. The step sizes
∆x = 3/69 , ∆t = 0.1∆x2 /κ
have been used where
κ = 2 ⋅ 10−5 m2 /s
denotes the thermal diffusivity of air. As an initial condition, the temperature inside
the room was ﬁxed to the outdoor value TE .
Once the temperature distribution is known the heat current densities accord-
ing to
̂
j = −λ n∇T
can be computed (λ = thermal conductivity of air³) and integrated along the window
and the surface of the stove. The temporal evolution is depicted in Figure 6.15. For
the stationary (equilibrium) case, both currents should be equal. The discrepancy of
about 10 percent seems to have numerical reasons (resolution too low).

heat ﬂux from heating

heat ﬂux through window

Fig. 6.15: Heat current densities integrated over
window, and heating in arbitrary units as a
function of time.

3 Thermal diffusivity and thermal conductivity are different material properties but linked by the re-
lation κ = λ/ρc with density ρ and speciﬁc heat c.
166 | 6 Partial differential equations I, basics

6.2.3.3 Von Neumann stability analysis

In the iteration rules (6.58–6.60) the grid-dependent numerical parameter
∆t
s= κ
∆x2
occurs on which the stability of the method depends. To shed light on this, one de-
composes Tℓ into the eigenmodes of a linear chain of springs
(n)
Tℓ = A(n) eikℓ . (6.61)

Substitution into (6.58) and canceling eikℓ yields

A(n+1) = A(n) [1 + 2s(cos k − 1)] .

For a stable method the solution should asymptotically approach some stationary
value, which is only possible if
󵄨󵄨 (n+1) 󵄨󵄨 󵄨󵄨 (n) 󵄨󵄨
󵄨󵄨A 󵄨󵄨 ≤ 󵄨󵄨A 󵄨󵄨 .
󵄨 󵄨 󵄨 󵄨
As a consequence, the absolute value of the so-called ampliﬁcation factor

V k = 1 + 2s(cos k − 1) (6.62)

must be less than or equal to one for arbitrary k. This is the stability criterion of von
Neumann⁴. From (6.62) one finds V k ≤ 1 for all k and the stability limit is given as
V k ≥ −1, leading to
1
s≤ .
1 − cos k
Since this constitutes an upper bound for ∆t we must find the specific k that minimizes
s, giving
k = π , s = 1/2
and therefore the condition
1 ∆x2
∆t ≤ . (6.63)
κ 2
If ∆t is chosen as larger, the grid becomes numerically unstable first with the ‘most
dangerous mode’
e iπℓ ,
corresponding to a change of signs from one grid point to the next (Figure 6.16). The
relation (6.63) is typical for an explicit method. In general one has

∆t ∼ (∆x)n

if the time derivative is ﬁrst order and the highest occurring space derivative is of or-
der n.

4 John von Neumann, American mathematician, 1903–1957.

6.2 Finite differences | 167

Fig. 6.16: Typical numerical instability for

1 2 3 4 5 6 7 8 an explicit FD scheme, corresponding to the
‘most dangerous mode’ k = π in (6.61).

If the mesh has N grid points the number of equations is N and the number of time
iterations K ∼ 1/∆t ∼ 1/(∆x)n = N n . Then the total numerical effort turns out to be

NE ∼ N n+1 ,

which may restrict the application of fully explicit methods, in particular if higher
derivatives are around.
The same stability consideration can be done for the implicit scheme (6.59) for
which one obtains
1
Vk = ,
1 − 2s(cos k − 1)
an ampliﬁcation factor whose absolute value is less than or equal to one for all k.
Thus, the implicit method is unconditionally stable, and restrictions for the time step
are only due to accuracy constraints. The numerical effort is then independent on the
order of the appearing derivatives

NE ∼ N .

6.2.4 Hyperbolic PDEs, example: convection equation, wave equation

6.2.4.1 Convection equation

We begin with the most simple hyperbolic PDE in (1+1) dimensions, the convection
equation (6.6). For v > 0 the family of characteristics has a positive slope in the tx-
plane, Figure 6.1. Open Cauchy initial conditions of the form

u(x, 0) = u 0 (x) , u(0, t) = u a (t)

allow for a unique solution in 0 ≤ x ≤ L, t ≥ 0. An explicit FTCS scheme takes the

form
(n+1) (n) 1 (n) (n)
ui = u i − C (u i+1 − u i−1 ) , (6.64)
2
where we have introduced the Courant number
v∆t
C= . (6.65)
∆x
However, a von Neumann analysis shows the numerical instability already for arbi-
trarily small time steps C, ∆t > 0.
168 | 6 Partial differential equations I, basics

To construct a stable algorithm one can change the representation of the deriva-
tives, the differential quotients. The most simple way is the so-called upwind scheme.
Here, a space derivative is taken asymmetric in the ‘upstream’ direction. Instead of
(6.64) one obtains in the lowest order
(n) (n)
(n+1) (n) {u i − u i−1 , C, v > 0
ui = ui − C{ (6.66)
(n) (n)
{u i+1 − u i , C, v < 0

which can be extended to the more general case v = v(x) by substituting v with v i .
With a constant v the ampliﬁcation factor reads

|V|2 = 1 − 2|C|(1 − |C|)(1 − cos k)

and with |V|2 ≤ 1 one ﬁnds the Courant condition

|C| ≤ 1 . (6.67)
(n)
Substituting u i on the left-hand side of (6.64) by its average

(n+1) 1 (n) (n) 1 (n) (n)

ui = (u + u i−1 ) − C (u i+1 − u i−1 ) , (6.68)
2 i+1 2
leads to the Lax scheme, which also proves to be stable if the Courant condition (6.67)
holds.
For both upwind and Lax methods one pays for stability with artiﬁcial numerical
friction. Comparing (6.66) or (6.68) with the FTCS schemes of the parabolic transport
equation
∂ t u + v ∂ x u = D ∂2xx u ,

one ﬁnds for the upwind scheme

D = |v|∆x
and for the Lax scheme
∆x2
D=
.
2∆t
A second order scheme with respect to the time step is obtained when discretizing
the time derivative by
∂u(t) 󵄨󵄨󵄨󵄨
(n+1) (n−1)
u − ui
󵄨󵄨 = i (6.69)
∂t 󵄨󵄨i 2∆t
and results in
(n+1) (n−1) (n) (n)
ui = ui − C (u i+1 − u i−1 ) . (6.70)

The second order time discretization (6.69) applied on the diffusion equation leads to
an unstable scheme. For the convection equation on the other hand, it yields a much
6.2 Finite differences | 169

1 1 1

0.8 0.8 0.8

0.6 0.6 0.6

t
0.4 0.4 0.4

0.2 0.2 0.2

0 0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
x x x

Fig. 6.17: Numerical solutions of the convection equation with contour lines in the xt-plane. Left: up-
wind with ∆x = 1/1000, middle: upwind with ∆x = 1/100, right: leapfrog with ∆x = 1/1000.
Maximal Courant number C = 0.1. The leapfrog method shows no dissipation.

better stability behavior as the one-step schemes discussed before. Hence for the am-
pliﬁcation factor one ﬁnds

V = −iC sin k ± √1 − C2 sin2 k ,

which for all |C| ≤ 1 has an absolute value equal to one. At least in the case of the
convection equation, the method is free of any numerical dissipation.
Since time integration ‘jumps’ over the spacial grid line at each time step, the
method is also called the leapfrog scheme. Even and odd grid points (black and white
squares on a checkerboard) are decoupled and may drift apart in the course of the
iterations. This can be avoided by introducing a weak additional diffusion.
Figure 6.17 shows three numerical solutions of the convection equation with the
same initial condition

u 0 = e(x−0.1) /γ 2
2
, γ = 0.2 , ua = 0

and the same time-depending v(t) = 2 cos(πt). For the upwind scheme, one clearly
recognizes numerical dissipation ∼ ∆x that vanishes using leapfrog (right frame).

6.2.4.2 Wave equation

Wave equations are encountered in almost all ﬁelds of physics. We shall again start
with the most simple (1+1)-dimensional case (6.26). For constant Dirichlet boundary
conditions
u(0, t) = u a , u(L, t) = u b

an analytical solution can be written up as in (6.55), but now with

ncπ
g n (t) = A n sin ω n t + B n cos ω n t , ω n = ck n = .
L
170 | 6 Partial differential equations I, basics

Thus one has to determine twice as many coefficients as for the diffusion equation and
needs accordingly twice as many initial conditions. For physical problems one often
prescribes
u(x, 0) = u 0 (x) , ∂ t u(x, 0) = v0 (x) (6.71)

and ﬁnds after some short manipulation

L L
2 2
Bn = ∫ dx u 0 (x) sin k n x , An = ∫ dx v0 (x) sin k n x .
L ωn L
0 0

Discretizing the second time derivative with the help of (6.30) yields a leapfrog scheme
of the form
(n+1) (n) (n−1) ∆t2 (n) (n) (n)
ui = 2u i − u i + c2 2 (u i+1 − 2u i + u i−1 ) . (6.72)
∆x
(n+1)
As already seen in (6.70), this is a two-step method. To compute u i we must know
(n) (n−1)
u i and additionally u i . Let the initial conditions be given in the form (6.71). Then
(0) (−1)
we ﬁnd u i and u i from
(0) (−1) (0)
ui = u 0 (x i ) , ui = u i − v0 (x i )∆t .

All further values for n > 0 then follow by iterating (6.72).

Is the explicit forward scheme (6.72) numerically stable? Because this is a two-step
method, von Neumann’s stability analysis is a bit more involved. With the well-tried
mode ansatz
(n)
u j = A(n) eikj
follows from (6.72)

A(n+1) = 2A(n) − B(n) + 2q2 A(n) (cos k − 1)

(6.73)
B(n+1) = A(n)

with q = c∆t/∆x and the auxiliary variable B(n) . We write (6.73) in matrix form
(n+1) (n)
A A
( ) = M( )
B B

with
2 + 2q2 (cos k − 1) −1
M=( ) .
1 0
To achieve a stable algorithm, the absolute values of the two eigenvalues λ12 of M have
to be less than or equal to one. Due to

λ1 λ2 = Det M = 1
6.2 Finite differences | 171

this is only possible if λ1 = λ2 = 1 or the two λ i form a complex conjugated pair having
an absolute value of one. From this we derive
2
q2 ≤ .
1 − cos k
An upper limit for q and hence for the time step is provided by k = π (compare Fig-
ure 6.16) and results in q2 ≤ 1 or

∆t ≤ ∆x/c .

If we substitute the phase velocity c by the transport velocity v this is again the Courant
condition (6.67).
As an application, we wish to study a problem from ﬂuid mechanics. Based on the
Euler equations for an inviscid ﬂuid, one systematically derives the (two-dimensional)
shallow water equations:

∂ t h = −(h − f) ⋅ ∆Φ − (∇(h − f)) ⋅ (∇Φ)

1 (6.74)
∂ t Φ = −g ⋅ (h − h0 ) − (∇Φ)2 .
2
Φ and h depend on x, y and t, g = 9.81 m/s2 is the gravity acceleration, while
∆ = ∂2xx + ∂2yy and ∇ = (∂ x , ∂ y ) stand for the two-dimension Laplace and gradient op-
erator, respectively. The water is enclosed in the vertical direction by the two planes
z = f(x, y) (ground) and z = h(x, y, t) (surface; see Figure 6.18), and its horizontal
velocity at the surface is determined by the potential Φ as

v⃗H = ∇Φ .

Equations (6.74) are valid only for surface structures (waves) that are slowly varying
on the scale of the average water depth h0 . For a derivation of the shallow water equa-

Fig. 6.18: Sketch of the shallow water system. The water is conﬁned by the stationary ground at
z = f and the deformable surface at z = h.
172 | 6 Partial differential equations I, basics

tions we refer to ﬂuid mechanics textbooks, e.g., [3]. Like the underlying basic hydro-
dynamic equations, the shallow water equations are nonlinear. However, for small
wave amplitudes η
h(x, y, t) = h0 + η(x, y, t)

we may linearize (6.74) according to

∂ t η = −(h0 − f)∆Φ + (∇f ) ⋅ (∇Φ)

(6.75)
∂ t Φ = −gη .

Differentiating the ﬁrst equation with respect to time and inserting the second one
yields
∂2tt η − g ⋅ (h0 − f)∆η + g ⋅ (∇f ) ⋅ (∇η) = 0 .
If the slope of the ground is also small we can ﬁnally neglect the last term and arrive
at the (2+1)-dimensional wave equation

∂2tt η − c2 ∆η = 0 (6.76)

with a space dependent phase speed

c(x, y) = √g ⋅ (h0 − f(x, y)) .

Thus waves are propagating slower in shallow water, in agreement with experience.
What happens if a wave travels from deep water into more shallow regions, for in-
stance while approaching the shore? To ﬁnd out we solve (6.76) numerically applying
the scheme (6.72). For the ground we use a parabola in x direction:

f(x) = 0.9 x2

on the unit square. One can choose the scaling of space and time to obtain h0 = g = 1.
Neumann boundary conditions at x = 0, 1 and y = 0, 1 allow for unique solutions.
To create the waves we assume an oscillating δ-shaped source term on the right-hand
side of (6.76)
∼ δ(x − x0 )δ(y − y0 ) cos ωt ,
which can be realized numerically by setting a certain single grid point, e.g., in the
middle of the layer x0 = y0 = 1/2 to

u I/2,J/2 = cos ωt .

Figure 6.19 shows three snapshots at consecutive times as well as the correspond-
ing surface proﬁles along the line y = 1/2. If the waves reach the shore their speed
and their length decrease but their amplitude increases. The grid consists of I × J =
200 × 200 mesh points; the code can be found in [4].
6.3 Alternative discretization methods | 173

t = 0.2 t = 0.5 t = 3.0

Fig. 6.19: Wave propagation caused by an oscillating point source in the middle of the integration
domain. The water depth decreases with increasing x. Bottom row: cross section along the dashed
lines.

6.3 Alternative discretization methods

FD schemes can be developed and coded in a straightforward way with rather less
effort while proving reliable and robust for various applications. On the other hand,
more sophisticated methods have been developed to approximate differential equa-
tions with a system of algebraic equations that have higher accuracy and better con-
vergence. Although the effort in code writing can be higher at ﬁrst, this normally pays
off with the lower CPU consumption and faster performance.

6.3.1 Chebyshev spectral method

The method is again explained with one spacial dimension and can be generalized
afterwards. We follow mainly the ﬁrst parts of the recommended book by Guo et al. [5].
As already seen in Section 5.3, the idea behind spectral methods is to decompose the
desired solution u(x) into a (ﬁnite) sum of certain given functions φ i (x), called base
functions, Galerkin functions, or ansatz functions:
N
u(x) ≈ u (N) (x) = ∑ a i φ i (x) . (6.77)
i=0
174 | 6 Partial differential equations I, basics

Even the FD method ﬁts in this representation, taking the special base functions

{1 if x = x i
φ i (x) = { (6.78)
0 else
{
with the N grid points x i . However, here the notion FD has become more accepted.

6.3.1.1 Lagrange polynomials

For the Chebyshev spectral method one takes for the base the Lagrange polynomials
of Nth degree:
(N)
N
x − xj
φ i (x) = L i (x) = ∏ . (6.79)
x
j=0,j=i̸ i
− xj

As one easily may recognize, the Lagrange polynomials have the same property (6.78)
as the ‘FD base’, namely
(N)
L i (x j ) = δ ij (6.80)
with the Kronecker symbol δ ij . As for the FD methods x is only deﬁned at certain dis-
crete points x j , called mesh points or collocation points. For the Chebyshev–Gauss–
Lobatto method the mesh points are generated via
jπ
x j = − cos ( ) , j = 0...N (6.81)
N

and span the interval [−1, 1]. Note that the mesh points are no longer equidistant but
accumulate towards the borders, which improves the convergence and accuracy of the
method.
Figure 6.20 shows the four Lagrange polynomials for N = 3. All polynomials are
of third degree, the mesh points from (6.81) are computed as

x0 = −1 , x1 = −1/2 , x2 = 1/2 , x3 = 1 .

1
L0(3) L(3)
1 L2(3) L3(3)
0.8

0.6

0.4

0.2
Fig. 6.20: The four Lagrange polynomials for
–1 –0.5 0 0.5 x 1 N = 3. The four mesh points are located at
x0 = −1, x1 = −1/2, x2 = 1/2, x3 = 1 (dashed).
6.3 Alternative discretization methods | 175

In the following we write the numerical approximation (6.77) in the decomposition

N
(N)
u (N) (x) = ∑ u i L i (x) . (6.82)
i=0

Here, the N + 1-dimensional vector u⃗ represents the function u (N) (x) in the base of the
(N)
N + 1 Lagrange polynomials L i . Due to (6.80) the inverse relation

u i = u (N) (x i )

is immediately written down.

6.3.1.2 Differentiation matrix

Differentiating (6.82) with respect to x yields
(N)
du (N) (x) N dL i (x)
= ∑ ui . (6.83)
dx i=0
dx

Deﬁning
󵄨
(1) du (N) (x) 󵄨󵄨󵄨
ui ≡ 󵄨󵄨
dx 󵄨󵄨󵄨x=x i
we may write (6.83) in the form
u⃗ (1) = D u⃗ . (6.84)
The differentiation matrix D has the dimension (N + 1) × (N + 1). Its elements
󵄨
dL j (x) 󵄨󵄨󵄨󵄨
(N)
D ij = 󵄨󵄨
dx 󵄨󵄨󵄨󵄨
󵄨x=x i
can be computed explicitly as:

c i (−1)i+j {2 if k = 0, N
D ij = for i ≠ j = 0 . . . N with c k = {
cj xi − xj 1 else
{
xi (6.85)
D ii = − for i = 1...N −1
2(1 − x2i )
2N 2 + 1
D00 = −D NN = − .
6
For the case N = 3 one ﬁnds

−19/6 4 −4/3 1/2

−1 1/3 1 −1/3
D=( ) . (6.86)
1/3 −1 −1/3 1
−1/2 4/3 −4 19/6
176 | 6 Partial differential equations I, basics

The second derivative with respect to x is obtained by applying D twice on u:⃗

u⃗ (2) = D D u⃗ = D 2 u⃗ (6.87)

with
16 −28 20 −8
1 10 −16 8 −2
D2 = ( ) . (6.88)
3 −2 8 −16 10
−8 20 −28 16

6.3.1.3 Differential equations

We start with the stationary heat equation in one dimension including an internal heat
source Q:
d2 T
= −Q(x) . (6.89)
dx2
In addition we assume the Dirichlet conditions

T(−1) = Tℓ , T(1) = T r . (6.90)

To incorporate the boundary conditions the differentiation matrix D 2 has to be modi-

ﬁed. Again we consider the case N = 3. Instead of (6.88) we now use

1 0 0 0
̃2 10/3 −16/3 8/3 −2/3
D =( ) (6.91)
−2/3 8/3 −16/3 10/3
0 0 0 1

and obtain for the discretized form of (6.89)

1 0 0 0 T0 Tℓ
10/3 −16/3 8/3 −2/3 T1 −Q1
( )( ) = ( ) (6.92)
−2/3 8/3 −16/3 10/3 T2 −Q2
0 0 0 1 T3 Tr

with Q i = Q(x i ). The numerical task consists now in inverting D̃ and the determina-
2

tion of the solution T i . If, on the other hand, Neumann conditions are given on one or
on both boundaries, D 2 has to be changed accordingly. Take, for instance,

d x T|x=−1 = S , T(1) = T r ,

so the first line of D 2 has to be substituted by the first line of D (first derivative) and
instead of (6.92), we obtain the system

−19/6 4 −4/3 1/2 T0 S

10/3 −16/3 8/3 −2/3 T1 −Q1
( )( ) = ( ) . (6.93)
−2/3 8/3 −16/3 10/3 T2 −Q2
0 0 0 1 T3 Tr
6.3 Alternative discretization methods | 177

0.02

0.01

–0.01
Fig. 6.21: Temperature distribution (bold) cre-
ated by randomly chosen heat sources as a
–0.02 solution of (6.92) with N = 100 Chebyshev–
Gauss–Lobatto mesh points. The thin line
–1 –0.5 0 0.5 1 shows Q(x).

Figure 6.21 shows a solution of (6.92) but for N = 100 and the Dirichlet conditions,

T(−1) = T(1) = 0 .

The heat source was chosen randomly, but obeys the constraint
1

∫ dx Q(x) = 0 .
−1

Note that in the discretized form of the integral the variable step sizes (6.81) must be
accounted for, thus
1 N
∫ dx Q(x) ≈ ∑ Q i ∆x i , with ∆x i = x i − x i−1 .
−1 i=1

Next we consider the time-dependent heat equation (6.54) with the boundary con-
ditions (6.90) and the initial condition

T(x, 0) = T a (x) .

Discretizing the time derivative as in (6.56) yields the iteration rule

T⃗ (j+1) = M T⃗ (j) (6.94)

with
T⃗ (j) = (T0 (j∆t), T1 (j∆t), . . . T N (j∆t))
and the matrix
M = 1 + κ∆t D̃
2
178 | 6 Partial differential equations I, basics

with the modiﬁed matrix D̃ that takes for the special case N = 3 the form
2

0 0 0 0
̃2 10/3 −16/3 8/3 −2/3
D =( ) . (6.95)
−2/3 8/3 −16/3 10/3
0 0 0 0

The iteration is started with the initial vector

T⃗ (0) = (Tℓ , T1a , T2a , T r )

where initial and boundary conditions are incorporated.

The scheme (6.94) is fully explicit. With only a bit more effort an implicit method
can be constructed
1 T⃗ (j)
( − κ D̃ ) T⃗ (j+1) =
2
, (6.96)
∆t ∆t
where now the matrix between the brackets must be inverted.

6.3.2 Spectral method by Fourier transformation

6.3.2.1 Fourier modes

For periodic boundary condition

u(x) = u(x + L)

a natural base is provided by Fourier modes. Instead of (6.77) we write

N−1
u(x) = ∑ ũ n e−2πinx/L (6.97)
n=0

where i = √−1. The function u at the now equidistant mesh points x j = j∆x = jL/N is
given as
N−1
u j = ∑ ũ n e−2πinj/N (6.98)
n=0

and the inverse relation reads

1 N−1
ũ n = ∑ u j e2πinj/N . (6.99)
N j=0

The method is best suited for linear PDEs with constant coefficients but can also be
extended to certain nonlinear problems, as we shall see below.
6.3 Alternative discretization methods | 179

6.3.2.2 Stationary problems

Again we consider (6.89) for 0 ≤ x ≤ L. Inserting (6.97) yields

T̃ n = k −2 ̃
n Qn (6.100)

with the Fourier transforms T̃ n , Q̃ n of T(x), Q(x) and

k n = 2πn/L .

To avoid a singularity at k = 0 one must require Q̃ 0 = 0. Due to

1 N−1
Q̃ 0 = ∑ Qj ,
N j=0

the mean of Q must be zero, which also follows by integrating (6.89) over x and sup-
posing periodic boundary conditions. Obviously the Laplace operator is in diagonal
form in Fourier space for arbitrary spatial dimensions and can be inverted easily. Back-
transformation via (6.97) yields T(x) in real space. The Fourier transforms can be exe-
cuted effectively applying the FFT-algorithm (Fast-Fourier-Transform), the numerical
effort thereby is ∼ N log N.
Taking the FD-representation of (6.89)

T j+1 − 2T j + T j−1 = −∆x2 Q j

and inserting the Fourier transform (6.98), a different result from (6.100) is obtained,
namely
∆x2
T̃ n = Q̃ n . (6.101)
2 − 2 cos(k n ∆x)

0.00020

0.00015

0.00010

(1)
0.00005

(2) Fig. 6.22: The difference between

0 (1): 1/(2N 2 (1 − cos(2πn/N)) and (2): 1/(4π 2 n2 )
0 10 20 30 40 50 for N = 100 becomes noticeable for larger n
n (smaller wave length).
180 | 6 Partial differential equations I, basics

Expanding the fraction with respect to ∆x,

∆x2 1 ∆x2
≈ 2 + + O(∆x4 ) ,
2 − 2 cos(k n ∆x) k n 12

we see that the difference vanishes with ∆x2 but can become important for larger val-
ues of k n , as in Figure 6.22.

6.3.2.3 Other boundary conditions

The procedure can be readily extended to homogeneous Dirichlet conditions

u(0) = u(L) = 0 .

Instead of (6.97) we simply take the sine transform

N−1
u s (x) = ∑ ũ n sin(πnx/L) . (6.102)
n=1

Inhomogeneous boundary conditions

u(0) = u ℓ , u(L) = u r (6.103)

can be also fulﬁlled with (6.102) by adding the homogeneous solution of (6.89)

d2 u h
=0, (6.104)
dx2
leading to
u(x) = u s (x) + u h (x) ,
where u h must satisfy the inhomogeneous boundary conditions (6.103). In one dimen-
sion the solution of (6.104) is
x
u h (x) = u ℓ + (u r − u ℓ ) ,
L
with possible extension to more dimensions. Thus, for the rectangle

0 ≤ x ≤ Lx , 0 ≤ y ≤ Ly

we have instead of (6.102)

u s (x, y) = ∑ ũ nm sin(πnx/L x ) sin(πmy/L y ) , (6.105)

n,m

and the homogeneous problem reads

∂2 ∂2
( 2
+ 2 ) u h (x, y) = 0 . (6.106)
∂x ∂y
6.3 Alternative discretization methods | 181

If one wants to fulﬁll the inhomogeneous boundary condition

u(0, y) = u(x, 0) = u(x, L y ) = 0 , u(L x , y) = f(y) ,

the exact solution of (6.106) reads

∞
u h (x, y) = ∑ A n sinh(k n x) sin(k n y) , k n = πn/L y
n=1

with
Ly
2
An = ∫ dy f(y) sin(πny/L y ) .
L y sinh(πnL x /L y )
0
For the discretized form we obtain
M−1
u hij = ∑ Ã n sinh(πni/M) sin(πnj/M)
n=1

with
M−1
2
Ã n = ∑ fℓ sin(πnℓ/M)
M sinh(πnN/M) ℓ=1
and fℓ = f(ℓ∆y). Here we have assumed equal step sizes

∆x = ∆y = L x /N = L y /M

for an N × M grid.

6.3.2.4 Time-dependent problems

We continue with the one-dimensional heat equation (6.54). An implicit iteration al-
gorithm is obtained taking the backward differentiation with respect to time
∂T T(x, t) − T(x, t − ∆t) T (j) (x) − T (j−1) (x)
= = (6.107)
∂t ∆t ∆t
which leads to
1 ∂2 T (j)
[ − κ 2 ] T (j+1) = . (6.108)
∆t ∂x ∆t
FD-discretization of the second derivative yields the scheme (6.59). Applying the
Fourier transform (6.97) the operator in the square brackets takes diagonal form and
can be immediately inverted:
−1 ̃ (j)
(j+1) 1 Tn
T̃ n = [ + κk 2n ] . (6.109)
∆t ∆t
If inhomogeneous Dirichlet conditions as (6.103) are given, the sine transform (6.102)
can be used again. In addition, the homogeneous problem to (6.108)
1 ∂2
[ − κ 2 ] T h (x) = 0 (6.110)
∆t ∂x
182 | 6 Partial differential equations I, basics

has to be solved, which can be done exactly in one dimension. One ﬁnds

T h (x) = A sinh(αx) + B cosh(αx)

with α = (κ∆t)−1/2 and A, B from the boundary conditions (6.103)

Tr
A= − Tℓ coth(αL) , B = Tℓ .
sinh(αL)

6.3.3 Finite-element method

The finite-element method (FE) was developed to solve PDEs, mainly from fluid me-
chanics, on complex geometries. It belongs to the spectral methods with base func-
tions that are only nonzero in certain places, namely at the finite elements. Inside
each element a linear space dependence or a simple polynomial is assumed [6].
We shall introduce the method during the example of the two-dimensional
Laplace equation
∂2 u ∂2 u
+ =0 (6.111)
∂x2 ∂y2
with Dirichlet boundary conditions.

6.3.3.1 Grid and base functions

We are given an irregular, possibly multiply connected domain as in Figure 6.23. We
choose n inner mesh points or nodes

(x i , y i ) , i = 1...n

and m (outer) points lying on the boundaries. The domain is divided in triangles hav-
ing the inner and outer points as corners (vertices). In higher (d) dimensions, cells
with d + 1 corners are deﬁned. The approximate solution sought after can be written
similarly to (6.77) according to:
n
u(x, y) = ∑ φ i (x, y) , (6.112)
i

where φ i is nonzero only in those cells containing the node x i , y i as a vertex. There,
one adjusts
φ i (x i , y i ) = u i = u(x i , y i )
with u i as the function value (node value) at the node i. Let the points (x i , y i ), (x k , y k ),
(xℓ , yℓ ) span triangle number j, Figure 6.24. We split the function φ i into

φi = ∑ αj , (6.113)
j ϵ neighbors(i)
6.3 Alternative discretization methods | 183

Fig. 6.23: Irregular geometry with n = 1894 inner and m = 242 outer points. The domain, here
multiply connected, is paved with ﬁnite elements that are triangles for the two-dimensional case.

uℓ

ui
αj

Fig. 6.24: u i , u k , u ℓ span the element j inside

which the base functions α j are linear and given
as (6.114). φ i is composed of all (here ﬁve) α j
uk and is continuous along the grid lines.

where α j is nonzero only in the triangle j and there linear, thus

α j (x, y) = a j + b j x + c j y . (6.114)

The sum over j in (6.113) runs over all triangles having x i , y i as vertex and shown in
Figure 6.24. The coefficients a j , b j , c j , different in each triangle, follow (6.112) to form
184 | 6 Partial differential equations I, basics

three equations:
1 xi yi aj ui
(1 xk y k ) (b j ) = (u k ) (6.115)
1 xℓ yℓ cj uℓ
and, after an inversion of the matrix:

aj x k yℓ − xℓ y k xℓ y i − x i yℓ xi yk − xk yi ui
1
(b j ) = ( y k − yℓ yℓ − y i y i − y k ) (u k ) . (6.116)
D
cj xℓ − x k x i − xℓ xk − xi uℓ

Here, D is the determinant:

D = x i y k + x k yℓ + xℓ y i − x k y i − xℓ y k − x i yℓ .

The coefficients a j , b j , c j and hence the element functions α j depend linearly on the
node values u i .

6.3.3.2 Variational problem

Equation (6.111) can be derived from an extremum principle. Taking the functional
1
S[u] = ∫ dx dy [(∂ x u)2 + (∂ y y)2 ] (6.117)
2
one ﬁnds for its functional derivative
δS ∂2 u ∂2 u
=− 2 − 2 .
δu ∂x ∂y
Hence, a function u for which S has an extremum is a solution of the Laplace equation
(6.111). But if u is decomposed as (6.112, 6.113) in the element functions α j we ﬁnd for
the triangle j due to (6.114) simply

(∂ x u)j = b j , (∂ y u)j = c j .

Then (6.117) turns into

1
S(u i ) = ∑ F j (b 2j + c2j ) , (6.118)
2 j

where the sum runs over all triangles and F j denotes the surface area of triangle j.
The extremum of S is found by equating all its derivatives with respect to u i to zero:

∂S ∂b j ∂c j
= ∑ F j (b j + cj )=0. (6.119)
∂u i j
∂u i ∂u i

Since b j and c j depend linearly on u i , (6.119) has the form of a linear, inhomogeneous
system of equations for the n inner node values u i . The m boundary points thereby
play the role of the inhomogeneities.
6.3 Alternative discretization methods | 185

6.3.3.3 Algorithm
The tasks of an FE-algorithm can be divided into three parts:

(a) Mesh generation. The location of the nodes is chosen as appropriate to the domain
and the geometry of the boundaries. The node coordinates

xi , yi , i = 1...n +m

are stored and the boundary points are marked, e.g., by sorting as the ﬁrst or the last
m pairs. To evaluate (6.116) later, for each point i one must know the amount as well
as the ordinal number of its neighbor points (the points with a common triangle). This
can be achieved introducing an array NN(i, j), where NN(i, 0) denotes the amount k
of neighbors and NN(i, j), j = 1 . . . k the corresponding ordinal numbers.

(b) Laplace solver. The linear system (6.119) has the form
n m
∑ L ij u j = ∑ R ik u rk , i = 1...N . (6.120)
j k

Here, L is an n × n, R an n × m matrix, u j denotes the inner points, and u rk the given

outer boundary points (we restrict ourselves here to Dirichlet conditions). Since each
point is connected only to a few surrounding points the matrix L is sparse.

(c) Evaluation. To visualize the results one can plot, for instance, contour lines. Let
u i , u j , u k be the node values at the corners of a certain triangle. The contour line u = h
crosses the triangle if

min(u i , u j , u k ) < h < max(u i , u j , u k ) . (6.121)

For a triangle where (6.121) is fulfilled one computes the intersections of h with its
edges and connects them with a line. Normally, two intersection points on two differ-
ent edges exist. The cases where h is exactly equal to a value on one or more corners
can be treated separately.
Figure 6.25 shows the solution of the two-dimensional Laplace equation on the
domain of Figure 6.23, built of n = 1894 inner and m = 242 outer points.
To facilitate mesh generation, an unambitious ‘semiautomatic’ program
mesh_generator is presented in [4]. A ppm-file (portable pixmap) containing the ge-
ometry of the domain in the form of the boundaries is needed, created with a standard
graphics program, e.g., xfig. Then the Laplace equation is solved on the prescribed
domain by the program laplace_solver. An output file is generated containing the
node values, which can be plotted by grid_contour. A plot as shown in Figure 6.25
is the result. More detailed instructions can be also found in [4] as well as in Ap-
pendix D.2.
186 | 6 Partial differential equations I, basics

ur = −1

ur = 1

ur = −1

1
=
ur

ur = 0

Fig. 6.25: Numerical solution for the Laplace equation with the annotated Dirichlet conditions
and three point charges in the interior. The contour lines correspond to the potential values
u = 0.001/0.05/0.2/0.4/0.6/0.9 (solid) und u = −0.001/ − 0.05/ − 0.1/ − 0.2/ − 0.4 (dashed).

6.4 Nonlinear PDEs

In the next chapter we shall investigate in more detail several nonlinear problems as
applications and examples. Here, we wish to concentrate on the Ginzburg–Landau
equation.

6.4.1 Real Ginzburg–Landau equation

Based on the original idea of Landau⁵ [7] the Ginzburg–Landau equation serves as a
phenomenological model describing phase transitions. A macroscopic quantity, for
instance the magnetization of a ferromagnetic material, is described by the help of an
order parameter q(t). The values q = ±q0 correspond to the magnetic state where all
elementary magnets are parallel and point in the same direction, and q ≈ 0 denotes
the nonmagnetic state above the Curie temperature Tc where the spins are not ordered.

5 Lew Dawidowitsch Landau, Russian physicist, 1908–1968.

6.4 Nonlinear PDEs | 187

If there is no preferred direction in space, the free energy F should not depend on the
sign of q and F(q) = F(−q). Expanding F up to the fourth order yields
1 2 1 4
F(q, T) = F(0, T) + αq + βq . (6.122)
2 4

If β > 0 there exists a minimum of F at q = 0 and α > 0, at q = ±q0 = (−α/β)1/2

for α < 0. Obviously, α = 0 can be considered as the critical point where a phase
transition between the ordered, ferromagnetic state and the disordered, nonmagnetic
state takes place. Thus it is easy to relate α to temperature:
Tc − T
α∼− = −ε
Tc
where ε is called the ‘bifurcation parameter’ or ‘control parameter.’ To achieve a dy-
namical description of the phase transition one assumes further that F is minimized
in the course of time. Thus a gradient dynamics model is obtained having the form

dq dF
=− = εq − βq3
dt dq

or, after appropriate scaling of q:

dq
= εq − q3 . (6.123)
dt
The three stationary solutions of (6.123)

q0s = 0 , s
q12 = ±√ ε (6.124)

correspond to the different phases; see Figure 6.26. By the natural extension

q(t) 󳨀→ q(x, t)

Fig. 6.26: Pitchfork bifurcation at ε = 0. For

ε > 0 the nonmagnetic state becomes unstable
and the two stable states q s = ±√ε branch off.
This is a typical scenario for a phase transition
of the second order.
188 | 6 Partial differential equations I, basics

a space-time dependent magnetization is introduced, allowing for the description of

dislocations and magnetic domains separated by Bloch walls. Each wall and each dis-
location increases the total free energy. The ground state minimizing F corresponds
still to one of the homogeneous solutions (6.124). A sensitivity of F with respect to
dislocations and walls is achieved by adding the ‘Ginzburg’ term⁶ ∼ (∇q)2 to (6.122).
Integration over the spatial coordinates yields the free energy functional
1 1 1
F[q] = ∫ dx dy { αq2 + βq4 + D ((∂ x q)2 + (∂ y q)2 )} . (6.125)
2 4 2
A

As above a gradient dynamics is obtained, we now apply the functional derivative

∂q δF ∂F ∂F ∂F
=− =− + ∂x ( ) + ∂y ( ) ,
∂t δq ∂q ∂(∂ x q) ∂(∂ y q)
which eventually leads to the real Ginzburg–Landau equation:

∂ t q = εq + D ∆q − q3 .

Finally, scaling the lengths with √D results in the normal form:

∂ t q = εq + ∆q − q3 . (6.126)

6.4.2 Numerical solution, explicit method

With (6.126) we have a PDE that is nonlinear with the dependent variable q. A fully
explicit scheme (FTCS) as an extension of (6.60) is obtained adding the extra terms
evaluated at time t:
(n) (n) (n) (n) (n)
(n+1) (n) (n) (n) 3
q i,j+1 + q i+1,j − 4q i,j + q i−1,j + q i,j−1
q i,j = q i,j + ∆t (εq i,j − (q i,j ) + ) . (6.127)
∆x2

To check the numerical stability we must linearize around one of the stable solutions,
e.g., q s = √ε:
q ij = √ε + u ij .
For small u the linear system
(n) (n) (n) (n) (n)
(n+1) (n) (n)
u i,j+1 + u i+1,j − 4u i,j + u i−1,j + u i,j−1
u i,j = u i,j + ∆t (−2εu i,j + ) (6.128)
∆x2

results. A von Neumann analysis similar to Section 6.2.3.3 but now in two dimensions
(n)
u m,n = A(n) ei(k x m+k y n)

6 Witali Lasarewitsch Ginsburg, Russian physicist, 1916–2009.

6.4 Nonlinear PDEs | 189

leads to
A(n+1) = V A(n)
with the ampliﬁcation factor
1
V = 1 + 2 [−ε + (cos k x + cos k y − 2)] .
∆x2
The stability condition |V| ≤ 1 yields an upper limit for the time step

∆x2
∆t ≤ . (6.129)
ε∆x2 + 4
In practice, the inequality
ε ∆x2 ≪ 1

holds and one may estimate

∆x2
,∆t ≤ (6.130)
4
which is the same result already found for the two-dimensional diffusion equation.
Hence the diffusion part of (6.126) is crucial for the numerical stability and not the
nonlinearities.

6.4.3 Numerical solution, semi-implicit method

For a fully implicit method all terms inside the brackets on the right-hand side of
(6.127) have to be known to the ‘new’ time t+∆t. Hence a nonlinear system of equations
had to be solved which would have been possible only iteratively. An easier realizable
procedure is obtained if only the linear parts of the right-hand side are considered at
t + ∆t whereas the nonlinearity remains explicit at t. Instead of (6.127) we arrive at the
semi-implicit scheme
(n+1) (n+1) (n+1) (n+1) (n+1)
(n+1) (n) (n+1) (n) 3
q i,j+1 + q i+1,j − 4q i,j + q i−1,j + q i,j−1
q i,j = q i,j + ∆t (εq i,j − (q i,j ) + ) .
∆x2
(6.131)
The same stability analysis as above yields now the upper time step limit
1
∆t ≤ , (6.132)
2ε
independently of the spatial step size ∆x.
Coding in MATLAB we can use the toolbox ‘gallery’ [8], providing several standard
matrices, amongst others the FD-version of the Laplacian in two dimensions, as shown
in Figure 6.8 (a). Running the script
190 | 6 Partial differential equations I, basics

clear;
kplot=[5,10,20,30,60,100,150,200,300]; % at k=kplot(m) make pic. m
n=200; dx=0.5; dt=1; kend=300; eps=0.1; % mesh size, steps,epsilon
psi=zeros(n); % 2D-array for plotting

lap=-gallery('poisson',n)/dx^2; % Laplace discretizing

m=(1/dt-eps)*speye(n^2)-lap; % matrix M

phi=rand(n^2,1)'-.5; % random init. cond.

ib=1; % count for figs.

for k=0:kend % time loop

phi=(phi/dt-phi.^3)/m; % matrix inversion

if k==kplot(ib) % plotting
subplot(3,3,ib); ib=ib+1;
psi(:)=phi; % reshape phi to 2D
contourf(psi);
axis off; title(sprintf('t = %5.1f',k*dt));
drawnow;
end

end

creates a time series of nine snapshots as shown in Figure 6.27.

6.4.4 Problems

The Fisher–Kolmogorov equation.

With pencil and paper:
1. The Fisher–Kolmogorov equation is a nonlinear diffusion equation having the
form:

∂ t u(x, t) = D ∂2xx u(x, t)+ αu(x, t)− βu 2 (x, t), u(x, t) ≥ 0, D, α, β > 0. (6.133)

By the help of an appropriate scaling of t, x, u, transform the Fisher–Kolmogorov

equation to the normal form

∂ t u(x, t) = ∂2xx u(x, t) + u(x, t) − u 2 (x, t) . (6.134)

6.4 Nonlinear PDEs | 191

Fig. 6.27: Numerical solution of the real Ginzburg–Landau equation (6.126) obtained with the
scheme (6.131) for ε = 0.1, time series.

2. Show with a linear stability analysis that the fixed point (stationary solution)
u (0) = 0 is unstable, and that the fixed point u (0) = 1 is stable.
3. Monotonic solutions that connect the two fixed points

u = u(x − vt) = u(ξ) , u(ξ → −∞) = 1 , u(ξ → ∞) = 0 , v>0 (6.135)

are denoted as fronts or kinks. Therefore the front moves with a certain velocity v
towards the ﬁxed point u = 0, i.e., to the right-hand side. Find a lower limit for v.
192 | 6 Partial differential equations I, basics

4. Show that for a certain v and κ

1
u(x − vt) = 2
, C>0 (6.136)
(1 + Ce−κ(x−vt))

is a solution of (6.134). Determine v and κ. What is the meaning of the (arbitrary

but positive) constant C?
5. Release the restriction u ≥ 0. With the help of the analogy to a one-dimensional
motion of a point mass in a potential, show that a stationary, localized solution
of (6.134) exists. Determine this solution by integrating the law of energy conser-
vation of the mass point.
6. Perform a linear stability analysis

u(x, t) = u 0 (x) + w(x) exp(λt)

about the solution u 0 (x) found in 5. and derive a stationary Schrödinger equation
for the disturbances w(x) of the form

(−d2xx + V(x)) w(x) = −λw(x) . (6.137)

Find the ground state by expanding V(x) up to the order x2 . What can you say
about the stability of u 0 (x)?

And to code:
1. Verify the results of 1–4 solving (6.134) numerically with an FTCS method and the
appropriate boundary conditions u(x = 0) = 1, u(x = L) = 0. Assume a large
enough L.
2. Solve the Schrödinger equation (6.137) writing for instance a MATLAB code. Ana-
lyze its spectrum and its eigenfunctions.

Bibliography

[1] A. P. S. Selvadurai, Partial differential equations in mechanics, Vol. 1, 2, Springer (2010).

[2] J. D. Jackson, Classical Electrodynamics, Wiley & Sons (1998).
[3] L. D. Landau and E. M. Lifshitz, Fluid Mechanics, Vol. 6, Butterworth–Heinemann, 2nd edn.
(1987).
[4] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
[5] W.Guo, G. Labrosse and R.Narayanan, The Application of the Chebyshev-Spectral Method in
Transport Phenomena, Springer (2012).
[6] T. I. Zohdi, A Finite Element Primer for Beginners: The Basics, Springer (2015).
[7] L. D. Landau, On the Theory of Phase Transitions, Zh. Eksp. Teor. Fiz. 7, (1937).
[8] D. J. Higham and N. J. Higham, MATLAB Guide, SIAM (2005).
7 Partial differential equations II, applications
Now we shall apply the methods and schemes learned in the last chapter to problems
from several ﬁelds of physics. Let us start with some simple quantum mechanical ex-
amples.

7.1 Quantum mechanics in one dimension

7.1.1 Stationary two-particle equation

We ﬁrst consider two interacting particles in one spatial dimension. The particles,
each with mass m, are located at x1 , x2 and connected with a spring having the spring
constant −α. The stationary Schrödinger equation thus reads

Ĥ Ψ(x1 , x2 ) = E Ψ(x1 , x2 ) ,

where Ψ is the two-particle wave function and the Hamiltonian is given as

ℏ2 ∂2 ∂2 1
Ĥ = − ( 2 + 2 ) − α (x1 − x2 )2 . (7.1)
2m ∂x1 ∂x2 2
We shall also look at the case of a repelling interaction α > 0. In addition the particles
are trapped in a potential well with length L and inﬁnitely high walls,

Ψ(0, x2 ) = Ψ(L, x2 ) = Ψ(x1 , 0) = Ψ(x1 , L) = 0 .

With the scaling

L π4 ℏ2 π2 ℏ2 ̃
x̃ i , α =
xi = α̃ , E = E (7.2)
π mL4 2mL2
the stationary Schrödinger equation takes the dimensionless form
∂2 ∂2 ̃ Ψ =0.
( + 2 + α̃ (x̃ 1 − x̃ 2 )2 + E) (7.3)
∂ x̃ 1 ∂ x̃ 2
2

In the following we leave the tildes. For the decoupled case α = 0 the exact solutions
simply read (N → ∞)

2 N
Ψ(x1 , x2 ) = ∑ cℓm sin ℓx1 sin mx2 (7.4)
π ℓ,m=1

where the coefficients cℓm are determined from the initial condition. For the energy,
one ﬁnds the levels
E = Eℓm = ℓ2 + m2 , (7.5)
which are two-fold degenerated if ℓ ≠ m, reﬂecting the symmetry x1 ↔ x2 . The
decoupled problem is similar to the single particle Schrödinger equation in a two-
dimensional potential well.

https://doi.org/10.1515/9783110515145-007
194 | 7 Partial differential equations II, applications

Inserting (7.4) in (7.3) with α ≠ 0 and multiplying with

2
sin kx1 sin nx2
π
we obtain after integrating over x1 , x2 the linear system

∑ M knℓm cℓm = E c kn (7.6)

ℓ,m

with the matrix elements

ππ
4α
M knℓm = (ℓ2 + m2 ) δℓk δ mn − ∫∫ dx1 dx2 (x1 − x2 )2 sin kx1 sin nx2 sin ℓx1 sin mx2
π2
00

{ 32αkℓmn −1 + (−1)k+ℓ + (−1)m+n − (−1)k+ℓ+m+n

{
{ − , k ≠ ℓ , m ≠ n
{
{ π2 (ℓ2 − k 2 )2 (m2 − n2 )2
{
{
{
{
{
{ 4αmn (1 + (−1)m+n )
{
{ − , k = ℓ , m ≠ n
{ (m2 − n2 )2
={
{
{ 4αkℓ (1 + (−1)k+ℓ )
{
{
{
{ − , k ≠ ℓ , m = n
{
{ (k 2 − ℓ2 )2
{
{
{
{ 3(n2 + k 2 ) − π2 k 2 n2
{ α + k 2 + n2 , k=ℓ, m=n.
{ 6k 2 n2
(7.7)

As happens quite often in quantum mechanics, one is confronted with the task to di-
agonalize a large matrix. To convert M into a two-dimensional matrix, one can conﬁne
each pairs of indices to one:

(k, n) → k 󸀠 = k + (n − 1)N (7.8)

and the same for (ℓ, m). N is the number of modes included in (7.4). The larger N, the
better spatial resolution of the wave functions is achieved. However, M has N 2 × N 2
elements and the numerical effort grows fast with N 4 .
Figure 7.1 shows the lower energy levels depending on α for N = 10. Note that
the degeneracy vanishes due to the coupling. On the other hand, for large α nearly
degenerated states occur again where a symmetric and an antisymmetric state belong
to (almost) the same energy value.
Figure 7.2 depicts contour lines of the 12 lowest states for α = 10. The probability
densities are largest near the corners of the line x1 + x2 = L, corresponding to the
furthest possible separation of the two particles. In Figure 7.3 we show the states for
attractive interaction, α = −10. Here, the probability density is maximal along the line
x1 = x2 , hence the particles prefer to be close to each other.
7.1 Quantum mechanics in one dimension | 195

10
Ψ22

Ψ00

E 0

Ψ12 + Ψ21 Ψ12 – Ψ21

–10

Ψ21 Ψ12
–20
–20 –10 0 10 20
α

Fig. 7.1: Energy levels as a function of α. The states Ψ ℓm , Ψ mℓ , degenerated for α = 0, split up. Inset:
by linear combination of two degenerated states (bottom), new states can be constructed (top).

Fig. 7.2: The 12 lowest states in the x1 , x2 -plane, from top left to bottom right, α = 10. The states
(1,2), (3,4) (5,6) are almost degenerated, compare Figure 7.1.
196 | 7 Partial differential equations II, applications

Fig. 7.3: The lowest states for α = −10. Here, degeneracy is completely removed.

7.1.2 Time-dependent Schrödinger equation

The time-dependent one-particle Schrödinger equation already appeared in Chapter 5.

We wish to continue our study again in one spatial dimension. Scaling of x as in (7.2)
leads to its dimensionless form
∂2 Ψ(x,̃ t)̃ ̃
̃ x,̃ t)̃ Ψ(x,̃ t)̃ = i ∂Ψ(x,̃ t) ,
− + U( (7.9)
∂ x̃ 2
∂t̃
where in addition time has been scaled with
π2 ℏ
t̃ = t
2mL2
and
2mL2
Ũ = 2 2 U .
π ℏ
In what follows we leave all tildes again.

7.1.2.1 The inﬁnitely high potential well

Again we consider a potential well with inﬁnitely high walls

{0 , 0<x<π
U(x) = {
∞, else .
{
Since the particle cannot penetrate into a domain with inﬁnite energy its wave func-
tion must be zero there. Then, continuity of the wave function yields the Dirichlet con-
ditions
Ψ(0, t) = Ψ(π, t) = 0 .
Separation of space and time dependence allows for an exact solution of the form
∞ ∞
Ψ(x, t) = ∑ A n e−iω n t f n (x) = ∑ A n e−in t sin nx ,
2
(7.10)
n n
7.1 Quantum mechanics in one dimension | 197

where the constants A n are determined by the initial condition. Obviously

ω n = n2 ϵ N , (7.11)
saying that the wave function is periodic in t
Ψ(x, t) = Ψ(x, t + T)
with the period time T = 2π, where T is also named recurrence time.
If Ψ is known, the mean values of position and momentum can be computed
π
1
⟨x⟩(t) = 2 ∫ dx Ψ ∗ x Ψ
N
0
π
(7.12)
1
⟨p⟩(t) = 2 ∫ dx Ψ ∗ (−2i∂ x )Ψ
N
0

with the norm

N 2 = ∫ dx |Ψ|2 .
0
As is easily shown by the help of the Schrödinger equation, N does not depend on time
and the norm is conserved.

7.1.2.2 Wave packet

Taking as initial condition at t = 0 a Gaussian wave packet having the width σ and its
maximum at x0 ¹
Ψ(x, 0) = e ik0 x e−(x−x0) /2σ ,
2 2
(7.13)
one finds for the means according to (7.12)
⟨x⟩(0) = x0 , ⟨p⟩(0) = 2k 0 .
If the wave packet is interpreted as a particle at time t = 0, x0 and k 0 correspond to
the classical initial conditions of position and velocity (or momentum).
We note that due to the dispersion relation (7.11) the wave packet at first flows apart
but then exactly regains its initial shape (7.13) after the recurrence time t = T.

7.1.2.3 Algorithm
With the help of the time-evolution operator
t 󸀠 ̂
̂ t0 ) = e−i ∫t0 dt H(t ) ,
󸀠 ∂2
U(t, Ĥ = − 2 + U(x, t) ,
∂x

1 However, the initial condition should also fulﬁll the boundary conditions. This is here only approx-
imately true, the wave packet should therefore be narrow enough and not too close to one of the walls
so that Ψ(0, 0) ≈ Ψ(π, 0) ≈ 0 holds.
198 | 7 Partial differential equations II, applications

the solution of the Schrödinger equation (7.9) can be formally written as

̂ t0 )Ψ(x, t0 ) .
Ψ(x, t) = U(t, (7.14)

Looking at the evolution for a small time interval ∆t = t − t0 and expanding with
respect to ∆t up to the ﬁrst order, the ‘short-time-evolution’ operator
t+∆t
̂ 󸀠)
dt 󸀠 H(t ̂
̂ + ∆t, t) = e−i ∫t
U(t ̂
= e−i H(t)∆t + O(∆t2 ) = 1 − i H(t)∆t + O(∆t2 ) (7.15)

results. Inserting in (7.14) yields the iteration rule

Ψ(x, t + ∆t) = (1 ̂
− i H(t)∆t) Ψ(x, t) , (7.16)
⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
Â

analogue to an explicit Euler scheme. But contrary to U,̂ the truncation Â is no longer
unitary, thus keeping the norm if Ψ

̂
⟨Ψ(x, t + ∆t) | Ψ(x, t + ∆t)⟩ = ⟨AΨ(x, ̂
t) | AΨ(x, ̂
t)⟩ = ⟨Ψ(x, t)|Â + A|Ψ(x, t)⟩

with
Â + Â = (1 + i H(t)∆t)
̂ ̂
(1 − i H(t)∆t) = 1 + Ĥ 2 ∆t2
only conserved up to the order ∆t. The trick to get rid of this issue is to formulate (7.15)
in a different way,

̂ ̂ ̂ ̂ −1 ̂
̂ + ∆t, t) =e−i H(t)∆t
U(t = e−i H(t)∆t/2e−i H(t)∆t/2 = [ei H(t)∆t/2] e−i H(t)∆t/2
−1
̂
= (1 + i H(t)∆t/2) ̂
(1 − i H(t)∆t/2) + O(∆t2 ) ,

leading now to the short-time-evolution operator of the form

̂ + ∆t, t) = (B̂ + )−1 B̂ + O(∆t2 )

U(t

with
B̂ = 1 − i H(t)∆t/2
̂ .

Due to
−1 +
Û Û + = [( B̂ + ) B]̂ [(B̂ + )−1 B]
̂ = (B̂ + )−1 B̂ B−1 B̂ + = (B̂ + )−1 B̂ + = 1

this is unitary in any order of ∆t. Hence instead of (7.16) it is much better to take the
norm-conserving rule
−1
Ψ(x, t + ∆t) = (B̂ + ) B̂ Ψ(x, t)

or
B̂ + Ψ(x, t + ∆t) = B̂ Ψ(x, t) . (7.17)
7.1 Quantum mechanics in one dimension | 199

At each iteration step B̂ + must be inverted to compute Ψ(x, t + ∆t), therefore (7.17) can
be considered an implicit method.
To evaluate (7.17), Ψ can be decomposed into a function system. The most straight-
forward way is to do a discretization by ﬁnite differences; see Section 6.2:

Ψ(x, t) → Ψ(x k , t) = Ψ k (t) .

With the representation (6.30) one gets (U = 0)

2i∆x2
− (− + 2) Ψ k (t + ∆t) + Ψ k+1 (t + ∆t) + Ψ k−1 (t + ∆t)
∆t
2i∆x2
=( + 2) Ψ k (t) − Ψ k+1 (t) − Ψ k−1 (t) (7.18)
∆t
or
⃗ + ∆t) = −L+ Ψ(t)
L Ψ(t ⃗ , ⃗ + ∆t) = −L−1 L + Ψ(t)
Ψ(t ⃗

where L is tridiagonal.
Figure 7.4 shows a numerical solution of (7.18) with ∆x = π/1000 and ∆t = 10−5 .
As an initial condition, the wave packet (7.13) with

x0 = 1 , k 0 = 40 , σ = 3/4 (7.19)

0.8

0.6

0.4

0.2

0
0 1 2 3
x

Fig. 7.4: xt-diagram obtained by integrating (7.18). Plotted are the contour lines of |Ψ(x, t)|2 .
200 | 7 Partial differential equations II, applications

Fig. 7.5: Mean and variance of x. After t = 2π the initial state is completely restored, and after t = π
the modes with even n take their initial values.

was utilized. It can be clearly seen how the wave packet ﬂows apart up to t ≈ 0.5. Then,
the particle can be hardly localized, which becomes also obvious from Figure 7.5. The
wave packet (7.13) after T is completely, and after T/2 partially, reconstructed. There,
the variance
Var(x) = ⟨x2 ⟩ − ⟨x⟩2
takes a minimum.

7.1.2.4 Step sizes

How can we estimate an upper limit for the step sizes ∆t and ∆x in (7.18)? Naturally,
they depend on the desired resolution and therefore on typical intrinsic time and
space scales of the problem at hand. Let us start with the time scales:
Considering the particular problem of the potential well with a localized initial
condition (7.13), there exist four characteristic times:
7.1 Quantum mechanics in one dimension | 201

(1) The recurrence time

T1 = T = 2π ≈ 6.28
(2) The circulation time (particle travels from one wall to the other)
T2 = π/2k 0 ≈ 0.04
(3) The ‘ﬂow-apart time’ until the width of the wave packet

σ 2 2T32
b = 2√ + 2
2 σ
is about equal to the length of the well π, hence
π2 σ2 σ4
T3 = √ − ≈ 0.8
8 4
(4) Validity of the Courant condition (6.65). In (7.10), the phase velocity of the wave
with amplitude A n is v p = n. We estimate the maximal occurring n = n m by re-
quiring
A n m = ϵ A k0 = ϵ
with a given ϵ ≪ 1 (accuracy). From
A n = e−σ (k−k0 )2 /2
= e−σ (n−k0 )2 /2
2 2

it follows that
2
n m = √− ln ϵ + k 0 .
σ2
The Courant condition (6.65) is valid for an upper T4 of
∆x
T4 = .
nm
To determine ∆x one must deﬁne a certain spacial resolution. If we assume that
the shortest occurring wave length λ0 = 2π/n m is resolved with 10 mesh points,
we ﬁnd
2π
∆x = (7.20)
10n m
or
2π
T4 = .
10n2m
Taking the parameters (7.19) and ϵ = 10−4 , we obtain n m = 46 and
T4 ≈ 3 ⋅ 10−4 .

Thus, a hierarchy of disparate characteristic times of the form

T4 ≪ T2 ≪ T3 ≪ T1
is found. Eventually, the upper limit of the time step reads
∆t < T4 ,
and from (7.20) for ∆x,
∆x < 0.014 .
202 | 7 Partial differential equations II, applications

7.2 Quantum mechanics in two dimensions

To exemplify a two-dimensional problem we next turn to the behavior of a charged

particle (e.g., an electron) in a plane perpendicular to a stationary and uniform mag-
netic ﬁeld.

7.2.1 Schrödinger equation

Let the motion of the particle with mass m and charge q be restricted on the xy-plane
perpendicular to a given magnetic field
B⃗ = B ê z .
The classical analogue provides closed, circular trajectories having the angular fre-
quency ω c = qB/m, named cyclotron frequency. Along the canonical formalism, the
Hamiltonian is modified by the minimal substitution
1 ̂⃗ 2
Ĥ = ( p̂⃗ − q A)
2m
̂
with the operator A⃗ belonging to the vector potential of the magnetic field B⃗ such that
B⃗ = curl A.⃗ Due to gauge invariance A⃗ can be only determined up to an expression ∇ϕ.
Thus we can take for instance the symmetric form
y
1
A⃗ = B (−x ) .
2
0
The scaling of Section 7.1 yields the dimensionless Schrödinger equation
1 2 2
[−∆ + iB0 (y∂ x − x∂ y ) + B (x + y2 )] Ψ(x, y, t) = i∂ t Ψ(x, y, t) (7.21)
4 0
with the 2D-Laplacian ∆ = ∂ xx + ∂ yy and
qL2
B0 = B.
π2 ℏ
We assume again the particle located in a square potential well of length L with in-
finitely high walls, leading to the boundary conditions
Ψ(0, y, t) = Ψ(π, y, t) = Ψ(x, 0, t) = Ψ(x, π, t) = 0 .

7.2.2 Algorithm

The reasoning of Section 7.1.2.3 is valid just as well in two dimensions. Discretizing
again with ﬁnite differences on an N × N-grid with step size ∆x,
Ψ(x, y, t) → Ψ(x k , yℓ , t) = Ψ k,ℓ (t)
7.2 Quantum mechanics in two dimensions | 203

yields from (7.17) the iteration rule

2i∆x2
( − 4 − β 2 (x2k + y2ℓ )) Ψ k,ℓ (t + ∆t)
∆t
+ (1 − iβyℓ )Ψ k+1,ℓ (t + ∆t) + (1 + iβyℓ )Ψ k−1,ℓ (t + ∆t)
+ (1 + iβx k )Ψ k,ℓ+1 (t + ∆t) + (1 − iβx k )Ψ k,ℓ−1 (t + ∆t)
2i∆x2
=( + 4 + β 2 (x2k + y2ℓ )) Ψ k,ℓ (t)
∆t
− (1 − iβyℓ )Ψ k+1,ℓ (t) − (1 + iβyℓ )Ψ k−1,ℓ (t)
− (1 + iβx k )Ψ k,ℓ+1 (t) − (1 − iβx k )Ψ k,ℓ−1 (t) (7.22)

with the abbreviation β = B0 ∆x/2. Taking again a vector notation as in (7.8)

Ψ k,ℓ = Φ k󸀠 , k 󸀠 = k + (ℓ − 1)N , (7.23)

we can write (7.22) as

⃗ + ∆t) = −L+ Φ(t)
L Φ(t ⃗ .
However, the matrix L is no longer tridiagonal and cannot be inverted quickly apply-
ing the Thomas algorithm. It is still sparse, so one could use either sparse matrix al-
gorithms or iterative schemes as introduced in Section 6.2.2.2.

7.2.3 Evaluation

Figures 7.6 and 7.7 show the temporal evolution of a wave packet with and without a
magnetic ﬁeld. The initial condition is chosen with
⃗
Ψ(x, y, 0) = ei k0 x⃗ e−(x−⃗ x⃗ 0 ) /2σ 2
2
(7.24)
where
k⃗ 0 = (100, 70) , x⃗ 0 = (π/2, π/2) , σ = 1/20 .

Fig. 7.6: Temporal evolution of the wave packet (7.24), computed with the scheme (7.22), snapshots
at equidistant times without magnetic ﬁeld. Plotted are the contour lines of |Ψ|2 as well as the ‘tra-
jectory’ x⃗m (t) = ⟨Ψ| x|Ψ⟩.
⃗
204 | 7 Partial differential equations II, applications

Fig. 7.7: If the magnetic ﬁeld is switched on (B 0 = −300), the particle travels along segments of a
circle.

The FD-grid consists of 200 × 200 points, and the step sizes are
∆x = π/199 , ∆t = 5 ⋅ 10−6 .
The solid black lines depict the mean of position. Clearly one recognizes at least in the
beginning and as long as the wave packet is still localized, the classically expected
path, including reﬂections at the side walls.

7.3 Fluid mechanics: ﬂow of an incompressible liquid

Fluid dynamics or hydrodynamics is a branch of continuum mechanics and based

on classical mechanics. Space is thought to be completely filled with a continuum,
namely the fluid, which can be a gas or a liquid. The state of the fluid is described by
certain fields: v(⃗ r,⃗ t), the flow field, ρ(r,⃗ t) the density, and P(r,⃗ t), the pressure. For
compressible fluids a material law of the form P = P(ρ) closes the description. For the
more general, nonisothermal case, a further state variable, the temperature T(r,⃗ t) is
necessary.

7.3.1 Hydrodynamic basic equations

Here we shall concentrate on incompressible ﬂuids with ρ = ρ 0 = const. and start with
the isothermal case T = const. Conservation of momentum and total mass leads to the
Navier–Stokes equations and the continuity equation, respectively [1]
1
∂ t v⃗ + (v⃗ ⋅ ∇) v⃗ = ν∆ v⃗ − ∇P , (7.25a)
ρ0
∇ ⋅ v⃗ = 0 , (7.25b)
where ν denotes the, here constant, kinematic viscosity².

2 If ν is not constant, for instance for non-Newtonian ﬂuids or nonisothermal problems, (7.25a) be-
comes more involved. We refer to textbooks of hydrodynamics, e.g., [1].
7.3 Fluid mechanics: ﬂow of an incompressible liquid | 205

In three dimensions the incompressibility condition (7.25b) is identically satisﬁed

by the ansatz
v⃗ = curl(Ψ ê z ) + curl curl(Φ ê z ) . (7.26)
The two expressions on the right-hand side are denoted as toroidal and poloidal ve-
locity ﬁelds. The pressure is eliminated by taking the curl of (7.25a) and one obtains
two nonlinearly coupled, scalar equations for the ﬁelds Ψ and Φ; see Section 7.3.4.2
below.

7.3.1.1 Plane incompressible ﬂow: the stream function

In two dimensions (7.26) simplifies to
∂y Ψ
v⃗ = curl(Ψ ê z ) = ( ) . (7.27)
−∂ x Ψ
The field Ψ(x, y, t) is called stream function, and the curves with Ψ(x, y, t) = const. are
the stream lines. Due to (t = const.)
dΨ = ∂ x Ψ dx + ∂ y Ψ dy = −v y dx + v x dy
one finds along a stream line where dΨ = 0:
dy v y
= .
dx v x
Hence, for the stationary case ∂ t Ψ = 0, the stream lines are identical with the particle
paths³.

7.3.1.2 Vorticity
The vorticity of a given flow is defined by the curl
ξ ⃗ = curl v⃗ .
In two dimensions and with (7.27) this simplifies to the scalar field
ξ = −∆Ψ . (7.28)

7.3.1.3 Algorithm and boundary conditions

In two dimensions, the curl of (7.25a) has only a z-component. After some short ma-
nipulations this component takes the form of a transport equation for ξ :
∂ t ξ + v x ∂ x ξ + v y ∂ y ξ = ν∆ξ . (7.29)
To determine v⃗ from Ψ one must invert (7.28). The following steps have to be performed
at each time step:

3 In ﬂuid dynamics the notion ‘particle’ stands rather for a volume element of the continuum swim-
ming with the ﬂow.
206 | 7 Partial differential equations II, applications

(1) t := 0. Choose an initial condition ξ = ξ(t = 0).

(2) ⃗
Invert (7.28) applying a Poisson solver, determine Ψ(t), and then from (7.27), v(t).
(3) Iterate (7.29) (FTCS, Galerkin, etc.) and compute ξ(t + ∆t).
(4) t := t + ∆t. If t < tend , go to (2).

Boundary conditions are necessary in (7.29) as well as for the inversion of (7.28). Along
a rigid no-slip wall both components of v⃗ must vanish. We take as an example, the wall
at x = 0. From v⃗ = 0 it follows from (7.27)
∂ x Ψ(x = 0, y) = ∂ y Ψ(x = 0, y) = 0 .
Hence, Ψ must be a constant along the wall and since only derivatives of Ψ occur we
may put
Ψ(x = 0, y) = 0 , ∂ x Ψ(x = 0, y) = 0 . (7.30)
But now we have two conditions for Ψ, namely Dirichlet and Neumann, but none for
the vorticity ξ . To end up with a well-deﬁned problem one must somehow ‘distribute’
the conditions to the two arrays Ψ and ξ [2].
Equation (7.28) can be uniquely inverted with the Dirichlet condition Ψ = 0. To
include also the Neumann condition, we expand Ψ at x = 0 up to the second order
and make use of (7.30):
1
Ψ(∆x, y) = ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
Ψ(0, y) + ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
∂ x Ψ(0, y) ∆x + ∂ xx Ψ(0, y)∆x2 . (7.31)
=0 =0
2

If Ψ is constant along the wall, its second derivative ∂ yy Ψ also vanishes. Then (7.28)
reduces to
ξ(0, y) = −∂ xx Ψ(0, y)
and, substituted in (7.31), a Dirichlet condition for ξ can ﬁnally be stated:
2Ψ(∆x, y)
ξ(0, y) = − . (7.32)
∆x2

7.3.2 Example: driven cavity

7.3.2.1 The system

Let a fluid (2D) be confined by four walls having the length L. Let the upper wall move
to the right side with a constant velocity U0 , the other walls are in rest. For all walls,
no-slip conditions apply. In particular at the upper wall one has v x = U0 , v y = 0 or
Ψ =0, ∂ y Ψ = U0 .
Since ∂ y Ψ does not vanish at y = L, the Dirichlet condition (7.32) for ξ reads there
2Ψ(x, L − ∆x) 2U0
ξ(x, L) = − − . (7.33)
∆x2 ∆x
Figure 7.8 shows the system and the boundary conditions.
7.3 Fluid mechanics: flow of an incompressible liquid | 207

L Ψ = 0, ξ = −2Ψ(x, y = L − Δ x)/Δ x 2 −2U0/Δ x

Ψ = 0, ξ = −2Ψ(x = L − Δ x, y)/Δ x 2
Ψ = 0, ξ = −2Ψ(x = Δ x, y)/Δ x 2

Ψ = 0, ξ = −2Ψ(x, y = Δ x)/Δ x 2 L

Fig. 7.8: Driven cavity with boundary conditions. The upper wall moves with the constant velocity U 0
to the right. Inside the ﬂuid (gray) vorticies are created.

7.3.2.2 Dimensionless equations and boundary conditions

In ﬂuid mechanics one normally works with dimensionless variables and equations.
For our problem, the scaling
L ̃
v⃗ = U0 v⃗̃ , (x, y) = (L x,̃ L y)̃ , t= t
U0
is nearby and leads instead of (7.28, 7.29) to the system (all tildes are again removed)
1
∂t ξ + vx ∂x ξ + vy ∂y ξ = ∆ξ
Re (7.34)
ξ = −∆Ψ

with the Reynolds number⁴

U0 L
Re =
ν
as the only parameter left (characteristic number). The boundary condition at the top
lid (7.33) after scaling takes the form
2Ψ(x, 1 − ∆x) 2
ξ(x, 1) = − − .
∆x2 ∆x

4 Osborne Reynolds, British physicist, 1842–1912.

208 | 7 Partial differential equations II, applications

7.3.2.3 Algorithm
The discretization of x and y is achieved using ﬁnite differences with step size ∆x.
We refer to the algorithm of Section 7.3.1.3. For step (2) the ADI-method introduced in
Section 6.2.2.2 is implemented. Here we wish to explain step (3), the numerical inte-
gration of the vorticity equation (7.34), in more detail, see also [2]. Substitution of the
time derivative by the forward differential quotient of the ﬁrst order yields the one-step
scheme⁵
ξ(t + ∆t) − ξ(t) 1
= −∂ x (u(t󸀠 )ξ(t󸀠 )) − ∂ y (v(t󸀠 )ξ(t󸀠 )) + ∆ξ(t󸀠 ) , (7.35)
∆t Re
which can be, depending on t󸀠 , an explicit (t󸀠 = t) or an implicit (t󸀠 = t + ∆t) scheme.
Introducing the variation
δξ = ξ(t + ∆t) − ξ(t) (7.36)
we may formulate (7.35) as

δξ = ∆t (A x + A y ) ξ(t󸀠 ) (7.37)

where the matrix operators A x , A y are deﬁned as

1
Ax = L − L x (u)
R e xx
1
Ay = L − L y (v) ,
R e yy
with
ξ i+1,j − 2ξ i, j + ξ i−1,j u i+1,j ξ i+1,j − u i−1,j ξ i−1,j
L xx ξ = , L x (u)ξ = ,
∆x2 2∆x
ξ i,j+1 − 2ξ i, j + ξ i,j−1 v i,j+1 ξ i,j+1 − v i,j−1 ξ i,j−1
L yy ξ = , L y (v)ξ = .
∆x2 2∆x
A semi-implicit method (Crank–Nicolson) converges more quickly and has a higher
accuracy compared to a fully explicit one. Instead of (7.37) we choose
1 1
δξ = ∆t (A x + A y ) ξ(t) + ∆t (A x + A y ) ξ(t + ∆t) , (7.38)
2 2
which can be converted with (7.36) into
1
[1 − ∆t (A x + A y )] δξ = ∆t (A x + A y ) ξ(t) . (7.39)
2
The expression in the square brackets can be factorized
1 1 1
[ . . . ] = (1 − ∆t A x ) (1 − ∆t A y ) − ∆t2 A x A y ,
2 2 4

5 Given ∇ ⋅ v⃗ = 0 it follows u∂ x ξ + v∂ y ξ = ∂ x (uξ) + ∂ y (vξ).

7.3 Fluid mechanics: ﬂow of an incompressible liquid | 209

where the last term is of order ∆t2 and will be self-consistently neglected. The inver-
sion of the square bracket in (7.39) can now be achieved in two consecutive steps where
in each step only a tridiagonal matrix must be inverted.
Step (I):
1
(1 − ∆t A x ) δξ ∗ = ∆t (A x + A y ) ξ(t) . (7.40)
2
Applying the Thomas algorithm, δξ ∗ can be computed efficiently.
Step (II):
1
(1 − ∆t A y ) δξ = δξ ∗ . (7.41)
2
As for the ADI-method, i, j must be exchanged to obtain A x and A y in tridiagonal form.
Finally, ξ(t + ∆t) can be determined from

ξ(t + ∆t) = ξ(t) + δξ .

The complete code can be found in [3].

7.3.2.4 Results
We present results on a 200 × 200 grid with ∆x = 1/199, ∆t = 1.25 ⋅ 10−4 , and
for two different Reynolds numbers. The stream lines at several times are depicted
in Figures 7.9 and 7.10. As it is known from the literature [4], the stationary vortices for
smaller R e become oscillatory unstable at about R e ≈ 8000, what can be seen from
the ﬁgures. In Figure 7.11 we show the total deviation (7.36)
200
δ ξ ̄ (t) = ∑ |δξ ij (t)| (7.42)
i,j=1

over time for the two runs. For R e = 5000 a stationary state is reached after t ≈ 100.

2.53 15.16 99.54

Fig. 7.9: Driven cavity, time series for R e = 5000. The state in the right frame is stationary. Times are
in units of τ = 2s (water, cavity length 10 cm, U 0 = 5 cm/s).
210 | 7 Partial differential equations II, applications

2.53 12.64 27.58

37.49 49.89 62.29

75.51 88.86 99.54

Fig. 7.10: Time series for R e = 10 000. Here the stream lines remain time-dependent, τ = 1s, U 0 = 10
cm/s.

To get a feeling for the time and length scales, we go back to dimensionful quanti-
ties and take water as the fluid with ν = 10−6 m2 /s. For R e = 5000 and L = 0.1 m
a lid speed of U0 = 0.05 m/s results. For the time scale, one finds τ = L/U0 = 2 s.
At R e = 10 000 (Figure 7.10) and the same cavity size, one has U0 = 0.1 m/s and
τ = 1 s.
7.3 Fluid mechanics: flow of an incompressible liquid | 211

0.8

Re = 10000
0.6

0.4

0.2 Re = 5000

0
0 10 20 30 40 50 60 70 80 90 100
t

Fig. 7.11: The integrated deviations (7.42) over time. For R e = 10 000 the ﬂow ﬁeld remains time-
dependent.

7.3.3 Thermal convection: (A) square geometry

Now we can come back to the question of the temperature distribution in a heated
room with insulating walls from Section 6.2.3.2. There we assumed pure thermal dif-
fusion caused by heat conduction of the air in rest and found that it would take many
days to obtain a somewhat homogeneous temperature across the room. By an exten-
sion of the code developed in the previous section we are now able to account for the
flow field of the air caused by the temperature gradients, a process also named convec-
tion. Contrary to the former system from Section 6.2.3.2 we simplify the situation and
assume isolating walls on the left, right, and on the ceiling. Along the lower side we
prescribe a fixed and constant temperature T0 , corresponding to floor heating. Win-
dow and stove are removed.

7.3.3.1 Basic equations

Figure 7.12 sketches the system and the boundary conditions. For the air remaining in
rest, the heat equation with the initial condition
T(x, y, t = 0) = 0
can be solved analytically:⁶
T(x, y, t) = T0 (1 − erf(y/2√κt)) . (7.43)

6 The boundary condition at y = L is thereby only approximately fulﬁlled, however the error remains
small as long as t << τ.
212 | 7 Partial differential equations II, applications

L Ψ = 0, ξ = −2Ψ(x, y = L − Δ x)/Δ x 2, ∂yT = 0

Ψ = 0, ξ = −2Ψ(x = L − Δ x, y)/Δ x 2, ∂xT = 0

Ψ = 0, ξ = −2Ψ(x = Δ x, y)/Δ x 2, ∂xT = 0

Ψ = 0, ξ = −2Ψ(x, y = Δ x)/Δ x 2, T = T0 L

Fig. 7.12: A room uniformly heated from below and its boundary conditions.

Here, κ again denotes the thermal diffusivity of the air and erf(x) stands for the error
function. The solution (7.43) describes a temperature front moving slowly from the
ﬂoor to the ceiling by thermal diffusion. The time needed to cross the room is in the
order of the thermal diffusion time

τ = L2 /κ (7.44)

and amounts for L = 3 m about 5 days, as already computed in Section 6.2.3.2. To ac-
count for the air ﬂow, the diffusion equation must be extended to the transport equa-
tion
∂ t T + v⃗ ⋅ ∇T = κ∆T .
The air begins to move due to buoyancy. Warm air is lighter and tends to move up-
wards; cold air descends in a vertical gravitation ﬁeld. Assuming a linear relation
(Boussinesq approximation) between density ρ and temperature

ρ(T) = ρ 0 (1 − α(T − T0 ))

where
1 dV 󵄨󵄨󵄨󵄨
α= 󵄨
V dT 󵄨󵄨󵄨T=T0
7.3 Fluid mechanics: ﬂow of an incompressible liquid | 213

is the thermal expansion coefficient, on the ride-hand side of the Navier–Stokes equa-
tion (7.25a) an additional volume force density occurs
ρ(T)g
− ê y = −g (1 − α(T − T0 )) ê y
ρ0

with the gravitation acceleration g = 9.81 m/s2 . Since no reference velocity U0 exists,
the scaling we choose here differs from that of Section 7.3.2.2:
L ⃗
v⃗ = ṽ , (x, y) = (L x,̃ L y)̃ , t = τ t̃ , T = T0 Θ
τ
with τ from (7.44). The complete system thus reads (tildes removed)
1
[∂ t ξ + v x ∂ x ξ + v y ∂ y ξ ] = ∆ξ + R ∂ x Θ (7.45a)
Pr
ξ = −∆Ψ (7.45b)

∂ t Θ + v⃗ ⋅ ∇Θ = ∆Θ (7.45c)

with the Prandtl number⁷ Pr = ν/κ and the Rayleigh number⁸

gαL3 T0
R= . (7.46)
κν

7.3.3.2 Algorithm and results

The method outlined in Section 7.3.2.3 is easily extended to the nonisothermal case.
Coupling of temperature and vorticity is achieved by adding the term
Θ i+1,j − Θ i−1,j
R ∆t
2∆x
on the right-hand side of (7.40). The heat equation (7.45c) can be iterated fully explicitly
by an FTCS method as outlined in Section 6.2.3.2. This yields an upper bound for the
time step
1
∆t < ∆x2 .
4
To obtain more accurate results, ∆t should be much smaller. If we compute R for the
values of air (α as for an ideal gas at T ≈ 0°C:

α = 1/273 K , ν = 14 ⋅ 10−6 m2 /s , κ = 20 ⋅ 10−6 m2 /s ,

one ﬁnds for L = 3 m and T0 = 20°C the huge value of

R ≈ 7 ⋅ 1010 .

7 Ludwig Prandtl, German physicist, 1875–1953.

8 3rd Baron Rayleigh (John W. Strutt), British physicist, 1842–1919.
214 | 7 Partial differential equations II, applications

3.8 min 8.0 min 9.4 min

11.1 min 13.5 min 15.0 min

16.1 min 18.0 min 20.3 min

Fig. 7.13: Evolution of the temperature ﬁeld, time series for R = 108 . The ﬂow becomes turbulent
soon. The times stated are valid for air (τ = 0.45 ⋅ 106 s).

The air ﬂow becomes turbulent almost immediately and small-scaled vortices are cre-
ated, which demand an extremely high resolution in space; see also Section 7.3.3.4
below. Since the code should run on a normal PC, we take the much smaller value
R = 108 for which a 200 × 200 grid

∆x = 1/199 , ∆t = 0.0003∆x2 ≈ 0.75 ⋅ 10−8

proves to be sufficient. Figure 7.13 depicts contour lines of the temperature ﬁeld at sev-
eral times; see also Figure 1.4. The diffusion front (7.43) becomes unstable after about
7.3 Fluid mechanics: ﬂow of an incompressible liquid | 215

Fig. 7.14: Averaged temperature of the ceiling (7.47) for convection (R = 108 ) and pure diffusion
(R = 0). For this Rayleigh number, convection leads to an at least ﬁfty times larger heat transport
than diffusion alone.

4 minutes and a vortex flow occurs that entrains and quickly distributes the tempera-
ture field in the whole room. Thereby, the heat transport from the floor to the ceiling
becomes much more effective, which can be clearly seen at the time dependence of
the mean ceiling temperature
1
1 200
Θ1 (t) = ∫ dx Θ(x, y = 1, t) = ∑ Θ i,200 (t) (7.47)
200 i=1
0

shown in Figure 7.14.

The whole process is transient and comes to an end in the long-time limit. Due to
the insulating walls the thermodynamic equilibrium is reached for t → ∞. Then the
air has the uniform temperature T0 everywhere, no buoyancy exists, and the ﬂow will
have come to rest by viscous damping.

7.3.3.3 Linear stability analysis

It is difficult to compute the stability of the time and space-dependent diffusion front
(7.43). If we modify the top boundary conditions into

T(x, y = L) = 0 , (7.48)

the so-called heat conduction state

Θ(0) = 1 − y , v⃗ = 0 , (7.49)
216 | 7 Partial differential equations II, applications

i.e., a linear temperature proﬁle with the air in rest, is a solution of (7.45). Linearizing
the dimensionless equations around this state yields the system
1
∂ t ξ = ∆ξ + R ∂ x Θ̃ (7.50a)
Pr
ξ = −∆Ψ (7.50b)
∂ t Θ̃ = ∆ Θ̃ − ∂ x Ψ (7.50c)

with Θ̃ = Θ − Θ(0) . Idealizing the boundary conditions for the vorticity and stream
function as⁹
ξ =0, Ψ =0
at all four boundaries, the linear problem (7.50) can be solved exactly with

ξ = A sin(nπx) sin(mπy) e λt (7.51a)

λt
Ψ = B sin(nπx) sin(mπy) e (7.51b)
Θ̃ = C cos(nπx) sin(mπy) e λt . (7.51c)

Inserting this into (7.50) yields the solvability condition

λ2 R n2
+ λπ2 (n2 + m2 )(1 + 1/Pr) + π4 (n2 + m2 )2 − 2 =0 (7.52)
Pr n + m2
for the growth rate λ of the disturbances. For R > 0, the polynomial (7.52) possesses
two real roots, and one of them is always negative. The other has a zero for
π4 (n2 + m2 )3
R cmn = . (7.53)
n2
Hence the solution (7.49) becomes unstable ﬁrst with the mode m = n = 1 as soon as
c
R ≥ R c = R11 = 8π4 ≈ 779 .

7.3.3.4 Frozen time analysis

Now we turn to the fully time dependent diffusion front (7.43) and examine its linear
stability. Numerical solutions of the full problem (Figure 7.13) reveal that the front be-
comes unstable rather soon and the upper boundary y = L can be shifted to inﬁnity.
Hence we can treat the problem in the half space

0≤y≤∞

and the scaling length L used above is no longer of relevance. Instead, we take now
the space and time scaling
ℓ2
(x, y) = (ℓx,̃ ℓy)̃ , τ=
κ

9 This leads for v⃗ to the free boundary conditions that actually could be only valid at a plane and free
surface.
7.3 Fluid mechanics: ﬂow of an incompressible liquid | 217

with
κν 1/3
ℓ=( )
gαT0
and obtain a system similar to (7.45) but now only with one parameter, the Prandtl
number, left:
1
[∂ t ξ + v x ∂ x ξ + v y ∂ y ξ ] = ∆ξ + ∂ x Θ (7.54a)
Pr
ξ = −∆Ψ (7.54b)

∂ t Θ + v⃗ ⋅ ∇Θ = ∆Θ . (7.54c)

Performing a linear stability analysis about the (scaled) base state (7.43)

T (0) (y, t) = 1 − erf(y/2√t) (7.55)

results in a linear system with an explicit time dependence. The trick of the ‘frozen
time’ analysis, or approximation, is to decouple the time scales of the base state and
that of its disturbances. Thus we may consider the time in (7.55) as ‘frozen’ and call
it t∗ , which brings an additional control parameter into play. Substituting

Ψ = Φ(z) e ikx eλt , Θ = T (0) (y, t∗ ) + ikϑ(z) e ikx eλt (7.56)

in (7.54) yields after linearization the generalized eigenvalue problem

λ
(k 2 Φ − Φ󸀠󸀠 ) = −Φ󸀠󸀠󸀠󸀠 + 2k 2 Φ󸀠󸀠 − k 4 Φ − k 2 ϑ
Pr (7.57)
λϑ = ϑ󸀠󸀠 − k 2 ϑ + (∂ y T (0) ) Φ

with
1 y2
∂ y T (0) = − e− 4t∗
√πt∗
with primes denoting y-derivatives. The remaining task is to determine the critical
time t∗c for which the largest eigenvalue λ becomes positive and the perturbations (7.56)
may start to grow exponentially. This can be achieved by discretizing (7.57), e.g., with
finite differences and solving the linear system by a standard routine as DGGEV, for
instance; see Figure 7.15. Again, the infinite y-domain must be approximated by a finite
length L, which however, must be large compared with the size of the diffusion front
(7.55), thus
2√t∗ ≪ L
must be checked self-consistently.
If we use the realistic values for a room as stated in Section 7.3.3.2 and leading to
R = 7 ⋅ 1010 , we find ℓ ≈ 0.73 mm and τ ≈ 0.026 s. Thus, the front would be unstable
almost immediately at t c ≈ 0.08 s and the typical wave length of the growing vortices
is estimated with Λ = 2π/k c ≈ 2.5 cm.
218 | 7 Partial differential equations II, applications

stable
t*c

k
Fig. 7.15: Stability chart found numerically by solving (7.57). Modes above the zero line λ = 0 (bold)
are unstable. Instability occurs ﬁrst at t c∗ ≈ 3.2 with k c ≈ 0.18.

7.3.4 Thermal convection: (B) Rayleigh–Bénard convection

The Rayleigh number computed from (7.46) for a normal room is larger than the crit-
ical one by a factor of about 108 . Since R ∼ L3 this changes if one goes to the ‘lab-
oratory scale.’ Taking for instance L = 1 cm and instead of air a 50 cs silicone oil
ν = 5 ⋅ 10−5 m/s2 , α = 0.7 ⋅ 10−3 1/K, a Rayleigh number of
R ≈ 1400
results for a vertical temperature difference of one degree, a value rather close to the
critical one.

7.3.4.1 Linear stability analysis

The seminal experiments devised by H. Bénard¹⁰ at the beginning of the 20th century
were performed in a rectangular geometry with¹¹
Lx ≫ Ly .

10 Henri C. Bénard, French physicist, 1874–1939.

11 For details see e.g., the monograph by E. L. Koschmieder, Bénard Cells and Taylor Vortices, Cam-
bridge University Press (1993).
7.3 Fluid mechanics: ﬂow of an incompressible liquid | 219

Fig. 7.16: Sketch of typical convection cells in a liquid

heated uniformly from below for R slightly above R c , cor-
responding to the mode (7.58) with m = 1, k = k c . The
distance of two rolls is πL y /k c ≈ 1.4L y for idealized free
boundary conditions and ≈ L y for realistic no-slip bound-
aries.

Hence, lateral boundaries play a negligible role and the linearized system is instead
of with (7.51), better solved by a plane wave ansatz of the form

ξ = A sin(mπy) e ikx eλt (7.58a)

ikx λt
Ψ = B sin(mπy) e e (7.58b)
Θ̃ = C sin(mπy) e ikx eλt . (7.58c)

Periodic lateral boundary conditions are fulﬁlled for certain wave numbers
2π
kn = n.
Lx
For large L x , k can be considered continuous, and we can leave the index n. Periodic
structures in x direction with wave length Λ = 2π/k are obtained, as in Figure 7.16.
With (7.58) one ﬁnds instead of (7.53) the relation
(k 2 + m2 π2 )3
R cm (k) = (7.59)
k2
from which the minimal R c for m = 1 and k = k c = π/√2 is computed as
27π4
Rc = ≈ 658 .
4

7.3.4.2 Fully nonlinear problem in three room dimensions

The linear analysis provides information on the critical Rayleigh number and on the
wave number, and therefore on the size of the growing flow patterns. On the other
hand, nothing can be said on the saturation, orientation, or selection of the convec-
tion rolls beyond instability. Two perpendicular roll sets could, for instance, form a
regular square pattern, and three sets with a relative orientation of 120 degrees may
build the experimentally encountered hexagonal cells. Only a nonlinear analysis of
the complete hydrodynamic equations may answer these questions, which we shall
tackle now in three room dimensions.
Due to the vanishing divergence of v⃗ a full description of an incompressible fluid
is possible with two scalar fields, as already mentioned in Section 7.3.1:

∂y Ψ ∂x ∂z Φ
v⃗ = (−∂ x Ψ ) + (∂ y ∂ z Φ ) . (7.60)
0 −∆2 Φ
220 | 7 Partial differential equations II, applications

Here, ∆ denotes the complete 3D-Laplacian and ∆2 = ∂2xx + ∂2yy its horizontal ver-
sion. From here on, the vertical component is denoted with z. Forming the curl of the
Navier–Stokes equations (7.25a) its z-component reads:
1 1
{∆ − ∂ t } ∆2 Ψ(r,⃗ t) = − [∇ × ((v⃗ ⋅ ∇)v)]
⃗ . (7.61)
Pr Pr z
An equation for Φ is obtained by twice applying the curl, according to
1 1
{∆ − ∂ t } ∆∆2 Φ(r,⃗ t) = −R ∆2 Θ(r,⃗ t) − [∇ × ∇ × ((v⃗ ⋅ ∇)v)]
⃗ . (7.62)
Pr Pr z
For the temperature field one gets
{∆ − ∂ t } Θ(r,⃗ t) = −∆2 Φ(r,⃗ t) + (v⃗ ⋅ ∇)Θ(r,⃗ t) . (7.63)
The necessary boundary conditions for Φ and Ψ follow from the ansatz (7.60). We fo-
cus on laterally periodic boundary conditions. No-slip conditions at the lower and
upper boundaries require the vanishing of all three velocity components, leading to
Φ(r,⃗ t) = Ψ(r,⃗ t) = ∂ z Φ(r,⃗ t) = 0 , z = 0, 1 . (7.64)
For the temperature field, a general (Robin) condition applies
∂ z Θ(r, t) = ±Bi Θ(r, t) z = 0, 1 , (7.65)
with another dimensionless parameter Bi, called Biot number¹². It describes the ratio
of thermal conductivities of the boundary plates to that of the liquid. If the boundaries
are perfect conductors (metal) Bi → ∞, a bad or nearly insulating material like glass
or plastic is reflected by Bi ≪ 1.
The system (7.61–7.65) completely describes the convection of a Newtonian liquid
in the Boussinesq approximation. For the sake of simplicity we shall concentrate fur-
ther on the limit of a large, or better infinite, Prandtl number. This is a very good ap-
proximation for viscous fluids like many oils, at least not too far from threshold R c .
If Pr → ∞ (7.61) reduces to
∆∆2 Ψ(r,⃗ t) = 0 ,
which allows, in combination with the boundary conditions in (7.64), only the trivial
solution
Ψ =0.
Thus, the infinite Prandtl number case is completely described by the two scalar
fields Φ and Θ. This considerably reduces and simplifies the dynamics of the possible
structures. Near the instability, one usually observes a transient temporal behavior
leading after an initial phase asymptotically to a fixed point consisting of more or less
parallel oriented convection rolls (large Bi) or squares (small Bi). For larger values of
R further instabilities occur and more involved, partially time-dependent structures
can be seen that we shall not further discuss here.

12 Jean-Baptiste Biot, French physicist, 1774–1862.

7.3 Fluid mechanics: ﬂow of an incompressible liquid | 221

7.3.4.3 Algorithm, pseudospectral method

We summarize the equations for inﬁnite Pr in three dimensions:

∆2 ∆2 Φ(r,⃗ t) = −R ∆2 Θ(r,⃗ t) (7.66a)

{∆ − ∂ t } Θ(r,⃗ t) = −∆2 Φ(r,⃗ t) + (v⃗ ⋅ ∇)Θ(r,⃗ t) . (7.66b)

In two spatial dimensions the main task is the inversion of band matrices. In three
room dimensions another method is more suited, the so-called pseudospectral scheme.
Here, the system is split into its linear and nonlinear parts. The linear parts are treated
in Fourier space and can be inverted easily, the nonlinear ones are computed in po-
sition space. This trick avoids the expensive computation of convolution sums in
Fourier space.
We scale the position coordinates so that

0 ≤ x ≤ Lx , 0 ≤ y ≤ Ly , 0≤z≤1.

If periodic lateral boundary conditions are given, a Fourier decomposition with re-
spect to the horizontal coordinates x, y using plane waves as in Section 6.3.2 is nearby.
We apply the 2D-Fourier transforms
J L
⃗
Θ(r,⃗ t) = ∑ ∑ Θ(̃ k⃗ jℓ , z, t) e−i k jℓ x⃗
j ℓ
(7.67)
J L
̃ k⃗ jℓ , z, t) e −i k⃗ jℓ x⃗
Φ(r,⃗ t) = ∑ ∑ Φ(
j ℓ

with r ⃗ = (x, y, z) and the 2D vectors

x⃗ = (x, y) , k⃗ jℓ = (k x , k y )jℓ = (2πj/L x , 2πℓ/L y ) .

Substitution in (7.66) and discretization of the time derivative by (we leave the indices
at k)⃗
Θ(̃ k,⃗ z, t + ∆t) − Θ(̃ k,⃗ z, t)
∂ t Θ(̃ k,⃗ z, t) ≈
∆t
leads to the semi-implicit system¹³
1 ̃ k,⃗ z, t + ∆t) = 1 Θ(
( + k 2 − ∂2zz ) Θ(̃ k,⃗ z, t + ∆t) + k 2 Φ( ̃ k,⃗ z, t) − F(̃ k,⃗ z, t) (7.68a)
∆t ∆t
2
(∂2zz − k 2 ) Φ( ̃ k,⃗ z, t + ∆t) + R Θ(̃ k,⃗ z, t + ∆t) = 0 (7.68b)

with F(̃ k,⃗ z, t) as the 2D-Fourier transform of the nonlinearity in (7.66b)

F(r,⃗ t) = (v⃗ ⋅ ∇)Θ(r,⃗ t) .

13 We replace ∆ 2 with −k2 , which is strictly speaking only correct for L x , L y → ∞, see Section 6.3.2.
222 | 7 Partial differential equations II, applications

Discretizing z by ﬁnite differences

z = z n = (n − 1)∆z , n = 1...N , ∆z = 1/(N − 1)

(7.68) can be formulated as an inhomogeneous system of equations

M k Q(⃗ k,⃗ t + ∆t) = P(⃗ k,⃗ t) (7.69)

with the 2N-component vectors

Q(⃗ k,⃗ t) = (Θ̃ 1 (k,⃗ t), Φ̃ 1 k,⃗ t) , . . . , Θ̃ N (k,⃗ t), Φ̃ N (k,⃗ t))
(7.70)
P(⃗ k,⃗ t) = (Θ̃ 1 (k,⃗ t)/∆t − F̃ 1 (k,⃗ t), 0 , . . . , Θ̃ N (k,⃗ t)/∆t − F̃ N (k,⃗ t), 0)

where the node values are abbreviated as

Θ̃ n (k,⃗ t) = Θ(̃ k,⃗ z n , t)

and the same for Φ̃ and F.̃

The boundary conditions at z = 0, 1 must be incorporated into the 2N × 2N mesh
point matrix M k . Due to the alternate sorting of the node values Θ̃ n und Φ̃ n , the ma-
trix M k is a band matrix with four upper and four lower diagonals; see Table 7.1. The
detailed listing can be found in [3].

7.3.4.4 Results
(A) Linear analysis: The critical Rayleigh number R c where the liquid starts to move
and patterns are formed depends on the Biot number. To determine R c we perform a
linear stability analysis of (7.68) just removing the quadratic terms F̃ in (7.68a). To ﬁnd
the critical point we put ∂ t Θ = 0, i.e., we cancel in M k the terms including ∆t. Writing
in (7.68b) the expression with R on the right-hand side, (7.69) can be formulated as a
generalized eigenvalue problem

M 󸀠k w⃗ = −R D w⃗ (7.71)

where
{1 for i even and j = i − 1
D ij = { .
0 else
{
The system (7.71) is then solved numerically by a suitable LAPACK routine (e.g.,
SGGEV). Figure 7.17 shows the smallest eigenvalue corresponding to the critical R
needed for the onset of convection, both for a ﬁnite and an inﬁnite Biot number and
depending on k.
Tab. 7.1: The band matrix M k has the band width 9.

Ak − a k2 −1/∆z2 0 0 0 ...
R B k + 1/∆z4 0 Ck 0 1/∆z4 0 0 ...
(−1/∆z2
( 0 Ak k2 −1/∆z2 0 0 ... )
)
( 0 Ck R Bk 0 Ck 0 1/∆z4 0 0 ... )
( )
( )
( 0 0 −1/∆z2 0 Ak k2 −1/∆z2 0 0 ... )
( )
( 0
( 1/∆z4 0 Ck R Bk 0 Ck 0 1/∆z4 0 ... )
)
( ... | | )
( )
Mk = ( )
( ... | ------- bandwidth ------- | )
( )
( ... )
( )
( )
( ... )
( )
( ... )
( )
( ... )
... 0 −1/∆z2 0 Ak − a k2
( ... 1/∆z4 0 Ck R B k + 1/∆z4 )

with the abbreviations:

1 2 1 4k 2 6 2k 2 4
Ak = + k2 + 2 , a= , B k = k4 + + , Ck = − −
∆t ∆z ∆z2 (Bi∆z + 1) ∆z2 ∆z4 ∆z2 ∆z4
7.3 Fluid mechanics: ﬂow of an incompressible liquid |
223
224 | 7 Partial differential equations II, applications

3000

Bi → ∞
2000
R

1000
Bi = 0.5
Fig. 7.17: Critical Rayleigh number where con-
0 vection sets in. Both R c and k c decrease with Bi.
1 2 3 4 5 In particular one ﬁnds R c ≈ 1700 for Bi → ∞,
k and R c ≈ 900 for Bi = 0.5.

Fig. 7.18: Temperature ﬁeld at the midplane z = 1/2 at several times. Left row: Bi → ∞,
R = 1900, t = 75 (top), 150 (bottom), right row: Bi = 0.5 R = 1000, t = 75 (top), 1000 (bottom).
The patterns of the bottom frames are not yet stationary. For t → ∞ one obtains parallel rolls (left)
or a perfect square pattern (right), respectively.
7.4 Pattern formation out of equilibrium | 225

(B) Nonlinear analysis: Figure 7.18 shows the temperature distribution along the mid
plane of the layer, computed by a numerical simulation of (7.68). As an initial condi-
tion, a randomly distributed temperature ﬁeld

Θ ijk (t = 0) = r ijk , −0.1 ≤ r ijk ≤ 0.1

has been applied. The grid consists of 128 × 128 × 20 mesh points, the step sizes are

∆t = 0.05 , ∆x = 0.2 .

From experiment as well as from theory one knows that for small Biot numbers
squares are encountered, for larger ones rolls (or stripes). The hexagons observed
in Bénard’s original work result only if the surface is free and surface tension as a
function of temperature is considered in addition. This is called the Marangoni effect
which we cannot discuss here in more detail.

7.4 Pattern formation out of equilibrium

7.4.1 Reaction-diffusion systems

Based on a system of chemical reactions with K reactants, from the law of mass action
one can systematically derive the reaction equations in the form of a set of nonlinear,
ordinary differential equations for the concentrations w i (t) of the reactant i:

d t w i = f i (w1 , . . . w K , p) , i = 1...K . (7.72)

Here, p conﬁnes a set of externally ﬁxed control parameters as pressure, temperature,

or certain other concentrations that are kept constant. If in addition one allows for spa-
tial diffusion of the reactants, the concept has to be extended to spatially depending
concentration ﬁelds w i (r,⃗ t). Instead of (7.72) one obtains a system of coupled nonlin-
ear PDEs, the so-called reaction-diffusion equations:

∂ t w i = D i ∆w i + f i (w1 , . . . w K , p) , i = 1...K (7.73)

where D i denotes the diffusion constant of species i.

In animate and in inanimate nature one often encounters periodic or nearly pe-
riodic patterns, Figure 7.19 [6]. A. Turing¹⁴ [7] theoretically examined equations of the
form (7.73) describing a hypothetical chemical reaction that should evolve during the
embryonic phase of certain animals. After a spatial structure of reactants is formed
these reactants should drive the pigment cells and account for the coloring of coat,
skin, or scales [8]. Turing recognized that a minimal model of only two equations
(K = 2) is sufficient to obtain spontaneous pattern formation in the form of periodic
structures in space if the two diffusion constants are within a certain range.

14 Alan Turing, British mathematician, computer scientist, and theoretical biologist, 1912–1954.
226 | 7 Partial differential equations II, applications

Fig. 7.19: Following Alan Turing, coat, skin, or scale patterns on certain animals are created in their
embryonic phase by a chemical nonequilibrium reaction. Randomly placed defects and dislocations
in the otherwise rather regular structures seem to conﬁrm Turing’s hypothesis. Courtesy of Marlies
Bestehorn.

7.4.1.1 Turing instability

Hence we shall further concentrate on the model (7.73) with K = 2 and start with one
spatial dimension:

∂ t w1 (x, t) = D1 ∂2xx w1 (x, t) + f1 (w1 , w2 ) (7.74a)

∂ t w2 (x, t) = D2 ∂2xx w2 (x, t) + f2 (w1 , w2 ) . (7.74b)

We assume that a spatially uniform ﬁxed point exists determined by

f1 (w01 , w02 ) = 0 , f2 (w01 , w02 ) = 0 . (7.75)

Expanding the f i in (7.74) up to the linear order we obtain for the deviations u i (x, t)
from the ﬁxed point w0i the system

∂ t u 1 (x, t) = D1 ∂2xx u 1 (x, t) + a1 u 1 (x, t) + b 1 u 2 (x, t) (7.76a)

∂ t u 2 (x, t) = D2 ∂2xx u 2 (x, t) + a2 u 2 (x, t) + b 2 u 1 (x, t) . (7.76b)

Here we have introduced the abbreviations (Jacobi matrix elements)

∂f1 󵄨󵄨󵄨󵄨 ∂f1 󵄨󵄨󵄨󵄨 ∂f2 󵄨󵄨󵄨󵄨 ∂f2 󵄨󵄨󵄨󵄨
a1 = 󵄨󵄨 , b 1 = 󵄨󵄨 , a2 = 󵄨󵄨 , b 2 = 󵄨 . (7.77)
∂w1 󵄨󵄨w0 ∂w2 󵄨󵄨w0 ∂w2 󵄨󵄨w0 ∂w1 󵄨󵄨󵄨w0
The ansatz
u j (x, t) = v j exp(λt + ikx) (7.78)
7.4 Pattern formation out of equilibrium | 227

turns (7.76) into the linear eigenvalue problem

(M − λ 1) v⃗ = 0 (7.79)

with the 2 × 2 matrix

−D1 k 2 + a1 b1
M=( ) .
b2 −D2 k 2 + a2

The solvability condition for (7.79) provides the characteristic polynomial

λ2 + λ(D1 k 2 + D2 k 2 − a1 − a2 ) + (D1 k 2 − a1 )(D2 k 2 − a2 ) − b 1 b 2 = 0 . (7.80)

Instead of solving (7.80) for λ we may also ask for parameter settings a i , b i where a sta-
ble (old) state (λ < 0) becomes unstable (λ > 0). Hence the critical point is determined
by λ = 0. From (7.80) we ﬁnd

(D1 k 2 − a1 )(D2 k 2 − a2 ) − b 1 b 2 = 0 ,

or, solved for a1 :

b1 b2
a1 = D1 k 2 − . (7.81)
D2 k 2 − a2
Let a1 > 0. Since the two diffusion constants D i are surely positive, this can only be
valid for all k if
b 1 b 2 < 0 , und a2 < 0
holds. Now consider a1 as a control parameter that can be increased from outside by
varying e.g., temperature or pressure. The mode becoming unstable at ﬁrst then has
a wave number k = ±k c that minimizes (7.81), (a1 (k c ) = a1c ). The zero of the derivative
of (7.81) with respect to k yields

a2 b1 b2 1 a1 a2
k 2c = + √− = ( + ) . (7.82)
D2 D1 D2 2 D1 D2

For a1 > a1c the ﬁxed point w0i becomes unstable and the mode (7.78) with k = k c
grows exponentially in time. This spatially periodic structure is nowadays called Tur-
ing instability or Turing structure.

7.4.1.2 Hopf instability

To compute the critical point we simply assumed λ = 0 in (7.80). But there is another
way for an instability to occur: λ may be complex valued and the mode oscillates in
time. Then one has either a damped (old, stable state) or an exponentially increasing
oscillation, depending on the sign of the real part of the eigenvalue λ. Thus the critical
point is now determined by Re(λ) = 0 or

λ = iω (7.83)
228 | 7 Partial differential equations II, applications

with a real valued frequency ω. Inserting (7.83) into (7.80) and separating real and
imaginary parts one obtains the two equations

ω2 = (D1 k 2 − a1 )(D2 k 2 − a2 ) − b 1 b 2 (7.84a)

0 = (D1 + D2 )k 2 − a1 − a2 . (7.84b)

Equation (7.84b) determines the critical point depending on the wave number:

a1 = (D1 + D2 )k 2 − a2 . (7.85)

Obviously the mode with k = 0 becomes unstable at ﬁrst as soon as

a1 ≥ −a2 .

Hence the oscillatory mode comes with a long-wave, even homogeneous, spatial struc-
ture, corresponding to
kc = 0 . (7.86)
As already noted in Section 3.2, instabilities belonging to a pair of conjugate complex
eigenvalues are also denoted as Hopf instabilities, and the frequency at the critical
point is named Hopf frequency. It follows from (7.84a) with k = 0:

ω c = √a1 a2 − b 1 b 2 = √−a21 − b 1 b 2 . (7.87)

To obtain a real valued frequency, b 1 b 2 < −a21 must hold. Let b 1 < 0 and b 2 > 0. Then
the substance w1 creates via the coupling b 2 the substance w2 , while w2 diminishes
w1 via b 1 . This is why (7.74) is also denoted as an activator-inhibitor system with w1 as
the concentration of the activator and w2 as that of the inhibitor.
Temporal oscillations of a spatially uniform state were ﬁrst experimentally ob-
served by Boris Belousov in 1950 and are now named Belousov–Zhabotinskii reac-
tions.

7.4.1.3 Codimension-two
But what happens now if one of the control parameters, say a1 , is increased over the
threshold? The uniform stationary state becomes unstable, but how so exactly is de-
termined by the other parameters. Depending on the values of a1 , b 1 , and b 2 as well
as on the diffusion constants, the homogeneous state will give way either to a time pe-
riodic (Hopf) or to a space periodic (Turing) structure. From mathematics, the notion
codimension is familiar. The codimension of an object is deﬁned as the difference of
the dimension of the room in which the object lives and the dimension of the object
itself. A plane in the three-dimensional space possesses the codimension 3 − 2 = 1.
Instability conditions like (7.81) or (7.85) determine the value of one parameter related
to the others. They deﬁne an n −1-dimensional subspace of the n-dimensional param-
eter space and therefore have the codimension one.
7.4 Pattern formation out of equilibrium | 229

Fig. 7.20: Sketch of Turing and Hopf instability for the two-equation system (7.76). If increasing a1
and a2 < a2CD2 , the Turing instability comes ﬁrst (a), if a2 > a2CD2 the Hopf instability occurs directly
above threshold (b). For (c), a2 = a2CD2 .

But now we can adjust the parameters in such a way that the Turing and Hopf modes
are unstable at the same a1 . This additionally restricts the parameter setting into an
n − 2-dimensional subspace. A ‘double’ instability of this kind is named codimension-
two instability (CD2); see also Figure 7.20.
Inserting the critical wave number (7.82) into (7.81), one ﬁnds the smallest neces-
sary value for a1 to get a Turing instability:

D1 D1
a1T = a2 + 2√−b 1 b 2 . (7.88)
D2 D2
In the same way one obtains for the oscillatory instability from (7.85) with (7.86)

a1H = −a2 . (7.89)

Equating the two expressions ﬁnally leads to a condition for a2 :

√−b 1 b 2 D1 D2
a2CD2 = −2 , a1CD2 = −a2CD2 . (7.90)
D1 + D2
This value is denoted as codimension-two-point. Obviously, for a2 > a2CD2 the Hopf
mode becomes unstable ﬁrst; for a2 < a2CD2 the Turing mode is observed at threshold.
For a2 ≈ a2CD2 one expects the simultaneous emergence of both modes and a particu-
larly rich spatio-temporal behavior of the solutions of the fully nonlinear equations.

7.4.1.4 The Brusselator

During the 60s, a model based on a hypothetical chemical reaction scheme was de-
vised by Prigogine¹⁵ and coworkers at the University of Brussels. The model was moti-

15 Ilya Prigogine, Russian–Belgian physical chemist, 1917–2003.

230 | 7 Partial differential equations II, applications

vated by the chemical oscillations discovered about ten years before by Belousov¹⁶ and
consisted in its original form of two coupled ODEs (K = 2). Later, diffusion terms were
added and spatio-temporal pattern formation was examined extensively. The system
has the form of (7.74) and may show, depending on parameters, either Hopf or Turing
instabilities.

(A) Equations and linear analysis. For the Brusselator one ﬁnds

f1 = A − (B + 1) w1 + w21 w2 , f2 = B w1 − w21 w2 , (7.91)

and the only ﬁxed point is given by

w01 = A , w02 = B/A .

Since there are only two free parameters (A, B), the four coefficients a i , b i of the Ja-
cobian (7.77) are not independent from each other and read

a1 = B − 1 , b1 = A2 , a2 = −A2 , b 2 = −B . (7.92)

If A is considered to be ﬁxed and B is deﬁned as a variable control parameter, the

relations (7.81, 7.85) can be solved for B:

(T) A2 + (D1 A2 + D2 )k 2 + D1 D2 k 4
Bc =
D2 k2
(H)
Bc =(D1 + D2 )k 2 + A2 + 1 . (7.93)

From (7.82) we obtain for the Turing mode the critical wave number
A
k 2c = . (7.94)
√D1 D2

The conditions (7.88, 7.89) turn into

2
(T) D1 (H)
B =
T
B c (k c ) = (A√ + 1) , B H = B c (0) = A2 + 1 , (7.95)
D2

from which one ﬁnds the codimension-two point by equating B T = B H :

2√D1 D2 (D1 + D2 )2
A CD2 = , B CD2 = . (7.96)
D2 − D1 D2 − D1
Figure 7.21 shows the parameter plane for D1 = 1/2, D2 = 1. Depending on A, both
instability types are possible.

16 Boris P. Belousov, Russian chemist, 1893–1970.

7.4 Pattern formation out of equilibrium | 231

Fig. 7.21: Parameter plane for the Brusselator

with D2 = 2D1 . On the left-hand side of the
dashed line the homogeneous state becomes
oscillatory unstable; on the right-hand side
one observes Turing patterns.

(B) Semi-implicit pseudospectral method. Next we wish to integrate numerically the

fully nonlinear system applying a semi-implicit pseudospectral scheme as already
done in Section 7.3.4.3. Chemical reactions are often performed on thin gels and a two-
dimensional representation is natural. It is of further advantage to transform the basic
system to the deviations u i from the ﬁxed point

u 1 = w1 − A , u 2 = w2 − B/A

as in (7.76). For u i = u i (x, y, t) one derives the system

∂ t u 1 = D1 ∆u 1 + (B − 1) u 1 + A2 u 2 + f(u 1 , u 2 ) (7.97a)
2
∂ t u 2 = D2 ∆u 2 − B u 1 − A u 2 − f(u 1 , u 2 ) (7.97b)

with the nonlinearity

B
f(u 1 , u 2 ) = ( + u 2 ) u 21 + 2A u 1 u 2 . (7.98)
A
Similar to (7.69), the semi-implicit formulation leads to an inhomogeneous system of
equations
1 1
M k u⃗ k (t + ∆t) = u⃗ k (t) + f k (u 1 (t), u 2 (t))( ) (7.99)
∆t −1
with
u⃗ k = (u k1 , u k2 )
and u k , f k as the Fourier transform of u and f . Since the third dimension is absent,
M k is here a simple 2 × 2-matrix
1
∆t + D1 k2 + 1 − B −A2
Mk = ( ) , (7.100)
B 1
∆t + D2 k2 + A2

which can be inverted explicitly:

∆t + D 2 k + A
1 2 2
1 A2
M −1
k = ( ) .
Det M k −B 1
+ D1 k2 + 1 − B
∆t
232 | 7 Partial differential equations II, applications

t = 200 t = 200

Fig. 7.22: Depending on the value of A, for the plane (2D) Brusselator either (asymptotically station-
ary) Turing structures (left, A = 0.8), or spatially large-scaled temporally oscillating Hopf patterns
(right, A = 0.35) emerge. For D1 /D2 = 0.1 it follows that A CD2 ≈ 0.7.

t = 15 t = 26

t = 34 t = 50

Fig. 7.23: Spreading fronts can be seen for other parameters in the oscillatory region. They are sim-
ilar to the chemical nonequilibrium patterns observed experimentally in the Belousov–Zhabotinsky
reaction. Time series for D1 /D2 = 0.5, A = 0.2, B = 1.2.
7.4 Pattern formation out of equilibrium | 233

(C) Results. We begin with D1 = 0.1, D2 = 1. For A above the codimension-two-

point A CD2 ≈ 0.7 we obtain Turing patterns mostly in the form of more or less regular
hexagons, including some point and line defects. Below the CD2-point, large-scaled
regions in space oscillating with about the Hopf frequency can be seen, in agreement
with the linear theory, as in Figure 7.22.
Closer to the CD2-point a complex interchange of Turing and Hopf modes is ob-
served; compare also Figure 1.3 from the introduction. Qualitatively different struc-
tures, namely the experimentally known target or spiral patterns, are found for less
differing diffusion constants, for instance D1 = 0.5, D2 = 1, and much farther in the
Hopf region, A = 0.2, B = 1.2; see Figure 7.23.
We conclude this section presenting a MATLAB code for the semi-implicit pseu-
dospectral scheme based on the built-in 2D fast-Fourier-transform FFT2.

clear
n=512; % mesh points
% parameter input
d=input('d1/d2? ');
acd2=2.*sqrt(d)/(1.-d) % cd-2 point
a=input('value for a? ');
bh=1.+a^2 % threshold Hopf mode
bt=(1.+a*sqrt(d))^2 % threshold Turing mode
b=input('value for b? '); ba=b/a;

xkc=sqrt(a/sqrt(d)); dx=2.*pi/xkc/15.; dx2=dx^2; dt=0.1; % step sizes

dkx2=(2.*pi/n/dx)^2;

for k=1:n; % coefficients of M

for j=1:n;
xk2=((k-n/2)^2+(j-n/2)^2)*dkx2;
det=(1./dt+d*xk2+1.-b)*(1./dt+xk2+a^2)+b*a^2;
m22(k,j)=(1./dt+d*xk2+1.-b)/det;
m11(k,j)=(1./dt+xk2+a^2)/det;
m12(k,j)=a^2/det;
m21(k,j)=-b/det;
end
end
m11=ifftshift(m11); m12=ifftshift(m12);
m22=ifftshift(m22); m21=ifftshift(m21);

u1=0.2(rand(n)-.5); u2=0.2(rand(n)-.5); % initial conditions

234 | 7 Partial differential equations II, applications

t=0; tend=1000; n=0;

for t=0:dt:tend; n=n+1; % time loop
fn=(u1.^2).*(ba+u2)+2.*a*u1.*u2; % nonlinearity
rhs1=u1/dt+fn; rhs2=u2/dt-fn; % rhs in real space

fs1=fft2(rhs1); fs2=fft2(rhs2); % Fourier transform

rhs1=m11.*fs1+m12.*fs2; % comp. in Fourier space
rhs2=m21.*fs1+m22.*fs2;
u1=real(ifft2(rhs1)); u2=real(ifft2(rhs2)); % back to real space

if(n==100) % plotting
n=0; contour(u1); drawnow; t
end
end

7.4.2 Swift–Hohenberg equation

7.4.2.1 Motivation
Turing structures originate from instabilities where the fastest growing, most unstable
modes have a ﬁnite wave number k c . If we evaluate for instance (7.80) for the Brusse-
lator, a growth rate λ(k 2 ) is obtained in the super-critical domain B > B c as shown in
Figure 7.24. The function λ(k 2 ) seems to be rather involved, including a square root. If

λ
0.01
rg
be
en

k2
oh

0
−H

2 2.5 3 3.5
ift
Sw

–0.01
r
lato
sse

k2c
–0.02
Bru

–0.03

–0.04

Fig. 7.24: The basic idea behind the Swift–Hohenberg equation is to expand the eigenvalue λ(k2 ) of
a Turing instability, here for the Brusselator, in the neighborhood of the critical point, leading to the
polynomial (7.101). Parameters: D1 = 0.1, D2 = 1, A = 0.8, B c = 1.57, B = 1.01B c .
7.4 Pattern formation out of equilibrium | 235

we restrict ourselves only to the region of the unstable (linearly growing) modes, we
may expand λ at B c , k 2c , according to:

λ ≈ a(B − B c ) − b(k 2 − k 2c )2 (7.101)

with 󵄨
∂λ 󵄨󵄨󵄨󵄨 1 ∂2 λ 󵄨󵄨󵄨
a= 󵄨 , b=− 󵄨󵄨 .
∂B 󵄨󵄨󵄨B c ,k2c 2 ∂(k ) 󵄨󵄨󵄨B c ,k2c
2 2

In position space one may simply replace k 2 with −∆ and obtain a linear PDE whose
solution reﬂects the instability behavior (7.101):

∂ t Ψ(x, y, t) = a(B − B c )Ψ(x, y, t) − b(∆ + k 2c )2 Ψ(x, y, t) . (7.102)

After appropriate scaling of time and space, (7.102) takes the form

∂ t Ψ(x, y, t) = εΨ(x, y, t) − (∆ + 1)2 Ψ(x, y, t) (7.103)

with the sole control parameter ε ∼ B − B c . To account for a nonlinear saturation of

the linearly unstable modes one adds to (7.103) the most simple terms that guarantee
global stability (bounded solutions):

∂ t Ψ(x, y, t) = εΨ(x, y, t) − (∆ + 1)2 Ψ(x, y, t) + A Ψ 2 (x, y, t) − Ψ 3 (x, y, t) . (7.104)

Therewith we have derived the Swift–Hohenberg equation [9], nowadays a standard

model of pattern formation. It describes the spatio-temporal pattern evolution close to
the critical point of a (monotonic) Turing instability¹⁷. A similar ‘normal form’ can be
derived for an oscillatory (Hopf) bifurcation and yields a complex Ginzburg–Landau
equation; see Problems.
The Swift–Hohenberg equation can be formulated as a gradient system:
δL[Ψ]
∂t Ψ = − . (7.105)
δΨ
Here, δ/δΨ denotes the functional derivative and L is the functional
1 2A 3 1 4
L[Ψ] = − ∫ dx dy {εΨ 2 − [(∆ + 1)Ψ]2 + Ψ − Ψ } , (7.106)
2 3 2
also called the Lyapunov functional. As one easily shows it follows from (7.105) that

dt L ≤ 0 (7.107)

and L decreases monotonically as long as a stationary state (minimum or secondary

minimum of L) is reached asymptotically. The property (7.107) can be used as a check
as well as a stop criterion for any numerical method.

17 The original work was for A = 0. The extension with a square term was ﬁrst proposed and studied
by H. Haken [6].
236 | 7 Partial differential equations II, applications

7.4.2.2 Algorithm
For periodic boundary conditions it is easy to construct a pseudospectral scheme as
already done in Section 7.4.1.4. Since we now have only one equation to iterate, the
matrix inversion turns into a simple division in Fourier space:
f k (Ψ(t))
Ψ k (t + ∆t) = (7.108)
1/∆t − ε + (1 − k 2 )2
with the Fourier transform f k of
Ψ
f(Ψ) = + A Ψ2 − Ψ3 .
∆t
Another possibility is the discretization in position space with ﬁnite differences. Let
N be the number of mesh points in each direction (for simplicity we assume a square
geometry). We organize the node values in a one-dimensional vector Φ k , k = 1 . . . N 2
as in (7.23) and formulate a large (N 2 ) system of equations
N2
Φ i (t)
∑ M ij Φ j (t + ∆t) = + A Φ2i (t) − Φ3i (t)
j
∆t

where M has to be inverted. Of course M is a rather large matrix with (N 2 )2 elements

but most of them are zero and special routines for sparse matrices can apply.
MATLAB is suited to treat sparse problems. A simple code could look like that:

clear;
n=200; dx=0.5; dx2=dx^2; dt=3; tend=1000; eps=0.1; a=0.;
lap=-gallery('poisson',n); lap2=lap*lap; % Laplace and biharmonic FD in 2D
pot=speye(n^2)+lap/dx2; % matrix needed for Lyapunov potential
m=(1/dt-eps+1)*speye(n^2)+2*lap/dx2+lap2/dx2^2; % matrix M
psi=zeros(n);

phi=rand(1,n^2)-0.5; % random initial values

k=0;
for t=0:dt:tend % time loop till tend
k=k+1;
phi=(phi/dt-phi.^3+a*phi.^2)/m; % matrix inversion
lyap=eps*phi.^2-.5*phi.^4+2./3.*a*phi.^3-(phi*pot).^2; % potential
lyap_plot(k)=-0.5*dx2*sum(lyap); t_plot(k)=t; % plot values
7.4 Pattern formation out of equilibrium | 237

5 200

0 150

–5 100

–10 50

–15
0 50 100 150 200 50 100 150 200

Fig. 7.25: Numerical solution of the Swift–Hohenberg equation at t = 200 for ε = 0.1, A = 0 (right)
and Lyapunov functional (7.106) over t (left), computed on a 200 × 200 grid and for a random-dot
initial condition.

10 200

0 150

–10 100

–20 50

–30
0 50 100 150 200 50 100 150 200

Fig. 7.26: The same as in Figure 7.25 but for A = 0.5.

if(mod(k,10)==0) % plott each 10 time steps

psi(:)=phi; % compute 2D Psi from Phi
subplot(1,2,1); plot(t_plot,lyap_plot)
subplot(1,2,2); contourf(psi)
drawnow; t
end
end

It produces output as shown in Figures 7.25, 7.26. After t = 200 the depicted patterns
are not completely stationary but still evolving towards their ﬁxed points. Performing
a linear analysis around a perfectly regular stripe solution, it turns out that at a critical
value
√3ε
Ac =
2
238 | 7 Partial differential equations II, applications

stripes become unstable and give way to hexagons already found for the Brusselator. If
the symmetry Ψ → −Ψ is broken, i.e., A ≠ 0, one always ﬁnds hexagons directly above
threshold ε = 0 since A c = 0. Thus, hexagons are the typical (generic) structure for a
Turing instability with no further symmetry restrictions. As a secondary instability for
larger ε stripes, or for more involved nonlinearities, squares may also occur.

Boundary conditions. The Swift–Hohenberg equation is of the fourth order in the

spatial derivatives and thus needs two independent boundary conditions along each
side. Taking the gallery MATLAB call, the matrix ‘lap’ contains the 2D Laplace FD-
representation of Figure 6.8a, cut at the boundaries. This corresponds to Ψ = 0 along
all sides. The biharmonic operator ‘lap2’ in the form of Figure 6.9 is obtained by squar-
ing the matrix ‘lap.’ Then a second boundary condition is automatically built in of the
form of ∆Ψ = 0, corresponding together with the ﬁrst condition to ∂ ξξ Ψ = 0 where ξ
stands for the variable perpendicular to the boundary.
Very often, the Swift–Hohenberg equation is solved with the boundary conditions

Ψ =0, n̂ ⋅ ∇Ψ = 0 (7.109)

with n̂ normal to the boundaries. Conditions of the form (7.109) can be realized by
modifying the diagonal elements of the matrix ‘lap2’ that belong to the points next to
the four sides, according to

for j=1:n:n^2
j1=j+n-1;
lap2(j,j)=lap2(j,j)+2; lap2(j1,j1)=lap2(j1,j1)+2;
end
for j=1:n
j1=j-n^2-n-1
lap2(j,j)=lap2(j,j)+2; lap2(j1,j1)=lap2(j1,j1)+2;
end

This code section must be included in the code on page 236 before computing the
matrix ‘m’ in line 5. The modiﬁcation of the diagonal elements can be understood by
assuming a ‘virtual line’ outside of the integration domain, as explained in more detail
in Section 6.2.2.1.

7.4.3 Problems

Similar to the derivation in Section 7.4.2.1, a model equation valid for the weakly non-
linear case close to an oscillatory (Hopf) instability can be found. This equation is also
named the complex Ginzburg–Landau equation and reads:

∂ t Ψ(x, t) = [ε + iω0 + (1 + ic1 ) ∂2xx ] Ψ(x, t) − (1 + ic3 )|Ψ(x, t)|2 Ψ(x, t) (7.110)
Bibliography | 239

where ε, ω0 , c i ϵ R and Ψ ϵ C. Assume periodic boundary conditions in the domain

0 ≤ x ≤ L:
Ψ(x = 0, t) = Ψ(x = L, t) .
1. Without loss of generality, the frequency ω0 can be put to zero. Why? Show that
for ε > 0 an exact solution of (7.110) reads

Ψ = A exp(iΩt) (7.111)

with A, Ω ϵ R. Determine A and Ω.

2. Examine the stability of the solution (7.111). Show that it becomes unstable if

c1 c3 < −1 . (7.112)

3. Solve (7.110) for ε = 0.1 numerically using an FTCS scheme in the stable regime,
i.e., for
c1 = 1/2 , c3 = −1/2 ,
and in the unstable regime, i.e., with

c1 = 1/2 , c3 = −4 .

Take L = 800, ε = 0.1 and the step sizes ∆x = 0.8 und ∆t = 0.05. As the
initial condition, assume a random dot distribution of Ψ(x, 0) in the interval
0.9A . . . 1.1A. Represent the solution graphically in a space-time diagram.
4. If (7.112) is fulﬁlled, (7.110) possesses spatio-temporal solutions that are chaotic.
Examine them by determining their largest Lyapunov exponent Λ for different val-
ues of c3 and ﬁxed c1 = 1/2. Plot Λ as a function of c3 .

Bibliography

[1] L. D. Landau and E. M. Lifshitz, Fluid Mechanics, Vol. 6, Butterworth–Heinemann, 2nd edn.
(1987).
[2] C. A. J. Fletcher, Computational Techniques for Fluid Dynamics 2, Springer (2013).
[3] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
[4] C. H. Bruneau, The 2D lid-driven cavity problem revisited, M. Saad, Computers & Fluids 35, 326
(2006).
[5] P. N. Swarztrauber and R. A. Valent, FFTPACK5 – a FORTRAN library of fast Fourier transforms,
http://www.netlib.org/fftpack, February 11, 2018.
[6] H. Haken, Synergetics: Introduction and Advanced Topics, Springer (2012).
[7] A. M. Turing: The chemical basis of morphogenesis, Phil. Trans. R. Soc. London B 237, 37 (1952).
[8] J. D.Murray, Mathematical Biology, Springer (2008).
[9] J. Swift, P. C. Hohenberg, Hydrodynamic ﬂuctuations at the convective instability, Phys. Rev. A15,
319 (1977).
8 Monte Carlo methods (MC)
The terms ‘Monte Carlo methods’ or ‘Monte Carlo simulations’ refer to a variety of nu-
merical methods mainly based on random numbers. On the one hand these can be
problems arising from physics where ﬂuctuations or other random processes play a
central role, for example thermal motion in statistical physics, trajectories of particles
in disordered media, or equilibrium phase transitions. On the other hand, mathemat-
ical problems like the approximate computation of high-dimensional integrals or the
determination of extreme values in a high-dimensional variable space can be often
only tackled by MC methods.

8.1 Random numbers and distributions

8.1.1 Random number generator

Since random numbers play a crucial role, we wish to start with their properties and
their realization on a computer. Because all processes on computers are in principle
deterministic, one should better talk about ‘pseudorandom numbers’. A pseudoran-
dom number generator is implemented in FORTRAN95. The statement

CALL RANDOM_NUMBER(x)

assigns to the real variable x an equally distributed random number in the interval

0≤x<1. (8.1)

If x is deﬁned as a real valued array, all its elements are ﬁlled with different random
numbers.
Random number generators are based on maps of the form

x i = f(x i−1 , x i−2 , . . . x i−n ) . (8.2)

A simple, however not applicable, example with n = 1 is the logistic map somewhere
in the chaotic region, studied in Chapter 1. FORTRAN95 uses different maps with a
much larger n, producing much better and much less correlated random sequences.
The value of n thereby depends on the compiler and operating system. The actual n
can be found by

CALL RANDOM_SEED(size=n),

for GNU-FORTRAN running under Linux one obtains n = 12. At each run the very same
values (x1 . . . x n ) are initialized and as a consequence exactly the same reproducible
random series is generated. This can be changed by the call

CALL RANDOM_SEED(put=ix),

https://doi.org/10.1515/9783110515145-008
8.1 Random numbers and distributions | 241

where ix is an integer array of length n, containing the initial values for the iteration
(8.2). If one wishes to create a different sequence for each run, one can link the initial
values to the system clock. A subroutine for the initialization is given below; see also
Appendix B:

SUBROUTINE RANDOM_INIT
INTEGER, DIMENSION(100) :: ix ! auxiliary array for initial values
CALL RANDOM_SEED(size=n)
CALL SYSTEM_CLOCK(ic)
ix=ic
CALL RANDOM_SEED(put=ix(1:n))
END

8.1.2 Distribution function, probability density, mean values

The distribution function Φ(x) denotes the probability to draw a random variable X
that is less than x. For the equal (rectangular) distribution (8.1) it follows

{
{ 0, if x < 0
{
Φ r (x) = { x , if 0 ≤ x < 1 . (8.3)
{
{
{1 , if 1 ≤ x

If we restrict ourselves to the domain [0, 1), (8.3) simply reads

Φ r (x) = x , x ϵ [0, 1) .
Another name for the distribution function is cumulative probability. The probability
density function (PDF)
dΦ(x)
ρ(x) = (8.4)
dx
specifies the probability to find x in the infinitesimal region x, x + dx. For the equal
distribution one has
{
{ 0 , if x < 0
{
ρ r (x) = {1 , if 0 ≤ x < 1 . (8.5)
{
{
{0 , if 1 ≤ x
If the PDF is known, the expectation value, or mean,

⟨x⟩ = ∫ dx x ρ(x) (8.6)

as well as the variance

Var(x) = ⟨(x − ⟨x⟩)2 ⟩ = ⟨x2 ⟩ − ⟨x⟩2 = ∫ dx x2 ρ(x) − ⟨x⟩2 (8.7)

and the standard deviation

σ(x) = √Var(x) (8.8)
242 | 8 Monte Carlo methods (MC)

can be computed. The integrals thereby run over the domain of deﬁnition of the ran-
dom variable. If, on the other hand, a series x1 . . . x k of randomly distributed numbers
is given, one evaluates mean and variance as
1 k 1 k 2
⟨x⟩ = ∑ xi , Var(x) = ∑ x − ⟨x⟩2 .
k i=1 k i=1 i
Particularly for the equal distribution in [0, 1) one ﬁnds
⟨x⟩ = 1/2 , Var(x) = 1/12 .

8.1.3 Other distribution functions

Based on the equally distributed random numbers x produced by a computer, how

can we create a random number y with a different, desired probability density ρ(y)?
We shall present two methods:

8.1.3.1 Acceptance-rejection
We seek for a random variable y in the region

a≤y<b (8.9)

with given PDF ρ(y). First we draw an equally distributed x1 ϵ [0, 1) and stretch it to
the interval (8.9)
y = a + (b − a) x1 .
This value should be accepted with the probability ρ(y)/ρ m , where ρ m denotes the
maximum of ρ. To this end one determines another equally distributed number x2
and accepts y if
x2 ≤ ρ(y)/ρ m . (8.10)
Otherwise, y is rejected and the process of drawing two new numbers x1 , x2 is repeated
as long as (8.10) is fulﬁlled. The method can be readily extended to higher dimensional
distributions ρ(x, y, . . . ). If one wishes to construct a distribution having long tails,
ρ(y)/ρ m can become rather small and many trials to ﬁnd an acceptable value y can
become necessary, making the method slow and inefficient.

8.1.3.2 Transformation method

Here one directly tries to ﬁnd an (analytical) function y = f(x). Since the cumulative
probability for both PDFs must be the same,

Φ(y) = Φ r (x) ,

we may compute f(x) for a given Φ and Φ r from (8.3) according to

Φ(f(x)) = Φ r (x) = x . (8.11)

8.1 Random numbers and distributions | 243

1800

1600

1400

1200

1000

800

600

400

200

0
0 0.5 1 1.5 2 2.5 3 3.5

Fig. 8.1: Histogram of y from (8.13) for 100 000 randomly drawn numbers.

To demonstrate the method we give an example. Let us ﬁnd f for a variable y ϵ [0, π)
with the desired PDF
1
ρ(y) = sin y . (8.12)
2
y
Because Φ = ∫0 dy󸀠 ρ(y󸀠 ) = 12 (1 − cos y) we obtain from (8.11)

y = f(x) = arccos(1 − 2x) . (8.13)

The MATLAB script

n=100000;
y=acos(1-2*rand(1,n));
hist(y,100)

provides a histogram of y, Figure 8.1, where the PDF (8.12) can be identiﬁed clearly.

8.1.3.3 Special case: normal distribution

Quite often one needs normally or Gaussian distributed random numbers according
to
1
ρ(y) = exp(−(y − y0 )2 /b 2 ) , y ϵ (−∞, ∞) .
b√π
Here, y0 denotes the mean and b the width of the Gaussian distribution. The variance
is related to the width
Var(y) = b 2 /2 .
244 | 8 Monte Carlo methods (MC)

2000 3500
3000
1500 2500
2000
1000
1500

500 1000
500
0 0
–4 –2 0 2 4 –5 0 5

Fig. 8.2: Histograms of y from (8.14) for 100 000 random numbers, left for N = 2, right for N = 50.

(A) Central limit theorem. According to the central limit theorem of statistics, the
sum of inﬁnitely many equally distributed random numbers provides a normally
distributed number. Let x i be a series of equally distributed numbers in [0, 1). Then
N
1
y = y0 + a ∑ (x i − ) (8.14)
i
2

for increasing N approaches more and more a Gaussian distributed variable in the
domain
y0 − aN/2 ≤ y < y0 + aN/2
with mean ⟨y⟩ = y0 and variance

a2 N
Var(y) = ⟨(y − y0 )2 ⟩ = a2 N Var(x) = .
12
In practice a rather small N is quite often sufficient. If one is not interested in rather
small or large y (again the tails) one can produce quite accurately distributed num-
bers already for N ≈ 50; see Figure 8.2. A code producing an approximately Gaussian
distributed random number with given mean and variance could look like this:

REAL FUNCTION gauss(xm,var)

PARAMETER (n=50) ! number of drawn equally distr. numbers
c the variable gauss contains a pseudo-gaussian distributed random number
c with variance var and mean xm in the region
c xm-sqrt(3*var*n) ... xm+sqrt(3*var*n)
REAL :: xm,var,xi(n)
CALL RANDOM_NUMBER(xi) draw n equally distr. numbers in [0,1)
gauss=xm+SQRT(12.*var/FLOAT(n))*(SUM(xi)-FLOAT(n)/2.)
END FUNCTION
8.2 Monte Carlo integration | 245

(B) Box–Muller method. If a many-particle system is in thermal equilibrium at a

given temperature T, the speed of the individual particles are Maxwell–Boltzmann
distributed. In two spatial dimensions the transformation method can be applied
straightforwardly. The Maxwell–Boltzmann distribution density for the speed w =
√v2x + v2y reads with the scaling used in Section 4.5:
w
ρ(w) = exp(−w2 /2T) .
T
This can be integrated and provides the distribution function

Φ(w) = ∫ ρ(ω) dω = − exp(−w2 /2T) + c

with the integration constant c. From (8.11) we ﬁnd

w(x) = √−2T ln(c − x)

with equally distributed x ϵ [0, 1). Eventually the variable w should be in the interval
[0, ∞) and therefore c = 1. We can thus write the initial velocity for each particle as

v x (0) = w cos φ , v y (0) = w sin φ ,

where the angle φ should be equally distributed in [0, 2π). Finally we obtain for an
initial equilibrium velocity distribution with temperature T

v x (0) = √−2T ln(1 − x1 ) cos(2πx2 ) , v y (0) = √−2T ln(1 − x1 ) sin(2πx2 ) (8.15)

with the two independently drawn random numbers x1 , x2 in [0,1). The relations (8.15)
are denoted as the Box–Muller method.

8.2 Monte Carlo integration

8.2.1 Integrals in one dimension

It is common to compute integrals approximately by discretizing the region with N,

for the simplest case, equidistant mesh points:
b N
J = ∫ dx f(x) = ∆x ∑ g i f(x i ) + O(1/N k ) , (8.16)
a i

with
x i = (i − 1)∆x + a , ∆x = (b − a)/(N − 1) .
The order k of the error in (8.16) depends on the choice of the weights g i . Thus one
obtains the rectangle rule for

gN = 0 , gi = 1 , i = 1...N −1
246 | 8 Monte Carlo methods (MC)

with k = 1. For
g1 = g N = 1/2 , gi = 1 , i = 2...N −1
the trapezoidal rule yields k = 2 and for

g N = 1/3 , g1 = g3 = . . . g N−1 = 4/3 , g2 = g4 = . . . g N−2 = 2/3

the even more accurate Simpson rule is achieved having k = 4. Central to all formulas
is a systematic evaluation of the integrand f(x) at many points.
Instead of a systematic evaluation, Monte Carlo integration is based on the com-
putation of the integrand at N randomly chosen points x i in a ≤ x i ≤ b. Due to scaling
and shifting x can always be transformed to the domain a = 0, b = 1 and we shall
further consider only this case here. The integral (8.16) is thus computed with

1 N
J= ∑ f(x i ) + O(1/N k ) (8.17)
N i

with arbitrary x i ϵ [0, 1] drawn with a certain given probability density ρ(x). Compared
to (8.16) the numerical effort seems to be the same. One has to evaluate f for N different
x i . How large is the error exponent k in (8.17)?
Contrary to (8.16) the error is no longer systematic but rather depends on the ran-
dom series x i . We must determine the standard deviation of (8.17) with respect to dif-
ferent random series. Let

x ki , i = 1...N , k = 1...M

the kth series x i . Then we compute for the integral along (8.17) a mean value of

1 M
⟨J⟩ = ∑ Jk
M k

with
1 N
Jk = ∑ f(x ki ) .
N i

For the variance we obtain

1 M
Var(J) = ∑ (J k − ⟨J⟩)2
M k
2
1 M 1 N
= ∑ ( ∑ f(x ki ) − ⟨J⟩)
M k N i

1 M N
= ∑ ∑ (f(x ki ) − ⟨J⟩) (f(x kj ) − ⟨J⟩) .
MN 2 k ij
8.2 Monte Carlo integration | 247

If the individual numbers x ki are completely uncorrelated, the summands in the last
line will cancel for i ≠ j. Only the expressions with i = j remain and we obtain
1 M N 2
Var(J) = ∑ ∑ (f(x ki ) − ⟨J⟩) . (8.18)
MN 2 k i
On the other hand we can compute the variance of f :
1 M N 2
Var(f) = ∑ ∑ (f(x ki ) − ⟨J⟩) . (8.19)
MN k i

Comparing with (8.18) yields

Var(f)
Var(J) = (8.20)
N
or
σ(f)
σ(J) =. (8.21)
√N
Because the variance of f does not depend on N, the error in (8.17) goes as
1
∼ ,
√N
i.e., we found k = 1/2. This is worse than the most simple rectangular scheme in (8.16)
and seems not to favor MC integration.

8.2.2 Integrals in higher dimensions

But what happens in higher dimensional spaces? We study the n-dimensional integral
1 1

J n = ∫⋅ ⋅ ⋅ ∫ dx1 dx2 . . . dx n f(x1 . . . x n ) . (8.22)

0 0

Numerical computation based on a regular discretization demands for N mesh points

in each room dimension if the error should be of order 1/N k , leading to a total of
N T = N n function evaluations of the integrand f . However, the consideration leading
to (8.21) is independent of the number of room dimensions. Thus the error of MC inte-
gration still goes with 1/√N T where N T is the number of function evaluations. Hence
the MC integration for the same number of evaluations is becoming more accurate if
1 1
≤ a k/n
√N T NT
where a is a constant depending on the method. If N T ≫ a the estimation
n ≥ 2k
results. If we compare MC integration for instance with the Simpson rule (k = 4), it
will be paid off for a room dimension n > 8, and for the simple rectangle rule already,
for n > 2.
248 | 8 Monte Carlo methods (MC)

8.2.2.1 Example
As an example we wish to compute the integral
1 1

J n = ∫⋅ ⋅ ⋅ ∫ dx1 dx2 . . . dx n (x1 + x2 + . . . x n )2 . (8.23)

0 0

It can be solved exactly as

n n2
+ . J nA = (8.24)
12 4
Evaluation using the rectangle rule with N mesh points in each dimension yields
N N
2
J nR ≈ (∆x)n ∑ ⋅ ⋅ ⋅ ∑ (x i1 + x i2 + . . . x i n ) , x i = ∆x (i − 1) .
i 1 =1 i n =1

Monte Carlo integration provides

1 NT 2
J nM ≈ ∑ (ξ1 + ξ2 + . . . ξ n )
N T i =1
1

with the equally distributed, uncorrelated random numbers

0 ≤ ξi ≤ 1 .
To compare the results we take the same number of summands for each method, lead-
ing to
N = (N T )1/n .
The relative error in percent is given as
󵄨󵄨 R,M 󵄨
󵄨󵄨J n − J nA 󵄨󵄨󵄨
Er(%) = 100 ⋅ 󵄨 󵄨;
J nA
see Figure 8.3.

8.2.2.2 Simple sampling and importance sampling

To determine integrals of the form (8.16) with (8.17), we assumed equally distributed
sampling points x i ; this is also called simple sampling. If f(x) varies strongly, the de-
viation from the mean according to (8.21), and hence the error, can become large. On
the other hand we can rewrite (8.16) for an arbitrary, nonzero function ρ(x) as:
b b
f(x)
J = ∫ dx f(x) = ∫ dx [ ] ρ(x) . (8.25)
ρ(x)
a a

If ρ(x) is the probability density of x, we can interpret (8.25) as the mean ⟨f/ρ⟩ and
compute the variance
b
f(x) 2
Var(f/ρ) = ∫ dx [ ] ρ(x) − J 2 . (8.26)
ρ(x)
a
8.2 Monte Carlo integration | 249

Fig. 8.3: Numerical computation of the integral (8.23) for n = 1, 4, 7 room dimensions. Monte Carlo
integration compared to rectangle rule. Relative error in percentage over the total number of mesh
points N T = N n . For the rectangle rule N = 3 . . . 20 have been used in each dimension. The MC error
−1/2
goes with N T (straight line). For n = 1 the rectangle rule is superior, whereas for n = 4, MC is
already more precise.

Taking for ρ an appropriate PDF, (8.26) can be minimized. Vanishing functional

derivative δ/δρ of
b

F[ρ] = Var(f/ρ) + λ ∫ dx ρ
a

with λ as Lagrange parameter originating from the constraint (normalization)

∫ dx ρ(x) = 1 (8.27)
a

yields
−f 2 /ρ 2 + λ = 0

or
|f|
ρ= ,
√λ
where λ is to be determined with (8.27). The error becomes smaller the more the shape
of ρ resembles that of |f|. Adjusting the PDF according to the form of the integrand is
named importance sampling, and can be readily extended to higher dimensions.
250 | 8 Monte Carlo methods (MC)

8.3 Applications from statistical physics

Now let us turn to the other important field of Monte Carlo applications, namely to
problems where randomness and fluctuations play an intrinsic role. To this end we
first revisit many-particle systems. In Chapter 4 we examined such systems solving
Newton’s equations of motion for certain interaction potentials. Applying now the
methods of statistical physics, one considers an ensemble of macroscopically iden-
tical systems that differ microscopically, i.e., by different initial conditions of the par-
ticles. Averaging over the ensemble one can compute certain macroscopic values like
density or temperature. In thermal equilibrium one may also consider snapshots of
the same system at different times, making use of the identity of ensemble average
and time average.

8.3.1 Two-dimensional classical gas

8.3.1.1 Hard spheres model

We start with a two-dimensional gas of N hard spheres, or better hard discs, each hav-
ing the diameter d0 . Let the particles be randomly distributed in a certain given volume
V, say the unit square. We draw the positions r i⃗ from an equal distribution in the unit
square
r ⃗i = (x i , y i ) , x i ϵ [0..1] , y i ϵ [0..1] , i = 1 . . . N .
However, the particles must not overlap. Thus we have to choose the positions fulﬁll-
ing the constraints
d ij = √(x i − x j )2 + (y i − y j )2 ≥ d0 (8.28)
for each pair i ≠ j. The code section for the initialization could look as:

PARAMETER (n=1000) ! number of particles

REAL, DIMENSION(2,n) :: x ! positions
REAL, DIMENSION(2) :: xn ! new positions
....
d0 = ... ! sphere diameter, minimal distance
c draw n particle positions randomly
DO i=1,n
11 CALL RANDOM_NUMBER(xn) ! random position
DO j=1,i-1
IF(d(xn,x(:,j)).LT.d0) GOTO 11 ! distances ok?
ENDDO
x(:,i)=xn ! yes, accept position
ENDDO
....

The function d computes the Euclidean distance according to (8.28).

8.3 Applications from statistical physics | 251

To create a new conﬁguration one assigns fresh random coordinates x k , y k to

an arbitrarily chosen particle k = 1 . . . N. If all constraints (8.28) are fulfilled the
new position is accepted. Otherwise the coordinates are rejected and drawn again.
Hence one produces successively different ensemble members in accordance with the
macroscopic conditions V = const., N = const. Since there is no motion of particles,
a mechanical kinetic energy is not defined. Nevertheless, the interaction between the
particles can be easily considered. A hard core repulsion is already manifested in the
restriction (8.28). How can we model an additional attractive force? Let the interaction
for instance be expressed by an attractive Coulomb potential
α
u(r i⃗ − r ⃗j ) = − , α>0.
d ij
Then one computes the total energy for each configuration according to
N N
U(r1⃗ , . . . r ⃗N ) = ∑ ∑ u(r ⃗i − r ⃗j ) . (8.29)
i=1 j=i+1

One could be tempted to ﬁnd the ‘ground state’ with the lowest energy just by testing as
much as possible conﬁgurations. Assuming N = 1000, we had to vary 2000 variables
(3000 in 3D) in the region [0, 1]. To detect a minimum in such a high-dimensional
space seems to be rather hopeless.

8.3.1.2 Metropolis algorithm

A quite elegant method to an approximate solution of the problem is provided by the
Metropolis algorithm¹. In particular we shall here consider the canonical ensemble
where volume, particle number, and temperature are prescribed. A central part of the
statistical description is the partition sum [2]

Z(N, V, T) = ∑ exp(−E i /k B T) (8.30)

with Boltzmann’s constant k B and energy E i of the microstate i. The sum runs over
all macroscopically possible microstates. If one deals with continuous variables as
for the hard spheres gas model, the sum is converted into a 2N-dimensional integral
according to

Z(N, V, T) ∼ ∫ d2 r1⃗ ⋅ ⋅ ⋅ ∫ d2 r N⃗ exp(−U(r1⃗ . . . r ⃗N )/k B T) . (8.31)

If the partition sum is known, the probability in ﬁnding a state with energy E i in the
canonical ensemble is expressed by
1
ρi = exp(−E i /k B T) , (8.32)
Z

1 Devised by Metropolis, Rosenbluth, Rosenbluth, Teller and Teller in 1953 [1].

252 | 8 Monte Carlo methods (MC)

also denoted as Boltzmann distribution. With (8.32) we can compute mean values as

⟨Y⟩ = ∑ Y i ρ i , (8.33)
i

where again the sum runs over all states and has to be replaced by the appropriate
integrals for the case of continuous variables.
How can the sums or high-dimensional integrals for the computations of Z be
evaluated numerically? It is possible to apply MC-integration. Normally (8.32) is con-
centrated in a very narrow region around its mean

⟨E⟩ = ∑ E i ρ i
i

and simple sampling will not work effectively. A much better strategy is to draw only
states with E i ≈ ⟨E⟩. Hence one varies the state variables not arbitrarily but rather
in such a way that the system moves towards thermal equilibrium. Let X i be a set of
variables describing the system in the state i, i.e., the position coordinates of the gas
from Section 8.3.1.1. Now one seeks a sequence of states

X0 → X1 → . . . X n

that converges preferably quickly to equilibrium. Each state should be uniquely deter-
mined by its predecessor; such a sequence is denoted as a Markov chain. Let

P(X i → X j )

be the transition probability from state i into state j. A stationary distribution ρ(X) is
given if the condition

ρ(X i )P(X i → X j ) = ρ(X j )P(X j → X i ) ,

known as detailed balance, holds. For ρ as (8.32) one obtains

ρ(X j )
P(X i → X j ) = P(X j → X i ) = P(X j → X i ) exp(−∆E ji /k B T) (8.34)
ρ(X i )

with ∆E ji = E j − E i . If E i > E j , the transition i → j should always take place (approxi-

mation to the equilibrium):

P(X i → X j ) = 1 if ∆E ji < 0 .

From (8.34) one ﬁnds for the reverse transition j → i

P(X j → X i ) = exp(−∆E ij /k B T) and ∆E ij > 0 .

8.3 Applications from statistical physics | 253

Importance sampling with the Metropolis algorithm is built by the following steps:
1. Choose an initial configuration X0 . Compute E0 (X0 ). i = 1.
2. Construct the next configuration X i by modifying the position of an arbitrary par-
ticle. Compute E i (X i ) and ∆E i,i−1
3. If ∆E i,i−1 ≤ 0, accept the new configuration X i . i = i + 1. Go to 2.
4. If ∆E i,i−1 > 0, draw an equally distributed random number 0 ≤ r < 1. If r <
exp(−∆E i,i−1 /k B T), accept the new configuration X i . i = i + 1. Go to 2.
5. If r > exp(−∆E i,i−1/k B T), restore the old configuration X i = X i−1 . Go to 2.

We exemplify steps 1–5 with a code for MC simulation of the 2D gas with the interaction
potential given in (8.29). For the complete listing see [3].

Step 1 Initial position is computed with program section on page 250. Computation
of E0 :

E0=0.
DO i=1,n-1
DO j=i+1,n
E0=E0-1./d(x(1:2,i),x(1:2,j))
ENDDO
ENDDO

Step 2

100 CALL RANDOM_NUMBER(rk)

k=rk*FLOAT(n)+1 ! random particle k
Ek=0.
DO i=1,n
IF(i.NE.k) Ek=Ek-1./d(x(1:2,i),x(1:2,k)) ! contribution of k to E0
ENDDO
xn=x(1:2,k) ! store old config.
CALL RANDOM_NUMBER(x(1:2,k)) ! new random position of k
Ekn=0.
DO i=1,n ! constraints fulfilled? (hard spheres)?
IF(i.NE.k) THEN
IF(d(x(1:2,K),x(1:2,i)).LT.dmin) THEN
x(1:2,K)=xn
GOTO 100 ! No, then draw again
ENDIF
Ekn=Ekn-1./d(x(1:2,k),x(1:2,i)) ! contribution of k to E0
ENDIF
ENDDO
dE=Ekn-Ek ! delta E
254 | 8 Monte Carlo methods (MC)

Fig. 8.4: (N = 500), conﬁguration at T = 2000 (left frame) and drop formation at T = 20 (right).

Step 3

IF(dE.LT.0.) THEN
E0=E0+dE ! new E0
GOTO 100 ! next configuration
ENDIF

Step 4

CALL RANDOM_NUMBER(r)
IF(r.LT.EXP((-dE/t)) THEN
E0=E0+dE ! new E0
GOTO 100 ! next configuration
ENDIF

Step 5

x(1:2,k)=xn ! restore old config, E0 unchanged

GOTO 100 ! next draw

Figure 8.4 presents two particle conﬁgurations for high and low T. Scaling of temper-
ature and distances are such that α = k B = 1 and d0 = 0.01. Obviously, a circu-
lar assignment of the particles minimizes the energy for lower T and a single drop is
formed. For T = 2000 one obtains a gaseous state. The transition becomes visible if a
histogram of the particle distances is recorded as a function of temperature. From the
histogram one ﬁnds the average particle distance as

∑i r i H(r i )
⟨r⟩ = ;
∑i H(r i )

see Figure 8.5.

8.3 Applications from statistical physics | 255

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 200 400 600 800 1000 1200 1400 1600 1800 2000

Fig. 8.5: Mean particle distance ⟨r⟩ over temperature for N = 500, d 0 = 0.01.

Fig. 8.6: Drop formation under gravity for g = 103 (left) and g = 104 (right) at T = 20.

The inﬂuence of gravitation in the negative y-direction can be readily included by

adding the potential energy of the form
N
Up = g ∑ yi (8.35)
i

to (8.29). For g > 0 the particles ‘move’ downwards and eventually a drop having a
certain contact angle is formed on the lower wall. For larger g the drop is squeezed
and the contact angle becomes smaller; see Figure 8.6.

8.3.2 The Ising model

Certain metals such as iron or nickel show spatial regions where spins orient them-
selves in a parallel fashion and create a permanent magnetic ﬁeld, a property called
256 | 8 Monte Carlo methods (MC)

ferromagnetism. If the metal is heated above a critical temperature T c named Curie

temperature, polarization and magnetic ﬁeld vanish. Hence these metals undergo a
phase transition from a magnetic to a nonmagnetic state or vice versa with tempera-
ture as the control parameter and magnetization as the order parameter.
A simpliﬁed model describing ferromagnetism was devised by E. Ising² almost
100 years ago [4]. Thereby, N spins are located on a regular periodic lattice (in Ising’s
original work on a one-dimensional chain) and each spin has two possible orienta-
tions, say up and down, denoted by ±1. If only the nearest neighbor interaction is
taken into account, the energy is given by
N γ N
E(s1 , . . . s N ) = −ε ∑ ∑ s i s j − H ∑ s i , (8.36)
i jϵNN(i) i

where ε measures the coupling strength, NN(i) denote the indices of the nearest neigh-
bors and H stands for an external, homogeneous magnetic ﬁeld. The amount γ of
nearest neighbors depends on the room dimension n as well as on the geometry of
the periodic grid. One ﬁnds

{
{ 2 if n = 1, chain
{
{
{
{4 if n = 2, square lattice
{
{
{
γ = {6 if n = 2, hexagonal lattice
{
{
{
{ if n = 3, simple cubic lattice
{
{ 6
{
{
{8 if n = 3, body-centered cubic lattice.
Taking (8.36) we compute the partition sum (we shall use from here the abbreviation
β = 1/k B T)
Z(H, T) = ∑ ∑ ⋅ ⋅ ⋅ ∑ exp(−βE(s1 . . . s N )) , (8.37)
s1 s2 sN

where each sum runs only over the two values s i = −1, 1. Hence we have to evaluate 2N
expressions in total. With the partition sum at hand, we can compute the free energy
F(H, T) = −k B T ln Z(H, T) (8.38)
and from there further important macroscopic quantities like the internal energy
∂ F
U(H, T) = −k B T 2 ( ) , (8.39)
∂T k B T
the speciﬁc heat
∂U
C(H, T) = , (8.40)
∂T
or the overall magnetization
N
∂ F
M(H, T) = ⟨∑ s i ⟩ = − ( ) . (8.41)
i
∂H k B T

2 Ernst Ising, German physicist, 1900–1998.

8.3 Applications from statistical physics | 257

8.3.2.1 Mean-ﬁeld approximation

A rather crude ‘solution’ to the problem is achieved applying the mean-ﬁeld approx-
imation. Although this method in its simplest form yields inaccurate or even wrong
results, it shows at least qualitatively why a phase transition can take place and why
the critical temperature depends both on coupling strength and room dimension.
With (8.36) the energy of a single spin reads
γ
E s = −ε s ∑ s j − Hs = −εγs s̄NN − Hs , (8.42)
jϵNN(s)

with
γ
1
s̄NN = ∑ sj (8.43)
γ jϵNN(s)

as the mean of the neighbor spins. With the partition sum (two states, s = ±1)

̄ + βH) + exp(−βεγ s̄NN − βH) = 2 cosh(βεγ sNN

Z s = exp(βεγ sNN ̄ + βH)

the mean of s with given s̄NN

1 +1
s̄ = ̄ + βHs) = tanh(βεγ s̄NN + βH)
∑ s exp(βεγs sNN (8.44)
Z s s=−1

is computed. One now makes the strongly simpliﬁed assumption that the mean of the
neighbors (8.43) is equal to the mean (8.44) and obtains the transcendental equation

s̄ = tanh(βεγ s̄ + βH) (8.45)

from which s̄ and therewith the magnetization can be determined.

We start with the case H = 0 where the trivial solution s̄ = 0 exists. However, for

βεγ > 1 (8.46)

this solution becomes unstable and two new real branches ±s̄1 appear (three intersec-
tions of tanh(βεγ s)̄ and s̄ exist). The two new branches are identiﬁed with the ferro-
magnetic phases. The Curie temperature is given from (8.46)

k B T c = εγ .

It is proportional to the coupling strength ε and depends on the number of neighbors

and the room dimension. The magnetization is visualized easiest by solving (8.45) with
respect to T
εγ s̄
T=
k B atanh(s)̄
and plotting T over s̄ ϵ [−1, 1], Figure 8.7 left. In the moment where an external ﬁeld
is applied, a preferred direction exists. Then, one of the two nontrivial states becomes
258 | 8 Monte Carlo methods (MC)

s̅ s̅
1 1

0.5 0.5

0
1 2 3 4 –3 –2 –1 0 1 2 3
Tc H
–0.5 –0.5

–1 –1

Fig. 8.7: Left: magnetization for H = 0 over T . Right: magnetization over external ﬁeld H for T = 2.
The dashed line is computed from (8.45). It is unstable where its slope is negative and the true pro-
cess follows the arrows. Scaling: kB = ε = 1 and γ = 4, T c = 4.

more stable, and the other can still be metastable for a certain range of H, while a
hysteresis loop shows up: Figure 8.7, right frame.
Based on the mean-field approach, magnetic phases should occur also in one di-
mension (spin chain) at finite T = 2ε/k B . However, in real life this is not the case,
which we briefly demonstrate in the next section.

8.3.2.2 The one-dimensional Ising model

Consider a chain of N spins having equal distances. The relation (8.36) turns into
N−1 N
E = −2 ε ∑ s i s i+1 − H ∑ s i (8.47)
i i

and the partition sum can be computed exactly. For the sake of simplicity we shall
concentrate on the case H = 0 and substitute 2ε = ϵ. Using (8.47) one ﬁnds from
(8.37)

Z(T) = ∑ ∑ ⋅ ⋅ ⋅ ∑ exp(βϵs1 s2 ) exp(βϵs2 s3 ) . . . exp(βϵs N−1 s N ) . (8.48)

s1 s2 sN

The sum over s1 can be evaluated:

∑ exp(βϵs1 s2 ) = exp(βϵs2 ) + exp(−βϵs2 ) = 2 cosh(βϵs2 ) = 2 cosh(βϵ)

and shows to be independent on s2 . Next, the sum over s2 is executed in the same way
and so on, and ﬁnally one arrives at

Z(T) = 2N (cosh(βϵ))N−1 ≈ (2 cosh(βϵ))N . (8.49)

From (8.39) the internal energy

ϵ
U(T) = −ϵN tanh ( ) (8.50)
kB T
8.3 Applications from statistical physics | 259

0.5 1

0.4
M 0.5

0.3
C
0.2 –2 –1 0 1 2
H

0.1 –0.5

0
0 1 2 3 4 –1
T

Fig. 8.8: Left: speciﬁc heat per spin for the one-dimensional Ising model (H = 0); right: magne-
tization per spin depending on an external magnetic ﬁeld for T = 1 (solid) and T = 2 (dashed),
kB = ϵ = 1.

and from (8.40) the speciﬁc heat

ϵ2 N
C(T) = (8.51)
k B T 2 cosh(ϵ/k B T)
can be determined. The shape of C(T) could allude to a phase transition for a speciﬁc
T c ≈ 1, Figure 8.8, left. However, this is not the case as we shall see if we compute
the magnetization according to (8.41). To this aim one must know the partition sum
for H ≠ 0. This can be achieved also exactly but the computation is more involved; for
details we refer to the literature, e.g., [2]. Here, we only present the result (N ≫ 1):
N
Z(H, T) = exp(Nβϵ) [(cosh(βϵ) + √sinh2 (βH) + exp(−4βϵ)] . (8.52)

We ﬁnally get
N sinh(βH)
M(H, T) =
√sinh2 (βH) + exp(−4βϵ)

and from there

M(0, T) = 0 , for T > 0 .

No spontaneous magnetization occurs for ﬁnite temperature; see also Figure 8.8, right.
Applying thermodynamics, this can be understood by an estimate of the free en-
ergy
F = U − TS .
260 | 8 Monte Carlo methods (MC)

(a)

Fig. 8.9: One-dimensional chain of spins, (a) mag-

(b) netic ground state, (b) dislocation.

In a homogeneous magnetic phase all spins are aligned (Figure 8.9 (a)), the entropy
is S = 0, and the internal energy reads U = −Nϵ. If, on the other hand, a disloca-
tion as shown in Figure 8.9 (b) is included, the internal energy increases by 2ϵ, but in
the meantime the entropy increases also by k B ln N since there exist N possibilities to
place the dislocation. Thus, the free energy is in total changed by

∆F = 2ϵ − k B T ln N ,

which becomes negative for T > 2ϵ/(k B ln N). Since F decreases on the way to ther-
modynamic equilibrium and finally takes a minimum there, more and more disloca-
tions are created by thermal fluctuations and the order vanishes. In the thermody-
namic limit N → ∞ fluctuations at even infinitesimal temperature are large enough
to destroy a one-dimensional magnetic structure independently from the coupling
strength ϵ.
Why then does the mean-field approximation in one dimension yield a value of
T c = 2ε/k B ? If one assumes as in (8.45) for all spins the same mean value, the influence
of fluctuations is considerably diminished, leading to larger values of T c in general.
For only one spatial dimension, even a qualitatively wrong result can turn out.

8.3.2.3 The two-dimensional Ising model

Coupling is stronger in two room dimensions and may ﬁnally lead to ordered states at
ﬁnite T. A dislocation in 2D (a grain boundary) on an M × M lattice costs the internal
energy 2Mϵ and thus changes the free energy by

∆F = 2Mϵ − k B T ln M ,

an expression that is nonnegative for ﬁnite T and large enough M.

Onsager³ [5] was able to ﬁnd an exact solution of the two-dimensional Ising model
using transfer matrices. He derived for T c the relation
2ε ε
Tc = ≈ 2.269 .
k B asinh(1) kB
The derivation is however rather involved and lengthy and an exact solution in three
dimensions is still absent.

3 Lars Onsager, Norwegian-American physicist, 1903–1976.

8.3 Applications from statistical physics | 261

The Metropolis algorithm introduced in Section 8.3.1.2 can be readily applied on

the Ising model. We shall use a square grid built up by N = M × M spins. The method
can be extended in a straightforward manner to three dimensions.
We refer to the procedure given on page 253. The initial state (step 1) may consist
either of equally distributed (hot start-up) or of parallel orientated spins (cold start-
up). To compute E, (8.36) is evaluated with γ = 4. A new conﬁguration is created by
changing the direction of a randomly drawn spin. How can ∆E be computed if spin s k
changes to −s k ? Let the initial energy be
γ
E({s}) = −ε s k ∑ s j − H s k + rest , (8.53)
jϵNN(k)

and after changing s k

γ
E({s󸀠 }) = ε s k ∑ s j + H s k + rest , (8.54)
jϵNN(k)

where ‘rest’ denotes all terms that do not depend on s k . Subtracting the two equations
above yields
∆E = E({s󸀠 }) − E({s}) = 2s k h k (8.55)
with the abbreviation
γ
hk = ε ∑ sj + H . (8.56)
jϵNN(k)

In two dimensions the sum in (8.56) can only take one out of the ﬁve values −4,−2,0,2,4
and one ﬁnds
∆E = ±2(4ε + H, 2ε + H, H, −2ε + H, −4ε + H) .

The Boltzmann factors are needed only for the cases ∆E > 0. Assuming

|H| < 2ε

and taking the scaling

ε ̃
T= T, H = ε H̃
kB
the ﬁve Boltzmann factors

̃ T)̃ exp(−4T)̃ ,
exp(±2H/ ̃ T)̃ exp(−8T)̃ ,
exp(±2H/ ̃ T)̃
exp(−2H/

occur. It suffices to compute the four different exponential functions for one time at
the beginning of the program what increases its speed considerably.
A code section containing steps 2–5 of page 253 could look like this (for the com-
plete code see [3]):
262 | 8 Monte Carlo methods (MC)

ex(0)=0.5
ex(1)=EXP(-4./T) ! the four exp-functions
ex(2)=EXP(-8./T)
exh(1)=EXP(2.*H/T)
exh(2)=EXP(-2.*H/T)
....
c should the spin is(i,j) be changed?
in=-is(i,j)*(is(i,j+1)+is(i,j-1)+is(i+1,j)+is(i-1,j)) ! sum over
! 4 neighbors
IF(in.GT.0) THEN ! Delta E > 0, change!
is(i,j)=-is(i,j)
ELSE ! Delta E<=0
CALL RANDOM_NUMBER(r)
k=-in/2 ! pointer to ex
k1=(is(i,j)+3)/2 ! pointer to exh
IF(r.LT.ex(k)*exh(k1)) THEN ! r < Boltzmann fact., change!
is(i,j)=-is(i,j)
ENDIF
ENDIF

For H = 0, (8.56) is an integer and the case ∆E = 0 occurs frequently. For a faster
thermalization it is advantageous to allow for transitions with the probability 1/2. This
is why we put ex(0)=0.5 in the first code line.
Let us start with the case without external field H = 0. After about N steps, con-
figurations close to equilibrium are created. Figure 8.10 shows different spin configu-
rations depending on temperature. The magnetization is found from
N
1
⟨M⟩ = ⟨∑ s i ⟩ , (8.57)
N i

where the brackets denote an average of over 1000 different configurations, each
found after a preliminary iteration of 5N steps. Figure 8.11 left shows the typical form
of M for a phase transition already known from the mean-field approximation. Ac-
tually, the mean should be computed by (8.33) with (8.32). But since all states that
contribute belong to almost the same E i , the difference is small. The standard devia-
tion of E is smaller by two orders of magnitude than its mean.
In the same way as (8.57) one finds for the mean energy

1 K
⟨E⟩ = ∑ E({s k }) (8.58)
K k

with E({s k }) belonging to the conﬁguration k according to (8.36). Again the averaging
should be computed for many states after a certain number of preliminary iterations
8.3 Applications from statistical physics | 263

T = 10.0 〈M〉 = 0.000 T = 6.0 〈M〉 = 0.000 T = 4.0 〈M〉 = 0.000

T = 3.5 〈M〉 = 0.000 T = 3.0 〈M〉 = 0.001 T = 2.7 〈M〉 = 0.002

T = 2.5 〈M〉 = 0.001 T = 2.4 〈M〉 = 0.042 T = 2.3 〈M〉 = 0.042

T = 2.2 〈M〉 = 0.783 T = 2.1 〈M〉 = 0.869 T = 2.0 〈M〉 = 0.911

Fig. 8.10: Conﬁgurations for several T of an 80 × 80 grid. The critical point where magnetization
changes strongly is at T c ≈ 2.3.
264 | 8 Monte Carlo methods (MC)

1 2.5
0.9
0.8 2
0.7
0.6 1.5
0.5
〈M〉

〈C〉
0.4 1
0.3
0.2 0.5
0.1
0 0
1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 1.6 1.8 2 2.2 2.4 2.6 2.8 3
T T

Fig. 8.11: Average magnetization (left) and speciﬁc heat of the 80 × 80 grid. Both curves are typical
for a phase transition of the second kind at T c ≈ 2.3.

〈M〉 〈M〉
0.5
0.5

T H
0
1 2 3 4 5 6 7 8 −0.1 0.1

−0.5
−0.5

Fig. 8.12: Left: Imperfect bifurcation for an external magnetic field H = 0.03, computed from the
mean-field approach. Right: magnetization for T = 2 as function of the external field. Left branch:
H from 0.1 ⋅ ⋅ ⋅ − 0.1, right branch from −0.1 . . . 0.1.

following each change of the temperature. If ⟨E⟩ is known for two neighbored temper-
atures with difference δT ≪ T, the speciﬁc heat per spin can be determined by
1 ⟨E⟩(T + δT) − ⟨E⟩(T)
⟨C⟩(T) = .
N δT
Clearly in Figure 8.11, right frame, a maximum of ⟨C⟩ is recognized at the critical point.
Finally we wish to discuss the case H ≠ 0. Then the pitchfork bifurcation in Fig-
ure 8.7 left turns into an imperfect bifurcation, and Figure 8.12, left, shows the cor-
responding result from the mean-ﬁeld approximation, equation (8.45). The middle
8.4 Differential equations derived from variational problems | 265

branch is unstable, and for small T two stable solutions exist. Figure 8.12, right frame,
shows the magnetization found with the Metropolis algorithm for a fixed T = 2. The
magnetic field is first increased from −0.1 through 0.1, then reduced from 0.1 to −0.1. A
hysteresis loop appears.

8.4 Differential equations derived from variational problems

Many differential equations can be formulated by the help of a variational principle.

A (stationary) solution may then be determined by seeking the extreme values of a
certain functional.

8.4.1 Diffusion equation

We start with an example. The diffusion equation with internal sources reads:

∂ t Ψ(r,⃗ t) = ∆Ψ(r,⃗ t) + ρ(r,⃗ t) . (8.59)

Deﬁning the functional

1
F[Ψ] = ∫ d3 r ⃗ [ |∇Ψ|2 − ρ Ψ] (8.60)
2
V

we can write (8.59) in the form

δF
∂t Ψ = − (8.61)
δΨ
where δ . . . /δΨ denotes the functional derivative. If Ψ evolves according to (8.59), the
functional F is minimized. This is readily seen by forming its derivative

δF δF 2
dt F = ∂t Ψ = − ( ) = − (∂ t Ψ)2 ≤ 0 .
δΨ δΨ
A minimum of F then corresponds to a stationary solution of (8.59), the Poisson equa-
tion:
δF 󵄨󵄨󵄨󵄨
∆Ψ0 = −ρ and 󵄨 =0. (8.62)
δΨ 󵄨󵄨󵄨Ψ =Ψ0

8.4.1.1 Metropolis algorithm

Instead of solving numerically the PDE (8.59) by discretization in space and itera-
tion in time one could also try to ﬁnd a minimum of (8.60) applying for instance the
Metropolis algorithm. At ﬁrst Ψ is discretized, say by a regular M × M mesh having the
width ∆x (again we shall restrict ourselves to two spatial dimensions):

Ψ(x, y) → Ψ ij .
266 | 8 Monte Carlo methods (MC)

Thus one has to deal with an almost analogue problem to the Ising model, the only
difference is that the ‘spins,’ namely the node values of Ψ = Ψ ij , now may take con-
tinuous values. The Metropolis algorithm can be easily adjusted:
1. Choose an initial condition, e.g., random and equally distributed Ψ ij or Ψ ij = 0.
Compute F.
2. For the new configuration change one Ψ nm with randomly drawn n, m.
3. Compute the new F 󸀠 and ∆F = F 󸀠 − F.
4. If ∆F < 0, accept the new configuration.
5. If ∆F > 0, draw a random number r out of [0,1]. If r < exp(−∆F/T), accept the new
configuration; otherwise restore the old one.

What is here the role of the ‘temperature’? Analogue to thermodynamics, the possi-
bility to accept a Metropolis step with increasing F increases with T. Thus the system
may leave a secondary minimum to reach eventually the main minimum. Too large
temperatures however may yield rather noisy and not converging results and one has
to play around a bit to ﬁnd the appropriate T. Also a ‘time-dependent,’ slowly decreas-
ing temperature could help in some cases.

8.4.1.2 Implementation with ﬁnite differences

Approximating the derivatives and the integral in (8.60) with ﬁnite differences, the
functional F turns into a function of N = M × M variables
M
1 1
F(Ψ ij ) = ∑ [ (Ψ i+1,j − Ψ i−1,j )2 + (Ψ i,j+1 − Ψ i,j−1)2 − ρ ij Ψ ij ∆x2 ] . (8.63)
ij
8 8

If only one certain Ψ mn is changed, ∆F can be effectively evaluated just as (8.53) by

dividing (8.63) in terms which contain Ψ mn and the rest. After some manipulations
one arrives at
1 2 1
F = Ψ mn − Ψ mn (Ψ m+2,n + Ψ m−2,n + Ψ m,n+2 + Ψ m,n−2 ) − ρ mn Ψ mn ∆x2 + rest .
2 4
̃
Let Ψ mn be the new value at node m, n. Then the modiﬁcation of F reads
1 ̃2 2
∆F = (Ψ mn − Ψ mn )
2
1
+ (Ψ̃ mn − Ψ mn ) (−ρ mn ∆x2 − Ψ m+1,n − Ψ m−1,n − Ψ m,n+1 − Ψ m,n−1 ) .
4
In the last formula we have halved the step size and replaced the nodes m, n + 2 with
m, n + 1 and so on.

8.4.1.3 Results
Figure 8.13 shows a ‘development’ of the initial condition Ψ ij = 0 on a 100 × 100 grid
with ∆x = 1. Two point sources
ρ 20,20 = −1 , ρ 80,80 = 1
8.4 Differential equations derived from variational problems | 267

Fig. 8.13: Solution of the Poisson equation for two point charges applying the Metropolis algorithm.
The snapshots are taken from left to right after 106 , 108 , 2 ⋅ 109 variations of the node values and for
a temperature of T = 4 ⋅ 10−5 .

are given. At each step, the new node value is achieved by

Ψ̃ mn = Ψ mn + αξ (8.64)

with an equally distributed random variable ξ ϵ [−0.5, 0.5] and α = 0.1, while the
temperature is ﬁxed with T = 4 ⋅ 10−5 . The boundary values Ψ0,j , Ψ M+1,j , etc. are not
changed and remain as zero (Dirichlet condition).

8.4.2 Swift–Hohenberg equation

As demonstrated in Section 7.4.2, the Swift–Hohenberg equation may also be derived

from a functional (7.106). Thus we can again use the Metropolis algorithm. Changing
a node value Ψ mn now causes the change of the potential
∆L ε 1 4 10
2
= (Ψ̃ mn
2 2
− Ψ mn )( − + − 2 + 4 )
∆x 2 2 ∆x ∆x
2 8
+ (Ψ̃ mn − Ψ mn ) ( ( 2 − ) (Ψ m+1,n + Ψ m−1,n + Ψ m,n+1 + Ψ m,n−1 )
∆x ∆x4
2
+ 4 (Ψ m+1,n+1 + Ψ m−1,n+1 + Ψ m+1,n−1 + Ψ m−1,n−1 )
∆x
1
+ 4 (Ψ m,n+2 + Ψ m,n−2 + Ψ m+2,n + Ψ m−2,n ) )
∆x
1 1 4
− A(Ψ̃ mn 3 3
− Ψ mn ) + (Ψ̃ mn 4
− Ψ mn ).
3 4
The expression is more involved because (7.106) contains the Laplacian. Figure 8.14
shows two states on a 100 × 100 grid for different parameters, each obtained after
2 ⋅ 109 variations. The node modiﬁcations were done as in (8.64) but with α = 0.01,
∆x = 1/2, T = 2.5 ⋅ 10−4 . Figure 8.15 depicts the corresponding potentials. The tem-
perature chosen in Figure 8.16 is obviously too high.
268 | 8 Monte Carlo methods (MC)

Fig. 8.14: Two solutions of the Swift–Hohenberg equation by variation of the Lyapunov potential
(7.106) with the help of the Metropolis algorithm. Left frame: ε = 0.1, A = 0, right frame: ε = 0.1,
A = 0.5. As in Section 7.4.2, one ﬁnds stripes for A = 0 and hexagons for A = 0.5.

0.8 1
0.6 0.5
0.4 0
–0.5
0.2
–1
0
–1.5
–0.2 –2
–0.4 –2.5
–0.6 –3
–0.8 –3.5
200 600 1000 1400 1800 200 600 1000 1400 1800

Fig. 8.15: Lyapunov functional over the number of variations (in millions) for the two parameter sets
of Figure 8.14, T = 2.5 ⋅ 10−4 .

12.6

12.2

11.8

11.4 Fig. 8.16: Lyapunov functional for ϵ = 0.1,

A = 0, but now for T = 2.5 ⋅ 10−3 . At higher
11
temperatures the ﬂuctuations are clearly larger
and it takes much longer until an equilibrium
10.6
0 400 800 1200 1600 2000 (stable) state is reached.
Bibliography | 269

Bibliography

[1] N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller and E. Teller, Equation of State Calcula-
tions by Fast Computing Machines, J. Chem. Phys. 21, 1087 (1953).
[2] K. Huang, Statistical Mechanics, Wiley & Sons (1987).
[3] FORTRAN codes at http://www.degruyter.com/books/978-3-11-051513-8
[4] E. Ising, Beitrag zur Theorie des Ferromagnetismus, Z. Phys, 31, 253 (1925).
[5] L. Onsager, Crystal statistics I. A two dimensional model with order-disorder transition, Phys.
Rev. 65, 117 (1944).
A Matrices and systems of linear equations
In this appendix, we concentrate on methods for the solution to large linear systems
of equations. These can be homogeneous

A ⋅ x⃗ = 0 (A.1)

or inhomogeneous
A ⋅ x⃗ = b⃗ , (A.2)
where x,⃗ b⃗ denote vectors in RN and A is an N × N square matrix.
We begin with some properties and deﬁnitions of square matrices.

A.1 Real matrices

Let A be a real-valued N × N matrix, A ij ∈ R.

A.1.1 Eigenvalues and eigenvectors

Each vector v⃗ i in RN for which

A ⋅ v⃗ i = λ i v⃗ i with v⃗ i , λ i ∈ C (A.3)

holds is called the eigenvector of A and belongs to the eigenvalue λ i . The eigenvalues
are either real valued or come as conjugate-complex pairs; the same is found for the
eigenvectors.

A.1.2 Characteristic polynomial

The polynomial of degree N

P(λ) = det(A − λ 1) = (λ1 − λ)(λ2 − λ) . . . (λ N − λ) (A.4)

is called the characteristic polynomial of A. Its N zeros λ i are the eigenvalues of A and
are also denoted as the spectrum of A.
Multiple zeros: Let λ i be a k i -fold zero of P,

P(λ) = (λ1 − λ)k1 (λ2 − λ)k2 . . . (λ s − λ)k s .

The eigenvalue λ i then has the algebraic multiplicity k i .

https://doi.org/10.1515/9783110515145-009
A.1 Real matrices | 271

A.1.3 Notations

– The transposed matrix A T is obtained by interchanging rows and columns of A:

A Tij = A ji . (A.5)

– The inverse matrix A−1 is deﬁned as

A−1 ⋅ A = A ⋅ A−1 = 1 . (A.6)

It is computed by solving the linear, inhomogeneous system of equations (A.6), in

components:
N
∑ A ij A−1
jk = δ ik . (A.7)
j

– For a symmetric matrix A = A T holds.

– For an orthogonal matrix A −1 = A T holds. Because of (A.7) all rows and all columns
are pairwise orthogonal.
– A normal matrix commutes with its transposed, A ⋅ A T = A T ⋅ A. Symmetric and
orthogonal matrices are normal.

A.1.4 Normal matrices

Let A be a normal matrix. An orthogonal matrix T exists with

T −1 ⋅ A ⋅ T = B (A.8)

where B is diagonal, B ij = b i δ ij . A and B are called similar and have the same char-
acteristic polynomial and thus the same spectrum λ i = b i . The eigenvectors of A (A T )
form the columns (rows) of the transformation matrix T (T −1 ).

A.1.4.1 Symmetric matrices

A symmetric N × N matrix has the following properties:
1. All N eigenvalues and eigenvectors are real valued.
2. Eigenvectors belonging to different eigenvalues are orthogonal.
3. If an eigenvalue has the algebraic multiplicity k (k-fold degenerated), it has k
linearly independent eigenvectors that can be orthogonalized applying Gram–
Schmidt orthogonalization.
4. From 2. and 3. follows: the N eigenvectors form a complete orthogonal system in
RN , i.e., they span the complete RN .
5. The transformation matrix T in (A.8) is orthogonal.
272 | A Matrices and systems of linear equations

A.1.4.2 Orthogonal matrices

The spectrum of an orthogonal matrix M is located on the unit circle in the complex
plane:
|λ j | = 1 , λ j ∈ C

or
λ j = exp(iφ j ) , φj ∈ R .
Because of det M = ∏Ni λ i it immediately follows with A.1.1

det M = ±1 .

Example: the rotational matrix in two dimensions

cos θ − sin θ
D=( )
sin θ cos θ

has the characteristic polynomial

(cos θ − λ)2 + sin2 θ = 0

and the eigenvalues

λ12 = exp(±iθ) .

A.2 Complex matrices

Let A be a complex N × N matrix, A ij ∈ C.

A.2.1 Notations

The adjoint matrix A+ is obtained by interchanging rows and columns and forming
the conjugate-complex of each element

A+ij = A∗ji . (A.9)

For the inverse matrix A −1 (A.6) and (A.7) hold.

Special matrices:
– The self-adjoint matrix with A = A+ . Self-adjoint matrices are also named Hermi-
tian.
– The unitary matrix with A −1 = A + .
– The normal matrix with A ⋅ A+ = A+ ⋅ A.
– Self-adjoint and unitary matrices are also normal.
A.2 Complex matrices | 273

A.2.2 Jordan canonical form

Each arbitrary, complex or real-valued, N × N matrix A can be transformed by an in-

vertible matrix C (linear transformation) into a matrix having Jordan canonical form,
or for short, into a Jordan matrix:
J 1 (λ1 ) 0 0 0 ...
0 J 2 (λ2 ) 0 0 ...
C −1 ⋅ A ⋅ C = J = ( 0 0 J 3 (λ3 ) 0 ... ) (A.10)
...
( 0 ... J m (λ m ))
with the matrices (Jordan blocks)
λi 1 0
λi 1
J i (λ i ) = λ i , ( ) , (0 λi 1) , etc. (A.11)
0 λi
0 0 λi
of length
1, 2, 3, etc.
– J i (λ i ) contributes to the spectrum of A with the sole (algebraic 1,2,3,. . . -fold) eigen-
value λ i .
– The geometric multiplicity of the eigenvalue λ i is equal to the number of the lin-
early independent eigenvectors belonging to λ i . It is thereby equal to the number
of Jordan blocks with λ i in J.
– A normal matrix has the following properties:
– Algebraic and geometric multiplicity are equal.
– All Jordan blocks have the length one.
– All normal matrices are diagonalizable.

Example:
The matrix
λ1 1 0 0
0 λ1 0 0
J=( )
0 0 λ1 0
0 0 0 λ2
is already in Jordan canonical form and has the characteristic polynomial
P(λ) = (λ1 − λ)3 (λ2 − λ) .
It follows for the eigenvalues

λ1 λ2
algebraic multiplicity 3 1
geometric multiplicity 2 1
274 | A Matrices and systems of linear equations

Hence, J possesses only three eigenvectors, two belonging to λ1 and one to λ2 . Thus,
the eigenvectors do not form a complete orthogonal system.

A.3 Inhomogeneous systems of linear equations

We begin with N-dimensional systems having the form

A ⋅ x⃗ = b⃗ . (A.12)

A.3.1 Regular and singular system matrices

The system (A.12) possesses a unique solution if the system matrix A is invertible. It
reads
x⃗ = A−1 ⋅ b⃗ . (A.13)
Hence, det A ≠ 0 and A is called a regular matrix. Then the rank of A is

rank(A) = N ,

and A is built up by N linearly independent rows (and columns).

Conversely, if det A = 0 and rank(A) = N − K, there are K linearly dependent rows
and A is named singular. The kernel of A

ker(A)

denotes the K-dimensional subspace of RN spanned by the vectors

v⃗1 , . . . , v⃗ K

for which
A ⋅ v⃗ i = 0 , i = 1...K .

The kernel is also called the null space. For a singular A, depending on b⃗ there exists
either no or inﬁnitely many solutions to (A.12). There is no solution if b⃗ is partially or
completely in ker(A):

(a) no solution if v⃗+i ⋅ b⃗ ≠ 0 for at least one i ≤ K

(b) inﬁnitely many solutions if v⃗+i ⋅ b⃗ = 0 for all i ≤ K .

Here, v⃗ +i denotes the eigenvectors of A+ (or the left-sided eigenvectors of A):

(A+ − λ i 1) ⋅ v⃗+i = 0 ,

and λ1 , . . . λ K = 0, λ K+1 , . . . λ N ≠ 0.
A.3 Inhomogeneous systems of linear equations | 275

For case (b), the solutions are found by expanding x⃗ and b⃗ into the base v⃗ i :
N N N
x⃗ = ∑ y i v⃗ i , b⃗ = ∑ β i v⃗ i = ∑ β i v⃗ i
i=1 i=1 i=K+1

with β i = v⃗ +i ⋅ b.⃗ From (A.12) we obtain:

N N N N
∑ y i A v⃗ i = ∑ y i λ i v⃗ i = ∑ y i λ i v⃗ i = ∑ β i v⃗ i
i=1 i=1 i=K+1 i=K+1

and because the v⃗ i are linearly independent

v⃗+i ⋅ b⃗
yi = , i = K +1...N
λi
must hold. The other y i , i = 1 . . . K remain undetermined.

A.3.2 Fredholm alternative

Hence, the solutions of (A.12) read either

x⃗ = A −1 ⋅ b⃗ , if rank(A) = N

or, given that b⃗ ∈ ̸ ker(A),

N v+i ⋅ b⃗ K
x⃗ = ∑ v⃗ i + ∑ y i v⃗ i , if rank(A) = N − K .
i=K+1
λi i=1

These two possibilities are named Fredholm alternative.

A.3.3 Regular matrices

The matrix inversion to numerically solve (A.12) can be achieved by applying some
standard routines provided e.g., by the LAPACK library. Depending on the form of A
different routines are to be considered, a documentation is found for instance in¹

http://www.netlib.org/lapack/explore-html/ﬁles.html

The different names of the routines are logically derived and composed from their
functions, data types, etc., for details see

http://www.netlib.org/lapack/individualroutines.html

1 Date: November 2017.

276 | A Matrices and systems of linear equations

To solve (A.12), the matrix A does not need to be inverted completely, it suffices
to transform it into a triangular shape, applying Gauss elimination (for details see
e.g., [1]). Once (A.12) is in triangular form

A󸀠 ⋅ x⃗ = b⃗ 󸀠 (A.14)

with
A󸀠11 . . . A󸀠1N
A󸀠22 . . .
A󸀠 = ( A󸀠33 . . ) , (A.15)
.
0 A󸀠NN
the system (A.14) is easily solved by backward substitution and we obtain

b 󸀠i − ∑Nk=i+1 A󸀠ik x k
x i := , (A.16)
A󸀠ii

where the index i = N, N − 1, . . . 1 runs backwards. If the matrix in (A.15) is regular,

det A 󸀠 = ∏Ni A󸀠ii ≠ 0 and as a consequence all A󸀠ii ≠ 0.

A.3.4 LU decomposition

LU decomposition provides a direct, noniterative solution method of an inhomoge-

neous system of linear equations. The matrix A is written as the product of two matri-
ces
L⋅U =A (A.17)
where L is a lower (only nonzero elements on the diagonal and below) and U an upper
(nonzero elements on the diagonal and above) triangular matrix. Taking a 4×4 matrix
as an example, we have

L11 0 0 0 U11 U12 U13 U14 A11 A12 A13 A14

L21 L22 0 0 0 U22 U23 U24 A21 A22 A23 A24
( )⋅( )=( ) .
L31 L32 L33 0 0 0 U33 U34 A31 A32 A33 A34
L41 L42 L43 L44 0 0 0 U44 A41 A42 A43 A44
(A.18)
Then (A.12) can be written as

A ⋅ x⃗ = (L ⋅ U) ⋅ x⃗ = L ⋅ (U ⋅ x)⃗ = b⃗ . (A.19)

Introducing the vector

y⃗ = U ⋅ x⃗ (A.20)
we ﬁrst solve
L ⋅ y⃗ = b⃗ , (A.21)
A.3 Inhomogeneous systems of linear equations | 277

which, because of the triangular form of L, is simple:

i−1
1 [
y i := b i − ∑ L ij y j ] . (A.22)
L ii
[ j=1 ]
Here we imply that the sum is zero for i = 1 and for y j occurring on the right-hand side
the values computed in the same loop are used (forward substitution).
As a second step, we solve
U ⋅ x⃗ = y⃗ (A.23)
by backward substitution just as in (A.14).
The main task is to ﬁnd the decomposition (A.17), or, in other words, to determine
the matrices L and U. This is done by solving the inhomogeneous system
K
∑ L ik U kj = A ij , i = 1...N , j = 1...N , (A.24)
k=1

where because of the special structure of L and U the sum runs only up to

K = min(i, j) .

These are N 2 equations for 2× N 2+N = N 2 + N unknowns and the system is underdeter-
2

mined, meaning the LU decomposition is not unique. Thus, we can arbitrarily make
the N elements in the diagonal of L 1:

L ii = 1 , i = 1...N . (A.25)

A direct solution of (A.24) is given by the Crout algorithm [3]:

1. j = 1.
2. Compute the j coefficients
i−1
U ij := A ij − ∑ L ik U kj , i = 1...j . (A.26)
k=1

3. Compute the N − j coefficients

j−1
1
L ij := [A ij − ∑ L ik U kj ] , i = j +1...N . (A.27)
U jj k=1

4. j := j + 1. If j ≤ N go to 2.

Following the algorithm step-by-step, it turns out that on the right-hand sides only
elements are needed that have already been computed in prior steps.
278 | A Matrices and systems of linear equations

A.3.4.1 Pivoting
A division occurs in (A.27), which may cause numerical problems if the corresponding
U jj is small. One may show that Crout’s algorithm (as other elimination methods too)
is unstable if U jj becomes small compared to other matrix elements. A solution is pro-
vided by reordering the system. One may interchange arbitrarily rows and columns
but then has to swap also the elements of the vectors x⃗ and b⃗ accordingly. Finding
an appropriate permutation that leads to a large-valued divisor (the pivot element in
(A.27)) is called ‘pivoting.’ One distinguishes between full and partial pivoting. In the
first case, rows and columns of A are interchanged pairwise, in the latter only the rows.
How can we find the optimal pivot element? Normally it suffices to take that ele-
ment with the largest absolute value. To compare elements in different rows, the ma-
trix A should be scaled first. One can use the largest element in each row and scale
this to one, i.e., one computes for each row i a scaling factor

s−1
i = max |A ij | j

and then scales

A󸀠ij = s i A ij , b 󸀠i = s i b i .

A.3.5 Thomas algorithm

If the matrix A has many zeros, the methods can often be simpliﬁed considerably.
The special case of a tridiagonal matrix (only nonzeros in the diagonal, and in one
upper and lower diagonal) occurs often from ﬁnite differences discretization and can
be solved quite effectively. Let
A ⋅ x⃗ = y⃗ ,
with a tridiagonal A given as

b1 c1 0
a2 b2 c2
( . . . )
A=(
(
) .
) (A.28)
ai bi ci
. . .
(0 aN bN )

In a ﬁrst step, (A.28) is reduced to triangular form

1 c󸀠1 0
1 c󸀠2
( . . )
A󸀠 = (
(
) ,
) (A.29)
1 c󸀠i
. .
(0 1)
A.4 Homogeneous systems of linear equations | 279

which is achieved by substituting

c1 y1
c󸀠1 = , y󸀠1 = (A.30)
b1 b1
and
ci y i − a i y󸀠i−1
c󸀠i := , y󸀠i := , i = 2...N . (A.31)
b i − a i c󸀠i−1 b i − a i c󸀠i−1
The system
A󸀠 x⃗ = y⃗ 󸀠
can then be readily solved by (A.16):

x N = y󸀠N and x i := y󸀠i − x i+1 c󸀠i , i = N − 1, . . . 1 . (A.32)

The receipt (A.30-A.32) is known as the ‘Thomas algorithm’ and can be extended
straightforwardly to matrices having more than one upper and lower diagonal.

A.4 Homogeneous systems of linear equations

A.4.1 Eigenvalue problems

Now we turn to the homogeneous equations (A.3)

B ⋅ x⃗ − λ x⃗ = 0 , (A.33)

that can be brought into the form (A.1) by substituting B − λ 1 = A. The system (A.33)
has nontrivial solutions only if the determinant

det(B − λ 1) = 0

vanishes (solvability condition). As already stated in Section A.1.2, this yields the char-
acteristic polynomial P(λ) whose N zeros correspond to the eigenvalues of B. To each
eigenvalue λ i , an eigenvector v⃗ i is assigned so that

B ⋅ v⃗ i = λ i v⃗ i , i = 1...N , (A.34)

where not all eigenvalues must be different, see Section A.2.2.

A.4.2 Diagonalization

The central task is to ﬁnd eigenvalues and eigenvectors for a given matrix. As outlined
in Section A.2.2, each matrix can be transformed into Jordan canonical form:

C−1 ⋅ B ⋅ C = J .
280 | A Matrices and systems of linear equations

Since the transformation does not change the spectrum, the eigenvalues of A and J
are equal and can simply be read off from the diagonal of J:

λ i = J ii .

Thus, the spectrum is known if an appropriate C has been determined. If J is diagonal,

J ij = λ i δ ij ,

B is said to be diagonalizable and its desired eigenvectors are identical with the
columns of the matrix C (for matrices which can not be brought into diagonal form
there exist special methods which we shall not discuss here).
How can we obtain the transformation C? In practice, two different methods are
applied, sometimes also a combination of both methods. We shall present a short
overview of both, for more details see the literature, e.g., [1] or [2].

A.4.2.1 Transformation method

The idea is to ﬁnd different invertible transformation matrices P i that change and sim-
plify the original matrix in a speciﬁc way. This can be the zeroing of certain elements
(Jacobi transformation), or of complete rows or columns (Householder transforma-
tion). Thus, one has
B 󸀠 = P−1 −1 −1
k . . . P2 P1 B P1 P2 . . . P k .

The resulting matrix B󸀠 is similar to B and must not already be in diagonal form. Any-
way, it should be easier to handle than the original matrix. Finally, the better condi-
tioned B󸀠 can be further diagonalized by factorization.

A.4.2.2 Factorization method

Assume that the matrix B can be factored into a right matrix F R and a left one F L ac-
cording to
B = FL ⋅ FR . (A.35)
A new matrix can be built by commutation

B󸀠 = FR ⋅ FL . (A.36)

Conversely, multiplying (A.35) with F −1

L from the left side yields

F R = F −1
L ⋅B,

and, inserting this into (A.36),

B󸀠 = F −1
L ⋅ B ⋅ FL . (A.37)

Hence, B and B󸀠 are similar and have the same spectrum. Next, one constructs the
sequence
B (n+1) = F −1
nL ⋅ B
(n)
⋅ F nL = F nR ⋅ F nL (A.38)
A.4 Homogeneous systems of linear equations | 281

with B (0) = B and F nL , F nR as factors of B(n) . One can show that B(n) converges for
n → ∞ to a triangular matrix if the factorization (A.35) fulﬁlls certain properties [2].
This will be described in greater detail in the next section.

A.4.2.3 LU factorization
Now we use in (A.35) the LU decomposition introduced in detail in Section A.3.4

B=L⋅U.

The iteration rule (A.38) thus reads

B(n+1) = L−1
n ⋅B
(n)
⋅ Ln = U n ⋅ Ln (A.39)

and one complete iteration step looks as follows:

1. Factorize B in L and U.
2. Compute B󸀠 = U ⋅ L.
3. Put B := B 󸀠 .
4. If B is not in triangular form, go to 1.

At each step B comes closer to an upper triangular matrix. If B has reached this form
it will not change further, hence each upper triangular matrix is a ﬁxed point of the
map (A.39). This is evident because an upper triangular matrix B factorizes in a trivial
way into an upper triangular matrix U = B and the unit matrix as a ‘lower triangular
matrix’ L. Then one has L U = U L and B = B 󸀠 .

A.4.2.4 QR factorization
Not every matrix can be LU decomposed. Conversely, the so-called QR decomposition

B=Q⋅R

with R again as the upper triangular matrix but now an orthogonal matrix Q is always
possible. Instead of (A.39) we obtain

B(n+1) = Q −1
n
⋅ B(n) ⋅ Q n = R n ⋅ Q n . (A.40)

We list the following properties without proof, for details see [2]:
– If all eigenvalues of B have different absolute values, B (n) converges for n → ∞ to
an upper triangular matrix. The eigenvalues are ordered with increasing absolute
value in the diagonal.
– If k eigenvalues have the modulus |λ i |, i = 1 . . . k, B(n) converges to an upper
triangular matrix with the exception of a diagonal block matrix of order k whose
eigenvalues are λ i .
282 | A Matrices and systems of linear equations

A.4.3 Application: zeros of a polynomial

When given a polynomial of degree N in the form

N
P(x) = ∑ a k x k , (A.41)
k=0

whose N complex zeros

P(x i ) = 0 , xi ∈ C , i = 1...N

are to be determined, several methods are available. Here we introduce an algorithm

leading to a linear eigenvalue problem. First, we normalize (A.41)

b k = a k /a N , k = 0...N −1

and write the polynomial as

N−1
̃
P(x) = xN + ∑ bk xk ,
k=0

having of course still the same zeros as P. We construct the N × N-Frobenius matrix

0 −b 0
1 0 −b 1
( 1 0 −b 2 )
( )
F=(
(. . ) ) . (A.42)
(. . )
. .
( 1 −b N−1 )

One can show that

̃
det(F − x 1) = (−1)N P(x)
holds, i.e., the N eigenvalues of F correspond to the zeros of P. The matrix (A.42)
has Hessenberg form, this is a triangular matrix with an additional secondary diag-
onal. Such matrices can be effectively diagonalized, e.g., using the LAPACK routine
DHSEQR:

SUBROUTINE zeros(a,n,z)
C computes all zeros of the polynomial
C p = a(n)*x^n + a(n-1)*x^(n-1) ... a(0) = 0
C a(0:n) [in]: coefficients (not changed) real*8
C z(n) [out]: the complex zeros, complex*16
Bibliography | 283

C Double precision version

IMPLICIT REAL*8 (a-h,o-z)
COMPLEX*16 z(n)
DIMENSION a(0:n),zr(n),zi(n),frob(n,n),work(1000)
an=a(n); a=a/an ! normalization of polynomial
C Frobenius matrix (Hessenberg)
frob=0.
frob(1:n,n)=-a(0:n-1)
DO i=1,n-1
frob(i+1,i)=1.
ENDDO
C Lapack call, computes the eigenvalues of frob
CALL DHSEQR('E','N',n,1,n,frob,n,zr,zi,dumy,1,work,1000,info)
z=CMPLX(zr,zi)
a=a*an ! re-do normalization
END

Bibliography

[1] L. Hogben (editor), Handbook of Linear Algebra, Chapman & Hall (2007).
[2] J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, Springer (2010).
[3] W. H. Press, B. P. Flannery, S. A. Teukolsky and W. T. Vetterling, Numerical Recipes, Cambridge
Univ. Press (2007).
B Program library
B.1 Routines

We wish to supply and to describe some useful and often needed routines. They can be
compiled separately and linked by specifying the corresponding object ﬁle (.o) during
the compilation process of the main program. If the routines are gathered e.g., in a ﬁle
named fortlib.f, one compiles once with

gfortran -c fortlib.f

The option -c tells the compiler not to create an executable but rather generate an ob-
ject file that is named for this case fortlib.o. If a routine from the library is used, one
must modify the script file used for compilation, (e.g., make_f95 from Section 1.2.4),
by adding the fortlib object file. Finally, the script file should look like this (if LAPACK
is also needed):

gfortran -O2 $1.f -o $1 fortlib.o -L/usr/lib/ -lpgplot -llapack

The complete library contains the following sources:

Name purpose see book page

graphics
init graphics initialization 8
contur contour lines 56
contur1 contour lines
ccontu contour lines (colored) 199
image bitmap (colored) 20
ccircl initialize color circle 20
Runge–Kutta
rkg Runge–Kutta 67
drkg Runge–Kutta (double precision)
drkadt Runge–Kutta, adaptive time step (double prec.) 71
miscellaneous
tridag Thomas algorithm for tridiagonal matrix 278
ctrida Thomas algorithm for complex tridiag. matrix
dlyap_exp computation of (all) Lyapunov exponents 104
schmid Schmidt–Gram orthogonalization
volum volume spanned by vectors (function)
deter determinant of a matrix (function)
random_init initialization of random generator 241

https://doi.org/10.1515/9783110515145-010
B.2 Graphics | 285

In the following, we only present the comment headers of the respective routines.
Where not speciﬁed otherwise, implicit type declaration is assumed and real
variables are in single precision (real*4). The complete listing is provided in
‘http://www.degruyter.com/books/978-3-11-051513-8’.

B.2 Graphics

All routines in this section resort to programs of the PGPLOT library (see chapter 1).

B.2.1 init

SUBROUTINE init(x0,x1,y0,y1,size,aspect,frame)
c initializes a graphics window
c x0,x1 [In] x-region
c y0,y1 [In] y-region
c size [In] size of window in inches
c aspect [in] aspect ratio (y/x)
c frame [In] if frame='Y' or 'y' a box is plotted
c

B.2.2 contur

SUBROUTINE contur(a,idim,jdim,n)
c Plots n contour lines of an array a. The contour values are chosen
c equidistant between min(a) and max(a).
c If n=0: only the a=zero line is plotted.
c a(idim,jdim) [In] 2D array
c idim, jdim [In] dimensions of a
c n [In] number of lines

B.2.3 contur1

SUBROUTINE contur1(a,idim,jdim,n)
c plots contour lines as 'contur'
c n=0 [In] only zero line plotted
c n>0 n lines between 0 and max(a) (equidistant)
c n<0 -n lines between min(a) and 0 (equidistant)
c rest see 'contur'
286 | B Program library

B.2.4 ccontu

SUBROUTINE ccontu(a,idim,jdim,n)
c plots contour lines as 'contur'
c the lines are plotted with ascending color indices from 2 through
c 2+n-1.
c rest see contur

B.2.5 image

SUBROUTINE image(a,idim,jdim,ic0,ic1,a0,a1)
c plots a colored bitmap of a real*4 array 'a'
c a(idim,jdim) [In] array whose values are pixeled
c idim, jdim [In] dimensions of a
c ic0, ic1 [In] defines the color range (ic0=min(a), ic1=max(a))
c a0,a1 [In] minimum and maximum for color values,
c if both =0 min(a) and max(a) is used.

B.2.6 ccircl

SUBROUTINE ccircl(i0,i1)
c loads the color register from i0 through i1 with a periodic color circle.

B.3 Runge–Kutta

All four routines are based on the 4th-order Runge–Kutta scheme for ODEs.

B.3.1 rkg

SUBROUTINE rkg(y,t,n,dt,eq)
c integrates the ODE system defined in eq (external) over one step
c [t,t+dt] by 4th order Runge--Kutta
c y [In/Out] dependent variables y(n)
c t [In] independent variable
c n [In] number of eqs.
c dt [In] step size
c eq [In] user-suppl. subroutine, defines the ODE system
B.4 Miscellaneous | 287

B.3.2 drkg

SUBROUTINE drkg(y,t,n,dt,eq)
C same as rkg, but in DOUBLE PRECISION (y,t,dt)

B.3.3 drkadt

SUBROUTINE drkadt(y,t,n,dt,eps,eqs)
c same as drkg, but with adaptive step size
c REAL*8 y [In/Out] dependent variables y(n)
c REAL*8 t [In] independent variable
c n [In] number of eqs.
c REAL*8 dt [In/Out] step width, to be computed
c REAL*8 eps [In] desired accuracy, dt is adapted such
c that the error is < eps
c eqs [In] user-suppl. subroutine
c resorts to routine drkg

B.4 Miscellaneous

B.4.1 tridag – Thomas algorithm

SUBROUTINE tridag(a,b,c,d,n)
c solves a tridiagonal inhomogeneous algebraic system
c a(i)*x(i-1) + b(i)*x(i) + c(i)*x(i+1) = d(i), i=1..n
c using the Thomas algorithm
c a(n) [In] lower diagonal
c b(n) [In] diagonal, attention:b is overwritten!
c c(n) [In] upper diagonal
c d(n) [In/Out] r.h.s. (in), overwritten by solution x(n) (out)

B.4.2 ctrida

SUBROUTINE ctrida(a,b,c,d,n)
c same as tridag, but for a complex valued system (a,b,c,d)
288 | B Program library

B.4.3 dlyap_exp – Lyapunov exponents

SUBROUTINE dlyap_exp(eqs,eql,dt,tprel,tend,ntau,n,nfmax,fly)
c computes the nfmax largest Lyapunov exponents of the system 'eqs'
c all in DOUBLE PRECISION
c eqs [In] user-suppl. subroutine, defines the complete ODE system
c eql [In] user-suppl. subroutine, defines the linearized ODE system
c REAL*8 dt [In] step size for RK4
c REAL*8 tprel [In] preliminary iterations from when 'fly' is computed
c REAL*8 tend [In] maximum t for computation
c ntau [In] after tau=ntau*dt Lyap. exp. update
c n [In] number of eqs.
c nfmax [In] number of computed Lyap exp.
c REAL*8 fly [In/Out] in: initial values for main trajectory
c Out: fly(i) contains the i-th Lyap. exp.
c routines needed: drkg, schmid, volum

B.4.4 schmid – orthogonalization

SUBROUTINE schmid(x,n,m)
c orthonormalizes the m vectors x(i,j)
c by a Schmidt--Gram algorithm
c REAL*8 x(i,j) [In/Out], i=1..n (components of vector j),
c j=1..m
c routine needed: norm

SUBROUTINE norm(x,n)
c normalizes the vector REAL*8 x(n), L2-norm

B.4.5 FUNCTION volum – volume in n dimensions

REAL*8 FUNCTION volum(x,n,m)

c computes the volume of an m-dimensional parallelepiped spanned by
c the m vectors x(i,j) in m-dimensional vector space
c REAL*8 x(i,j) [In], i=1..n (components of vectors j), j=1..m
c REAL*8 volum [Out] volume
c routine needed: deter
B.4 Miscellaneous | 289

B.4.6 FUNCTION deter – determinant

REAL*8 FUNCTION deter(a,n)

c computes the determinant of an n x n matrix
c REAL*8 deter [Out] determinant
c REAL*8 a(n,n) [In] matrix
c n [In] dimensions of a

B.4.7 random_init – random numbers

SUBROUTINE random_init
c initializes the FORTRAN95 random number generator RANDOM_NUMBER
c with the system clock. No parameters
C Solutions of the problems
We show solutions to the exercises, in particular to those solved with ‘paper and pen-
cil.’

C.1 Chapter 1

Problems 1.3.3

1. We search a solution of x = f (2) (x) = f(f(x)) with f(x) = ax(1 − x),

x = af(x)(1 − f(x))
= a2 x(1 − x)(1 − ax(1 − x)) = a2 x(1 − (1 + a)x + 2ax2 − ax3 ) . (C.1)
Two particular solutions are already given, namely those solving x = f(x):
x=0, x = 1 − 1/a
and the degree of (C.1) can be lowered by two by polynomial long division. The re-
maining polynomial is of 2nd degree
ax2 − (1 + a)x + 1 + 1/a = 0 , (C.2)
its roots correspond to the period two cycle asked for:
1+a 1 √ 2
x p1 ,p2 = ± a − 2a − 3 . (C.3)
2a 2a
Real solutions exist only for a nonnegative radicant, which is equivalent to
a≥3.
2. We study the stability of (C.3) in the region a > 3. A linear stability analysis yields
for the perturbations ϵ of (C.3)
ϵ n+1 = A ϵ n
with the ampliﬁcation factor
d (2) 󵄨󵄨󵄨󵄨
A= f (x)󵄨󵄨 .
dx 󵄨󵄨 x p ,p
1 2

Forming the derivative of the second iterate (C.1) we obtain

A = −4a3 x3 + 6a3 x2 − 2(1 + a)a2 x + a2 .
Using (C.2) we ﬁnd after some calculations
A = −a2 + 2a + 4 .
The periodic solution (C.3) becomes unstable if
|A| > 1 .
This is the case if either a < 3, which is excluded from the existence of (C.3), or if
a > 1 + √6 .

https://doi.org/10.1515/9783110515145-011
C.2 Chapter 2 | 291

C.2 Chapter 2

Problems 2.1.4

1. For K = 0:
y n+1 = y n , x n+1 = x n + y n ,
horizontal rows of dots, which are dense if y n is irrational, otherwise points with gaps.

2. The chain length after N iterations is given as

ℓ = x N − x0 ,

thus for K = 0
ℓ = N y0 ,
a linear function of the initial value y0 . For small values of K the length deviates from
the straight line for certain rational y0 = 1/M, M integer, and locks in the form of steps
of a staircase. For larger K this effect increases and chaotic regions occur in addition;
see Figure C.1.

104 104

8000 8000

6000 6000
K = 0.2 K = 0.5
4000 4000

2000 2000

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

104 104

8000 8000

6000 6000
K = 0.7 K = 1.0
4000 4000

2000 2000

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

Fig. C.1: Chain length as function of y 0 for several K after 10 000 iterations. For larger K an irregular
staircase (devil’s staircase) occurs.
292 | C Solutions of the problems

C.3 Chapter 3

Problems 3.1.2

We study the map x n+1 = f(x n ) with

1 1
f(x) = a ∆t x (1 − + − x)
a a∆t
and
df 1 1
= a ∆t (1 − + − 2x) .
dx a a∆t
In particular we wish to compute the stability of the fixed point
xs = 1 − 1/a
and find for the amplification factor
df 󵄨󵄨󵄨󵄨
A= 󵄨 = 1 + (1 − a)∆t .
dx 󵄨󵄨󵄨xs
For xs > 0, one must have a > 1 and therefore A ≤ 1. The numerical stability limit is
thus given by A = −1, or
2
∆t = .
a−1

C.4 Chapter 4

Problems 4.3.2

̃ 0 + ∆x)) on the left-hand side of (4.22) can be written as a Taylor

1. The expression f(y(x
expansion with respect to g:
̃ 0 + ∆x)) = f(y(x0 ) + g)
f(y(x
df 1 d2 f 2
= f(y(x0 )) + g+ g + O(∆x3 ) .
dy 2 dy2
Now taking g from the expansion
1
̃ 0 + ∆x) = y(x0 ) + y󸀠 ∆x + y󸀠󸀠 ∆x2 + O(∆x3 ) ,
y(x
⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
2
≡g

we obtain up to the order ∆x2

df 1 1 d2 f 󸀠2 2
̃ 0 + ∆x)) = f(y(x0 )) +
f(y(x (y󸀠 ∆x + y󸀠󸀠 ∆x2 ) + y ∆x + O(∆x3 ) .
dy 2 2 dy2
Inserting into (4.22) yields
∆x2 df 󸀠 ∆x3 1 df 󸀠󸀠 1 d2 f 󸀠2
̃ 0 + ∆x) = y(x0 ) + ∆x ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
y(x f(y(x0 )) + y + [ y + y ] +O(∆x4 ) .
2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟
dy 2 dy2
2 ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
2 dy
y󸀠
y󸀠󸀠 1 󸀠󸀠󸀠
2y
C.4 Chapter 4 | 293

On the other hand, when expanding the exact solution we obtain

1 󸀠󸀠 2 1 󸀠󸀠󸀠 3
y(x0 + ∆x) = y(x0 ) + y󸀠 ∆x +
y ∆x + y ∆x + O(∆x4 ) .
2 6
Subtracting the two last equations ﬁnally yields the error
1 󸀠󸀠󸀠 3
̃ 0 + ∆x) − y(x0 + ∆x)| =
|y(x |y |∆x
12
which is of order ∆x3 .

2. The symplectic scheme of (4.5) reads

φ n+1 = φ n + ω n+1 ∆t
(C.4)
ω n+1 = ω n − (αω n + Ω20 φ n ) ∆t
or
φ φ
A( ) = B( )
ω n+1 ω n
with
1 −∆t 1 0
A=( ) , B=( ) .
0 1 −Ω20 ∆t 1 − α∆t
Thus, Q s in (4.21) reads
Q s = A−1 B .

Problems 4.3.3.3

1. Meaning of the expressions:

α 1 : increase of prey by reproduction, exponential growth
α 2 : limitation of prey by predator, eating
β 1 : decease of predator, exponential decrease
β 2 : increase of predator due to prey food, eating

After scaling:
t = τ t̃ , n i = γ i ñ i
with
τ = 1/β 1 , γ1 = β 1 /β 2 , γ2 = β 1 /α 2
one arrives at the form (4.27) with the sole remaining (control) parameter
a = α 1 /β 1 .
2. The ﬁxed points are computed from
0 = a ñ 1 − ñ 1 ñ 2
0 = −ñ 2 + ñ 1 ñ 2
as
294 | C Solutions of the problems

(i) ñ 1 = ñ 2 = 0
Stable for a < 0, node, unstable for a > 0, saddle node.

(ii) ñ 1 = 1 , ñ 2 = a.

This fixed point exists only for a > 0 since ñ i as a concentration is by definition posi-
tive.
Linearizing:
ñ 1 1 u1
( ) = ( ) + ( ) e λt
ñ 2 a u2
leads after insertion in (4.27) to the solvability condition
λ 1
det ( ) = λ2 + a = 0
−a λ
and
λ12 = ±i√a .
Hence the second fixed point is always marginally stable (Re λ = 0), but has a nonva-
nishing imaginary part. This is denoted as a center.

3. Time derivative of W yields

d ñ 1 /dt d ñ 2 /dt
dW/dt = d ñ 1 /dt + d ñ 2 /dt − −a
̃n1 ñ 2
If we insert (4.27), the right-hand side vanishes, showing that W is conserved.

4. Numerical solutions are obtained straightforwardly using Runge–Kutta; see Fig-

ure C.2.

0 Fig. C.2: Numerical solution for a = 1 and differ-

0 1 2 3 4 5 ent initial conditions, n2 over n1 .
C.4 Chapter 4 | 295

Problems 4.4.5

1. Transformation to a constantly rotating frame of references turns (4.55) into (tildes

removed):
ẋ = v
ẏ = w
x (C.5)
v̇ = −2Ω0 w + Ω20 x −
rα
y
ẇ = 2Ω0 v + Ω20 y −
rα
The system (C.5) possesses infinitely many fixed points lying on the circle √x20 + y20 =
−2/α
r0 = Ω0 , each corresponding to the same circular orbit in the nonrotating frame.
Without loss of generality we can take the fixed point:
x 0 = r0 , y0 = 0 , v0 = w0 = 0 .
Linearization around this point yields for the deviations q⃗ the system
λ q⃗ = J q⃗
with the Jacobi matrix
0 0 1 0
0 0 0 1
J=( 2 ) . (C.6)
αΩ0 0 0 −2Ω0
0 0 2Ω0 0
The eigenvalues of J read
λ12 = ±Ω0 √α − 4 , λ34 = 0 .
The circular orbit is unstable if one of the eigenvalues has a positive real part, which
is the case for α > 4. If we compute the total energy of the planet in the nonrotating
system it reads
α−4 1
E0 = ,
2(α − 2) r0α−2
a value becoming positive if α > 4. Thus, the planet being initially on a circle could
leave the system and vanish at infinity, independently of r0 (this reasoning applies of
course only if the potential energy goes to zero for r → ∞, i.e., for α > 2). For α < 4
one finds
λ12 = ±iω
with
ω = Ω 0 √4 − α
and the planet will oscillate around the circular orbit when slightly disturbed. The
condition for a closed orbit is resonance
k
ω = Ω0 ,
ℓ
296 | C Solutions of the problems

where k and ℓ are two coprime integers. The orbit is closed after one revolution if ℓ = 1,
leaving the condition for α

α = 4 − k2 , k = 1, 2, 3, . . .

In fact there are inﬁnitely many sharp values for α where circular orbits stay closed
when slightly perturbed. We mention the ﬁrst three ones, together with the behavior
of the force on distance and the potential:
1. α = 3: Kepler potential, F ∼ −1/r2 , U ∼ −1/r.
2. α = 0: 2D-harmonic oscillator, F ∼ −r, U ∼ r2 .
3. α = −5: anharmonic oscillator, F ∼ −r6 , U ∼ r7 .

To show numerically the existence of closed orbits we integrate the system (C.5) with
the initial conditions

x(0) = 1 , y(0) = 0 , v(0) = 0 , w(0) = 1 + ϵ

for 100 revolutions and average the x(t i ) − x(0) value after each revolution i:

1 100
⟨δx⟩ = ∑ |x(t i ) − x(0)|
100 i

where t i is found from y(t i ) = 0 and w(t i ) > 0. The small deviation ϵ = 0.001 starts
the instability. The result for α = −6 through α = 4 is shown in Figure C.3. Clearly,
the deviations after one revolution get very small for the selected values α = −5, 0, 3,
indicating a closed orbit.

0.1000000
⟨δx⟩

0.0100000

0.0010000

0.0001000
Kepler

0.0000100 anharmonic
oscillator harmonic
oscillator
α
0.0000010
–6 –5 –4 –3 –2 –1 0 1 2 3 4

Fig. C.3: Averaged deviation from initial value x as a function of α.

C.4 Chapter 4 | 297

2. is left for the reader.

3. The Lagrangian points L1 through L3 are located on the x-axis. Thus it suffices to
evaluate the Jacobi potential (4.46) at y = 0:

1−μ μ x2
V(x, 0) = − − − =0.
|x + μ| |x + μ − 1| 2

Zeroing its derivative leads to the conditions for the ﬁxed points
1−μ μ
∂ x V(x, 0) = (x + μ) + (x + μ − 1) − x = 0 .
|x + μ|3 |x + μ − 1|3

After some calculations one ﬁnds a 5th degree polynomial:

P μ (x) = − x5 + x4 (2 − 4μ) + x3 (6μ − 6μ 2 − 1)

+ x2 (6μ 2 − 4μ 3 − 2μ) + x(2μ3 − μ 4 − μ 2 )
+ sign(x + μ) [x2 (1 − μ) + x(4μ − 2μ2 − 2) + (1 − μ)3 ]
+ μ sign(x + μ − 1)(x + μ)2 . (C.7)

Due to the moduli one must distinguish between the three cases
1. x < −μ, left of m2 , sign(x + μ) = −1, sign(x + μ − 1) = −1
2. −μ < x < 1 − μ, between m2 and m3 , sign(x + μ) = 1, sign(x + μ − 1) = −1
3. 1 − μ < x, right of m3 , sign(x + μ) = 1, sign(x + μ − 1) = 1.

The roots of (C.7) can be computed for instance using the program zeros of Sec-
tion A.4.3. Figure C.4 shows the Lagrangian points depending on the mass relation μ.

0.5

–0.5

–1
Fig. C.4: Solid: x-coordinates of the three La-
grangian points L 1 , L 2 , L 3 over the mass param-
0 0.2 0.4 0.6 0.8 1 eter μ; dashed: position of the two main bodies.
298 | C Solutions of the problems

Problems 4.5.6

1. Thermalization: Multiplying (4.74) with v⃗ i and summation over all particles for
F⃗ i = 0 yields
1 d
∑ v2 = −γ ∑ v2i . (C.8)
2 dt i i i

From the equipartition theorem in 2D it follows

1
∑ v2 = NT
2 i i

and then from (C.8)

d t T = −2γT = −2γ0 (T − Ts )
or
T − Ts ∼ exp(−2γ0 t) .

2. 2D-equilibrium conﬁgurations:
(i) If only the four nearest neighbors (blue) are considered, the potential energy reads

U GQ = 4 U(r)

for a square lattice, and

U GH = 6 U(r)
for a hexagonal grid, where we take U from (4.59). The equilibrium positions are
found from
dU GQ,H
=0 (C.9)
dr
as r Q = r H = 21/6 ≈ 1.12246.
(ii) Including also the red particles, we ﬁnd

U GQ = 4 U(r) + 4 U(√2r)

and
U GH = 6 U(r) + 6 U(√3r)
and from the equilibrium condition (C.9)

r Q = (65/36)1/6 ≈ 1.10349 , r H = ((27 + 1/27)/14)1/6 ≈ 1.11593 .

(iii) Including all plotted particles, we ﬁnally obtain

U GQ = 4 U(r) + 4 U(√2r) + 4 U(2r)

and
U GH = 6 U(r) + 6 U(√3r)) + 6 U(2r) ,
C.4 Chapter 4 | 299

1/6 1/6
1 + 2−6 + 2−12 1 + 3−6 + 2−12
r Q = (2 ) ≈ 1.10100, r H = (2 ) ≈ 1.1132.
1 + 2−3 + 2−6 1 + 3−3 + 2−6
The more stable conﬁguration is that with the lower energy. Computing U G for the
given equilibrium positions, one ﬁnds for all cases

U GH < U GQ ,

independently on the number of neighbors taken into account. Hence the lattice
should manifest a hexagonal geometry for low temperatures, in agreement with
Figure 4.14, left frame.

Problems 4.7.5

1. Linearizing (4.102) with respect to the upper rest position yields

u̇ 1 = u 2
u̇ 2 = −Ω20 (1 + a sin ωt) ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟
sin(π + u 1 )
=−u1

or
ü − Ω20 (1 + a sin ωt) u = 0 (C.10)
with u = u 1 . The same scaling as in Section 4.7.4 leads to

ü − (p + 2b sin 2t)u = 0 . (C.11)

Substituting the ansatz

u(t) = u 0 cos(t + α) e λt ,
and neglecting the term sin(3t + α), one obtains

(λ2 − 1 − p) (cos t cos α − sin t sin α)

− 2λ (sin t cos α + cos t sin α) − b (sin t cos α − cos t sin α) = 0
or
sin t [−(λ2 − 1 − p) sin α − (2λ + b) cos α]
+ cos t [(λ2 − 1 − p) cos α − (2λ − b) sin α] = 0 .

The two square brackets must vanish, leading to the system

−λ2 + 1 + p −2λ − b sin α

( )( )=0.
−2λ + b λ2 − 1 − p cos α

Its solvability condition constitutes a polynomial of 2nd degree in λ2 having the roots

λ2 = p − 1 ± √b 2 − 4p .
300 | C Solutions of the problems

Re λ = 0

p
Fig. C.5: The upper rest position is stable in the
1 shaded area.

For a nonpositive Re(λ), λ2 must be real valued and λ2 ≤ 0. The ﬁrst condition leads
to
b ≥ 2√p ,
the latter to
b ≤ 1+p.
In addition, 0 ≤ p ≤ 1 must hold. This results in a stability diagram sketched in Fig-
ure C.5.

2. The damped Mathieu equation is transformed to the dimensionless normal form

̃ =0
ü + β u̇ + (p + 2b sin 2t)u (C.12)

by the scaling of Section 4.7.4 and β = 2α/ω. To ﬁnd the Floquet exponents one nu-
merically integrates the system (4.104), but now with

0 1
L(t)̃ = ( ) , (C.13)
−(p + 2b sin 2t)̃ −β

from t ̃ = 0 through t ̃ = T = π. Determining the monodromy matrix according to (4.106)

one ﬁnds the Floquet multipliers (4.107).
Figure C.6 shows the contour lines max(|σ 1 |, |σ 2 |) = 1 that mark the onset of in-
stability. The tips of the tongues no longer reach the p-axis as in Figure 4.30 but are
shifted to ﬁnite critical amplitudes that are necessary to overcome friction.
C.5 Chapter 5 | 301

Fig. C.6: Stability chart for the damped Mathieu equation (C.12) with β = 0.1.

C.5 Chapter 5

Problems 5.6

1. Exact solution of (5.4).

Let α > 0. The set (5.4) is written dimensionless with the scaling

̃ ,
t = t/α (x, y) = (x,̃ y)̃ ⋅ g/α 2

and reads

ẍ̃ = −ẋ̃ + β̃ ỹ (C.14)

ÿ̃ = −ẏ̃ − 1 . (C.15)

Only one parameter β̃ = β/α 2 is left. We leave all tildes. Different ways to solve the
system are possible. One can compute y from (C.15) and obtains
T
y(t) = f(t) − t (C.16)
f(T)
with
f(t) = 1 − e−t .
302 | C Solutions of the problems

The boundary conditions y(0) = y(T) = 0 are already implemented in (C.16). From
(C.14) and (C.15) y can be eliminated:

d4t x + 2d3t x + d2t x = −β .

Integrating twice yields

β 2
ẍ + 2ẋ + x = − t +At+B.
2
Putting
β 2
x(t) = X(t) + a + b t −
t
2
with A = b − 2β, B = a + 2b − β, leads to the homogeneous equation

Ẍ + 2Ẋ + X = 0 ,

which is solved by
X(t) = (c t − a) e−t .
Here, x(0) = 0 is already included. Thus one ﬁnds for x
β 2
x(t) = (c t − a) e−t + a + b t − t . (C.17)
2
The coefficients b, c are determined by inserting (C.17) and (C.16) in (C.14) and com-
paring the coefficients:
βT
c= , b=c+β.
f(T)
The remaining coefficient a follows from the boundary condition x(T) = L:
T 2 β(1 + 3e−T ) L − βT
a=− + .
2f 2 (T) f(T)
2. Solution of the nonlinear Schrödinger equation.
The solution given is veriﬁed straightforwardly by insertion.
Discussion for arbitrary C: The integral found by separation of variables from
(5.45) has no analytic form. We discuss a ‘graphic solution’ coming from the analogy
to a classical one-dimensional motion of a mass point in a potential.
Writing (5.42) in the form
dV(Φ)
Φ󸀠󸀠 = −EΦ + γ Φ3 = − (C.18)
dΦ
with
E 2 γ 4
V(Φ) = Φ − Φ , (C.19)
2 4
(C.18) is identical with the equation of motion of a mass point with m = 1 in the poten-
tial (C.19) (x plays the role of time, Φ that of the position). Equation (5.45) corresponds
to the conservation of energy and C/2 denotes the total energy
C 1 󸀠 2
= (Φ ) + V(Φ) .
2 2
C.6 Chapter 6 | 303

V V V V
E2 /4γ

Φ Φ Φ Φ
E 2 /4γ

(a) (b) (c) (d)

Fig. C.7: The four different potentials. Nonlinear waves can exist for (a), (b) and (d).

We examine the four cases

(a) E > 0, γ > 0, Figure C.7 (a). For C < E2 /2γ, nonlinear oscillations exist around
Φ = 0. They have the square amplitude

Φ2m = (E − √E2 − 2γC) /γ (C.20)

and correspond to plane, nonlinear waves in position space. For C > E2 /2γ or
C < 0, Φ diverges for x → ±∞.
(b) E > 0, γ < 0, Figure C.7 (b). Nonlinear waves for C > 0.
(c) E < 0, γ > 0, Figure C.7 (c). Diverging solutions for all C.
(d) E < 0, γ < 0, Figure C.7 (d). If E2 /2γ < C < 0, nonlinear waves about Φ = ±√E/γ.
For C > 0 nonlinear waves about Φ = 0 with square amplitude (C.20).

C.6 Chapter 6

Problems 6.4.4

1. We obtain (6.134) after scaling

̃ ,
t = t/α x = x̃ √D/α , u = ũ ⋅ (α/β) .

2.
(i) Linearization with u 0 = 0, u = u 0 + w:

∂ t w = ∂2xx w + w

yields with w ∼ e λt e ikx

λ(k) = 1 − k 2 .
The ﬁxed point is unstable for any plane wave with |k| < 1.
(ii) Linearization about u 0 = 1:
∂ t w = ∂2xx w − w
yields
λ(k) = −1 − k 2 .
The ﬁxed point is stable for all k.
304 | C Solutions of the problems

3. Inserting u = u(ξ) in (6.134) leads to

−v∂ ξ u = ∂2ξξ u + u − u 2 .
Since u → 0 for ξ → ∞, we can linearize for large ξ according to:
−v∂ ξ u = ∂2ξξ u + u .

The ansatz u ∼ e−σξ , σ > 0 yields

1
v=σ+
σ
and hence a lower limit for the speed:
vmin = 2 for σ=1.
4. Inserting (6.136) in (6.134) leads to
2C e−κξ (κ 2 − 1 − κv) − C2 e−2κξ (2κv + 4κ 2 + 1) = 0 .
Since this must be valid for all ξ , the expressions in the brackets must vanish sepa-
rately and one computes
v = −5/√6 ≈ −2.04 , κ = 1/√6 .
By the substitution
C = e κx0
it becomes evident that C shifts the front to the right side by x0 .

5. The stationary form of (6.134) can be written as

dV(u)
u 󸀠󸀠 = u 2 − u = −
du
with
u2 u3
V(u) = −
2 3
and, as in problem 5.6(2), interpreted as one-dimensional motion in the potential V.
The ‘energy law’ thus reads
1 󸀠2
(u ) + V(u) = E . (C.21)
2
For the special value E = 1/6, the ‘motion’ is along the homoclinic orbit of the ﬁxed
point u = 1. Thereby, u goes from u = 1 to u = −1/2 and back to u = 1; see Figure C.8.
For E = 1/6, (C.21) may be integrated by the separation of variables:

du 2u + 1
x − x0 = √3 ∫ = 2 atanh (√ ) .
√1 − 3u 2 + 2u 3 3
After some short manipulations one obtains the stationary solution
3
u 0 (x) = 1 − 2
, (C.22)
2 cosh ((x − x0 )/2)
corresponding to a localized solution at x = x0 with u 0 (x → ±∞) = 1 and u 0 (x0 ) =
−1/2.
C.6 Chapter 6 | 305

V = 1/6

u = –1/2 u=1 u

Fig. C.8: Potential and homoclinic orbit for

E = 1/6.

6. We examine the stability of (C.22) with x0 = 0. Linearization of (6.134) about u 0

yields (6.137) with the potential
3
V(x) = 2u 0 − 1 = 1 − .
cosh2 (x/2)
Expanding
3 2
V(x) ≈ −2 + x ,
4
turns (6.137) into the equation of the quantum mechanical harmonic oscillator:
3 2
−d2xx w + x w = (2 − λ)w .
4
The ground state is a Gaussian w ∼ exp(−(√3/4) x2 ), for its ‘energy’ one ﬁnds (2 − λ) =
√3/2 or λ = 2 − √3/2 ≈ 1.13. Hence, λ > 0 and the solution (C.22) is proved to be
unstable.
The following MATLAB code computes the lowest three states of (6.137) with
d x w(−L/2) = d x w(L/2) = 0 with L = 50 and n = 1000 mesh points using a ﬁnite
difference scheme. The output is shown in Figure C.9.

clear;
n=1000;
dx=50/(n+1); dx2=dx^2;
x=-25+dx:dx:25-dx;
a=-1/dx2*ones(n-1,1); % tridiagonal matrix
for k=1:n
b(k)=2/dx2+1-3/cosh(x(k)/2)^2;
end
b(1)=b(1)-1/dx2; b(n)=b(n)-1/dx2;
m=diag(a,1)+diag(a,-1)+diag(b,0); % boundary conditions
[v,e]=eigs(m,3,'sa'); % compute eigenvalues and vectors
plot(x,v(1:n,1),x,v(1:n,2),x,v(1:n,3)); % output
-diag(e)
306 | C Solutions of the problems

0.2

0.15 1
0.1
3
0.05
0

–0.05
2
–0.1

–0.15
–25 –20 –15 –10 –5 0 5 10 15 20 25

Fig. C.9: The three lowest states.

The values for λ are printed and read λ = 1.25, 0, −0.75. Note that the second
(marginal, λ = 0) mode corresponds to a lateral shift of u 0 with w ∼ d x u 0 . It can
be easily shown that w = d x u 0 is an exact solution of (6.137) with λ = 0.

C.7 Chapter 7

Problems 7.4.3

1. With the transformation

Ψ = Ψ̃ exp(iω0 t)
ω0 vanishes from (7.110).
Inserting (7.111) in (7.110) yields (from here ω0 = 0):

iΩ = ε − (1 + ic3 ) A2

or, after separating real and imaginary parts,

ε − A2 = 0 , Ω + c3 A2 = 0

and from there

A = √ε , Ω = −c3 ε .

2. Linear stability analysis about (7.111). Inserting

Ψ = (A + w(x, t)) exp(iΩt)

in (7.110) and linearization with respect to w gives

ẇ = (1 + ic1 ) ∂2xx w − (1 + ic3 ) ε (w + w∗ )

ẇ ∗ = (1 − ic1 ) ∂2xx w∗ − (1 − ic3 ) ε (w + w∗ ) .
C.7 Chapter 7 | 307

With w ∼ exp(λt) exp(ikx) one arrives at the linear system

(λ + (1 + ic1 ) k 2 + (1 + ic3 ) ε) w + (1 + ic3 ) ε w∗ = 0

(1 − ic3 ) ε w + (λ + (1 − ic1 ) k 2 + (1 − ic3 ) ε) w∗ = 0 .

Its solvability condition is a 2nd degree polynomial in λ having the roots

λ1,2 = −k 2 − ε ± √ε2 − c21 k 4 − 2c1 c3 εk 2 . (C.23)

Expansion with respect to k 2 yields

λ = −(1 + c1 c3 ) k 2 + O(k 4 )

for small k. Thus, λ > 0 for arbitrarily small k if

1 + c1 c3 < 0 .

All plane wave disturbances with

0 ≤ k 2 ≤ k 2c

and
1 + c1 c3
k 2c = −2ε
1 + c21
are growing exponentially in time, Figure C.10. This is called Benjamin–Feir instability.

Fig. C.10: Largest λ from (C.23) for 1 + c 1 c 3 < 0

(solid) and 1 + c 1 c 3 > 0 (dashed).

3. Figure C.11 shows the real part of Ψ in an xt-diagram for c1 = 1/2, c3 = −3. It can be
easily seen that the initial condition (7.111) becomes unstable and after t ≈ 250 gives
way to a spatio-temporal chaotic evolution.

4. The largest Lyapunov exponent Λ is computed determining ﬁrst a numerical solu-

tion Ψ N (x, t) as in 3. This is the reference trajectory; compare also Section 4.6.3.1. In
308 | C Solutions of the problems

2000

1500

1000

500

0 Fig. C.11: xt-diagram of Re(Ψ) from t = 0

0 200 400 600 800 through t = 2000.

parallel, one integrates the linearized DEQ for the complex valued deviations from the
reference trajectory u(x, t):

∂ t u(x, t) = [ε+(1 + ic1 ) ∂2xx ] u(x, t)

− (1 + ic3 )[2|Ψ N (x, t)|2 u(x, t) + Ψ N (x, t)2 u ∗ (x, t)]

which is obtained by substituting

Ψ(x, t) = Ψ N (x, t) + u(x, t)

into (7.110) and linearizing around Ψ N . For u(x, 0) we choose a randomly distributed
but normalized initial condition

∑ |u(x j , 0)|2 = 1 , with x j = j∆x .

After a certain time δt (e.g., δt = 200∆t) the growth of |u| in the interval δt is com-
puted:
1/2

s(δt) = [∑ |u(x j , δt)| . 2]

[ j ]
Thereafter u(x, δt) is again normalized. The continued numerical integration then
yields s(2δt) and so on. At the end, the largest Lyapunov exponent is found as the
mean value of all local exponents according to

1 K 1
Λ = lim ∑ ln s(kδt) . (C.24)
K→∞ K k=1 δt

For each single value of c3 the integration is carried on up to t = 34 000. After chang-
ing (increasing or decreasing) c3 , the evaluation of the corresponding Lyapunov ex-
ponent according to (C.24) is started after an initial run until t = 15 000.
C.7 Chapter 7 | 309

Fig. C.12: Largest Lyapunov exponent Λ of the complex Ginzburg–Landau equation over c 3 for
c 1 = 1/2. A transition to chaos showing a wide hysteretic region results.

Figure C.12 depicts Λ over c3 for a ﬁxed c1 = 1/2. From (7.112) chaotic dynamics is
expected for c3 ≤ −2. Obviously, the behavior of Λ depends on the previous states
and a large hysteretic region shows up in which both chaotic and regular dynamics
may emerge at the same values of c3 .
D README and a short guide to FE-tools
D.1 README

File with description of the program examples on

http://www.degruyter.com/books/978-3-11-051513-8

The codes are ordered along their appearance in the book chapters.
In each subdirectory a script file

make_f95

exists for compilation.

In addition the packages

pgplot, installed in /usr/lib/

lapack, installed in /usr/lib/lapack/
fftpack, installed in /usr/lib/ (only for chapter 7)

are required.

In this directory:

README this file (see also appendix D.1)

fortlib.f the library described in appendix B
fortlib.o object files, linked by make_f95

TI stands for Terminal Input

------------
chap2/

lyap_2d.f
computes the largest Lyapunov exponent of the standard map,
sect. 2.2.3.
TI: K (control parameter)

https://doi.org/10.1515/9783110515145-012
D.1 README | 311

dbc_slp.f
decimal to binary converter from sect. 2.5.1.4
TI: following the learning phase, input of the decimal number to
be converted

kohonen_colormap.f
color map after Kohonen as shown in Figure 2.16.
TI: none

tsp.f
Traveling Salesman Problem from sect. 2.5.2.3
TI: R (radius of the initial circle)

------------
chap3/

henon_heiles.f
Plots the Poincaré section and trajectories from the Henon-Heiles
model from sect. 3.3.3.1, compare Figure 3.5. The value 'e' for the
energy can be modified in the code.
TI: initial values set by mouse

------------
chap4/

reduc_3_b_p.f
Plots and computes trajectories of the reduced three body problem
from sect. 4.4.4.4.
mu=1/4, can be modified in the code.
TI: value of energy 'e'. Then set initial value by mouse in allowed
region (red).

molecular_dynamics.f
Molecular dynamics simulation from sect. 4.5.4. On the left-hand side
particles are depicted in config. space; the right-hand side shows
the pair correlation function.
Simulations are also possible in a constant gravity field.
TI: temperature (0.1 - 2.0), gravitation (ca 1), tend (e.g. 200)
312 | D README and a short guide to FE-tools

pendulum_lyapunov_exp.f
Bifurcation diagram (bottom) and two Lyapunov exponents of the
driven pendulum from sect. 4.6.3.6.
TI: region for a, resolution

------------
chap6/

shallow_water.f
Solution of 2D shallow water eqs. from sect. 6.2.4.2 with FTCS.
Here for two oscillating point sources.
TI: none

grid_contur.f
mesh_generator.f
laplace_solver.f
Program tools for FE, see separate short guide in fe_guide.pdf

house.grd, house.knd, house.ppm

kidney.grd, kidney.knd, kidney.ppm
example files for FE

fe_guide.pdf
guide for the FE codes, see also appendix D.2

------------
chap7/

driven_cavity.f
solution of the hydrodynamic basic equations from sect. 7.3.2.3.
Plot of stream lines as in Figures 7.9., 7.10
TI: Reynolds number

benard_convection.f
Solution of the hydrodynamic eqs. from sect. 7.3.4.2 in 3D following
sect. 7.3.4.3.
Plot of temperature field. Rayleigh number, Biot number, tend
can be changed in the code.
Note: the library FFTPACK is needed (see make_95)
TI: none
D.2 Short guide to ﬁnite-element tools from Chapter 6 | 313

------------
chap8/

mc_gas.f
compute the equilibrium of a many-particle system based on the
Metropolis algorithm from sect. 8.3.1.2. In addition: constant
gravitation.
TI: temperature, gravitation. Note: units as in Figures 8.4 - 8.6

ising.f
solution of the Ising model with Metropolis algorithm from
sect. 8.3.2.3. temperature values are scanned, mean magnetization is
computed.
TI: external magnetic field

D.2 Short guide to ﬁnite-element tools from Chapter 6

The three codes for grid generation, Laplace solving, and evaluation:

program purpose [I]nput/[O]utput

mesh_generator generates a 2D-triangular mesh [I]: ppm-file domain boundary
[O] grid file
laplace_solver solves Laplace equation [I]: grid file
with Dirichlet b.c. [O] node file
grid_contur plots contour lines [I] node file, grid file

Example ﬁles:

name purpose created by

kidney.ppm, house.ppm domain bound. Xfig (or similar); see Figure D.1
kidney.grd, house.grd grid files mesh_generator
kidney.knd, house.knd node files laplace_solver

D.2.1 mesh_generator

TI stands for ‘Terminal Input’.

TI: name of ppm-ﬁle (P6-format, binary, one byte/pixel), contains the domain
boundary created by a graphics program, e.g., Xﬁg. The dimension of the bitmap can
be as large as 5000 x 5000.
314 | D README and a short guide to FE-tools

Fig. D.1: The two domain boundaries ‘house.ppm’ and ‘kidney.ppm.’ The latter also contains inner
boundaries (multiply connected domain).

TI: number of mesh points in the x-direction, deﬁnes the resolution of the triangular
grid. Reasonable values for the examples are 30–60.

The triangular grid is plotted (red). The outer boundary points are marked in blue. If
some are missing they must be added manually using the mouse.
If ﬁnished, type ‘s’ on keyboard. All mesh points left and right from the external
boundaries are then deleted automatically. Boundary points are now marked by cir-
cles.

Mouse modes:
There are three modes, which one can toggle between using the keys ‘d, r, z.’

Key ‘d’: delete mode: Clicked points near the mouse cursor are deleted.
Key ‘r’: boundary mode: Clicked points close to the mouse cursor are marked as
boundary points. Inner boundaries (example ‘kidney’) have to be marked by hand
and afterwards drawn to the desired position. Here, single points can also be set
(point charges).
Key ‘z’: draw mode: Clicked points near the mouse cursor can be drawn, e.g., onto
the boundary. The new location is obtained by positioning the mouse cursor and
repeated clicking. Attention: avoid crossing lines!

Key ‘s’: TI for grid ﬁle name (output, .grd) and termination of the program.
D.2 Short guide to ﬁnite-element tools from Chapter 6 | 315

D.2.2 laplace_solver

TI: grid ﬁle name.

In the code the boundary node values have to be deﬁned one by one. The number of
corresponding mesh points is obtained from mesh_generator.

TI: node ﬁle name. Here, the node values are stored after computation.

D.2.3 grid_contur

TI: node ﬁle name. Then request for the value of the contour line(s) to be plotted.
Termination by Ctrl-d.

What could be improved?

The package can be extended and improved in various ways. Here, we only note a few
ideas:
– Handling of the boundary conditions. Instead of setting the (Dirichlet) boundary
values in the code laplace_solver it could be better to define them in an extra file,
or by mouse input.
– Other boundary conditions (Neumann, mixed).
– Automatic detection of the inner boundaries.
– Extension to the Poisson equation.
– Extension to time-dependent problems (diffusion equation, Schrödinger equa-
tion).
– Extension to 3D.
Index
A correlation dimension 107
acceptance-rejection method 242 Courant number 167
activator-inhibitor system 228 Crank–Nicolson method 67, 208
ADI method 159 Crout algorithm 277
algebraic multiplicity 270 cyclic reduction 161
anharmonic oscillator 127 cyclotron frequency 202
apsidal precession 76
autonomous system 43 D
decimal to binary converter 33
B degree of freedom 41
ballistic flight 118 detailed balance 252
Barnsley ferns 23 differential quotient 153
Belousov–Zhabotinskii reaction 228 discretization
bifurcation diagram 97, 105 – biharmonic operator 154
biharmonic operator 154 – Laplace operator 154
boundary conditions 117, 149
distribution function 241
– Dirichlet 150, 152
driven cavity 206
– Neumann 150, 152, 157
drop formation 255
– Robin 150, 157
dynamical system 41
Boussinesq approximation 212
box-counting method 25, 108
E
Box–Muller method 90, 245
eigenvalue problems 279
Brusselator 229
elliptic PDE 149
Burgers equation 146
embedding dimension 110
butterfly effect 15
ensemble 250
C Euler method 60
Campbell–Baker-Hausdorff relation 52 expectation value 241
canonical ensemble 88, 251
canonical equations 50 F
capacity dimension 107 finite differences 118, 153
celestial mechanics 73 finite element
characteristics 144 – evaluation 185
Chebyshev spectral method 173 – Laplace solver 185
Chebyshev–Gauss–Lobatto 174 – mesh generation 185
Chirikov map 13 – method 182
Circle map 13 fixed point 43
codimension-two 228 Floquet
collocation method 129 – exponent 111
color maps 36 – theorem 111
conservative system 58 flow field 204
contact angle 255 flow-apart time 201
convection 211 FORTRAN 1
– equation 145, 167 fractal dimension 25, 106
– fully nonlinear problem 219 fractals 20
– linear stability analysis 215 Fredholm alternative 275
– Rayleigh–Bénard 218 Frenkel–Kotorova model 12

https://doi.org/10.1515/9783110515145-013
318 | Index

frozen time analysis 216 L

FTCS scheme 163 Lagrange polynomials 174
Lagrangian points 81
G LAPACK 5
Galerkin method 129 Laplace operator 154
Gauss–Seidel method 158 Lax scheme 168
geometric multiplicity 273 leapfrog scheme 169
Ginzburg–Landau equation 186 learning process 28
– complex 238 least-squares method 129
gnuplot 26 Lennard-Jones potential 86
gradient dynamics 187 linear separability 32
linear stability analysis 7, 43
H Linux 4
Hamilton function 50 logistic map 7
Hamiltonian system 50 logistic ODE 41
hard spheres 250 Lorenz attractor 49
harmonic oscillator 125 Lorenz equations 48
heat equation 151, 162, 176
Lotka–Volterra system 72
Hebb’s learning rule 29
LU decomposition 276
Hénon–Heiles model 54
LU factorization 281
Heun’s method 64
Lyapunov exponent 98
homoclinic orbit 60
– higher order 102
Hopf instability 43, 227
– logistic map 16
hyperbolic PDE 149
hysteresis 265 – multidimensional maps 17
– Standard map 18
Lyapunov functional 235
I
Lyapunov time 106
implicit declarations 10
implicit method 63
importance sampling 248, 253
M
incompressible ﬂuids 204
initial conditions 149 Markov chain 252
Ising model 255 mathematical pendulum 59, 95
– one-dimensional 258 Mathieu equation 114
– two-dimensional 260 MATLAB 1
Maxwell–Boltzmann distribution 90
J mean-ﬁeld approximation 257
Jacobi Metropolis algorithm 251
– matrix 18, 43 – 2D-gas 253
– method 158 – diffusion equation 265
– potential 80 – Ising model 260
Jordan canonical form 273 – Swift–Hohenberg equation 267
microcanonical ensemble 88
K molecular dynamics 86
Kepler problem 73 monodromy matrix 111
kernel 274 Monte Carlo integration 245
Kirkwood gap 77 Monte Carlo methods (MC) 240
Kohonen algorithm 35 moonshot 136
Index | 319

N S
Navier–Stokes equations 204 Schmidt–Gram method 104
neural networks 27 Schrödinger equation 122
Newton–Raphson 133 – nonlinear 134
No-Intersection Theorem 44 – stationary 125
normal distribution 243 – stationary two-particle 193
null space 274 – time-dependent 196
self-organized maps 34
O self-similar 23
orbit-type diagram 85 separatrix 60
Oscar II 76 shallow water equations 171
over-relaxation 158 shooting method 140
Sierpinski triangle 22
simple sampling 248
P
speciﬁc heat 92
pair correlation function 91
spectral method 178
parabolic PDE 149
spiral patterns 233
parametric instability 112
standard deviation 241
partition sum 251
Standard map 13
PDF 241
Stark effect 123, 130
perceptron 28
strange attractor 47
PGPLOT 5
stream function 205
pivoting 278
subdomain method 129
Poincaré
Swift–Hohenberg equation 235
– map 56
– Metropolis algorithm 267
– section 56, 97
symplectic method 51, 66, 89
Poincaré–Bendixson Theorem 45
Poisson bracket 51
T
Poisson equation 156
Takens, embedding theorem 110
Prandtl number 213
target patterns 233
pressure 204
Thomas algorithm 278
probability density 241
three-body problem 76
pseudospectral method 221, 231
– reduced 79
time-evolution operator 41, 197
Q transformation method 242
QR factorization 281 transport equation 212
quasilinear differential equations 40 traveling salesman problem 37
Trojans 81
R Turing instability 226
random numbers 240
Rayleigh number 213 U
reaction-diffusion systems 225 upwind scheme 168
reconstruction 109
recurrence time 197 V
regularization 82 Van der Pol oscillator 46
Reynolds number 207 variance 241
Rosenblatt rule 29 variational problem 184
Runge–Kutta method 67 velocity ﬁeld 205
– with adaptive step size 71 Verlet algorithm 53
320 | Index

von Neumann stability analysis 166 Z

vorticity 205 zeros of a polynomial 282

W
wave equation 169
wave packet 197

Advanced Mathematical Methods in Theoretical Physics - Gernot Schaller
No ratings yet
Advanced Mathematical Methods in Theoretical Physics - Gernot Schaller
144 pages
Lectures On AM
100% (4)
Lectures On AM
384 pages
Diffyqs PDF
No ratings yet
Diffyqs PDF
371 pages
Notes On Diffy Q's Jiri Lebl PDF
No ratings yet
Notes On Diffy Q's Jiri Lebl PDF
466 pages
Diffyqs
No ratings yet
Diffyqs
466 pages
Diffyqs
No ratings yet
Diffyqs
466 pages
AM Core 2021
No ratings yet
AM Core 2021
279 pages
Introduction to PDEs Lecture Notes
100% (4)
Introduction to PDEs Lecture Notes
226 pages
Notes On Diffy Qs-Differential Equations For Engineers
No ratings yet
Notes On Diffy Qs-Differential Equations For Engineers
315 pages
Partial Differential Equations and Fourier
100% (1)
Partial Differential Equations and Fourier
431 pages
Linear Partial Diferential Equations and Fourier Theory PDF
No ratings yet
Linear Partial Diferential Equations and Fourier Theory PDF
431 pages
Differential Equations
No ratings yet
Differential Equations
351 pages
Differentials Textbook (Diffy Q'S)
No ratings yet
Differentials Textbook (Diffy Q'S)
315 pages
PDE Textbook
No ratings yet
PDE Textbook
310 pages
Contents
No ratings yet
Contents
4 pages
Computational Physics
No ratings yet
Computational Physics
322 pages
Differential Equations Text
No ratings yet
Differential Equations Text
238 pages
LectureNotesMATH2048 PDF
No ratings yet
LectureNotesMATH2048 PDF
148 pages
Math Diff PDF
No ratings yet
Math Diff PDF
315 pages
Notes On Diffy Qs PDF
No ratings yet
Notes On Diffy Qs PDF
315 pages
Diffyqs 2 PDF
No ratings yet
Diffyqs 2 PDF
315 pages
Phys 430 Python
100% (1)
Phys 430 Python
120 pages
(Cambridge Monographs On Applied and Computational Mathematics) Benedict Leimkuhler, Sebastian Reich - Simulating Hamiltonian Dynamics-Cambridge University Press (2005)
0% (1)
(Cambridge Monographs On Applied and Computational Mathematics) Benedict Leimkuhler, Sebastian Reich - Simulating Hamiltonian Dynamics-Cambridge University Press (2005)
396 pages
Ordinary and Partial Differential Equations
100% (1)
Ordinary and Partial Differential Equations
384 pages
1 9780898719482 FM
No ratings yet
1 9780898719482 FM
17 pages
22 BBook
No ratings yet
22 BBook
173 pages
PDEs for Applied Mathematics Students
No ratings yet
PDEs for Applied Mathematics Students
227 pages
Computational Physics Lecture Notes
No ratings yet
Computational Physics Lecture Notes
129 pages
Computational Physics: Prof. Matthias Troyer Eth Z Urich, 2005/2006
No ratings yet
Computational Physics: Prof. Matthias Troyer Eth Z Urich, 2005/2006
49 pages
Computational Physics Lecture Notes
100% (3)
Computational Physics Lecture Notes
129 pages
An Introduction To Numerical Methods For The Physical Sciences
No ratings yet
An Introduction To Numerical Methods For The Physical Sciences
168 pages
PDE Solution Manual
No ratings yet
PDE Solution Manual
126 pages
Lectures On Differential Equations: Department of Mathematics University of California Davis, CA 95616
No ratings yet
Lectures On Differential Equations: Department of Mathematics University of California Davis, CA 95616
169 pages
Advanced Engineering Mathematics
No ratings yet
Advanced Engineering Mathematics
290 pages
Adzievski, Kuzman Siddiqi, A. H - Introduction To
No ratings yet
Adzievski, Kuzman Siddiqi, A. H - Introduction To
645 pages
Craig A. Tracy - Lectures On Differential Equations
No ratings yet
Craig A. Tracy - Lectures On Differential Equations
165 pages
Notes On Diffy Qs
No ratings yet
Notes On Diffy Qs
252 pages
Student's Solutions Manual To Accompany Fundamentals of Differential Equations
20% (5)
Student's Solutions Manual To Accompany Fundamentals of Differential Equations
748 pages
Solucion Ejercicios Impares Ed Nagle
100% (1)
Solucion Ejercicios Impares Ed Nagle
748 pages
WEG TPD500 Instruction Manual 1S5PNT500 EXP-ETH-PN-TPD500 ENITPT en It PT
No ratings yet
WEG TPD500 Instruction Manual 1S5PNT500 EXP-ETH-PN-TPD500 ENITPT en It PT
13 pages
CC Project Proposal
No ratings yet
CC Project Proposal
10 pages
Splitsleeve
No ratings yet
Splitsleeve
11 pages
Energies: Recent Advances in Transcritical CO (R744) Heat Pump System: A Review
No ratings yet
Energies: Recent Advances in Transcritical CO (R744) Heat Pump System: A Review
35 pages
M.Sc Computer Science Syllabus 2021-22
No ratings yet
M.Sc Computer Science Syllabus 2021-22
88 pages
Unit 5 Notes
No ratings yet
Unit 5 Notes
17 pages
Business Research Methodology
No ratings yet
Business Research Methodology
2 pages
NCHRP w172
No ratings yet
NCHRP w172
64 pages
Digital Calibrator/Thermometer Guide
No ratings yet
Digital Calibrator/Thermometer Guide
44 pages
manualRMFLS506D E FRMFL-150020200020Manual 20230413143754 PDF
No ratings yet
manualRMFLS506D E FRMFL-150020200020Manual 20230413143754 PDF
16 pages
FINnet 2.0: Transforming FIU India
No ratings yet
FINnet 2.0: Transforming FIU India
3 pages
WS PartsBook 2016
No ratings yet
WS PartsBook 2016
542 pages
434 10.PLC 10120
No ratings yet
434 10.PLC 10120
3 pages
MAT1110 Lecture Notes Sets 1
No ratings yet
MAT1110 Lecture Notes Sets 1
32 pages
Lec 2
No ratings yet
Lec 2
19 pages
MB Webcode 2024 en
No ratings yet
MB Webcode 2024 en
40 pages
IDL Programming Courses Overview
No ratings yet
IDL Programming Courses Overview
4 pages
Microstrip Patch Antenna Array: Design and Simulation of Edge Fed
No ratings yet
Microstrip Patch Antenna Array: Design and Simulation of Edge Fed
5 pages
Literature Review Outline Guide
100% (3)
Literature Review Outline Guide
6 pages
FCS Unit 3
No ratings yet
FCS Unit 3
5 pages
VSD Commissioning
No ratings yet
VSD Commissioning
2 pages
Move Things With CSS Jhey Tompkins 2020
No ratings yet
Move Things With CSS Jhey Tompkins 2020
78 pages
ERA De-Clipper Manual
No ratings yet
ERA De-Clipper Manual
15 pages
TM-2263 AVEVA Marine (12 Series) Programmable Macro Language (Basic) Rev 1.0 PDF
No ratings yet
TM-2263 AVEVA Marine (12 Series) Programmable Macro Language (Basic) Rev 1.0 PDF
178 pages
Victaulic L020070GE0 Specification Sheet
No ratings yet
Victaulic L020070GE0 Specification Sheet
3 pages
V80Hx30 - Economizer
No ratings yet
V80Hx30 - Economizer
2 pages
Scope of Work
100% (6)
Scope of Work
7 pages
Solar Document
No ratings yet
Solar Document
36 pages
MO9 Design and Produce Digital Embroidery
No ratings yet
MO9 Design and Produce Digital Embroidery
148 pages
SAP S4HANA 2020 - Release Information and Restrictions For EWM in SAP S4HANA 2020
No ratings yet
SAP S4HANA 2020 - Release Information and Restrictions For EWM in SAP S4HANA 2020
4 pages