An Introduction to Spectral Methods

J. Frauendiener
Institut für Astronomie und Astrophysik
Abteilung Theoretische Astrophysik
Universität Tübingen
joerg.frauendiener@uni-tuebingen.de

SFB-TR7 Gravitational Wave Astronomy

Spectral Methods J. Frauendiener

Contents

• Introductory example

• Method of weighted residuals

• Spectral approximation

• Orthogonal polynomials on [−1, 1]

• Example

SFB-TR7 Gravitational Wave Astronomy 1

Spectral Methods J. Frauendiener

Introductory Example

Consider the one-dimensional advection equation:

∂u ∂u
(t, x) = (t, x) x ∈ [0, 2π], t ∈ [0, T ),
∂t ∂x
u(0, x) = u0(x) x ∈ [0, 2π]
u(t, 0) = u(t, 2π) t ∈ [0, T ).

u is periodic → Fourier series:

∞
u(t, x) = ∑ ûk (t)eikx
k=−∞

SFB-TR7 Gravitational Wave Astronomy 2

Spectral Methods J. Frauendiener

• Approximate
N/2−1
u(t, x) ≈ ∑ ûk (t)eikx
k=−N/2
• Insert into PDE
N/2−1
∂ûk ikx N/2−1
∑ ∂t e = ∑ ik û k e ikx
,
k=−N/2 k=−N/2

• Extract equations by integrating against e−imx

∂ûm
= imûm, m = −(N/2) : N/2 − 1
∂t

• Solve
ûm(t) = eimt ûm(0), m = −(N/2) : N/2 − 1.
• Reconstruct
N/2−1
u(t, x) ≈ ∑ ûk (0) eik(x+t)
k=−N/2

SFB-TR7 Gravitational Wave Astronomy 3

Spectral Methods J. Frauendiener

N = 16, u0(x) = exp(sin(x))

SFB-TR7 Gravitational Wave Astronomy 4

Spectral Methods J. Frauendiener

Method of Weighted Residuals (MWR)

Consider a PDE with boundary condition (M a spatial operator)

ut = Mu, on U ⊂ Rm
Bu = 0, on ∂U

An approximate solution is a function ū for which the residual

R := ūt − Mū

is small.

SFB-TR7 Gravitational Wave Astronomy 5

Spectral Methods J. Frauendiener

What is “small”?

• Assume ū is in a finite-dimensional sub-space VN of a Hilbert space H

• Expand ū in a basis (φ0, φ1, . . . , φN ) (trial functions) of VN :

N
ū = ∑ uiφi
i=0

• Use test functions (χ0, χ1, . . . , χN ) to probe for the smallness of the
residual R = ūt − Mū

hχn|Ri = 0, for n = 0, 1, . . . , N

SFB-TR7 Gravitational Wave Astronomy 6

Spectral Methods J. Frauendiener

Choice of trial functions determines the method

• Finite differences: low order polynomials on stencils

(e.g. 2nd -order polynomials on 3 point stencils)

• Finite elements: smooth functions on local sub-domains of U

(e.g. cubic polynomials on tetrahedra)

• (Pseudo) spectral methods: global smooth functions on U

(e.g. trigonometric polynomials)

SFB-TR7 Gravitational Wave Astronomy 7

Spectral Methods J. Frauendiener

Choice of test functions determines the type of spectral method

• Galerkin method:
– χn = φn
– all φn satisfy the boundary conditions B(φn) = 0
– residual is orthogonal to VN

R ⊥ VN ,

– all (non-linear) terms are evaluated in ‘coefficient space’

– equations for the expansion coefficients

• Tau method:
– χn = φn
– boundary conditions are imposed as additional equations
– equations for the expansion coefficients

SFB-TR7 Gravitational Wave Astronomy 8

Spectral Methods J. Frauendiener

• Pseudo-spectral method:
– χn = δ(x − xn)
R(xn) = 0
– two representations of ū, ‘coefficients’ and ‘values’

N
ū(xn) = ∑ uiφi(xn)
i=0

– differentiate in coefficient space

– evaluate non-linearities in value space
– obtain equations for the expansion coefficients ui

• Collocation method: same as pseudo-spectral method, but obtain

equations for the values ū(xn).

SFB-TR7 Gravitational Wave Astronomy 9

Spectral Methods J. Frauendiener

How to choose the trial functions?

• Periodic problems: trigonometric polynomials e(ik·x) (FFT)

• Non-periodic problems: orthogonal polynomials

– bounded interval: Chebyshev polynomials (FCT)
– bounded interval: Legendre polynomials
– unbounded interval: Laguerre, Hermite polynomials
– sphere: (spin weighted) spherical harmonics

SFB-TR7 Gravitational Wave Astronomy 10

Spectral Methods J. Frauendiener

Spectral approximation

The trigonometric polynomials φk (x) = eikx are orthogonal on [0, 2π] with the
inner product Z 2π
hφk |φl i = φk (x)φl (x) dx = 2π δkl
0
For a function u : [0, 2π] → C define the continuous Fourier coefficients by

1
Z 2π
ûk = u(x)e−ikx dx, k∈Z
2π 0

The Fourier series of u is the formal series

S(u) = ∑ ûk φk.

k∈Z

Convergence?

SFB-TR7 Gravitational Wave Astronomy 11

Spectral Methods J. Frauendiener

The Nth order truncated Fourier series is

N/2−1
PN (u) = ∑ ûk φk
k=−N/2

PN orthogonal projector onto VN = span{φk : −N/2 ≤ k < N/2}

Proposition: For every square integrable function u: PN u(x) → u(x) for
N → ∞ almost everywhere.
How fast?
For u sufficiently smooth

max |u(x) − PN u(x)| =

x∈[0,2π]
∑ |ûn|
|n|&N/2

⇒ Decay of ûn with n

SFB-TR7 Gravitational Wave Astronomy 12

Spectral Methods J. Frauendiener

For u ∈ C 1 compute
Z 2π i i
Z 2π
2π ûn = u(x)e−inx dx = [u(2π) − u(0)] − u0(x)e−inx dx = O (n−1).
0 n n 0

Hence, for u ∈ C 2 and periodic ûn = O (n−2), . . .

Evanescent error: If u ∈ C ∞(S1) then n−pûn → 0 for any p ∈ N.

SFB-TR7 Gravitational Wave Astronomy 13

Spectral Methods J. Frauendiener

Discrete Fourier expansion

For pseudo-spectral and collocation method one needs to switch efficiently
between between coefficients and values:
Consider the N collocation points
2π j
xj = , j = 0, 1, . . . , N − 1
N
The discrete Fourier coefficients are

1 N−1
ũk = ∑ u(x j ) e−ikx j
N j=0

Discrete orthogonality allows inversion

N/2−1
u(x j ) = ∑ ũk eikx j
j=−N/2

SFB-TR7 Gravitational Wave Astronomy 14

Spectral Methods J. Frauendiener

The trigonometric polynomial of degree N/2

N/2−1
IN u(x) = ∑ ũk eikx
j=−N/2

interpolates the function u at the points x j , i.e., IN u(x j ) = u(x j ).

Computation of the coefficients by FFT.

SFB-TR7 Gravitational Wave Astronomy 15

Spectral Methods J. Frauendiener

Relation between interpolation and truncation?

Suppose PN u(x j ) → u(x j ) then

u(x j ) = ∑ ûm eimx j

m∈Z

and
! " #
1 N−1 1 N−1 1 N−1 i(m−k)x j
ũk = ∑ u(x j ) e −ikx j
= ∑ ∑ ûm e imx j
e −ikx j
= ∑ ûm ∑ e
N j=0 N j=0 m∈Z m∈Z N j=0
| {z }
=1 for m=k+NZ
∞
ũk = ûk + ∑ ûk±mN
m=1

SFB-TR7 Gravitational Wave Astronomy 16

Spectral Methods J. Frauendiener

Aliasing error

IN u = PN u + RN u

The difference between the truncated and interpolating polynomials is due

to the higher modes which alias the low order modes

k = −10 k = −2 k=6

RN u ⊥ u − PN u ⇒ ||u − IN u||2 = ||u − PN u||2 + ||RN u||2 ≥ ||u − PN u||2

Kreiss and Oliger (1979) The aliasing error is asymptotically of the same
order as the truncation error.

SFB-TR7 Gravitational Wave Astronomy 17

Spectral Methods J. Frauendiener

Differentiation

u(x) = ∑ ûkeikx =⇒ Du(x) = u0(x) = ∑ ikûkeikx

k∈Z k∈Z

Two possibilities:

(i) in coefficient space: used for Galerkin method

D : ûk 7→ ikûk .

Truncation commutes with differentiation: (PN u)0 = PN u0.

SFB-TR7 Gravitational Wave Astronomy 18

Spectral Methods J. Frauendiener

(ii) in physical space: used for collocation

1. compute ũk from the values u j = u(x j ) (FFT)

1 N−1
ũk = ∑ u(x j ) e−ikx j
N j=0

2. use D in coefficient space:

(1)
ũk 7→ ũk = ikũk

3. with these coefficients compute the values (iFFT)

N/2−1
∑
(1)
DN u(x j ) = ũk eikx j
k=−N/2

SFB-TR7 Gravitational Wave Astronomy 19

Spectral Methods J. Frauendiener

In general, the Fourier collocation derivative DN u 6= PN u0

The matrix DN : u j 7→ DN u(x j ) is skew symmetric, e.g.

 
0 1 0 −1
1 −1 0 1 0
D4 = 
 
2 0 −1 0 1
1 0 −1 0

SFB-TR7 Gravitational Wave Astronomy 20

Spectral Methods J. Frauendiener

Orthogonal Polynomials on [−1, 1]

Sturm-Liouville (eigen-value) problems provide a Hilbert space H with a

scalar product Z 1
hχ|φi = χ(x)φ̄(x)w(x) dx
−1

Weight function w determines the class of polynomials

w(x) = 1 Legendre
1
w(x) = √ Chebyshev
1 − x2
w(x) = (1 − x)α(1 + x)β Jacobi

Polynomials φk are orthogonal with respect to h · | · i and generate H .

SFB-TR7 Gravitational Wave Astronomy 21

Spectral Methods J. Frauendiener

Function u : [−1, 1] → C can be expanded as

∞
u(x) = ∑ ûkφk(x)
k=0

where φk is the polynomial of degree k.

The (continuous) coefficients are determined from

hu|φk i 1
Z 1
ûk = = u(x)φk (x)w(x) dx
hφk |φk i hφk |φk i −1

transformation to coefficient space involves evaluation of the integrals

Integrals cannot be computed exactly: approximation?

SFB-TR7 Gravitational Wave Astronomy 22

Spectral Methods J. Frauendiener

Gauß integration

Approximate
Z 1 N
f (x)w(x) dx ≈ ∑ wi f (xi)
−1 i=0
If the collocation points xi’s are the N + 1 zeros of the polynomial φN+1 and
if the weights wi are the solution of the linear system

N Z 1
∑ wi (xi) k
=
−1
xk w(x) dx
i=0

then the approximation is exact for all polynomials of degree ≤ 2N + 1

SFB-TR7 Gravitational Wave Astronomy 23

Spectral Methods J. Frauendiener

Gauß-Lobatto integration

Want to include the boundaries −1 = x0, x1, . . . xN−1, xN = 1 for collocation.

If x0, x1, . . . , xN are the zeros of the polynomial

q = φN+1 + aφN + bφN−1

where (a, b) are chosen so that q(±1) = 0 and if the weights wi solve the
linear system
N Z 1
∑(xi) k
wi =
−1
xk w(x) dx
i=0
then
N Z 1
∑ p(xi) wi = −1
p(x)w(x) dx
i=0
for every polynomial with degree ≤ 2N − 1.

SFB-TR7 Gravitational Wave Astronomy 24

Spectral Methods J. Frauendiener

Define the discrete coefficients:

1 N N
ũk = ∑ u(xi)φk (xi)wi, with γk = ∑ φk (xi)2 wi
γk i=0 i=0

and the interpolating polynomial

N
IN u = ∑ ũk φk
k=0

It satisfies
IN u(x j ) = u(x j ).

SFB-TR7 Gravitational Wave Astronomy 25

Spectral Methods J. Frauendiener

Let PN u be the truncation of the Fourier series

N
PN u = ∑ ûk φk
k=0

Then in general PN u 6= IN u: aliasing error

PN u = IN u + RN u

Same results hold for truncation error and aliasing error as for
trigonometric polynomials.

SFB-TR7 Gravitational Wave Astronomy 26

Spectral Methods J. Frauendiener

Properties of Chebyshev polynomials

Tn(x) = cos(nt), for t = arccos(x)

T0(x) = 1
T1(x) = x
T2(x) = 2x2 − 1
T3(x) = 4x3 − 3x
T4(x) = 8x4 − 8x2 + 1
T5(x) = 16x5 − 20x3 + 5x

SFB-TR7 Gravitational Wave Astronomy 27

Spectral Methods J. Frauendiener

Orthogonality
1 dx π
Z
Tn(x)Tm(x) √ = cmδmn, c0 = 2, cl = 1 otherwise
−1 1 − x2 2

Three term recurrence

Tn+1(x) − 2xTn(x) + Tn−1(x) = 0,

Gauß-Lobatto collocation points

xk = cos(kπ/N),

Boundary values
Tn(±1) = ±1,
Product formula
1

TmTn = Tm+n + T|m−n|
2

SFB-TR7 Gravitational Wave Astronomy 28

Spectral Methods J. Frauendiener

Recurrence for derivatives

0 0
Tn+1 Tn−1
− = 2Tn,
n+1 n−1

Differentiation matrix Tn0 = ∑i DimTi

 
0 1 0 3 0 5 0
0 0 4 0 8 0 12
 
0 0 0 6 0 10 0 
 
D6 = 
0 0 0 0 8 0 12
0 0 0 0 0 10 0 
 
0 0 0 0 0 0 12
0 0 0 0 0 0 0

Gauß-Lobatto integration can be performed be a Fast Cosine Transform

(FCT)

SFB-TR7 Gravitational Wave Astronomy 29

Spectral Methods J. Frauendiener

Example: Heat equation with collocation method

Solve on [−1, 1]

u̇(t, x) = u00(t, x), u(0, x) = u0(x), u(t, ±1) = 0.

Approximation
N
uN = ∑ ai Ti
i=0
Collocation: with uN (t, xk ) = uk (t), u00N (t, xk ) = u00k (t)

R(xk ) = u̇k (t) − u00k (t) = 0, k = 0, 1, . . . , N

(Method of lines)

SFB-TR7 Gravitational Wave Astronomy 30

Spectral Methods J. Frauendiener

Boundary condition: replace R(x0) = 0 and R(xN ) = 0 by uN (±1) = 0

Differentiate

by FCT to coefficient space, multiplying with D2N and iFCT back

amounts to a N-th FD scheme applied to uk .

Example with N = 8, u0(x) = sin(πx).

SFB-TR7 Gravitational Wave Astronomy 31

Spectral Methods J. Frauendiener

SFB-TR7 Gravitational Wave Astronomy 32

Spectral Methods J. Frauendiener

Example: Heat equation with Galerkin method

Solve on [−1, 1]

u̇(t, x) = u00(t, x), u(0, x) = u0(x), u(t, ±1) = 0.

Trial functions: ψk = Tk+2 − Tk for k ≥ 0

 
−1 0 0
ψ0 = T2 − T0


  0 −1 0 
ψ1 = T3 − T1 ψi = S j iTj ,
 
S=
1 0 −1
 0 1 0
ψ = T −T 

2 4 2
0 0 1

Note: the ψk satisfy the boundary conditions ψk (±1) = 0

SFB-TR7 Gravitational Wave Astronomy 33

Spectral Methods J. Frauendiener

Approximation
N
uN = ∑ ak(t) ψk = ak(t)ψk
k=0
Differentiation matrix:
D2Tm = Tm00 = Dl mTl
Residual:
RN = u̇N − u00N = ȧk (t)ψk − ak ψ00k
Galerkin: test functions = trial functions

0 = hψm|RN i = ψm|ȧ ψk − a ψk = hψm|ψk i ȧ − ψm|D ψk a

k k 00 k 2


 k

Evaluate scalar products

hψm|ψk i = S mTj |S k Tl = S j mSl k hTj |Tl i = S j mSl k g jl = St GS = S∗S

j l



SFB-TR7 Gravitational Wave Astronomy 34

Spectral Methods J. Frauendiener

similarly:

ψm|D ψk = S mS k Tj |D Tl = S j mSl k hTj |Dnl Tni = S j mSl k Dnl g jn = S∗DS

2 j l 2




ODE for coefficients

(S∗S)ȧ = (S∗DS)a
0 ψm = a0 S m Tk
initial condition: u0 = uk0Tk = am m k

u0 = Sa0 ⇒ S∗u0 = (S∗S)a0

For N = 3:
 
2 0 0 0 0    
0 1 0 0 0 −2 0 1 0 0 3 0 −1
S∗ =  0 −1 0 1 0 S∗S =  0 2 0 
 
G= 0 0 1 0 0
0 0 0 1 0 0 0 −1 0 1 −1 0 2
0 0 0 0 1

SFB-TR7 Gravitational Wave Astronomy 35

Spectral Methods J. Frauendiener

Example with N = 8, u0 = sin(πx)

SFB-TR7 Gravitational Wave Astronomy 36

Spectral Methods J. Frauendiener

Important issues, but not discussed here

• Spectral methods in higher dimensions

• Spherical harmonics

• Stability issues

• Non-linearities

• Domain decomposition

SFB-TR7 Gravitational Wave Astronomy 37

Spectral Methods J. Frauendiener

References

J. P. Boyd, Chebyshev and Fourier Spectral methods, 2nd edition, Dover,

2001.

C. Canuto, M. Y. Hussaini et al, Spectral Methods in Fluid Dynamics,

Springer-Verlag, 1988.

B. Fornberg, A Practical Guide to Pseudospectral Methods, Cambridge

University Press, 1996.

D. Gottlieb and S. A. Orszag, Numerical Analysis of Spectral Methods,

SIAM, 1977.

L. N. Trefethen, Spectral Methods in MATLAB, SIAM, 2000(?).

SFB-TR7 Gravitational Wave Astronomy 38