Numerical Diff and Integration
Numerical Diff and Integration
MTL 107
Harish Kumar
(hkumar@iitd.ac.in)
f (n)
f (x) = f (x0 ) + f ′ (x0 )(x − x0 ) + · · · + (x − x0 )n + Rn
n!
f (n+1) (x1 )
where Rn = (n+1)! (x − x0 )n+1 .
First order Derivative
▶ Derivative of a function:
f (x0 + h) − f (x0 )
f ′ (x0 ) = lim
h→0 h
if the limit exists, otherwise not defined.
▶ Assume f to be C 2 (a, b),then using Taylor’s Theorem:
h2 ′′
f (x0 + h) = f (x0 ) + hf ′ (x0 ) + f (ξ)
2
for some ξ.
▶ Rearranging terms:
f (x0 + h) − f (x0 ) h ′′
f ′ (x0 ) = − f (ξ).
h 2
First order Derivative: Two Point Formulas
▶ Forward Difference:
f (x0 + h) − f (x0 )
f ′ (x0 ) ≈ ,
h
with error
|h| ′′
|f (ξ)|.
2
▶ Similarly, we have Backward Difference:
f (x0 ) − f (x0 − h)
f ′ (x0 ) ≈ ,
h
with error
|h| ′′
|f (ξ)|.
2
First order Derivative: Three Point Formulas
h2 ′′ h3
f (x0 + h) = f (x0 ) + hf ′ (x0 ) + f (x0 ) + f (3) (ξ1 )
2 3!
for some ξ1 .
▶ Similarly,
h2 ′′ h3
f (x0 − h) = f (x0 ) − hf ′ (x0 ) + f (x0 ) − f (3) (ξ2 )
2 3!
for some ξ2 .
First order Derivative: Three Point Formulas
h2 (3) h3 (3) h3
1
f (ξ) = f (ξ2 ) + f (3) (ξ1 )
6 2 3! 3!
1
f ′ (x0 ) ≈ (−3f (x0 ) + 4f (x0 + h) − f (x0 + 2h)) ,
2h
with error
h2 (3)
|f (ξ)|
3
for some ξ in (x0 , x0 + 2h).
▶ Both three point formulas are second order accurate.
▶ Replacing h with −h we can get other side formula.
First order Derivative: Five-Point Formulas
▶ Central formula:
1
f ′ (x0 ) ≈ (f (x0 − 2h) − 8f (x0 − h) + 8f (x0 + h) − f (x0 + 2h)) ,
12h
with error
h4 (5)
|f (ξ)|
30
for some ξ in (x0 − 2h, x0 + 2h).
▶ Exercise: One sided five point formula.
Second order Derivative: Three Point Formulas
▶ Note that,
h4 (4)
f (x0 +h)+f (x0 +h) = 2f (x0 )+h2 f ′′ (x0 )+ (f (ξ1 )+f (4) (ξ2 ))
24
▶ Rearranging,
1
f ′′ (x0 ) ≈ (f (x0 − h) − 2f (x0 ) + f (x0 + h)))
h2
with error,
h2 (4)
|f (ξ)|
12
for some ξ in (x0 − h, x0 + h).
▶ Exercise: A five point formula for second derivative.
Richardson Extrapolation
▶ Changing h to 2h we get,
1
f ′′ (x0 ) = (f (x0 − 2h) − 2f (x0 ) + f (x0 + 2h))
4h2
4h2 (4) 16h4 (6)
− f (x0 ) − f (x0 ) + O(h5 ).
12 360
▶ Multiplying first formula with 4, subtracting the other one and
dividing by 3, we get,
1
f ′′ (x0 ) =
12h2
(−f (x0 − 2h) + 16f (x0 − h) − 30f (x0 ) + 16f (x0 + h) − f (x0 + 2h))
Richardson Extrapolation
h4 (4)
|f (ξ)|,
90
for some ξ in (x0 − 2h, x0 + 2h).
▶ Exercise: Repeat the same procedure to derive a five point
central difference formula from three point central difference
formulas for first order derivatives.
Round-Off error Instability
and
f (x0 − h) = f¯(x0 − h) + e(x0 − h)
where f¯ is the floating point representation of f with error e.
Round-Off error Instability
f (x0 + h) − f (x0 − h) ε h2
|f ′ (x0 ) − | ≤ + M.
2h h 6
▶ What is the optimal value of h?
Round-Off error Instability
1. NewtonCotes Rules
▶ Trapezoidal rule
▶ Simpson’s rule
2. Romberg integration
3. Gauss quadrature
4. Adaptive quadrature
References
▶ U. Ascher and C. Greif: Numerical methods. 15.1 − 15.5.
▶ C. Moler: Numerical Computing with Matlab.
▶ W. Gander, M. Gander, and F. Kwok: Scientific
Computing.Springer 2014.
Introduction
f (a) + f (b)
T =h
2
The accuracy of a quadrature rule can be predicted in part by
examining its behavior on polynomials.
Order of a quadrature rule is the degree of the lowest degree
polynomial that the rule does not integrate exactly.
The orders of both midpoint and trapezoidal rule are 2.
Newton-Cotes rule
Then
Z b Z b n
X Z b n
X
f (x)dx ≈ Pn (x)dx = f (xi ) Li (x)dx = f (xi )wi .
a a i=0 a i=0
b b 1
b−x x −a
Z Z Z
h
dx = dx = h ξdξ = .
a b−a a b−a 0 2
So,
h
T = (f (a) + f (b)).
2
Simpson’s rule
2 1
S= M+ T
3 3
It turns out that Simpson’s rule also integrates cubics exactly, but
not quartics, so its order is four.
Newton-Cotes methods with order up to 8 exist.
Even higher order methods can be computed, but some of their
weights wi get negative which is undesirable for potential round-off.
Summary of basic quadrature rules
(b−a) ′′ (ξ )
Trapezoidal 2 [f (a) + f (b)] −f 12
2
(b − a)3
(b−a) 4 (ξ )
Simpson 6 [f (a) + 4f ( a+b
2 ) + f (b)] −f 90
3
( b−a
2 )
5
b
(b − a)h2
Z
f (x)dx − T (h) = | f ′′ (ξ) |, ξ ∈ [a, b].
a 12
Exercise
Composite trapezoidal rule
h h 1 1
T ( ) = ( y0 + y1 + ... + y2n−1 + y2n ),
2 2 2 2
h 1 1
= ( y0 + y2 + ... + y2n−2 + y2n )
2 2 2
h
+ (y1 + y3 + ... + y2n−1 )
2
1 h
= T (h) + (y1 + y3 + ... + y2n−1 )
2 2
function T=trapez(f,a,b,tol)
Numerical Methods for Computational Science and Engineering
Composite Rules
Trapezoidal rule
function T=trapez(f,a,b,tol);
% TRAPEZ(f,a,b,tol) tries to integrate int_a^b f(x) dx
% to a relative tolerance tol using the composite
% trapezoidal rule.
%
h = b-a; s = (f(a)+f(b))/2;
tnew = h*s; zh = 1; told = 2*tnew;
fprintf(’h = 1/%2.0f, T(h) = %12.10f\n’,1/h, tnew)
while abs(told-tnew)>tol*abs(tnew),
told = tnew; zh = 2*zh; h = h/2;
s = s+sum(f(a+[1:2:zh]*h));
tnew = h*s;
fprintf(’h = 1/%2.0f, T(h) = %12.10f\n’,1/h, tnew)
end;
T = tnew;
h
S(h) = (y0 + 4y1 + 2y2 + 4y3 + ... + 2y2n−2 + 4y2n−1 + y2n ).
3
Composite Simpson’s rule(cont.)
Exercise
Composite Simpson’s rule(cont.)
function S=simpson(f,a,b,tol);
% SIMPSON (f,a,b,tol) tries to integrate int_a^b f(x) dx
% to a relative tolerance tol using the composite
% Simpson rule.
%
h = (b-a)/2; s1 = f(a)+f(b); s2 = 0;
s4 = f(a+h); snew = h*(s1+4*s4)/3;
zh = 2; sold = 2*snew;
fprintf(’h = 1/%2.0f, S(h) = %12.10f\n’,1/h, snew)
while abs(sold-snew)>tol*abs(snew),
sold = snew; zh = 2*zh; h = h/2; s2 = s2+s4;
s4 = sum(f(a +[1:2:zh]*h));
snew = h*(s1+2*s2+4*s4)/3;
fprintf(’h = 1/%2.0f, S(h) = %12.10f\n’,1/h, snew)
end
S = snew;
Let Ri,0 = T (hi ) and hi+1 = hi /2. Then, for the errors,
Eliminate K1 − term :
3 15
Ei,0 − 4Ei+1,0 = K2 hi4 + K3 hi6 + ...
4 16
Romberg method(cont.)
This procedure eliminating higher and higher order error terms can
be continued, yielding the Romberg method:
4j Ri+1,j−1 − Ri,j−1
Ri+1,j =
4j − 1
Romberg method(cont.)
with
4j R j+1,j−1 − Ri,j−1
Ri+1,j = 0 ≤ j ≤ i ≤ k.
4j − 1
Same example again
1
xe x e −2
Z
I = dx = = 0.3591409142...
0 (x + 1)2 2
0.33978522855738
0.35308386657870 0.35751674591915
0.35751519587192 0.35899230563633 0.35909067628414
0.35872647716421 0.35913023759497 0.35913943305888 0.35914020697
0.35903678355577 0.35914021901962 0.35914088444793 0.35914090748
0.35911484861929 0.35914087030714 0.35914091372630 0.35914091419
0.35913439576246 0.35914091147685 0.35914091422149 0.35914091422
columns 4 to 6
0.35914091023295
0.35914091421734 0.35914091422123
0.35914091422950 0.35914091422951 0.35914091422952
Another example
R1√ 2
Let’s integrate 0 xdx = 3 by Romberg method
[I T]=romberg(@sqrt,0,1,1e-8)
Z 1 Z 1
dx = 2 = 1w1 + 1w2 xdx = 0 = ξ1 w1 + ξ2 w2
−1 −1
1 1
2
Z Z
x dx = = ξ12 w1 + ξ22 w2
2
x 3 dx = 0 = ξ13 w1 + ξ23 w2
−1 3 −1
Gauss Quadrature(cont)
The solution is
√ √
{w1 = 1, w2 = 1, ξ1 = 1/ 3, ξ2 = −1/ 3}
Then,
Z 1
g (x)Φn+1 (x)dx = 0, ∀g ∈ Pn .
−1
f ′′ (ξ)
h3/2
I − T (h) = h3 ≤ = 6.357.10−7 < 10−5 .
12 48
...
% Initialization
c = (a + b)/2;
fa = F(a,varargin{:});
fc = F(c,varargin{:});
fb = F(b,varargin{:});
% Recursive call
[Q,k] = quadtxstep(F, a, b, tol, fa, fc, fb, varargin{:});
fcount = k + 3;