Machine Learning Formulas Basic
Machine Learning Formulas Basic
Collection of Formulae
Notation
Matrix of order m x n
a 11 a 12 a 1n
···
a 21 a 22 a 2n
···
A=
..
.
a m1 a m2 ··· a mn
Vector of dimension n
x1
x2
x=
..
.
xn
Transpose
B = AT
bi j = a ji
( AB)T = BT AT
The matrix is symmetric if a i j = a ji .
Determinant
a 11 a 12 ··· a 1n
a 21 a 22 ··· a 2n
det A = p Ap = ..
.
a n1 a n2 ··· a nn
If A is of order 2 x 2, then
det A = a 11 a 22 − a 12 a 21
In general
n
X
det A = a i j (−1)i+ j det Mi j
i=1
n
X
= a i j (−1)i+ j det Mi j
j=1
where Mi j is the matrix one obtains if row i and column j are removed from
the matrix A.
Inverse
A−1 A = A A−1 = I (det A ,= 0)
If A is of order 2 x 2, then
− a12
−1 1 a 22
A =
det A − a 21
a 11
1
In general,
1
A−1 = CT
det A
where the elements in C are given by
c i j = (−1)i+ j det Mi j
Ax = λx
2
Dynamical systems
State-space equations
dx
= Ax + Bu
dt
y = C x + Du
Zt
x( t) = e At x(0) + e A( t−τ ) Bu(τ ) dτ
0
Weighting function
Zt
y( t) = h( t − τ )u(τ ) dτ
0
h( t) = Ce At B + Dδ ( t)
Transfer function
Y (s) = G(s)U (s)
G(s) = C (s I − A)−1 B + D = L h( t)
) *
Frequency response
u( t) = sin ω t
y( t) = a sin(ω t + φ )
a = pG(iω )p
φ = arg G(iω )
Linearization
If the nonlinear system
dx
= f ( x, u)
dt
y = ( x, u)
∆ x = x − x0
∆u = u − u0
∆ y = y − y0
d∆ x f f
= ( x0 , u 0 ) ∆ x + ( x0 , u0 ) ∆u
dt x u
∆y = ( x0 , u 0 ) ∆ x + ( x0 , u0 ) ∆u
x u
3
State-space representations
1. Diagonal form
λ1 0 β1
λ2 β2
dz
= z+
.
.
u
dt
. .
.
.
0 λn βn
y = γ1 γ2 . . . γn z + Du
− a1 1 0 . . . 0 b1
− a 0 1 0 b
2
2
dz
.
.
=
.
.
z+
.
.
u
dt
− an−1 0 0 1
− an 0 0 0 bn
y = 1 0 0 . . . 0 z + Du
− a1 − a2 . . . − an−1 − an
1
1 0 0 0
0
dz
= 0 1 0 0 +
z u
.
.
dt
.. .
.
0
0 0 1 0
y = b1 b2 . . . bn z + Du
b1 sn−1 + b2 sn−2 + · · · + bn
G(s) = D +
sn + a 1 sn−1 + · · · + a n
4
The Laplace transform
Operator lexicon
2 F (s + a) e− at f ( t) Damping
I
f ( t − a) t−a>0
3 e− as F (s) Time delay
0 t−a<0
1 1s2
4 F ( a > 0) f ( at) Scaling in t-domain
a a
1 1t2
5 F ( as) ( a > 0) f Scaling in s-domain
a a
Zt
6 F1 (s) F2 (s) f1 ( t − τ ) f2 (τ ) dτ Convolution in t-domain
0
Z c+i∞
1
7 F1 (σ ) F2 (s − σ ) dσ f1 ( t) f2 ( t) Convolution in s-domain
2πi c−i∞
d n F (s)
11 (− t)n f ( t) Differentiation in s-domain
dsn
Zt
1
12 F (s) f (τ ) dτ Integration in t-domain
s 0
Z∞
f ( t)
13 F (σ ) dσ Integration in s-domain
s t
5
Transform lexicon
1 1 δ ( t) Dirac function
1
2 1 Step function
s
1
3 t Ramp function
s2
1 1 2
4
s3 2 t Acceleration
1 tn
5
sn+1 n!
1
6 e− at
s+a
1
7 t · e− at
(s + a)2
s
8 (1 − at) e−at
(s + a)2
1
9 1
T e− t/T
1 + sT
a
10 sin at
s2 + a2
a
11 sinh at
s2 − a2
s
12 cos at
s2 + a2
s
13 cosh at
s2 − a2
1 1
1 − e− at
! "
14 a
s(s + a)
1
15 1 − e− t/T
s(1 + sT )
1 e−bt − e− at
16
(s + a)(s + b) a−b
6
Transform lexicon, continued
s ae− at − be−bt
17
(s + a)(s + b) a−b
a
18 e−bt sin at
(s + b)2 + a2
s+b
19 e−bt cos at
(s + b)2 + a2
1
20
s2 + 2ζ ω 0 s + ω 20
1
ζ =0 ω0 sin ω 0 t
1 2
e−ζ ω 0 t sin ω 0
p
ζ <1 √1 1 −ζ2 t
ω0 1−ζ 2
ζ =1 te−ω 0 t
1 2
e−ζ ω 0 t sinh ω 0
p
ζ >1 √1 2 ζ2 − 1 t
ω0 ζ −1
s
21
s2 + 2ζ ω 0 s + ω 20
1 2
e−ζ ω 0 t sin ω 0
p
0 ≤τ ≤π : ζ <1 √1 1 −ζ2 t +τ
1−ζ 2
√
ω0 1−ζ 2
τ = arctan −ζ ω 0
ζ =0 cos ω 0 t
ζ =1 (1 − ω 0 t) e−ω 0 t
a 1
sin( at − φ ) + e−bt sin φ
! "
22 ! " √
s2 + a2 (s + b) 2
a +b 2
a
φ = arctan
b
7
Laplace transform table, continued
s 1
cos( at − φ ) − e−bt cos φ
! "
23 ! " √
s2 + a2 (s + b) a2 + b2
a
φ = arctan
b
ab ae−bt − be− at
24 1+
s(s + a)(s + b) b− a
a2
25 1 − (1 + at) e− at
s(s + a)2
a
26 t − 1a (1 − e− at )
s2 (s + a)
ω 20
28
s(s2 + 2ζ ω 0 s + ω 20 )
1 2
e−ζ ω 0 t sin ω 0
p
0<ζ <1 1− √1 1 −ζ2 t +φ
1−ζ 2
φ = arccos ζ
ζ =0 1 − cos ω 0 t
1 1 n − at
29 t e
(s + a)n+1 n!
8
G G
G(s)=(1+sT), 1
1+sT
20 T=1
.
01 2 5 1 2 5 10
1 0
Radians/s
0.5
0.2
90
0.1
0.05
0.02
180
0.01
9
10
10
90
.
01 2 5 1 2 5 10
1 0
Radians/s
0.5
0.2
90
0.1
0.05
0.02
180
0.01
G G
2
ω
G(s)= 0
s2 + 2ζω0s + ω02
20 ω0=1
2 ζ=0.2
.
01 2 5 1 2 5 10
1 0
Radians/s
0.5 ζ=0.5
ζ=1.0
0.2
90
0.1
ζ=1.0
0.05 ζ=0.5
ζ=0.2
ζ=0.1
ζ=0.05
0.02 ζ=0.0
180
0.01
11
Stability
s + a1 a1 > 0
s2 + a 1 s + a 2 a 1 > 0, a2 > 0
3 2
s + a1 s + a2 s + a3 a 1 > 0, a 2 > 0, a 3 > 0, a1 a2 > a3
Routh’s algorithm
Consider the polynomial
Assume that the coefficients a i , bi are real and that a 0 is positive. Form the
table
a0 a1 a2 . . .
b0 b1 b2 . . .
c0 c1 c2 . . .
d0 d1 d2 . . .
..
.
where
c 0 = a 1 − a 0 b1 /b0
c 1 = a 2 − a 0 b2 /b0
..
.
d 0 = b1 − b0 c 1 /c 0
d 1 = b2 − b0 c 2 /c 0
..
.
The number of sign changes in the sequence a 0 , b0 , c 0 , d 0 · · · equal the num-
ber of roots for the polynomial F (s) in the right half plane Re s > 0. All the
roots of the polynomial F (s) lie in the left half plane if all numbers a 0 , b0 ,
c 0 , d 0 , . . . are positive.
12
Stability margins
0.8
0.6 1/ Am
0.4
0.2
G0 (ω o )
0
φm
−0.2
−0.4
−0.6
−0.8
G0 (ω c )
−1
1
10
ωc
0
10
Gain
1
Am
−1
10
−2
10
−1 0
10 10
−50
−100
Phase
−150
φm
−200 ωo
−250
−1 0
10 10
Frekvens [rad/s]
Gain margin:
Am = 1/ pG0 (iω 0 )p
Phase margin:
φ m = π + arg G0 (iω c )
Delay margin:
L m = φ m /ω c
13
State feedback and Kalman filtering
State feedback
If the system
dx
= Ax + Bu
dt
y = Cx
has the control law
u = − Lx + {r r
then the closed-loop system is given by
dx
= ( A − BL) x + B{r r
dt
y = Cx
Kalman filtering
Assume that only the output signal y can be directly measured. Introduce
the model
d x̂
= A x̂ + Bu + K ( y − C x̂)
dt
The reconstruction error x̃ = x − x̂ satisfies
d x̃
= ( A − KC ) x̃
dt
14
Lead-lag compensation
Lag compensator
s+a 1 + s/ a
G K (s) = =M M>1
s + a/ M 1 + sM / a
The rule of thumb
a = 0.1ω c
guarantees that the phase margin is reduced by less than 6○ .
Lead compensator
s+b 1 + s/b
G K (s) = K K N = KK N>1
s + bN 1 + s/(bN )
The maximum phase advance is given by the figure below:
60
50
40
∆φ
30
20
10
0
1 2 3 4 5 6 7 8 9 10
15
Simple PID tuning rules
a t
Controller K Ti Td
P 1/ a
PI 0.9/ a 3b
PID 1.2/ a 2b 0.5b
Controller K Ti Td
P 0.5 K 0
PI 0.45 K 0 T0 /1.2
PID 0.6 K 0 T0 /2 T0 /8
16
The Lambda method
The Lambda method is based on a step response experiment where the static
gain K p , a deadtime L, and a time constant T are determined according to
the following figure
Measurement signal
63 %
∆y
L T
Control signal
∆u
where
∆y
Kp =
∆u
The controller parameters for a PI controller are:
1 T
K=
Kp L + λ
Ti = T
The controller parameters for a PID controller in series and parallel form,
respectively, are:
1 T 1 L/2 + T
K′ = K=
K p L/2 + λ K p L/2 + λ
Ti′ = T Ti = T + L/2
L TL
Td′ = Td =
2 L + 2T
17