3D Data Processing
From Objects to Camera
Alberto Pretto
Agenda
● Homogeneous coordinates
● Axis-Angle representation
● Review of pinhole camera model and
perspective projection
2
Homogeneous Coordinates
● Converting to homogeneous coordinates
● Converting from homogeneous coordinates
3
Homogeneous Coordinates
● Line l : ax+by=d or ax+by+c=0
With [a,b]T unit vector:
pT . [a,b] = d
|p|cos(θ) = d
4
Homogeneous Coordinates
● Uniform treatment of points and lines
● Line-point incidence: lT p=0
5
Homogeneous Coordinates
● 2D Line between two 2D points
– Point x on l must be linear combination of p1
and p2:
6
Homogeneous Coordinates
● Join = cross product
7
Homogeneous Coordinates
● Join of 3D points
8
Inverse and Homogeneous Representation
10
Rotation Matrix Constraints
● Not all 3 × 3 matrices are in SO(3)
● The nine entries are not free
parameters
– 6 constraints since:
– Only 3 degrees of freedom (DoF):
for instance, we can decompose
our rotation matrix in three rotations
around the axes (Euler angles)
[Source: Wikipedia]
11
Exponential Map
● Consider an object that rotates over time with rotation
at time t defined by R(t) ∈ SO(3). z
y
z y
x x
● Compute the derivatives w.r.t. time: Skew-symmetric!
● A skew-symmetric matrix is completely defined by some
ω ∈ R3:
We assume here that
12
this matrix is constant
Exponential Map
● From Calculus:
● This result should hold for any column of
● Assuming R(0) = I:
● If we consider ω already multiplied by t:
● The above matrix exponential defines a map (exponential
map) from the space of all the skew symmetrix matrices so(3)
to SO(3):
"Lie algebra"
"Lie group"13
Exponential Map
so(3) is the tangent space at the identity of SO(3)
Identity matrix
so(3) (Lie algebra): space of all
skew symmetrix matrices n R9
SO(3) (Lie group) space of all
rotations matrices in R9
14
Axis-Angle Representation
● Assume R(t0) = I:
● For any object point p we can compute the tangent
linear velocity vector that defines the rotational
p = [x, y, z]T
motion:
● ω defines the axis of the instantaneous rotation and its ω
norm the magnitude of the rotational velocity z p
● Euler's rotation theorem: any rotation in 3D is ω×p y
equivalent to a pure rotation around a single fixed x
axis.
● We can represent any rotation by means of:
– A unit vector: axis of rotation ==> orientation of ω Axis–angle
– An angle θ: magnitude of the rotation around the the axis representation
==> norm of ω 15
Axis-Angle Representation
so(3)
SO(3)
16
Rodrigues' Formula
How to explicitly compute R from ω?
● Rodrigues' formula provides an algorithm to compute in
closed form the exponential map from so(3) to SO(3)
without actually computing the full matrix exponential.
● For a given rotation matrix R , there are infinite ω such that
17
Rodrigues' Formula Proof Hints
Define:
Since ω unit vector:
18
Inverse Map
● From Rodrigues' formula it is possible to
derive the inverse map to compute ω from
R:
● If R=I, the magnitude of the rotation is zero
so we can chose the orientation arbitrarily
● Again, not one-to-one map
19
Distance Between Rotations
● Given two matrices R1 and R2, how to
compute the angular distance θ between
rotations?
20
Review of Pinhole Camera Model
and Perspective Projection
21
Standard coordinate system
Standard Coordinate System
● Uppercase coordinates (X,Y,Z): 3D camera reference
frame, centered in the optical center
● Lowercase coordinates (x,y): 2D image reference frame
P
Optical
center
Principal
xis
point ala Z
t ic
Op
y
f
x O
p X
Im
ag
ep Y
lan 22
e
From 3D to 2D
A 3D point P = (XP, YP, ZP) is “projected” on the 2D
image plane into the 2D point p=(x,y)
P
y
f
x O
p X
Im
ag
ep Y
lan
e
Side view 23
From 3D to 2D
A 3D point P = (XP, YP, ZP) is “projected” on the 2D
image plane into the 2D point p=(x,y)
P
p
Z
y
f x
O y
x
p X
Im
ag
ep Y
lan
e
Side view 24
From 3D to 2D
A 3D point P = (XP, YP, ZP) is “projected” on the 2D
image plane into the 2D point p=(x,y)
P
p
Z
x
O y
Side view 25
From 3D to 2D
A 3D point P = (XP, YP, ZP) is “projected” on the 2D
image plane into the 2D point p=(x,y)
Side view 26
From 3D to 2D
The depth (i.e., the Z-coordinate) of each
point is lost! All the points that
All the points that lie in this line
lie in this line project onto the
project onto the same pixel
same pixel P
p
Z
f
x
O y
Y
Side view 27
Homogeneous coordinates
Homogeneous coordinates
28
Map to Pixels
We need to convert an image coordinates
from meters to pixels.
u
P
p
Z
f
x
O y
v
Y Note: In general, pixels do not
have a perfect square shape 29
Map to Pixels
Remember that the top left pixel has (0,0)
coordinates (i.e., in pixel space, the origin is
not the center of the image)
uc and vc are the coordinates of the principal
point in pixels and wp and hp are the pixel
width and height in meters, respectively.
30
Map to Pixels
●
Substituting
● Defining
31
Map to pixels
Map to Pixels
● Using homogeneous coordinates:
Perspective
projection matrix,
or camera matrix
32
From World to Camera
● In general, coordinates of a 3D point are not
specified in current camera frame:
● Before projecting a point onto the image plane, we
need to change its coordinates from the world
frame to the current camera coordinate system.
33
Putting all together
● From world frame to pixels:
● Sometimes this equation is rewritten in an alternative way:
● Where [R|t] is a 3x4 matrix composed by the rotation
matrix followed by the translation vector, and K is a 3 × 3
matrix holding the intrinsic parameters:
34
Normalized (or Canonical) Camera
● The camera matrix can be simplified if we assume that
f = 1. This special camera is called normal or canonical
camera:
● Projecting a 3D point into a canonical camera is trivial:
E.g., Meters 35
“Denormalize” a canonical camera
● The intrinsic parameters matrix K can be used to map
points from the canonical camera to a specific
camera:
● An alternative representation of the camera projection
process is provided by the following steps:
– Transform from world to camera frame using T;
– Project into the canonical camera using An;
– Denormalize the projection to the target camera using its K.
36
“Normalize” a real camera
● Conversely, we can “convert” a real
camera in a normalized camera:
37
Lens Distortion
● The pinhole camera model in
not enough to describe real
cameras:
– Lens is usually not perfect and
not perfectly aligned with the
imaging sensor.
● Lens distortion is modeled by
means of non-linear functions
defined by a set of intrinsic
(or simply, distortion)
parameters.
38
Projection Process with Distortion
● Transfrom from world to camera
frame
● Project into the canonical camera
● Apply the non linear distortion
● Denormalize the projection to the
target camera using its K.
39
