CPSC 425: Computer Vision
Lecture 2: Image Formation
(unless otherwise stated slides taken or adapted from David Lowe, Bob Woodham, Jim Little, Fred Tung and Leon Sigal )
1
Menu for Today (January 12, 2022)
Topics:
— Image Formation — Projection
— Cameras and Lenses — Human eye (as camera)
Readings:
— Today’s Lecture: Forsyth & Ponce (2nd ed.) 1.1.1 — 1.1.3
— Next Lecture: Forsyth & Ponce (2nd ed.) 4.1, 4.5
Reminders:
— Complete Assignment 0 (ungraded) by Wed, January 19
— Google Colab Tutorials next week
— TA and Office hours are posted and2 will start on Monday, January 17
Today’s “fun” Example
Photo credit: reddit user Liammm
3
Today’s “fun” Example: Eye Sink Illusion
Photo credit: reddit user Liammm
4
Today’s “fun” Example: Eye Sink Illusion
“Tried taking a picture of a sink draining, wound up with a picture of an eye instead”
Photo credit: reddit user Liammm
5
Surprising faces!
Photo credit: camaro5.com
6
Lecture 1: Re-cap
Types of computer vision problems:
— Computing properties of the 3D world from visual data (measurement)
— Recognition of objects and scenes (perception and interpretation)
— Search and interact with visual data (search and organization)
— Manipulation or creation of image or video content (visual imagination)
Computer vision challenges:
— Fundamentally ill-posed
— Enormous computation and scale
— Lack of fundamental understanding of how human perception works
7
Lecture 1: Re-cap
Computer vision technologies have moved from research labs into
commercial products and services. Examples cited include:
— broadcast television sports
— electronic games (Microsoft Kinect)
— biometrics
— image search
— visual special effects
— medical imaging
— robotics
… many others
8
Related Disciplines
Artificial Intelligence (AI)
Robotics
Computer Vision
Machine Human
Learning Scope of CPSC 425 Computer
Interaction
Image Processing
Graphics Geometric Reasoning
Recognition Medical
Imaging
Computational
Photography Neuroscience
Optics
Slide Credit: James Hays (GA Tech) 9
Related Disciplines: Vision and Graphics
Images Model
Vision
Graphics
Inverse problems: analysis and synthesis
(it is sometimes useful to think about computer vision as inverse graphics)
Slide Credit: Kristen Grauman (UT Austin) 10
Why Study Computer Vision?
It is one of the most exciting areas of research in computer science
Among the fastest growing technologies in the industry today
11
12
Wired’s 100 Most Influential People in the World
13
14
CVPR Attendance
15
Lecture 2: Goal
To understand how images are formed
16
What is Computer Vision?
Compute vision, broadly speaking, is a research field aimed to enable computers to
process and interpret visual data, as sighted humans can.
Sensing Device Interpreting Device
Image (or video) Interpretation
Image Credit: https://www.flickr.com/photos/flamephoenix1991/8376271918 blue sky,
trees,
fountains,
UBC, …
17
Overview: Image Formation, Cameras and Lenses
The image formation process that produces a particular image depends on
— Lighting conditions
source sensor
— Scene geometry
— Surface properties
— Camera optics normal
— Sensor properties eye
surface
element
Sensor (or eye) captures the amount of light reflected from the object
18
(small) Graphics Review
source
normal sensor
surface
element
19
(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:
source
normal sensor
Lambertian surface:
surface
element
constant, called albedo
20 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)
(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:
source
normal sensor
Lambertian surface:
surface
element
A Lambertian surface appears the same (brightness) from all directions.
21 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)
Lambertian sphere
“A surface that looks the same from any direction,
i.e., surface points are the same brightness”
(small) Graphics Review
Surface reflection depends on both the viewing and illumination
direction, with Bidirectional Reflection Distribution Function:
source
normal sensor
Lambertian surface:
surface
element
Mirror surface: all incident light reflected in one direction
23 Slide adopted from: Ioannis (Yannis) Gkioulekas (CMU)
Cameras
Old school film camera
Digital CCD/CMOS camera
Cameras
Old school film camera
Digital CCD/CMOS camera
Let’s say we have a sensor …
Digital CCD/CMOS camera
digital sensor
(CCD or
CMOS)
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
… and the object we would like to photograph
What would an image taken like this look like?
digital sensor
real-world
(CCD or
object
CMOS)
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging
digital sensor
real-world
(CCD or
object
CMOS)
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging
digital sensor
real-world
(CCD or
object
CMOS)
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging
digital sensor
real-world
(CCD or
object
CMOS)
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging
digital sensor
real-world
(CCD or
object
CMOS)
All scene points contribute to all sensor pixels
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Bare-sensor imaging
All scene points contribute to all sensor pixels
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Pinhole Camera
barrier (diaphragm)
pinhole
(aperture)
digital sensor
real-world
(CCD or
object
CMOS)
What would an image taken like this look like?
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Pinhole Camera
most rays are
blocked
digital sensor
real-world
(CCD or
object
CMOS)
one makes it
through
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Pinhole Camera
digital sensor
real-world
(CCD or
object
CMOS)
Each scene point contributes to only one sensor pixel
Slide Credit: Ioannis (Yannis) Gkioulekas (CMU)
Camera Obscura (latin for “dark chamber”)
principles behind the
pinhole camera or
camera obscura were Mozi – 470 to 390 BCE
first mentioned by
Chinese philosopher
Mozi (Mo-Ti) (470 to
390 BCE)
Reinerus Gemma-Frisius observed an eclipse of the sun at Louvain on January
24, 1544. He used this illustration in his book, “De Radio Astronomica et
Geometrica,” 1545. It is thought to be the first published illustration of a camera
Credit: John H., Hammond, “Th Camera Obscure, A Chronicle”
obscura.
First Photograph on Record
La table servie
Credit: Nicéphore Niepce, 1822
39
Pinhole Camera
A pinhole camera is a box with a small hole (aperture) in it
Forsyth & Ponce (2nd ed.) Figure 1.2
Pinhole Camera
A pinhole camera is a box with a small hall (aperture) in it
Forsyth & Ponce (2nd ed.) Figure 1.2
Image Formation
Forsyth & Ponce (2nd ed.) Figure 1.1
Credit: US Navy, Basic Optics and Optical Instruments. Dover, 1969
42
Elsa Dorfman and the large format camera
Credit: TuftsNow
Accidental Pinhole Camera
The pin hole camera is not just an abstraction, these are images of
“accidental” room-sized pinhole cameras that resulted from having blinds
with small holes in them.
44 Image Credit: Ioannis (Yannis) Gkioulekas (CMU)
What’s this?
Pinhole Camera (Simplified)
f’ is the focal length of the camera
Pinhole Camera (Simplified)
f’ is the focal length of the camera
Note: In a pinhole camera we can adjust the focal length, all this will do is change the size of the resulting image
Pinhole Camera (Simplified)
It is convenient to think of the image plane which is in from of the pinhole
What happens if object moves towards the camera? Away from the camera?
48
Perspective Effects
Far objects appear smaller than close ones
Forsyth & Ponce (2nd ed.) Figure 1.3a
Perspective Effects
Far objects appear smaller than close ones
Forsyth & Ponce (2nd ed.) Figure 1.3a
Perspective Effects
Far objects appear smaller than close ones
Forsyth & Ponce (2nd ed.) Figure 1.3a
Size is inversely proportional to distance
Perspective Effects
Parallel lines meet at a point (vanishing point)
Forsyth & Ponce (1st ed.) Figure 1.3b
52
Vanishing Points
Each set of parallel lines meets at a different point
— the point is called the vanishing point
53
Vanishing Points
Each set of parallel lines meet at a different point
— the point is called vanishing point
Sets of parallel lines one the same plane lead to collinear vanishing points
— the line is called a horizon for that plane
54
Vanishing Points
Slide Credit: David Jacobs
55
Vanishing Points
Slide Credit: David Jacobs
56
Vanishing Points
Each set of parallel lines meet at a different point
— the point is called vanishing point
Sets of parallel lines one the same plane lead to collinear vanishing points
— the line is called a horizon for that plane
Good way to spot fake images
— scale and perspective do not work
— vanishing points behave badly
57
Vanishing Points
One point perspective
Two point perspective
Vanishing Vanishing
point point
58 Slide Credit: Efros (Berkeley), photo from Criminisi
Perspective Aside
Image credit: http://www.martinacecilia.com/place-vanishing-points/
60
Perspective Aside
Image credit: http://www.martinacecilia.com/place-vanishing-points/
61
Properties of Projection
— Points project to points
— Lines project to lines
— Planes project to the whole or half image
— Angles are not preserved
— Incidences (intersections) are preserved
62
Properties of Projection
— Points project to points
— Lines project to lines
— Planes project to the whole or half image
— Angles are not preserved
Degenerate cases
— Line through focal point (pinhole or aperture) projects to a point
— Plane through focal point projects to a line
63
Projection Illusion
64
Perspective Projection
3D object point
Forsyth & Ponce (1st ed.) Figure 1.4
projects to 2D image point where
Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
67
Perspective Projection: Proof
3D object point
Forsyth & Ponce (1st ed.) Figure 1.4
projects to 2D image point where
Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
68
Aside: Camera Matrix
Camera Matrix
3D object point
Forsyth & Ponce (1st ed.) Figure 1.4
projects to 2D image point where
Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane,
image coordinate frame aligned with the camera coordinate frame
Aside: Camera Matrix
Camera Matrix
projects to 2D image point where
71
Aside: Camera Matrix
Camera Matrix
projects to 2D image point where
72
Aside: Camera Matrix
Camera Matrix
Pixels are squared / lens is perfectly symmetric
Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole
projects to 2D image point where
73
Aside: Camera Matrix
Camera Matrix
Pixels are squared / lens is perfectly symmetric
Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole
projects to 2D image point where
74
Aside: Camera Matrix
Camera Matrix
Pixels are squared / lens is perfectly symmetric
Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole
projects to 2D image point where
75
Aside: Camera Matrix
Camera Matrix
Pixels are squared / lens is perfectly symmetric
Sensor and pinhole perfectly aligned
Coordinate system centered at the pinhole
projects to 2D image point where
78
Aside: Camera Matrix
Camera Matrix
Camera calibration is the process of estimating parameters of the
camera matrix based on set of 3D-2D correspondences
(usually requires a pattern whose structure and size is known)
projects to 2D image point where
79
Perspective Projection
3D object point
Forsyth & Ponce (1st ed.) Figure 1.4
projects to 2D image point where
Note: this assumes world coordinate frame at the optical center (pinhole) and aligned with the image plane, image coord
Weak Perspective
Forsyth & Ponce (1st ed.) Figure 1.5
3D object point in projects to 2D image point
where and
81
Orthographic Projection
Forsyth & Ponce (1st ed.) Figure 1.6
3D object point projects to 2D image point
where
82
Summary of Projection Equations
3D object point projects to 2D image point where
Perspective
Weak Perspective
Orthographic
83
Projection Models: Pros and Cons
Weak perspective (including orthographic) has simpler mathematics
— accurate when object is small and/or distant
— useful for recognition
Perspective is more accurate for real scenes
When maximum accuracy is required, it is necessary to model additional
details of a particular camera
— use perspective projection with additional parameters (e.g., lens distortion)
84
Reminders
Readings:
— Today’s Lecture: Forsyth & Ponce (2nd ed.) 1.1.1 — 1.1.3
— Next Lecture: Forsyth & Ponce (2nd ed.) 4.1, 4.5
Szeliski (background): 2.1, 2.1.1, 2.1.5, 2.2.1, 2.2.2
Reminders:
— Complete Assignment 0 (ungraded) by Wed, Jan 19
— WWW: http://www.cs.ubc.ca/~little/cpsc425/assignments/Assignment0.html
85
