Digital Photography I
Optics & Sensors
EE367/CS448I: Computational Imaging
stanford.edu/class/ee367
Lecture 3
Gordon Wetzstein
Stanford University
Abelardo Morell
Pinhole Camera / Camera Oscura
Mo-Ti (Chinese Philosopher) 470-390 BC
Pinhole Camera / Camera Oscura
J. Vermeer “The Milkmaid”, 1658
Pinhole Camera / Camera Oscura
Credit: ©Toppan Printing Co., Ltd.
Original photo data (Het melkmeisje [The Milkmaid] by Johannes Vermeer) : J. Vermeer “The Milkmaid”, 1658
©Rijksmuseum Amsterdam. Purchased with the support of the Vereniging Rembrandt
Digital Photography - Overview
• optics
• aperture
• depth of field
• field of view
• exposure
• noise
• color filter arrays
• image processing pipeline
SLR Camera pentaprism
viewfinder
eyepiece
camera lens
mirror sensor
Camera Optics
Niepce “View from the Window at Le Gras”, 1826
1826
8h exp
Daguerrotype
• invented in 1836 by Louis Daguerre
• lenses focus light, better chemicals!
Daguerre “Boulevard du Temple”, 1838
10-12 mins
exposure
Lenses
• focus light
• magnify objects
Nimrud lens - 2700 years old
Lenses
1 1 1 S2 f
lensmaker’s = + magnification: M =− =
equation: f S1 S2 S1 f − S1
Lenses
S1<f: magnifying glass minification
Lenses - Aberrations
Lenses - Aberrations
Sharp Image Blurred Image due
to optical aberrations
1000 mm
500 mm
2.5°
5°
Field of View
350 mm 7.5°
250 mm 10°
135 mm 18°
85 mm 29°
50 mm 43°
35 mm 63°
28 mm 75°
8 mm 180°
Andrew McWilliams
1000 mm
500 mm
2.5°
5°
Field of View
350 mm 7.5°
250 mm 10° Hubble – what’s the focal length?
57.6 m or 52 arc secs = 0.0144° (CCD, 1024x1024)
135 mm 18°
85 mm 29°
50 mm 43°
35 mm 63°
28 mm 75°
8 mm 180°
Aperture
• aperture controls amount of light
focal plane
Aperture
• aperture controls amount of light
focal plane
Aperture
• unit: f-number N=f/D
focal plane
Aperture
• out of focus blur
circle of confusion
focal plane
Aperture
• out of focus blur
circle of confusion
focal plane
Depth of Field
depth of field: max circle of confusion
circle of confusion
focal plane
Circle of Confusion
S − S1
c = M ⋅D⋅
f S
D
M=
circle of confusion: c
S1 − f
focal plane: S1
S
Circle of Confusion
S − S1 Canon 5D Mark III: f=50mm, f/2.8 (N=2.8),
c = M ⋅D⋅ focused at 5m, pixel size=7.5um
S
circle of confusion in px
focal plane
depth of field
distance to camera in m
Depth of Field
http://photographywisdom.com/
Depth of Field & Motion Blur
London, Photography
Bokeh
artistic use coded aperture
two delighted blog Levin et al., SIGGRAPH 2007
Diffraction Limit
f-number
λ λ
• Ernst Abbe 1873: d= = ≈ λN
2nsin θ 2NA
spot radius (image space)
numerical aperture
diffraction Airy pattern
wikipedia
Diffraction Limit
f-number
λ λ
• Ernst Abbe 1873: d= = ≈ λN
2nsin θ 2NA
numerical aperture
• microscope objectives today: NA 1.4-1.6 à d=λ/2.8
• small f-number (large NA) = high resolution but shallow depth of field
• inherent tradeoff between “3D” information and 2D resolution
• space-bandwidth product (uncertainty principle)
Sensors
What’s a Pixel?
photon to electron converter
à photoelectric effect!
source: Molecular Expressions
wikipedia
What’s a Pixel?
• microlens: focus light on
photodiode
• color filter: select color channel
• quantum efficiency: ~50%
• fill factor: fraction of surface area
used for light gathering
source: Molecular Expressions
What’s a Pixel?
What’s a Pixel?
What’s a Pixel?
What’s a Pixel?
Most Common: Color Filter Arrays
Bayer pattern
any combination possible
wikipedia
tradeoffs?
Assorted Pixels
• Narasimhan & Nayar @ Columbia
• multiplex anything: polarization, color, time, ND, …
Exposure (shutter speed)
• exposure = irradiance * time (e.g. 1/250, 1/60, 1, 15, bulb)
wikipedia
¼ sec, f/3.3, ISO 200 2 sec, f/6.3, ISO 80
ISO (“film speed”)
sensor
sensitivity
–
analog gain
applied before
ADC!
bobatkins.com
Dynamic Range
• ratio between largest and smallest possible value
• bit depth also important! common bit depths: 12-14 bits RAW / 8
bits JPEG
high dynamic range
Kevin McCoy
Global Shutter vs. Rolling Shutter
All sensor pixels exposed at same time Row-by-row readout of image
• shorter exposure times per pixel
• motion artifacts
http://lfa.mobivap.uva.es/~fradelg/phd/notes/global-shutter.html
Photons to RAW Image
sensor defects
photon = fixed additive noise quantization
noise pattern noise “noise”
RAW image
photons
sensor amplifier ADC
(gain,ISO) (quantization)
Sensor Noise
• noise is (usually) bad!
• many sources of noise: heat, electronics, amplifier gain, photon to
electron conversion, pixel defects, read, …
• different noise follows different statistical distributions, two crucial ones:
• Gaussian
• Poisson
Gaussian Noise
• thermal, read, amplifier
• additive, signal-independent!
+ =
Photon or Shot Noise
• signal dependent
• Poisson
distribution:
λ k e− λ
f (k; λ ) =
k!
σ= λ
N photons: σ = N
2N photons: σ = 2 N
nonlinear!
wikipedia
Signal-to-Noise Ratio (SNR)
mean pixel value µ signal
SNR = =
standard deviation of pixel value σ noise
PQet
=
PQet + Dt + N r2
P = incident photon flux (photons/pixel/sec)
Q e = quantum efficiency
t = eposure time (sec)
D = dark current (electroncs/pixel/sec), including hot pixels
N r = read noise (rms electrons/pixel), including fixed pattern noise
Scientific Sensors
• e.g., Andor iXon Ultra 897: cooled to -100° C
• scientific CMOS & CCD
• reduce pretty much all noise, except for photon noise
Digital Photography
• optics
• aperture
• depth of field
• field of view
• exposure
• noise
• color filter arrays
• image processing pipeline
Digital Photography – Additional Resources
• What we left out: metering, autofocus, autoexposure, anti-aliasing filter,
IR filter (and probably much more)
• Stanford CS 178 – Digital Photography: slides, applets, and other
material online
• looking for a camera? check dpreview.com
Next: The Image Processing Pipeline
• RAW images
• demosaicking
• denoising
• deblurring
• white balancing
• gamma correction
• compression
Homework 2
• calculate and plot depth of field of different cameras (today’s
lecture)
• implement a simple image processing pipeline in Python and
explore demosaicking, denoising, etc. (next lecture)
References and Further Reading
• London, Upton, Stone, “Photography”, Pearson, 11th edition, 2013
• Stanford CS 178, “Digital Photography”, Course Notes http://graphics.stanford.edu/courses/cs178/
• wikipedia
