Lecture Notes 10

Image Sensor Optics

• Imaging optics
◦ Space-invariant model
◦ Space-varying model
• Pixel optics
◦ Transmission
◦ Vignetting
• Microlens

EE 392B: Image Sensor Optics 10-1

 Image Sensor Optics

Microlens

Imaging Optics

Pixel Optics

Image sensor optics consist of (a) imaging optics and (b) pixel optics

EE 392B: Image Sensor Optics 10-2

 Imaging Optics: Linear Space-Invariant Model

Y Y

Entrance Exit
Pupil Pupil
X X

Object Generalized Model Image

L (W/m2 sr) (f/#, T, R) I (W/m2)

A generalized model for rotationally symmetric imaging optics

J.W. Goodman, Introduction to Fourier Optics, 2nd Ed, Ch 6, pp. 126-171 (1996)

EE 392B: Image Sensor Optics 10-3

 Imaging Optics: Camera Equation

T ( λ ) R ( x, y ; λ ) x y 
I ideal ( x, y; λ ) ≅ π Lscene  , ; λ 
1 + 4 (1 + m ) ( f /# ) m m 
2 2

I ideal : ideal, unaberrated, geometric image irradiance distribution ( W m 2 )

Lscene : Lambertian object radiance distribution ( W sr m 2 )

m = − zi zo : magnification
T : transmittance
R : relative illumination factor (e.g., cos 4 fall-off)
f /# = f D : f-number

Imaging optics map 2D object radiance distribution (W/sr m2) into

2D image irradiance distribution (W/m2)
M.V. Klein and T.E. Furtak, Optics, , 2nd Ed, Ch 4, pp. 193-262 (1986)

EE 392B: Image Sensor Optics 10-4

 Resolution Limited by Diffraction

Plane Converging Plane Converging

wave wave wave wave

Perfect Diffraction
geometric limited
point point

Infinite Aperture Optics Finite Aperture Optics

Diffraction, which is caused at aperture edges, is the fundamental

reason why a plane wave can not be focussed into a perfect geometric
point

EE 392B: Image Sensor Optics 10-5

 Diffraction: Space Domain Description

Image formation process with incoherent illumination can be described

in the space domain by a convolution operation

I image ( x, y; λ ) = PSF ( x, y; λ ) ∗ I ideal ( x, y; λ )

I image : blurred, aberrrated, distorted image irradiance distribution

I ideal : ideal, unaberrated, geometric image irradiance distribution

PSF : point spread function, i.e., response of optical system in

image plane to a point excitation in object plane
∗: convolution operator

EE 392B: Image Sensor Optics 10-6

 Diffraction: Frequency Domain Description

Alternatively, image formation in shift-invariant systems can be viewed

as a linear filtering process in the frequency domain
F {I image ( x, y; λ )} = F {PSF ( x, y; λ )} ⋅ F {I ideal ( x, y; λ )}
= OTF ( f x , f y ; λ ) ⋅ F {I ideal ( x, y; λ )}

{
I image ( x, y; λ ) = F −1 OTF ( f x , f y ; λ ) ⋅ F {I ideal ( x, y; λ )} }
The optical transfer function (OTF) is normalized to preserve
radiometry. Its magnitude is the modulation transfer function (MTF)

F {PSF ( x, y; λ )}
MTF ( f x , f y ; λ ) = OTF ( f x , f y ; λ ) =
F {PSF ( x, y; λ )} f x = 0 , f y =0

EE 392B: Image Sensor Optics 10-7

 Example: Diffraction-limited Lens

1
f/# = 2.0, λ = 486 nm
0.9

0.8

0.7

0.6
MTF

0.5

0.4

0.3

0.2

0.1

0
0 200 400 600 800 1000 1200
Spatial frequency (lp/mm)

MTF(fr ) = π2 (ϕ − cos ϕ sin ϕ) with ϕ = cos−1(fr λf /#) and fr,cutof f = 1

λf /#

EE 392B: Image Sensor Optics 10-8

 System MTF

If the system is linear and space-invariant, the system MTF (optics +

sensor) in the frequency domain can then be easily computed

MTFsystem ( f x , f y ; λ ) = MTFoptics ( f x , f y ; λ ) ⋅ MTFgeometric ( f x , f y ) ⋅ MTFdiffusion ( f x , f y ; λ )

To obtain the image spectrum we apply the MTF as a linear filter

I image ( f x , f y ; λ ) = MTFsystem ( f x , f y ; λ ) ⋅ I object ( f x , f y ; λ )

EE 392B: Image Sensor Optics 10-9

 Modeling Real Imaging Lenses

Vignetting Distortion Illumination Effects

Wavefront Point Spread Modulation Transfer

Aberrations Function (PSF) Function (MTF)

P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)

EE 392B: Image Sensor Optics 10-10

 Imaging Optics: Linear Space-Varying Model

Y Y

Entrance Exit Ωk
Pupil Pupil
X X

Object Generalized Model Image

L (W/m2 sr) (f/#, T, R) I (W/m2)

Image plane is segmented into isoplanatic regions: Ω1, Ω2, . . ., Ωn

P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)

EE 392B: Image Sensor Optics 10-11

 Linear Space-Varying Image Formation

Let Ωk , k = 1, . . . , n be aplanatic image segments, then

I image ( x, y; λ ) = ∑ PSFΩk ( x, y; λ ) ∗ I ideal ,Ωk ( x, y; λ )

k

or
 
I image ( x, y; λ ) = F −1
{ }
∑ OTFΩk ( f x , f y ; λ ) ⋅ F I ideal ,Ωk ( x, y; λ ) 
 k 

P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)

EE 392B: Image Sensor Optics 10-12

 Example: Double Gauss f/2.0 Lens

P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005)

EE 392B: Image Sensor Optics 10-13

 Double Gauss f/2.0 Lens: MTF

P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005)

EE 392B: Image Sensor Optics 10-14

 Pixel Optics

Incident
Photons

Reflected
Transmitted Photons

Optical Efficiency
Photons

Rejected or
Accepted Scattered
Photons Photons

Efficiency
Uncollected

Quantum
Collected Electrons
Electrons

P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
H. Rhodes et al., IEEE Workshop on Microelectronics and Electron Devices, pp. 7-18 (2004)

EE 392B: Image Sensor Optics 10-15

 Incident Photons

-3
x 10
10

9
Photometric Exposure (lux-sec)

6
4000
5

3
2000
2
1000
1
2 3 4 5 6 7 8
Pixel size (µm)

P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 1-13 (2005)

EE 392B: Image Sensor Optics 10-16

 Pixel Transmittance

F+(z’) F-(z’)

Layer 0
I01
Layer 1 L1

Layer j-1 Transmission

Ij-1,j
Layer j Lj

Layer m Lm
Im,m+1
Layer m+1

F+(z”) F-(z)

F. Abeles, Ann. de Phys. 5, pp. 596-640 (1950)

P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-17

 Example: Transmittance 0.18µm CMOS

Transmittance 0.8

0.6

0.4 air
0.7 µm Si3N4

0.2 8.3 µm SiO2

Si
0
400 500 600 700 800
Wavelength (nm)

Pixel transmittance is λ-dependent (even for dispersion-free materials)

P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-18

 Example: Transmittance 0.18µm CMOS

0.8

0.6
Transmission T

Si3N4
0.4

SiO2

0.2
Si

0
0 20 40 60 80 100
Angle (degrees)

Pixel transmittance (λ-averaged) is approximately independent of angle

P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-19

 Pixel Vignetting

No Vignetting

Pixel Vignetting Pixel Cross-talk

P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-20

 Pixel Vignetting: Effect of Pixel Height

0.7
0 degrees
11 degrees
0.6
23 degrees

0.5
Pixel response

0.4

0.3

0.2

0.1

0
1 2 3 4 5 6 7
Number of metal layers

Reduction in optical efficiency as a function of the number of metal layers in a

0.18µm standard CMOS process (f/1.8 imaging lens)
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-21

 Pixel Vignetting: Effect of Technology Scaling

0.8
0 degrees
11 degrees
0.7 23 degrees
Pixel response

0.6

0.5

0.4

0.3
0.4 0.35 0.3 0.25 0.2 0.15
Feature size (µm)

Reduction in optical efficiency for a standard APS pixel with a 30% fill-factor
using 2 metal layers as a function of the feature size of CMOS technology
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)

EE 392B: Image Sensor Optics 10-22

 Optical Efficiency

• Definition:
◦ Optical efficiency (OE) is the ratio of the photons incident of the
substrate and the photons incident on the pixel surface
• Sources of photon loss:
◦ Back-reflections in dielectric stack (air-SiO2, SiO2-Si)
◦ Photons absorbed in dielectric stack (SiON)
⇒ Pixel transmittance T(λ,θ)
◦ Photons scattered away from pixel (pixel cross-talk)
◦ Photons rejected by metal
⇒ Pixel vignetting V(x,y,θ)
• Description:
◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ)

EE 392B: Image Sensor Optics 10-23

 Microlens

• Focus light onto photo-sensitive region – increases effective fill factor from
25-40% to 60-80% (and sensitivity by ≥ 2X)
• Less effective if photosensitive area is irregularly shaped

A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995) S.G. Wuu et al., “High
performance 0.25 µm CMOS color imager technology with non-silicide source/drain pixel,” IEDM Tech. Dig., pp.
705-708 (2000)

EE 392B: Image Sensor Optics 10-24

 Microlens Fabrication

• Lens material requirements:

◦ Highly transparent in the visible
light region
◦ Index of refraction > 1.59
◦ Can be applied below 500C
◦ No degradation or aging
◦ Semiconductor processing com-
patible
◦ Can be patterned with feature size
commensurate with the pixel size
• Lens materials are typically i-line or
DUV resists
• Base materials are acrylic-based resists,
polyimide resists, epoxy resists, poly-
organosiloxane, polyorganosilicate
EE 392B: Image Sensor Optics 10-25
 Microlens and the Main Lens

• Microlens is optimized for a specific main lens system

• Rays incident on the microlens form a cone with NA = sin ϕ
• NA varies as a function of the size and position of the exit pupil
• Principle ray at the periphery of the sensor has an angle δ, chief ray angle
(CRA), between the ray and the optical axis (δ depends on the position of
the pixel on the sensor)

A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)

EE 392B: Image Sensor Optics 10-26

 Microlens and F-Number

• High F-number: rays are parallel (NA ≈ 0)

• Low F-number: rays arrive at an angle (NA large) – microlens
effectiveness low

A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)

EE 392B: Image Sensor Optics 10-27

 Micro lens: How to concentrate photons

Micro-lens

Metal Wires

Photodiode

Shallow Pixel Deep Pixel

EE 392B: Image Sensor Optics 10-28

 Micro lens: How to concentrate photons

Dµ-lens Dµ-lens

fµ-lens
f'µ-lens

Shallow Pixel Deep Pixel

EE 392B: Image Sensor Optics 10-29

 Micro lens: How to concentrate photons

2NA
2NA'

Shallow Pixel Deep Pixel

q q
NA = 1 N A0 = 1
1+4(f /#)2 1+4(f /#0)2

EE 392B: Image Sensor Optics 10-30

 Micro lens: How to concentrate photons

2.44 f/# λ

2.44 f/#' λ

Shallow Pixel Deep Pixel

f 0
f
f /# = DµLens f /#0 = DµLens
µLens µLens

EE 392B: Image Sensor Optics 10-31

 Micro lens: How to concentrate photons

• Conservation of etendue
⇒ G = 2N Aimaging wpixel = 2N Aµlens wdiode

• Etendue limits light collection efficiency (NA)

⇒ Bigger NA allows bigger etendue
• Concentration depends on ratio of NA of microlens and imaging lens
⇒ For a 2x space concentration (wdiode = wpixel /2)
⇒ 2N Aµlens > N Aimaging

• Diffraction limits spot size (f-number)

⇒ Smaller f-number allows smaller spot size

EE 392B: Image Sensor Optics 10-32

 Micro lens: Concentration

(a) (b)
f/4.0 f/2.8
10
f/2.0
64 f/4.0
8
36 f/2.8
Concentration

Pixel height (µm)

16 f/2.0
6 f/1.4

4 f/1.4
4
0
10
8 10
6 8
6 2
4 4
2 2 2 4 6 8 10
Pixel size (µm) Pixel height (µm) Pixel size (µm)

P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)

EE 392B: Image Sensor Optics 10-33

 Pixel Position

• For large CRA (pixel away from the center of the lens, or exit pupil close
to the sensor), ray may not focus on the photo-sensitive region
• Effect: non-uniform sensitivity profile across image sensor

A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)

EE 392B: Image Sensor Optics 10-34

 Micro lens: Redirection (w/o offset)

Pixel width: 3µm Pixel Height: 8 µm

1
f/1.4
f/2.0
f/2.8
0.8 f/4.0

0.6
OE

0.4

0.2

0
0 5 10 15 20
θ (deg)
CR

P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)

EE 392B: Image Sensor Optics 10-35

 Micro lens: Redirection (with offset)

Pixel width: 3µm Pixel Height: 8 µm

1
f/4.0
f/2.8
0.8
Optical Efficiency

0.6
f/2.0

f/1.4
0.4
∆ = fml tan(θ)

0.2
fml

θ
0
0 5 10 15 20
θ (deg)

P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)

EE 392B: Image Sensor Optics 10-36

 Optical Efficiency: Summary

• Without microlens:
◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ)
• With microlens:
◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) ML(x,y,θ)
∗ where ML(x,y,θ) represents a correction factor to account for
the concentration and/or redirection performed by the microlens

EE 392B: Image Sensor Optics 10-37

 Improving OE

• Optimizing dielectric stack thickness

• Make sure dielectrics are not light absorbing
• Utilize different dielectric materials to achieve total internal reflection
• Add an airgap between pixels (total internal reflection)

EE 392B: Image Sensor Optics 10-38