References

COMPUTER VISION
Introduction to

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

“Digital Image Processing”, Rafael C.
Chapter 1: Gonzalez & Richard E. Woods,
Computer vision Addison-Wesley, 2002
 Much of the material that follows is taken
Overview of Computer Vision from this book
“Machine Vision: Automated Visual
ThS: TRẦN VĂN HÙNG Inspection and Robot Vision”, David
Email: tranvanhung@iuh.edu.vn Vernon, Prentice Hall, 1991
E-Learning: Trần Văn Hùng  Available online at:
ocw.fet.iuh.edu.vn homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/
Company
LOGO

Contents Course contents

• Basic Info • Stereo vision, Multi-view Chapter 1: Introduction to Computer vision

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

• Camera calibration stereo
• Capturing light • Structure from motion Chapter 2: Basic techniques in image processing
• Color • Object Recognition
Chapter 3: Image segmentation and
• Linear filters • Bag-of-features models feature extraction
• Edge detection • Face detection and
Chapter 4: Motion analysis and image
• Feature extraction: recognition
tracking processing
Corners and blobs • Segmentation
• Visual motion Chapter 5: Object recognition and
• Fitting: The Hough
classification
transform, Concepts • Tracking
and recipes • Deep Learning Chapter 6: Developing applications for
• Image alignment object recognition
• epipolar geometry
 Introduction Introduction Computer vision
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

1.1 Overview of Computer Vision

1.2 The related concepts

1.3 Fundamental steps in Image processing

1.4 Applications of computer vision

Introduction Computer vision Why study computer vision?

 Images and video are everywhere!
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Personal photo albums Movies, news, sports

Surveillance and security Medical and scientific images

 Why study computer vision? Why study computer vision?

 Vision is useful  “What if I don’t care about this wishy-washy recognition stuff?
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

I just want to make my robot go!”
 Vision is interesting
 Small Reason:
 Vision is difficult
 For measurement, other sensors are often better
 Half of primate cerebral cortex is devoted to visual (in DARPA Grand Challenge, vision was barely
used!)
processing
 For navigation, you still need to learn!
 Achieving human-level visual perception is  Big Reason:
 The goals of computer vision (what + where) are
probably “AI-complete”
in terms of what humans care about.

Slide credit: A. Efros

Origins of computer vision Connections to other disciplines

Artificial
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Intelligence

Machine
L. G. Roberts, Machine Robotics
Perception of Three Learning
Dimensional Solids, Ph.D.
thesis, MIT Department of
Electrical Engineering,
Computer Vision
1963.
Computer
Psychology
Graphics Neuroscience

Image
Processing
 Growth of the field 1. Early vision

 Basic image formation and processing

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

* =
Linear filtering
Edge detection

Cameras and sensors

Light and color

 Check out the list of CVPR 2017 corporate sponsors!

Feature extraction, feature tracking

2. “Mid-level vision” 3. Multi-view geometry

 Fitting and grouping

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Epipolar geometry Stereo

Fitting: Least squares Alignment

Hough transform
RANSAC Structure from motion 3D Photography
 4. Recognition 5. Additional Topics (time permitting)
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Segmentation

Image classification Video

Instance recognition, large-scale alignment

3D scene
understanding

Object detection Deep learning Images and text

The goal of computer vision The goal of computer vision

 To extract “meaning” from pixels

 To perceive the “world behind the picture”
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

153 156 148 152 149 147 139 146 142 150 146 144 137 125 120 119 136 146 151 164 172 175 183 188 196 200 205 208 214 214 219 217
159 151 150 148 140 138 139 129 119 104 86 82 89 97 107 115 118 130 128 132 128 144 160 168 179 188 200 208 213 220 212 214
149 146 153 147 147 146 132 99 73 78 87 96 105 120 138 151 145 157 163 171 165 161 146 126 157 184 190 201 215 212 214 214
145 150 154 148 148 126 93 67 72 78 96 107 117 127 131 134 127 154 166 167 183 194 200 195 143 140 175 190 197 203 206 207
151 153 151 147 120 85 67 75 84 83 94 92 81 78 78 91 83 117 126 144 178 200 201 203 208 175 127 159 185 196 195 206
146 144 139 123 79 66 74 83 79 69 64 62 58 50 46 54 54 66 60 80 86 108 141 191 184 200 187 123 144 175 198 199
135 130 115 87 64 77 90 79 78 85 81 63 55 57 56 53 70 62 61 68 59 58 84 105 168 194 196 183 131 151 185 197
128 116 92 71 82 94 103 101 83 101 88 66 70 90 80 42 39 53 88 73 76 82 116 87 97 144 188 195 190 166 171 203
135 120 84 83 108 127 135 115 100 92 79 49 85 74 59 0 0 0 50 69 52 79 157 141 100 84 136 187 206 204 189 200
144 103 91 115 139 147 127 91 87 80 72 44 61 84 25 0 0 0 50 181 45 69 142 164 167 113 93 130 193 199 208 203
139 102 123 143 137 131 109 85 93 84 68 47 77 86 31 0 3 0 51 156 53 75 141 169 199 151 171 108 143 181 199 208
141 135 153 142 114 104 97 97 83 98 77 42 77 96 79 21 0 23 58 46 56 77 155 199 212 161 194 193 164 187 202 205
160 172 164 141 128 112 98 95 100 96 91 73 68 86 75 73 64 65 54 69 77 115 190 212 193 181 174 188 210 194 202 207
179 189 160 140 139 116 97 97 108 103 110 99 75 80 72 83 50 55 54 95 98 174 205 185 179 188 185 190 193 217 217 224
189 183 152 130 121 105 105 117 114 108 107 115 110 81 85 85 87 81 81 124 183 202 175 180 178 171 173 204 225 215 219 225
178 161 149 135 120 115 122 129 137 145 131 121 125 115 109 91 92 111 132 159 173 170 184 176 184 190 191 217 210 226 228 223
187 159 139 127 125 115 118 121 121 131 133 134 140 137 134 139 140 152 141 154 170 163 195 194 176 198 216 209 219 224 223 226
185 164 140 122 116 110 109 108 113 118 115 116 123 127 135 148 154 162 165 170 171 160 183 198 201 210 223 216 221 222 221 226
188 175 150 130 118 117 113 110 108 115 117 123 130 132 138 150 157 158 174 182 189 186 198 221 224 221 227 221 223 218 218 222
187 179 158 141 124 127 125 127 126 129 130 135 139 141 150 165 175 172 185 195 207 210 212 226 229 222 224 224 223 218 219 221
188 184 172 159 138 135 135 143 143 143 144 146 145 147 160 174 184 191 199 207 211 213 217 224 227 223 223 221 221 218 224 223
192 191 187 174 153 139 140 147 146 149 157 162 160 159 165 174 181 198 201 210 212 216 223 224 225 225 220 215 217 215 224 224

What we see What a computer sees

 What kind of information can be extracted
The goal of computer vision
from an image?
Vision as measurement device
 To extract “meaning” from pixels Real-time stereo Structure from motion Multi-view stereo for
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

community photo collections

NASA Mars Rover

To perceive the “world behind the picture”

 What exactly does this mean?

 Vision as a source of metric 3D information Pollefeys et al.
Goesele et al.
• Amazing success story!
 Vision as a source of semantic information

What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Semantic information
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

tree
roof tree

sky chimney

building
building
window
door

trashcan car car

person
Outdoor scene
ground City European
…

Semantic information Geometric information

slide credit: Fei-Fei, Fergus & Torralba

 What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Challenges: viewpoint variation Challenges: illumination
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

What kind of information can be extracted
1.3 ĐẶC ĐIỂM CỦA THỊ GIÁC MÁY TÍNH
from an image?
Challenges: scale Challenges: deformation
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

 What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Challenges: occlusion Challenges: background clutter
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Challenges: object intra-class variation Challenges: local ambiguity
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

slide credit: Fei-Fei, Fergus & Torralba

 What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Challenges or opportunities? Depth cues: Linear perspective
 Images are confusing, but they also reveal the
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

structure of the world through numerous cues
 Our job is to interpret the cues!

Image source: J. Koenderink

What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Depth cues: Aerial perspective Depth ordering cues: Occlusion
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

 What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Shape cues: Texture gradient Shape and lighting cues: Shading
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Source: J. Koenderink

What kind of information can be extracted What kind of information can be extracted
from an image? from an image?
Position and lighting cues: Cast shadows Grouping cues: Similarity (color, texture, proximity)
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Source: J. Koenderink
 What kind of information can be extracted
from an image?
Contents digital image processing
Grouping cues: “Common fate”
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

This lecture will cover:
 What is a digital image?
 What is digital image processing?
 History of digital image processing
 State of the art examples of digital image
processing
 Key stages in digital image processing

Image credit: Arthus-Bertrand (via F. Durand)

What kind of information can be extracted

from an image?
What is a Digital Image?
Bottom line
A digital image is a representation of a two-
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024 dimensional image as a finite set of digital values,
called picture elements or pixels

• Perception is an inherently ambiguous problem

Many different 3D scenes could have given rise to a particular 2D picture
• Possible solutions
Bring in more constraints (more images)
Use prior knowledge about the structure of the world
• Need both exact measurements and statistical inference!
Image source: F. Durand
 What is a Digital Image? (cont…) What is a Digital Image? (cont…)

Pixel values typically represent gray levels, colours, Common image formats include:
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

heights, opacities etc  1 sample per point (B&W or Grayscale)
Remember digitization implies that a digital image  3 samples per point (Red, Green, and Blue)
is an approximation of a real scene  4 samples per point (Red, Green, Blue, and “Alpha”,
a.k.a. Opacity)
1 pixel

For most of this course we will focus on

grey-scale images

What is Digital Image Processing? What is DIP? (cont…)

The continuum from image processing to

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Digital image processing focuses on two major tasks
computer vision can be broken up into low-, mid-
 Improvement of pictorial information for human and high-level processes
interpretation
Low Level Process Mid Level Process High Level Process
 Processing of image data for storage, transmission and
Input: Image Input: Image Input: Attributes
representation for autonomous machine perception Output: Image Output: Attributes Output: Understanding

Examples: Noise Examples: Object Examples: Scene

Some argument about where image processing ends removal, image recognition,
understanding,
autonomous navigation
and fields such as image analysis and computer vision sharpening segmentation

start
In this course we will
stop here
 History of Digital Image Processing History of DIP (cont…)

Early 1920s: One of the first applications of Mid to late 1920s: Improvements to the
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

digital imaging was in the news- Bartlane system resulted in higher quality images
paper industry  New reproduction
 The Bartlane cable picture processes based
transmission service on photographic
Early digital image
 Images were transferred by submarine cable techniques
between London and New York  Increased number
 Pictures were coded for cable transfer and of tones in Improved
digital image Early 15 tone digital
reconstructed at the receiving end on a reproduced images image
telegraph printer

History of DIP (cont…) History of DIP (cont…)

1960s: Improvements in computing technology

1970s: Digital image processing begins to be
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

and the onset of the space race led to a surge of
used in medical applications
work in digital image processing
 1979: Sir Godfrey N.
 1964: Computers used to
Hounsfield & Prof. Allan M.
improve the quality of
Cormack share the Nobel
images of the moon taken
Prize in medicine for the
by the Ranger 7 probe
invention of tomography,
 Such techniques were used the technology behind
in other space missions A picture of the moon taken Computerised Axial Typical head slice CAT
including the Apollo landings by the Ranger 7 probe image
minutes before landing Tomography (CAT) scans
 History of DIP (cont…) Examples: Image Enhancement

1980s - Today: The use of digital image One of the most common uses of DIP techniques:
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

processing techniques has exploded and they are improve quality, remove noise etc
now used for all kinds of tasks in all kinds of areas
 Image enhancement/restoration
 Artistic effects
 Medical visualisation
 Industrial inspection
 Law enforcement
 Human computer interfaces

Examples: The Hubble Telescope Examples: Artistic Effects

Launched in 1990 the Hubble Artistic effects are

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

telescope can take images of used to make images
very distant objects more visually
appealing, to add
However, an incorrect mirror
special effects and to
made many of Hubble’s
make composite
images useless
images
Image processing
techniques were
used to fix this
 Examples: Medicine Examples: GIS

Take slice from MRI scan of canine heart, and find Geographic Information Systems
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

boundaries between types of tissue  Digital image processing techniques are used
 Image with gray levels representing tissue extensively to manipulate satellite imagery
density  Terrain classification
 Use a suitable filter to highlight edges  Meteorology

Original MRI Image of a Dog Heart Edge Detection Image

Examples: GIS (cont…) Examples: Industrial Inspection

Night-Time Lights of the Human operators are

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

World data set expensive, slow and
 Global inventory of unreliable
human settlement Make machines do the
job instead
 Not hard to imagine Industrial vision systems
the kind of analysis are used in all kinds of
that might be done industries
using this data Can we trust them?
 Examples: PCB Inspection Examples: Law Enforcement

Printed Circuit Board (PCB) inspection Image processing

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

techniques are used
 Machine inspection is used to determine that extensively by law enforcers
all components are present and that all solder
 Number plate
joints are acceptable
recognition for speed
 Both conventional imaging and x-ray imaging cameras/automated
are used toll systems
 Fingerprint recognition
 Enhancement of CCTV
images

Key Stages in Digital Image

Examples: HCI
Processing
Try to make human computer
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

interfaces more natural Image Morphological
 Face recognition Restoration Processing

 Gesture recognition
Image
Does anyone remember the Enhancement
Segmentation
user interface from “Minority
Report”?
Image Object
These tasks can be extremely Acquisition Recognition
difficult
Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing: Key Stages in Digital Image Processing:
Image Aquisition Image Enhancement
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Image Morphological Image Morphological
Restoration Processing Restoration Processing

Image Image
Segmentation Segmentation
Enhancement Enhancement

Image Object Image Object

Acquisition Recognition Acquisition Recognition

Representation Representation
Problem Domain Problem Domain
& Description & Description
Colour Image Image Colour Image Image
Processing Compression Processing Compression

Key Stages in Digital Image Processing: Key Stages in Digital Image Processing:
Image Restoration Morphological Processing
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Images taken from Gonzalez & Woods, Digital Image Processing (2002)
Image Morphological Image Morphological
Restoration Processing Restoration Processing

Image Image
Segmentation Segmentation
Enhancement Enhancement

Image Object Image Object

Acquisition Recognition Acquisition Recognition

Representation Representation
Problem Domain Problem Domain
& Description & Description
Colour Image Image Colour Image Image
Processing Compression Processing Compression
 Key Stages in Digital Image Processing: Key Stages in Digital Image Processing:
Segmentation Object Recognition
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Image Morphological Image Morphological
Restoration Processing Restoration Processing

Image Image
Segmentation Segmentation
Enhancement Enhancement

Image Object Image Object

Acquisition Recognition Acquisition Recognition

Representation Representation
Problem Domain Problem Domain
& Description & Description
Colour Image Image Colour Image Image
Processing Compression Processing Compression

Key Stages in Digital Image Processing: Key Stages in Digital Image Processing:
Representation & Description Image Compression
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Image Morphological Image Morphological
Restoration Processing Restoration Processing

Image Image
Segmentation Segmentation
Enhancement Enhancement

Image Object Image Object

Acquisition Recognition Acquisition Recognition

Representation Representation
Problem Domain Problem Domain
& Description & Description
Colour Image Image Colour Image Image
Processing Compression Processing Compression
 Key Stages in Digital Image Processing:
Colour Image Processing
Summary digital image processing

We have looked at:

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Image Morphological
Restoration Processing  What is a digital image?
 What is digital image processing?
Image
Segmentation  History of digital image processing
Enhancement
 State of the art examples of digital image
Image Object
processing
Acquisition Recognition
 Key stages in digital image processing
Next time we will start to see how it all works…
Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

Applications of computer vision Applications of computer vision

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Reading license plates, Monitoring for safety

Assistive technologies Entertainment Movie special effects
Factory inspection checks, ZIP codes (Poseidon)
(Sony EyeToy)

Digital cameras (face detection for setting focus, Visual search

Surveillance Autonomous driving, Driver assistance exposure) (MSR Lincoln)
robot navigation (collision warning, lane departure
warning, rear object detection)
 Applications of computer vision Applications of computer vision

“Simple” patterns
Reconstruction: 3D from photo collections
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Applications of computer vision Applications of computer vision

Recognition
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

Self-driving
cars

• Computer Eyesight Gets a Lot More

Accurate, NY Times Bits blog, August 18,
2014
• Building A Deeper Understanding of Images,
Google Research Blog, September 5, 2014
 Applications of computer vision
Computer Vision and Image Processing, tranvanhung@iuh.edu.vn. 2024

For more information on the

computer vision industry:

http://www.cs.ubc.ca/spider/lowe/vi
sion.html www.tvhungdhcn@gmail.com

Company
LOGO