CSE483 Computer Vision
Prof. Mahmoud Khalil
Summer 2023
1
Course Team
• Instructor : Prof. Mahmoud Khalil
• Email: mahmoud.khalil@eng.asu.edu.eg
• Lab TA:
• Email:
2
Text Books
• Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing,
4th Edition, Pearson Education, Inc. 2018., ISBN: 978‐9353062989
• Richard Sceliski, Computer Vision Algorithms and Applications,
Springer, 2011 (available online for free at:
http://szeliski.org/Book/), ISBN: 978‐1848829343
• OpenCV: https://opencv.org/
3
Course Contents
4
Marking Scheme
• Students will be evaluated based on the following:
Component % UEL
Student Activities (Project) 35 UEL
One Quiz 5
Midterm exam 20
Final exam 40 UEL
5
Visual Sciences
6
Image Processing ‐ Computer Vision
7
Course Outlines
8
Image Enhancement
9
Image Denoising
Salt & Pepper 3 X 3 Average Median
Noise
10
Image Enhancement ‐ Frequency Domain
Original Noisy image Fourier Spectrum
11
Edge Detection
12
Image Segmentation
Goal: identify groups of pixels that go together
13
Optical character recognition (OCR)
Technology to convert scanned docs to text
• If you have a scanner, it probably came with OCR software
Digit recognition, AT&T labs License plate readers
http://www.research.att.com/ http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
14
Face detection
• Almost all digital cameras detect faces
• Snapchat face filters
Smile detection
Sony Cyber-shot® T70 Digital Still Camera 16
Biometrics
Adapted from Anil Jain, Michigan State
17
Vision‐based biometrics
“How the Afghan Girl was Identified by Her Iris Patterns”
Read the story
wikipedia
18
Login without a password…
Face recognition systems now
Fingerprint scanners on
beginning to appear more widely
many new laptops, http://www.sensiblevision.com/
other devices
19
Video call eye gaze correction
Kuster et al., SIGGRAPH Asia 2012
• https://cgl.ethz.ch/publications/papers/paperKus12.php
Apple FaceTime
Attention Correction
Object recognition (in mobile phones)
Point & Find, Google Goggles
21
Building a Panorama
22
M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003
Feature descriptors
• Extraordinarily robust matching technique
• Can handle changes in viewpoint
• Up to about 60 degree out of plane rotation
• Can handle significant changes in illumination
• Sometimes even day vs. night (below)
• Fast and efficient—can run in real time
23
Steve Seitz
Interactive Games: Kinect
• Object Recognition:
http://www.youtube.com/watch?feature=iv&v=fQ59dXOo63o
• Mario: http://www.youtube.com/watch?v=8CTJL5lUjHg
• 3D: http://www.youtube.com/watch?v=7QrnwoO1‐8A
• Robot: http://www.youtube.com/watch?v=w8BmgtMKFbY
24
Medical imaging
Image guided surgery
3D imaging
Grimson et al., MIT
MRI, CT
25
Smart cars
• https://www.mobileye.com/
• Vision systems currently in high‐end BMW, GM, Volvo models
26
Slide content courtesy of Amnon Shashua
Google cars
http://www.nytimes.com/2010/10/10/science/10google.html?ref=artificialintelligence
27
AutoCars ‐ Uber bought CMU’s lab
28
Object recognition (in supermarkets)
How does it work? Think‐Pair‐Share
How does it work?
Thanks to Vivek Ramanujan
Sports
Virtual pitch markings Free viewpoint video
Sportvision first down line [Canon 2017]
Nice explanation on www.howstuffworks.com
JH
Sports video analysis
Tennis review system
http://www.hawkeyeinnovations.co.uk 35
Human shape capture
Human shape capture
Human shape capture
Human shape capture
Car Detection and Depth Estimation
40
Vision as a Source of Semantic Information
41
Why is vision so hard?
posed problem
[Sinha and Adelson 1993]
Adapted from Ali Farhadi, U of Washington 42
Challenges 1: view point variation
Michelangelo 1475-1564
Adapted from L. Fei-Fei, R. Fergus, A. Torralba
43
Challenges 2: illumination
Adapted from Fei-Fei Li
44
Challenges 3: occlusion
Magritte, 1957
Adapted from L. Fei-Fei,
R. Fergus, A. Torralba
45
Challenges 4: scale
Adapted from L. Fei-Fei,
R. Fergus, A. Torralba
46
Challenges 5: deformation
Xu, Beihong 1943
Adapted from L. Fei-Fei, R. Fergus, A. Torralba
47
Challenges 6: background clutter
Adapted from Fei-Fei Li
48
Challenges 7: intra-class variation
Adapted from L. Fei-Fei, R. Fergus, A.
49 Torralba
What do computers see?
50
Stages of computer vision
• Low‐level
image image
• Mid‐level
image features / attributes
• High‐level
features “making sense”, recognition
51
Low‐level
sharpening
blurring
Adapted from Linda Shapiro, U of Washington
52
Low‐level
Canny
original image edge image
Mid‐level
ORT
data
structure
edge image circular arcs and line segments
Adapted from Linda Shapiro, U of Washington
53
Mid‐level
K-means
clustering
(followed by
connected
component
analysis)
original color image regions of homogeneous color
data
structure
Adapted from Linda Shapiro, U of Washington
54
Low‐level to high‐level
low-level
edge image
mid-level
consistent
high-level line clusters
Adapted from Linda Shapiro, U of Washington
55
Visual recognition
Verification
Is this a car?
56
Visual recognition
Classification
:
Is there a car in this picture?
57
Visual recognition
Detection
:
Where is the car in this picture?
58
Visual recognition
Pose Estimation:
59
Visual recognition
Activity Recognition:
What is he What is he doing?
doing?
60
Visual recognition
Object Categorization:
Sky
Person
Tree
Horse
Car
Person
Bicycle
Road
61
Visual recognition
Segmentation
Sky
Tree
Car
Person
62
