ECE 4554 / ECE 5554

Computer Vision

Basic Image Operations

Virginia Tech
Fall 2023

[Credit to Prof. Creed Jones for several of these slides]

1
 Outline

• Intensity profiles

• Image histograms

• Pixelwise operations

• Two-dimensional spatial filters (introduction)

2
 Reminder: What computers see

243 239 240 225 206 185 188 218 211 206 216 225

242 239 218 110 67 31 34 152 213 206 208 221

243 242 123 58 94 82 132 77 108 208 208 215

235 217 115 212 243 236 247 139 91 209 208 211

233 208 131 222 219 226 196 114 74 208 213 214

232 217 131 116 77 150 69 56 52 201 228 223

232 232 182 186 184 179 159 123 93 232 235 235

232 236 201 154 216 133 129 81 175 252 241 240

235 238 230 128 172 138 65 63 234 249 241 245

237 236 247 143 59 78 10 94 255 248 247 251

234 237 245 193 55 33 115 144 213 255 253 251

248 245 161 128 149 109 138 65 47 156 239 255

190 107 39 102 94 73 114 58 17 7 51 137

23 32 33 148 168 203 179 43 27 17 12 8

17 26 12 160 255 255 109 22 26 19 35 24

Slide credit: Larry Zitnick

4
HISTOGRAMS

5
 An image histogram is a vector where each entry is
the count (or statistical frequency) of the corresponding intensity value

100 100 100 100 100 110 110 110 110 110
100 100 100 100 100 110 110 110 110 110
100 100 100 100 100 110 110 110 110 110
100 100 100 100 100 110 110 110 110 110
100 100 100 100 100 110 110 110 110 110
100 100 100 100 100 110 110 110 110 110 40

100 100 100 100 100 110 110 110 110 110 35

80 80 80 80 80 80 80 80 200 200 30
80 80 80 80 80 80 80 80 200 200 25
80 80 80 80 80 80 80 80 200 200 20

15

10

0
0 20 40 60 80 100 120 140 160 180 200 220 240

6
 Computing a histogram

def getHist(image, histlen):

rows,cols = image.shape
hist = np.zeros((histlen), int)
for r in range(rows):
for c in range(cols):
hist[image[r,c]] = hist[image[r,c]] + 1
return hist

. . .
img = cv2.imread('file.png', cv2.IMREAD_GRAYSCALE)
myHist = getHist(img, 256)

7
Noise and image smoothing (blur) cause the lobes in the histogram to spread;
below, Gaussian noise was added to pixel values

8
 Histograms can reveal images that are
underexposed, properly exposed, or overexposed

9
Rembrandt van Rijn, Jacob Blessing the Children of Joseph
10
Rembrandt van Rijn, Jacob Blessing the Children of Joseph
11
Jan Vermeer, Young Woman with a Water Pitcher
12
Jan Vermeer, Young Woman with a Water Pitcher
13
PIXELWISE OPERATIONS

14
 Pixelwise operations are common
(also called “point operations”)

𝐼𝐼𝑜𝑜𝑜𝑜𝑜𝑜 𝑟𝑟, 𝑐𝑐 = 𝑓𝑓(𝐼𝐼𝑖𝑖𝑖𝑖 𝑟𝑟, 𝑐𝑐 )

Examples:

𝐼𝐼𝑜𝑜𝑜𝑜𝑜𝑜 𝑟𝑟, 𝑐𝑐 = 0.5 × 𝐼𝐼𝑖𝑖𝑖𝑖 𝑟𝑟, 𝑐𝑐 + 10

𝐼𝐼𝑜𝑜𝑜𝑜𝑜𝑜 𝑟𝑟, 𝑐𝑐 = min(𝐼𝐼𝑖𝑖𝑖𝑖 (𝑟𝑟, 𝑐𝑐), 255)

(Typically your code would compute the output

by looping through all valid r and c values)

15
 Histogram equalization is the adjusting of intensity values
in an image so that the new histogram is approximately flat

Although the idea is to generate a flat histogram,

this rarely happens because of the limitations of mapping
from and to quantized intensity values

Something like
this is more
likely

16
 The usual reason for histogram equalization
is to give the image a nicer appearance
(but doesn’t help with the operations that we’ll discuss)

17
 Multi-image point operations produce a result by combining
pixels at the same locations in two (or more) source images:

𝐼𝐼𝑜𝑜𝑜𝑜𝑜𝑜 𝑟𝑟, 𝑐𝑐 = 𝑓𝑓(𝐼𝐼1 𝑟𝑟, 𝑐𝑐 , 𝐼𝐼2 𝑟𝑟, 𝑐𝑐 )

For example, let I and J refer to two different images:

• add(I, J) will add two images pixel by pixel

– we often divide by 2, or otherwise scale the output
– useful for noise reduction from a time sequence

• subtract(I, J) or diff(I, J) will form the difference of two images

– useful for detecting motion
– useful for checking whether two images are registered spatially

18
Averaging two or more
registered images is a
common method for
reducing noise

19
Pixelwise subtraction will reveal differences between two images;
BUT noise is enhanced, and the registration must be excellent

20
 Computing the pixelwise max or min of two images
can be used to reduce the effect of artifacts like glare,
or to find occasional defects (industrial inspection)

MAX

21
 Many other pixelwise operations are possible

• Pixel-by-pixel multiplication of two images can be used to perform masking

• Can also be done in conjunction with pixelwise Boolean operations (AND, OR)

22
2-DIMENSIONAL FILTERS
(INTRODUCTION)

23
 Image filtering

Common approach: for each pixel in an input image, compute one value at the
corresponding pixel location in an output image
– Often compute a function of that pixel’s local neighborhood
– Apply the same function at each pixel location
– Output image and input image are typically the same size

10 5 3 Some function
4 6 1 8
1 1 8
Local image data One output pixel

Slide credit: adapted from L. Zhang

 One motivation: Image denoising

Original image
(some noise is visible)

Credit: Fisher et al.

 One motivation: Image denoising

Original image Processed image

(some noise is visible) (noise has been reduced, although
some details have been blurred as well)

Credit: Fisher et al.

Credit: Fisher et al.
 First attempt at a denoising solution

• Let’s replace each pixel with the average of all the values in its neighborhood
• Assumptions:
– Expect pixels to be like their neighbors
– Expect noise processes to be independent from pixel to pixel

Slide credit: Kristen Grauman 28

 Example: box filter

g[⋅ ,⋅ ]
1 1 1

1 1 1

1 1 1

Slide credit: David Lowe

 g[⋅ ,⋅ ] = 1 1 1
1 1 1

Image filtering 1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0
?
0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0
?
0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0 50

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0 90

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0 50

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 g[⋅ ,⋅ ] = 1 1 1
1 1 1
1 1 1

Image filtering

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10

0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
Slide credit: S. Seitz k ,l
 Box filter

g[⋅ ,⋅ ]
What does it do?
• Computes an output image in which each 1 1 1
original pixel has been replaced by the
average of its local neighborhood 1 1 1

1 1 1
• Achieves a smoothing effect
(reduces spatial changes in intensity)

Slide credit: David Lowe

Smoothing with a box filter
 Correlation filtering
First, suppose that the averaging window size is (2k+1) x (2k+1):

Attribute uniform Loop over all pixels in neighborhood

check the cls video
weight to each pixel around image pixel F[i,j]

Now generalize to allow different weights depending on

neighboring pixel’s relative position:

nonuniform weights are allowed

Slide credit: Kristen Grauman
 check the cls video

Correlation filtering

This is called cross-correlation, denoted

Creates a new image G, replacing each pixel from F with a

linear combination of that pixel’s neighbors

The filter H[u,v] is also called a “kernel” or “mask” or

“operator” or “template”;
it specifies the weights in the linear combination
Slide credit: Kristen Grauman
 Filtering an impulse signal
What is the result of filtering the impulse signal (image) F
with the arbitrary kernel H?

0 0 0 0 0 0 0

?
0 0 0 0 0 0 0
a b c
0 0 0 0 0 0 0
d e f
0 0 0 1 0 0 0
g h i
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0

Slide credit: Kristen Grauman

 Convolution
– Flip the filter in both dimensions (bottom to top, right to left)
– Then apply cross-correlation

G=H*F F

Notation for H
convolution
operator

Slide credit: Kristen Grauman

 Convolution vs. correlation
Convolution

Cross-correlation

For a Gaussian or box filter, how will the outputs differ?

If the input is an impulse signal, how will the outputs differ?
Slide credit: Kristen Grauman
 Convolution has many nice properties
• Linearity:
filter(f1 + f2) =
filter(f1) + filter(f2)

• Shift Invariance:
same behavior regardless of
pixel location:
filter(shift(f)) = shift(filter(f))
 More properties of linear filters

• Commutative: a * b = b * a
– Conceptually no difference between filter and signal

• Associative: a * (b * c) = (a * b) * c
– Often apply several filters one after another: (((a * b1) * b2) * b3)
– This is equivalent to applying one filter: a * (b1 * b2 * b3)

• Distributes over addition: a * (b + c) = (a * b) + (a * c)

• Scalars can be factored out: ka * b = a * kb = k (a * b)

• Identity: for unit impulse e = […, 0, 0, 1, 0, 0, …],

a*e=a
 Practice with linear filters

?
0 0 0
* 0 1 0 =
0 0 0

Original

Slide credit: David Lowe (UBC)

 Practice with linear filters

0 0 0
* 0 1 0 =
0 0 0

Original Filtered
(no change)

Slide credit: David Lowe (UBC)

 Practice with linear filters

- ?
0 0 0 1 1 1
*( 0 2 0 1 1 1 ) =
0 0 0 1 1 1

(Note that filter sums to 1)

Original

Slide credit: David Lowe (UBC)

 Practice with linear filters

-
0 0 0 1 1 1
0 2 0 1 1 1
0 0 0 1 1 1

Original Sharpening filter

- Accentuates differences with local average

Slide credit: David Lowe (UBC)

Sharpening

Slide credit: David Lowe (UBC)

 Image filtering

• Linear filtering
– Function is a weighted sum (or difference) of pixel values

10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5
• Many uses! Local image data Kernel Modified image data
– Enhance an image
• Denoise, smooth, increase contrast, etc.
– Extract information from an image
• Intensity edges, distinctive points, texture, etc.
– Detect patterns in an image
• Template matching

Slide credit: Derek Hoiem

Other examples

-1 0 1
-2 0 2
-1 0 1
Sobel

Vertical Edge
(absolute value)
Other examples

1 2 1
0 0 0
-1 -2 -1
Sobel

Horizontal Edge
(absolute value)
 Coming soon

• More discussion of image filtering

• Coordinate transformations

56
 Summary

• Intensity profiles

• Image histograms

• Pixelwise operations

• Two-dimensional spatial filters (introduction)

• Cross-correlation, convolution

• Properties of linear filters

57