AI & Computer Vision, lesson 7 of 52

Filters, Thresholds & Edges: Classic Computer Vision

beginner20 min read

Introduction

Long before AI, computer vision worked with plain arithmetic on pixels, and those tricks still do half the work in every modern system. Today you'll blur, sharpen, threshold and trace edges, then chain them into a pipeline that pulls the white lines off a football pitch.

By the end of this lesson you'll be able to:

  • Blur an image with a Gaussian filter and explain why blur helps before other steps
  • Sharpen an image with a kernel and describe what a kernel does
  • Turn a grayscale image into black and white with a fixed, an Otsu and an adaptive threshold
  • Find edges with the Canny detector and tune its two thresholds
  • Clean a mask with erosion, dilation, opening and closing
  • Combine these steps into a short pipeline that pulls out the pitch lines

Working principle

Kernels: a small window sliding over the image

A kernel is a small grid of numbers, for example 3 by 3 or 5 by 5. Slide it over the image. At each position, multiply the kernel by the pixels under it, add up the results, and write that into the output. Different numbers give different effects.

The yellow 3 by 3 kernel slides over the picture. Each output pixel is the average of the 9 under it, so sharp jumps get softer: that's a blur.

FilterWhat it doesOpenCV
Gaussian blurAverages a pixel with its neighbours, with the nearest counting most. Removes grain and noise.cv2.GaussianBlur(img, (5, 5), 0)
SharpenBoosts the difference between a pixel and its neighbourscv2.filter2D(img, -1, kernel)
Median blurReplaces each pixel with the median of its neighbours. Great for salt-and-pepper speckles.cv2.medianBlur(img, 5)

A common sharpen kernel is [[0,-1,0],[-1,5,-1],[0,-1,0]]. The kernel size must be odd.

Thresholding

Thresholding turns each grey pixel into black or white by comparing it with a cut-off.

TypeCodeUse when
Fixedcv2.threshold(gray, 200, 255, cv2.THRESH_BINARY)The lighting is steady and you know the cut-off
Otsucv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)The picture has two clear groups (light and dark). OpenCV picks the cut-off for you.
Adaptivecv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, -10)The light changes across the picture (shadows on one half of the pitch)

Edges: Canny

Canny finds places where brightness changes sharply. Steps inside: blur, find gradients, thin the edges, then keep strong edges and weak edges that touch strong ones. You give it two numbers.

edges = cv2.Canny(blurred, 50, 150)   # low and high thresholds

Lower both and you get more (noisier) edges. Raise both and you only keep the strongest.

Morphology: cleaning masks

OperationEffectCode
ErosionShrinks white regions. Removes tiny white specks.cv2.erode(mask, kernel)
DilationGrows white regions. Closes small gaps.cv2.dilate(mask, kernel)
OpeningErosion then dilation. Removes specks without shrinking the shape.cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
ClosingDilation then erosion. Fills small holes.cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)

kernel = np.ones((3, 3), np.uint8) is a good start.

What you'll need

  • Your cv_course environment with Jupyter
  • A football photo, football_pitch.jpg (Cell 1 makes a noisy pretend pitch if you don't have one)

Build it

Run the cells in order. If there is no photo, Cell 1 makes a noisy fake pitch.

Cell 1: load, or make a noisy pitch

import cv2
import numpy as np
import matplotlib.pyplot as plt
 
img = cv2.imread("football_pitch.jpg")
 
if img is None:
    rng = np.random.default_rng(1)
    img = np.zeros((300, 400, 3), np.uint8)
    img[:] = (34, 142, 28)                                       # grass
    cv2.rectangle(img, (20, 20), (380, 280), (235, 235, 235), 3)  # touchline
    cv2.line(img, (200, 20), (200, 280), (235, 235, 235), 3)      # halfway line
    cv2.circle(img, (200, 150), 40, (235, 235, 235), 3)           # centre circle
    cv2.circle(img, (120, 200), 10, (200, 60, 30), -1)            # player
    noise = rng.normal(0, 18, img.shape)
    img = np.clip(img + noise, 0, 255).astype(np.uint8)           # grain
 
 
def show(images, titles):
    fig, axes = plt.subplots(1, len(images), figsize=(5 * len(images), 4))
    axes = np.atleast_1d(axes)
    for ax, im, t in zip(axes, images, titles):
        if im.ndim == 3:
            ax.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))
        else:
            ax.imshow(im, cmap="gray")
        ax.set_title(t)
        ax.axis("off")
    plt.show()
 
 
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
show([img, gray], ["Original", "Grayscale"])

Cell 2: blur and sharpen

blur3 = cv2.GaussianBlur(gray, (3, 3), 0)
blur9 = cv2.GaussianBlur(gray, (9, 9), 0)
median = cv2.medianBlur(gray, 5)
 
kernel = np.array([[0, -1, 0],
                   [-1, 5, -1],
                   [0, -1, 0]], dtype=np.float32)
sharp = cv2.filter2D(gray, -1, kernel)
 
show([gray, blur3, blur9, sharp], ["Gray", "Blur 3", "Blur 9", "Sharpened"])

Question: which blur size removes the grain? What does sharpening do to the grain?

Cell 3: thresholds

blurred = cv2.GaussianBlur(gray, (5, 5), 0)
 
_, fixed = cv2.threshold(blurred, 200, 255, cv2.THRESH_BINARY)
otsu_value, otsu = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
adaptive = cv2.adaptiveThreshold(blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
                                 cv2.THRESH_BINARY, 31, -10)
 
print("Otsu chose the cut-off:", otsu_value)
show([fixed, otsu, adaptive], ["Fixed 200", "Otsu", "Adaptive"])

Question: which one finds the lines best? Which one would you trust if one half of the pitch were in shadow?

Cell 4: Canny edges

edges_low = cv2.Canny(blurred, 30, 90)
edges_high = cv2.Canny(blurred, 100, 200)
edges_noblur = cv2.Canny(gray, 100, 200)
 
show([edges_low, edges_high, edges_noblur],
     ["Canny 30/90", "Canny 100/200", "Canny, no blur"])

Question: what happens to the edges if you skip the blur?

Cell 5: the pitch-line pipeline

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
 
# White = any hue, low saturation, high brightness (from lesson 4)
white = cv2.inRange(hsv, (0, 0, 190), (179, 60, 255))
 
kernel = np.ones((3, 3), np.uint8)
opened = cv2.morphologyEx(white, cv2.MORPH_OPEN, kernel)      # remove specks
closed = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, kernel)    # fill tiny gaps
lines = cv2.dilate(closed, kernel, iterations=1)              # thicken slightly
 
show([white, opened, lines], ["White mask", "After opening", "Final lines"])
print("Line pixels:", int(np.count_nonzero(lines)))

Question: what does opening remove that the raw white mask contained? What would closing do on its own?

Cell 6: overlay the lines on the photo

overlay = img.copy()
overlay[lines > 0] = (0, 0, 255)          # paint detected lines red
show([img, overlay], ["Original", "Detected lines in red"])

If it goes wrong

ErrorCauseFix
error: (-215) ksize.width > 0 && ksize.width % 2 == 1Even blur kernel sizeUse an odd number: 3, 5, 7
Canny gives pure noiseNo blur first, or thresholds too lowBlur, then raise both thresholds
Threshold output is all black or all whiteCut-off too high or too low for this photoCheck the histogram, or try Otsu
Adaptive threshold error about blockSizeblockSize must be odd and greater than 1Use 11, 21, 31
Opening removes the whole lineThe kernel is thicker than the lineUse a smaller kernel, or open with a thin kernel like (2, 2)
Lines mask picks up white shirts tooWhite shirts match the white rangeLimit to the pitch area, or filter by shape in lesson 8

Try it

  • Use cv2.HoughLinesP on the edges to find the straight lines and draw them
  • Try a tophat filter to keep thin bright lines in uneven light
  • Compare Cell 5 on a sunny photo and a shadowed photo

Check what you learnt

  1. Why do we blur before running Canny?

  2. When is an adaptive threshold better than a fixed one?

  3. What does erosion do to a white line?

  4. Why must a blur kernel size be odd, like 5 or 7?

  5. What does Otsu's method do?

Conclusion

Classic computer vision is arithmetic on pixels. A kernel slides over the image to blur or sharpen, a threshold turns greyscale into black and white, Canny finds edges, and morphology (erode, dilate, open, close) tidies masks. Chained together, they pull the pitch lines out of a photo with no AI at all.

What next?

Next: Contours & colour tracking. You'll turn masks into shapes and follow a coloured ball around the screen.