Object Detection: How YOLO Finds Things

Introduction

Your ball classifier answers one question: is there a ball? A real sports system needs more: where is every player, and where's the ball, in every frame. That's object detection. Today you'll learn how detectors work, what IoU and non-maximum suppression do, and run YOLO, one of the fastest detectors there is, on a football clip.

By the end of this lesson you'll be able to:

Say how detection differs from classification
Read what one detection contains: box, class, confidence
Calculate IoU and use it to decide whether a box is a hit
Explain confidence thresholds and non-maximum suppression (NMS)
Describe in plain words how YOLO "looks once" using grid cells at several scales
Run a pretrained YOLO model on a football clip and read its output
Say what a pretrained model can and cannot find (COCO classes)

Working principle

Classification versus detection

	Classification (lesson 18)	Detection (today)
Question	What is in this picture?	What is where?
Output	One label and a confidence	A list of: box, label, confidence
Objects per image	One main thing	Any number
Needs labels of	Whole images (or crops)	Boxes (what you drew in lesson 12)

What one detection looks like

class: sports ball    confidence: 0.82    box: x1=410, y1=280, x2=430, y2=300

The box is usually given by two corners, xyxy, in pixels. (Label files use centre and size as fractions: see lesson 12.)

IoU: is this box right?

Intersection over Union: overlap area ÷ combined area. 1.0 is a perfect match and 0 is no overlap. A common rule is that IoU of 0.5 or more counts as a hit. Two cautions: a box can be almost right and still miss (0.42 below), and a too-big box also scores badly.

Confidence threshold

The model gives every candidate a score. We keep only those above a threshold (for example 0.25).

Lower threshold: more detections and more false alarms
Higher threshold: fewer false alarms and more misses

It is the same precision-recall trade-off as lesson 19.

Non-maximum suppression (NMS)

A detector often draws several overlapping boxes around the same player. NMS cleans up: keep the highest-scoring box, throw away any other box that overlaps it too much (IoU above a threshold), then repeat with what's left.

The network makes 12 guesses, many around the same player.

Some newer YOLO versions are designed to avoid this extra step (they're called "NMS-free"), but the idea is still the best way to understand duplicates.

How YOLO looks once

YOLO splits the image into a grid and every cell predicts boxes for objects whose centres fall inside it. It does this at three scales at once:

Scale (stride)	Grid on a 640 by 640 image	Best at
32	20 × 20	Large objects (a goal, a close player)
16	40 × 40	Medium objects
8	80 × 80	Small objects (a distant player, the ball)

Modern YOLO is anchor-free: instead of starting from preset box shapes, each cell predicts the box edges directly. One pass through the network gives all boxes, which is why it is fast.

Pretrained models and COCO

A pretrained YOLO was trained on COCO, 80 everyday classes. It already knows person (id 0) and sports ball (id 32). It does not know goal or referee, and its "sports ball" is not always your ball in a pitch-side frame. That's why we fine-tune in lesson 24.

What you'll need

Your cv_course environment with Ultralytics: pip install ultralytics
A football clip (match.mp4), or the demo clip from lesson 5

Build it

Cell 1: IoU diagrams

Draws four prediction boxes against one "truth" box and saves four pictures (iou_0.jpg to iou_3.jpg). Green is the truth and red is the prediction.

import cv2
import numpy as np
 
 
def iou(a, b):
    """a, b = (x1, y1, x2, y2). Overlap area divided by the area covered by either box."""
    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
    inter = max(0, ix2 - ix1) * max(0, iy2 - iy1)
    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
    return inter / union if union > 0 else 0.0
 
 
truth = (100, 80, 220, 260)                       # the box a human drew
guesses = {
    "almost perfect": (105, 85, 225, 262),
    "shifted": (140, 100, 260, 280),
    "too big": (70, 50, 260, 300),
    "wrong place": (300, 80, 420, 260),
}
 
for i, (name, g) in enumerate(guesses.items()):
    img = np.full((320, 460, 3), 255, np.uint8)
    cv2.rectangle(img, truth[:2], truth[2:], (0, 160, 0), 2)           # green = truth
    cv2.rectangle(img, g[:2], g[2:], (0, 0, 220), 2)                    # red = prediction
    cv2.putText(img, f"{name}: IoU {iou(truth, g):.2f}", (10, 25), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 0), 2)
    cv2.imwrite(f"iou_{i}.jpg", img)
    print(f"{name:15s} IoU = {iou(truth, g):.2f}   counts as a hit at 0.5? {iou(truth, g) >= 0.5}")

Questions: why does "shifted" score less than 0.5 even though it clearly finds the same player? Which would be worse for a tracker, a box that is too big or one that is shifted?

Cell 2: confidence threshold and NMS

import numpy as np
 
 
def iou(a, b):
    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
    inter = max(0, ix2 - ix1) * max(0, iy2 - iy1)
    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
    return inter / union if union > 0 else 0.0
 
 
def nms(boxes, scores, iou_threshold=0.5):
    """Non-maximum suppression: keep the best box, drop its near-duplicates, repeat."""
    order = list(np.argsort(scores)[::-1])             # best first
    keep = []
    while order:
        best = order.pop(0)
        keep.append(best)
        order = [i for i in order if iou(boxes[best], boxes[i]) < iou_threshold]
    return keep
 
 
# Four boxes all claiming the same player, one separate ball, one weak guess on the grass
boxes = [(100, 80, 200, 260), (104, 84, 206, 262), (96, 78, 198, 256), (110, 90, 210, 270),
         (400, 300, 418, 318), (500, 50, 540, 120)]
scores = np.array([0.91, 0.88, 0.80, 0.55, 0.77, 0.12])
names = ["player", "player", "player", "player", "ball", "player"]
 
CONF_THRESHOLD = 0.25
print("1. before anything:", len(boxes), "boxes")
confident = [i for i in range(len(boxes)) if scores[i] >= CONF_THRESHOLD]
print(f"2. after confidence >= {CONF_THRESHOLD}:", len(confident), "boxes")
kept = nms([boxes[i] for i in confident], scores[confident], iou_threshold=0.5)
final = [confident[k] for k in kept]
print("3. after NMS:", len(final), "boxes ->", [(names[i], float(scores[i])) for i in final])
 
# What if the NMS threshold is too strict, or too loose?
for t in (0.1, 0.5, 0.95):
    k = nms([boxes[i] for i in confident], scores[confident], iou_threshold=t)
    print(f"NMS threshold {t}: keeps {len(k)} boxes")

Questions: at NMS threshold 0.95, why do duplicate boxes survive? What would happen to two players standing close together if the NMS threshold were 0.1? (Try it by moving the second player's box next to the first.)

Cell 3: the grids

import cv2
import numpy as np
 
W = H = 640
ball_centre = (437, 291)                       # where the ball's centre is, in pixels
img = np.full((H, W, 3), (40, 130, 40), np.uint8)
cv2.circle(img, ball_centre, 9, (255, 255, 255), -1)
 
for stride, colour in ((32, (255, 200, 0)), (16, (0, 200, 255)), (8, (255, 0, 255))):
    cells = W // stride
    col, row = ball_centre[0] // stride, ball_centre[1] // stride
    print(f"stride {stride:2d}: grid {cells} x {cells} = {cells * cells:5d} cells; ball centre falls in cell (col {col}, row {row})")
    if stride == 32:                           # draw only the coarse grid so the picture stays readable
        for k in range(cells + 1):
            cv2.line(img, (k * stride, 0), (k * stride, H), colour, 1)
            cv2.line(img, (0, k * stride), (W, k * stride), colour, 1)
    cv2.rectangle(img, (col * stride, row * stride), ((col + 1) * stride, (row + 1) * stride), colour, 2)
cv2.imwrite("grid.jpg", img)

Open grid.jpg. The coarse grid is drawn, and the one cell at each scale that "owns" the ball is outlined.

Question: which of the three cells is the smallest? Why does the finest grid help find a small ball?

Cell 4: a pretrained YOLO on a match clip

Install: pip install ultralytics. The first run downloads the weights. Use a football clip you are allowed to use (your own, with permission, as in lesson 14). Run python yolo_run.py match.mp4, or without an argument for the webcam. Press q to stop.

import sys
import time
 
import cv2
from ultralytics import YOLO
 
MODEL = "yolov8n.pt"            # downloads the first time. Newer family: "yolo26n.pt" (see the lesson)
SOURCE = sys.argv[1] if len(sys.argv) > 1 else 0      # a video file, or 0 for the webcam
CONF = 0.25                     # ignore detections below this confidence
CLASSES = [0, 32]               # COCO ids: 0 = person, 32 = sports ball
 
model = YOLO(MODEL)
print("The model knows", len(model.names), "classes. Person =", model.names[0], "| id 32 =", model.names[32])
 
cap = cv2.VideoCapture(SOURCE)
frames, counts = 0, {"person": 0, "sports ball": 0}
start = time.time()
 
while True:
    ok, frame = cap.read()
    if not ok:
        break
    result = model(frame, conf=CONF, classes=CLASSES, verbose=False)[0]
 
    for box in result.boxes:
        x1, y1, x2, y2 = (int(v) for v in box.xyxy[0].tolist())
        name = model.names[int(box.cls[0])]
        counts[name] += 1
        if frames % 30 == 0:                             # print a few so the output stays readable
            print(f"frame {frames}: {name} {float(box.conf[0]):.2f} at ({x1},{y1})-({x2},{y2})")
 
    cv2.imshow("YOLO", result.plot())                  # result.plot() draws boxes and labels for us
    frames += 1
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break
 
cap.release()
cv2.destroyAllWindows()
seconds = time.time() - start
print(f"{frames} frames in {seconds:.1f}s = {frames / seconds:.1f} FPS")
print("total detections:", counts)

To try the newest family, change MODEL to "yolo26n.pt" (check the Ultralytics documentation for the current names).

Questions: how many players did it find compared with how many there were? Did it find the ball? In which frames does it fail (blur, far away, partly hidden)? What FPS did you get on your laptop?

If it goes wrong

Error	Cause	Fix
`ModuleNotFoundError: ultralytics`	Not installed	`pip install ultralytics` in the same Python as the script
Weights fail to download	Blocked network	Download `yolov8n.pt` on another machine and put it beside the script
Very low FPS	CPU only	Expected. Use a smaller video, or wait for lesson 25.
No balls found	The ball is tiny, blurry or in a crowd	Normal for the pretrained model. Note it as evidence for fine-tuning.
Everyone is detected, including the crowd	`classes=[0, ...]` includes all people	That is correct. Crop the pitch area in lesson 32.
`cv2.imshow` window freezes	No `waitKey` or running without a display	Keep `cv2.waitKey(1)` in the loop. On a remote server write frames to a file.
Class id confusion	COCO ids differ from your own data's	The pretrained model uses COCO ids (0 and 32). Your trained model (lesson 24) will use yours.
IoU is higher than 1	Boxes not in corner format	Use `(x1, y1, x2, y2)`

Try it

Implement a function that returns the best-matching truth box for each prediction
Plot the number of detections per frame against the confidence threshold (0.1 to 0.9)
Try yolo26n.pt or yolo11n.pt and compare FPS and misses

Check what you learnt

What does a detector output for each object?
What does IoU measure?
What does non-maximum suppression (NMS) do?
Why is the ball hard for a detector?
Why isn't the pretrained model enough for our project?

Conclusion

A detector gives a box, class and confidence for every object. IoU measures how well a box fits, the confidence threshold drops weak guesses, and NMS removes duplicates. YOLO looks at the whole picture once, using grid cells at several scales, which is why it's fast enough for live video.

What next?

Next: Train a custom sports detector. You'll train YOLO on your own football data.