What is Computer Vision? See, Decide, Act

Introduction

When a football referee checks an offside on VAR, a computer looks at the video and draws a line across the pitch. How does it know where the players are? It can't "see" the way you do. All it gets from a camera is a huge grid of numbers.

Computer vision (CV for short) is the part of artificial intelligence that turns those numbers into meaning: that's a ball, that's a player, he's offside. In this course you'll build systems like that yourself, with Python, OpenCV and a webcam.

By the end of this lesson you'll be able to:

say what computer vision is, in your own words
name the three stages every CV system goes through: Perceive, Decide, Act
spot CV at work in sport, shops, hospitals, farms and factories
draw the pipeline for a football offside detector

A goalkeeper sees, decides and moves, in that order. Miss one stage and the goal goes in.

Working principle

What computer vision is

Normal programming tells a computer exactly what to do with data. Computer vision teaches a computer to pull meaning out of pixels.

Computer vision is the ability of a machine to find objects, people, text and events in images and video, and to act on what it finds automatically.

A CV system can:

detect and classify things: ball, player, goal, referee
track movement from one frame to the next: follow a player many times a second
measure and count: how many players are in the penalty box?
trigger decisions: raise an alert when the ball crosses the goal line
describe what happened: "Player 7 is offside"

The pipeline: Perceive, Decide, Act

Every CV system, from a self-driving car to a football analytics tool, works in the same three stages. You'll use this pipeline in every lesson of the course.

Stage	What happens	In an offside detector	Tools you'll use
1. Perceive	A camera captures pictures. A model finds the objects in each frame.	A webcam films the pitch. A model called YOLOv8 draws a box round every player and the ball.	Webcam, OpenCV, YOLOv8
2. Decide	Rules or a model work out what the objects mean.	Code checks: is any attacker past the last defender when the ball is played?	Python, coordinates, rules
3. Act	The system does something about it.	If offside: draw a red line, log the moment, play an alert sound.	OpenCV drawing, a log file, sound

Where you'll find CV today

Area	What CV does	What it saves people from doing
Sport	Tracks players and the ball, checks offside, analyses shots	Hours of tagging video by hand
Shops	Checkout-free stores, empty-shelf alerts	Queues at the till, walking the aisles to check stock
Hospitals	Sorts X-rays, spots unusual scans	Gets the worrying scans to a doctor first
Roads	Keeps a car in its lane, spots people crossing	Reacts faster than a human driver
Farms	Finds sick crops in drone photos	Walking thousands of hectares looking for disease
Factories	Spots faulty products on a production line	Checking every item by eye, at a speed no person can manage
The ocean	Finds floating plastic, checks coral health	Sending divers down to look

An image is just numbers

Here's the key idea for the whole course. To a computer, a photo is a grid of numbers. Each pixel holds three numbers from 0 to 255: how much blue, green and red light is in that spot. A webcam picture that's 640 pixels wide and 480 tall is 921,600 numbers, and a webcam sends about 30 of those pictures every second.

Computer vision is the art of finding patterns in those numbers. The computer doesn't understand that it's looking at a ball. It has learnt that a certain pattern of numbers usually means "ball".

Tap a pixel. print(frame[4, 5]) gives [40 30 220]: blue, green, red, from 0 to 255. OpenCV puts blue first.

What you'll need

For this lesson, nothing but your brain and a sheet of paper. From the next lesson on you'll need:

a laptop (Windows, macOS or Linux) with a webcam
Python 3.10 or newer (you'll install it in lesson 2)

Sneak peek: what the camera really sends

You'll run this yourself in the next lesson. For now, just read it and see how little code it takes to grab a picture from a webcam and look inside it.

import cv2                      # OpenCV: lets Python use cameras and images
 
cap = cv2.VideoCapture(0)       # open the first webcam on this computer
ret, frame = cap.read()         # grab one picture. ret says if it worked
print(frame.shape)              # (480, 640, 3): 480 rows, 640 columns, 3 colours
print(frame[240, 320])          # one pixel from the middle, e.g. [ 87 112 140 ]
cap.release()                   # always give the camera back when you're done

That last print shows one pixel: three numbers for blue, green and red. That is all a picture is.

Try it

Pick one of these systems and draw its pipeline on paper: three boxes joined by arrows, labelled PERCEIVE → DECIDE → ACT. In each box, write exactly what happens.

Smart checkout: a supermarket where shoppers pick up items and walk out without paying at a till.
Plant disease scanner: a farmer points a phone at a leaf, and the app says whether the plant is healthy.
Crowd counter: a stadium camera counts people coming in and warns security when a stand is full.
Offside detector: a camera spots an attacker in an offside position when the ball is played.
Sprint timer: webcams at the start and finish lines start and stop a timer as a runner crosses.

Harder: add a fourth box after ACT. Many real systems have a LEARN stage, where they get better from their own mistakes. What would your system learn from?

Check what you learnt

A shop wants to know when a shelf is empty. Which stage is the camera taking a picture of the shelf?
Does a CV system understand what it sees?
A webcam picture has the shape (480, 640, 3). What is the 3?
How is computer vision related to AI?
In an offside detector, what happens in the Act stage?

Conclusion

Computer vision turns pixels into decisions. Every system you'll build in this course follows the same pipeline: Perceive (the camera and the model find things), Decide (rules work out what it means) and Act (the system responds). And underneath it all, a picture is just a grid of numbers.

What next?

Next: Python & OpenCV: set up your computer-vision workshop. You'll install everything and see your own webcam feed in a Python window.