AI & Computer Vision, lesson 46 of 52

Formation & Tactical Analysis

advanced30 min read

Introduction

A team is like a flock of birds: nobody draws its shape, yet you can see it stretch when it attacks and squeeze when it defends. Today you'll measure that shape from your tracked positions: the lines (defence, midfield, attack), how wide and deep the team is, how compact, and how it changes with the ball.

By the end of this lesson you'll be able to:

  • Convert positions into depth for a team that attacks left or right
  • Find the lines of a formation by cutting at the biggest gaps
  • Measure width, depth and convex hull area
  • Estimate possession from the ball and the nearest player
  • Compare team shape with and without the ball
  • Make measured claims without telling the coach what to do

Working principle

Team shape is a statement about positions

A formation such as 4-4-2 means: four defenders, four midfielders, two forwards, plus the keeper. It is a label for a shape. From positions in metres (lesson 33) we can look at the shape directly instead of trusting a label.

Average position

Players move around. Their average position over a period is a good first summary: where they spend their time. It is the centre of the heatmap (lesson 41). Use the average over a minute or a phase of play, not the whole match, or you blur everything.

Finding the lines

To find "four, three, three":

  1. Take the outfield players (leave out the keeper).
  2. Measure how far up the pitch each one is. This depends on which way the team attacks, so we convert x into depth: distance from their own goal line.
  3. Sort by depth. Cut the sorted list at the biggest gaps. Two cuts make three lines.
  4. Count the players in each line, from back to front.

It is simple, and it works when lines are clear. It forces the number of lines you ask for, so it can't discover that a team has changed shape; and it cannot tell a flat midfield from a midfield with a "number 10".

Width, depth and compactness

MeasureMeaningHow we compute
WidthHow wide the team spreadsLargest minus smallest y of outfield players
DepthHow stretched from back to frontLargest minus smallest depth
AreaHow much ground is coveredArea of the convex hull

Small area = compact: hard to play through, but leaves space behind. Large area = stretched: more room for passes, more room for the opponent too. Neither is "right".

Who has the ball?

Tactics depend on the phase: with the ball, without it, or loose. Our proxy: the team of the nearest player, if within 6 metres. In our pretend match it matched the truth in 100% of frames; in a real match it will make mistakes (players close together, the ball in the air, a ball tracker that loses the ball). Treat the phases as estimates (lesson 36, lesson 40).

Directions and halves

Teams swap ends at half time, and our depth calculation needs the direction. Give each half its own direction, or the numbers will be mirrored. This is the kind of bug that makes a formation look "reversed" and a coach distrust the whole system.

Describe, don't prescribe

Computers can say: "this team had a width of 49 m with the ball and 35 m without". They cannot say "that was the right way to play". Tactics depend on the opponent, the score, the weather, the players' ages and the coach's plan. When you present results, say what you measured, how sure you are, and leave the judging to the people.

Be careful with claims about children's teams

Young players change position often and learn the game in different roles. Fixed labels ("a defender") can limit them. Use the data for the coach to discuss with the players, never to fix roles, rank players or select teams without human judgement (lesson 43).

What a camera can't see

A formation analysis needs all the players in view and a good calibration (lesson 33). A camera that sees only half the pitch cannot analyse a team's depth. If players are missing from the tracks, the shape is wrong. Report how many players you could see in each frame.

What you'll need

  • Tracked positions in pitch metres with team labels (or the sample CSV)
  • Your cv_course environment with SciPy: pip install scipy

Build it

Cell 1: two pretend teams with known shapes

Red plays 4-3-3 attacking right, blue plays 4-4-2 attacking left. A ball moves between players. When a team has the ball it pushes up and spreads wide; without it, it drops back and squeezes. Because we planted these behaviours, we can check whether the code finds them.

"""Two pretend teams with known formations, so we can check that the code finds them.
Red plays 4-3-3 and attacks to the right. Blue plays 4-4-2 and attacks to the left.
When a team has the ball it stretches forward and wide; without it, it squeezes together. Keeper = id 1 and 12."""
import json
 
import numpy as np
 
rng = np.random.default_rng(5)
FPS, MINUTES, NOISE = 5, 8, 1.2
W, H = 105.0, 68.0
 
 
def anchors(shape, attack_right=True):
    """x as fraction of the pitch length for each line, players spread across the width."""
    lines = {"4-3-3": [(0.28, 4), (0.45, 3), (0.65, 3)], "4-4-2": [(0.28, 4), (0.48, 4), (0.65, 2)]}[shape]
    out = [(0.05, 0.5)]                                           # goalkeeper
    for x, n in lines:
        for k in range(n):
            out.append((x, (k + 1) / (n + 1)))
    if not attack_right:
        out = [(1 - x, y) for x, y in out]
    return [(x * W, y * H) for x, y in out]
 
 
red = anchors("4-3-3", True)
blue = anchors("4-4-2", False)
n = FPS * 60 * MINUTES
frames = []
ball = np.array([W / 2, H / 2])
poss, carrier, loose_until = "red", None, -1
for i in range(n):
    if i % (FPS * 20) == 0:                                       # possession changes every 20 s
        poss = "blue" if poss == "red" else "red"
        carrier = None
    if carrier is None or i % (FPS * 3) == 0:                      # a new player gets the ball every 3 s (a pass)
        carrier = int(rng.integers(1, 11))                         # index 1..10 = an outfield player of the team with the ball
        if rng.random() < 0.10:
            loose_until = i + FPS * 2                              # sometimes the ball runs loose for 2 s
    players = []
    for team, anc, right in (("red", red, True), ("blue", blue, False)):
        has = (team == poss)
        for k, (ax, ay) in enumerate(anc):
            pid = k + 1 if team == "red" else k + 12
            if k == 0:
                x, y = ax, ay + (ball[1] - H / 2) * 0.15
            else:
                forward = (6.0 if has else -4.0) * (1 if right else -1)               # push up with the ball, drop without it
                spread = 1.20 if has else 0.85                                        # wider with the ball, narrower without
                x = ax + forward + (ball[0] - W / 2) * 0.10
                y = H / 2 + (ay - H / 2) * spread + (ball[1] - H / 2) * 0.10
            players.append({"id": pid, "team": team, "x": round(float(np.clip(x + rng.normal(0, NOISE), 0, W)), 2),
                            "y": round(float(np.clip(y + rng.normal(0, NOISE), 0, H)), 2)})
    cid = carrier + (0 if poss == "red" else 11)
    cp = next(p for p in players if p["id"] == cid)
    offset = rng.normal(0, 0.6, 2) + (np.array([9.0, 6.0]) if i < loose_until else 0)
    ball = np.array([cp["x"], cp["y"]]) + offset
    frames.append({"frame": i, "t": round(i / FPS, 2), "players": players,
                   "ball": {"x": round(float(np.clip(ball[0], 0, W)), 2), "y": round(float(np.clip(ball[1], 0, H)), 2)},
                   "true_possession": poss, "ball_loose": bool(i < loose_until)})
json.dump({"units": "metres", "fps": FPS, "frames": frames}, open("match_two_teams.json", "w"))
print("frames:", len(frames), "| red 4-3-3 (ids 1-11), blue 4-4-2 (ids 12-22)")

Cell 2: the tactics tools

"""Team shape from positions in metres: average positions, formation lines, width, depth, compactness, possession."""
import json
import math
from collections import defaultdict
 
import cv2
import numpy as np
 
PITCH_W, PITCH_H = 105.0, 68.0
 
 
def load_match(path="match_two_teams.json"):
    d = json.load(open(path))
    return d["fps"], d["frames"]
 
 
def average_positions(frames, team):
    """Mean (x, y) of each player of `team` over the given frames."""
    acc = defaultdict(list)
    for fr in frames:
        for p in fr["players"]:
            if p["team"] == team:
                acc[p["id"]].append((p["x"], p["y"]))
    return {pid: tuple(np.mean(v, axis=0)) for pid, v in acc.items()}
 
 
def depth_of(x, attack_right):
    """How far UP the pitch a point is for this team (0 = own goal line, 105 = opponent's goal line)."""
    return x if attack_right else PITCH_W - x
 
 
def split_lines(avg, attack_right=True, keeper_id=None, n_lines=3):
    """Group outfield players into lines by cutting at the biggest gaps in depth.
    Returns counts from back to front, for example [4, 3, 3]. The keeper is left out."""
    if keeper_id is None:                                         # the keeper is the player closest to their own goal
        keeper_id = min(avg, key=lambda pid: depth_of(avg[pid][0], attack_right))
    outfield = sorted(((depth_of(x, attack_right), pid) for pid, (x, y) in avg.items() if pid != keeper_id))
    depths = [d for d, _ in outfield]
    if len(depths) < n_lines:
        return [len(depths)]
    gaps = sorted(range(len(depths) - 1), key=lambda i: depths[i + 1] - depths[i], reverse=True)[:n_lines - 1]
    cuts = sorted(i + 1 for i in gaps)
    bounds = [0] + cuts + [len(depths)]
    return [bounds[i + 1] - bounds[i] for i in range(len(bounds) - 1)]
 
 
def formation_name(counts):
    return "-".join(str(c) for c in counts)
 
 
def hull_area(points):
    """Area (square metres) of the smallest convex shape around the points: how much ground the team covers."""
    if len(points) < 3:
        return 0.0
    pts = np.array(points, np.float32).reshape(-1, 1, 2)
    return float(cv2.contourArea(cv2.convexHull(pts)))
 
 
def shape_numbers(players, attack_right=True, keeper_id=None):
    """Width (side to side), depth (back to front) and hull area of the outfield players in ONE frame."""
    if keeper_id is None:
        keeper_id = min(players, key=lambda p: depth_of(p["x"], attack_right))["id"]
    out = [p for p in players if p["id"] != keeper_id]
    xs = [depth_of(p["x"], attack_right) for p in out]
    ys = [p["y"] for p in out]
    return {"width": max(ys) - min(ys), "depth": max(xs) - min(xs), "area": hull_area([(p["x"], p["y"]) for p in out])}
 
 
def possession(frame, max_distance=6.0):
    """Who has the ball? The team of the nearest player, if that player is within `max_distance` metres. Else None (loose ball)."""
    b = frame["ball"]
    if not b or not frame["players"]:
        return None
    nearest = min(frame["players"], key=lambda p: math.hypot(p["x"] - b["x"], p["y"] - b["y"]))
    return nearest["team"] if math.hypot(nearest["x"] - b["x"], nearest["y"] - b["y"]) <= max_distance else None
 
 
def shape_by_phase(frames, team, attack_right):
    """Average width, depth and area when the team has the ball, when the opponent has it, and when it is loose."""
    acc = defaultdict(list)
    for fr in frames:
        mine = [p for p in fr["players"] if p["team"] == team]
        if len(mine) < 4:
            continue
        who = possession(fr)
        phase = "loose" if who is None else ("with ball" if who == team else "without ball")
        acc[phase].append(shape_numbers(mine, attack_right))
    return {ph: {k: float(np.mean([r[k] for r in rows])) for k in ("width", "depth", "area")} | {"frames": len(rows)}
            for ph, rows in acc.items()}
 
 
def formation_over_time(frames, team, attack_right, fps, window_seconds=60):
    """The formation name in each window. A stable team gives the same name each time."""
    step = int(window_seconds * fps)
    return [formation_name(split_lines(average_positions(frames[i:i + step], team), attack_right))
            for i in range(0, len(frames) - step + 1, step)]
 
 
def draw(avg, attack_right, colour, canvas=None, scale=8, margin=20):
    w, h = int(PITCH_W * scale) + 2 * margin, int(PITCH_H * scale) + 2 * margin
    if canvas is None:
        canvas = np.full((h, w, 3), (50, 120, 50), np.uint8)
        cv2.rectangle(canvas, (margin, margin), (w - margin, h - margin), (255, 255, 255), 2)
        cv2.line(canvas, (w // 2, margin), (w // 2, h - margin), (255, 255, 255), 2)
    pts = {pid: (int(margin + x * scale), int(margin + y * scale)) for pid, (x, y) in avg.items()}
    keeper = min(avg, key=lambda pid: depth_of(avg[pid][0], attack_right))
    hull = cv2.convexHull(np.array([p for pid, p in pts.items() if pid != keeper], np.int32))
    cv2.polylines(canvas, [hull], True, colour, 2)
    for pid, p in pts.items():
        cv2.circle(canvas, p, 9, colour, -1)
        cv2.putText(canvas, str(pid), (p[0] - 7, p[1] + 4), cv2.FONT_HERSHEY_SIMPLEX, 0.4, (255, 255, 255), 1)
    return canvas

Cell 3: what the code finds

red  formation (attacking right): 4-3-3
blue formation (attacking left):  4-4-2
possession proxy agrees with the planted truth on 100% of frames
red  with ball     width  48.9 m   depth  41.1 m   area   1781 m2
red  without ball  width  35.1 m   depth  41.1 m   area   1275 m2
blue with ball     width  50.0 m   depth  40.7 m   area   1722 m2
blue without ball  width  36.1 m   depth  40.7 m   area   1227 m2

The formations were found correctly. The planted effect was found too: with the ball, red is about 14 m wider (48.9 against 35.1) and covers about 40% more ground. Depth did not change in our data because our pretend teams move up and down as a block; a real team's depth would change, and we would see it.

Cell 4: tests

import numpy as np
from tactics import *
 
bad = 0
def check(name, ok, extra=""):
    global bad; bad += not ok; print("PASS" if ok else "FAIL", name, extra)
 
fps, fr = load_match("match_two_teams.json")
red, blue = average_positions(fr, "red"), average_positions(fr, "blue")
check("red (attacking right) is found to play 4-3-3", formation_name(split_lines(red, True)) == "4-3-3", formation_name(split_lines(red, True)))
check("blue (attacking left) is found to play 4-4-2", formation_name(split_lines(blue, False)) == "4-4-2", formation_name(split_lines(blue, False)))
check("get the direction wrong and the answer is back to front", formation_name(split_lines(red, False)) != "4-3-3", formation_name(split_lines(red, False)))
check("four lines: 4-4-2 becomes 4-4-1-1 or similar, not a crash", sum(split_lines(blue, False, n_lines=4)) == 10 and len(split_lines(blue, False, n_lines=4)) == 4, split_lines(blue, False, n_lines=4))
 
# a school 7-a-side team: keeper + 2-3-1
seven = {1: (5, 34), 2: (22, 20), 3: (22, 48), 4: (45, 12), 5: (45, 34), 6: (45, 56), 7: (70, 34)}
check("7-a-side 2-3-1 found", formation_name(split_lines(seven, True)) == "2-3-1", split_lines(seven, True))
check("too few players for the lines does not crash", split_lines({1: (5, 34), 2: (30, 34)}, True, n_lines=3) == [1])
 
# formation is stable over time
names = formation_over_time(fr, "red", True, fps, 60)
check("red keeps the same formation in every minute", len(set(names)) == 1 and len(names) == 8, names)
 
# hull area
square = [(0, 0), (10, 0), (10, 10), (0, 10), (5, 5)]
check("hull of a 10 m square (with a point inside) is 100", abs(hull_area(square) - 100) < 1e-3, hull_area(square))
check("fewer than 3 points: area 0", hull_area([(0, 0), (1, 1)]) == 0.0)
 
# possession
frame = {"ball": {"x": 50, "y": 30}, "players": [{"id": 1, "team": "red", "x": 52, "y": 30}, {"id": 2, "team": "blue", "x": 61, "y": 30}]}
check("nearest player within 6 m has the ball", possession(frame) == "red")
frame["ball"]["x"] = 56
check("red 4 m away, blue 5 m away: red has it", possession(frame) == "red")
frame["ball"]["x"] = 40
check("ball more than 6 m from everyone is loose", possession(frame) is None)
check("no ball: None", possession({"ball": None, "players": frame["players"]}) is None)
truthful = [f for f in fr if not f["ball_loose"]]
agree = sum(possession(f) == f["true_possession"] for f in truthful) / len(truthful)
check("possession proxy agrees with the planted truth on non-loose frames", agree > 0.97, f"{agree:.1%}")
 
# planted effect recovered: wider and bigger with the ball
ph = shape_by_phase(fr, "red", True)
check("red is wider with the ball than without", ph["with ball"]["width"] > ph["without ball"]["width"] + 8, f"{ph['with ball']['width']:.1f} vs {ph['without ball']['width']:.1f}")
check("red covers more ground with the ball than without", ph["with ball"]["area"] > ph["without ball"]["area"] * 1.2, f"{ph['with ball']['area']:.0f} vs {ph['without ball']['area']:.0f}")
phb = shape_by_phase(fr, "blue", False)
check("blue shows the same pattern (direction handled)", phb["with ball"]["width"] > phb["without ball"]["width"] + 8)
 
img = draw(red, True, (40, 40, 220)); img = draw(blue, False, (220, 90, 30), img)
cv2.imwrite("formations.png", img)
check("picture drawn", img.shape == (584, 880, 3) and img.std() > 10)
print("failures:", bad)

All 18 should print PASS. Look at two of them closely: if you tell the code the wrong attacking direction, it gives a back-to-front answer (2-7-1 instead of 4-3-3), and asking for four lines splits 4-4-2 into 4-4-1-1. Neither is a bug; both are the code obeying what it was told. Teams swap ends at half time, so analyse each half separately.

Cell 5: draw it

import cv2
from tactics import load_match, average_positions, draw
 
fps, frames = load_match("match_two_teams.json")
red, blue = average_positions(frames, "red"), average_positions(frames, "blue")
img = draw(red, True, (40, 40, 220))                 # BGR: red
img = draw(blue, False, (220, 90, 30), img)          # BGR: blue, on the same pitch
cv2.imwrite("formations.png", img)

Open formations.png. The shapes drawn around each team are the convex hull: the smallest outline that contains all outfield players. Its area says how much ground the team covers.

Cell 6: with your own data

Your tracker (lesson 26) gives positions without teams. Two ways to add them:

  • By hand for now: write down which ids are which team from the first few frames.
  • By kit colour: crop each player, find the dominant shirt colour (k-means on the pixels), and split into two groups. This is a good extension. Referees and keepers will often form extra groups.

Put "team" on each player in the JSON, then use the same functions. Use the half-time frame number to split the match.

Questions: if a team is "4-3-3" in the average but changes constantly, is the label useful? What does a formation hide? Whose job is it to decide what a team should do: the data's, or the coach's? What could we measure to say "the team is pressing"?

If it goes wrong

ErrorCauseFix
Formation comes out back to frontWrong attacking directionSet attack_right per team and per half
Keeper counted as a defenderKeeper not found or not in viewPass keeper_id explicitly
4-3-3 becomes 4-2-4 sometimesLines not clear, or a player out of position that minuteUse a longer window; show the average positions and look
Width and depth look too bigA tracking glitch put a player far awayCheck for impossible speeds (lesson 45), remove glitch frames
Possession flips constantlyBall tracker loses the ballSmooth it, or require 3 frames in a row
One team has 9 playersPlayers lost by the trackerReport the count; don't analyse frames with fewer than 8
Teams mixed upSame-colour kits, or ids swappedLabel by hand; fix the tracks (lesson 26)
Hull area is zeroFewer than three players in the frameSkip those frames

Try it

  • Add a pressing measure: how fast the nearest defenders close the ball down
  • Split by half and compare both
  • Cluster kit colours to label teams automatically

Check what you learnt

  1. How do we find the lines (defence, midfield, attack)?

  2. Why leave the goalkeeper out of the shape?

  3. What is the convex hull area?

  4. Why does each team's attacking direction matter?

  5. Can these numbers tell a coach what to do?

Conclusion

Team shape is a set of measurements: average positions, the lines (found by the biggest gaps in depth), width, depth, compactness and the convex hull, all relative to the direction the team attacks. They describe what happened, clearly and comparably; deciding what it means is still the coach's job.

What next?

Next: Capstone kickoff: define your product. Unit 11 starts: your capstone project.