Auto Highlight Reel

Introduction

A video editor spends hours finding the good bits of a match. Your system already knows when the goals happened. Today you'll make it do the editing too: cut a clip around every goal, join them smoothly with captions and fades, and export a highlight reel ready to share (with permission).

By the end of this lesson you'll be able to:

Plan clip windows around events, with time before and after
Merge overlapping windows and clamp them to the video
Add captions, fades and title cards to clips
Use ffmpeg to keep sound and make a widely playable file
Test the planning logic, including the edge cases
Discuss consent, music rights and fairness for a shareable video

Working principle

Automation that makes something

So far automation lit an LED or wrote a number. Now it makes a finished piece of media: a video that tells the story of the match, with nobody editing it. The pipeline is the one from lesson 31 with a new last step:

video -> detector -> events (lesson 31) -> windows -> clips -> reel

Events are the cut points

The goal events you verified in lesson 40 are the most trustworthy cut points you have. A false alarm in the reel is embarrassing; a missed goal is a hole. That is why we spent lesson 36 and lesson 40 on reliability.

A window around each event

A goal clip that starts at the instant the ball crosses the line misses the best part: the shot and the run-up. So each event gets time before (3 to 5 seconds) and a shorter time after (2 to 3 seconds) for the reaction. The numbers are choices: watch a clip and adjust.

Merging

Two events close together would produce two clips that overlap, so the viewer would see the same seconds twice. We merge windows that overlap or are within a second of each other, and combine their labels. Merging is just sorting plus one comparison, and the tests prove the edge cases (clamping, exact gaps, empty input).

Seconds before each goal8 s

4 goals give 3 clips after merging, 47 seconds in total. Overlapping windows join so no second is shown twice.

Captions and fades

A small caption ("1/3 GOAL") tells viewers what they are watching. A short fade in and out hides the jump between clips. These are what make the reel feel finished, not just a pile of clips.

Length budget

A reel nobody watches is no reel. Pick a maximum length and drop clips to fit (max_total). Our rule is the simplest one: keep the earliest. A better rule would rank clips by importance (for example, a goal beats a near miss). Writing a scoring rule is a good extension, but notice that you are now deciding what counts as exciting, and the code will faithfully apply your opinion every time.

Sound and formats

OpenCV writes silent video, in a format some browsers cannot play. ffmpeg keeps the sound and writes H.264 with AAC, which nearly every device plays. For a reel you will share, use ffmpeg. For learning, OpenCV is enough.

Consent: a highlight reel is the most shareable thing you will make. Videos of children should not go online without the consent from lesson 14, and the permission for public sharing is a bigger step than permission to record. Ask separately.
Music and clips: do not add commercial music without a licence. Use silence, your own sounds or properly licensed music.
Label it: say "automatically generated" on the reel. It is true, and it keeps people honest about mistakes.
Fairness: if only the striker's goals make the reel, the keeper's saves are invisible. Think about whose story the reel tells (lesson 43).

What you'll need

Your match recording and goal events from lesson 39
Your cv_course environment
ffmpeg installed and on your PATH

Build it

No video of your own yet? Use the synthetic match: copy scene.py and detector.py (lesson 36) and events.py and goal_detector.py (lesson 31), then write match.mp4 once:

import cv2
from scene import make_clip, W, H
 
frames, _ = make_clip(n=300)
out = cv2.VideoWriter("match.mp4", cv2.VideoWriter_fourcc(*"mp4v"), 30, (W, H))
for f in frames:
    out.write(f)
out.release()

Cell 1: plan the clips

Events go in; windows come out. This is plain logic, with no video, so it is easy to test.

"""Auto highlight reel: events in, a short video out."""
import cv2
import numpy as np
 
 
def plan_windows(events, video_seconds, before=4.0, after=3.0, merge_gap=1.0, max_total=None):
    """events: list of (time_seconds, label). Returns a list of windows
    {'start', 'end', 'labels'} - overlapping or nearly touching windows are merged into one clip.
    max_total: if given, drop the lowest-priority windows (later ones last) until the reel fits."""
    wins = []
    for t, label in sorted(events):
        s, e = max(0.0, t - before), min(video_seconds, t + after)
        if e <= s:
            continue
        if wins and s <= wins[-1]["end"] + merge_gap:             # overlaps (or nearly) the previous clip
            wins[-1]["end"] = max(wins[-1]["end"], e)
            wins[-1]["labels"].append(label)
        else:
            wins.append({"start": s, "end": e, "labels": [label]})
    if max_total is not None:
        while wins and sum(w["end"] - w["start"] for w in wins) > max_total:
            wins.pop()                                           # simple rule: keep the earliest ones
    return wins
 
 
def caption(frame, text, y=None):
    out = frame.copy()
    h, w = out.shape[:2]
    y = y or h - 20
    (tw, th), _ = cv2.getTextSize(text, cv2.FONT_HERSHEY_SIMPLEX, 0.9, 2)
    cv2.rectangle(out, (10, y - th - 10), (20 + tw, y + 8), (0, 0, 0), -1)
    cv2.putText(out, text, (15, y), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (255, 255, 255), 2)
    return out
 
 
def title_card(w, h, text, frames, colour=(40, 20, 6)):
    card = np.full((h, w, 3), colour, np.uint8)
    (tw, th), _ = cv2.getTextSize(text, cv2.FONT_HERSHEY_SIMPLEX, 1.2, 2)
    cv2.putText(card, text, ((w - tw) // 2, (h + th) // 2), cv2.FONT_HERSHEY_SIMPLEX, 1.2, (255, 255, 255), 2)
    return [card] * frames
 
 
def render_reel(video_path, windows, out_path, title="Match highlights", fade_frames=6):
    cap = cv2.VideoCapture(video_path)
    fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
    w, h = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)), int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    out = cv2.VideoWriter(out_path, cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))
    written = 0
    for f in title_card(w, h, title, int(fps * 1.5)):
        out.write(f); written += 1
    for n, win in enumerate(windows, 1):
        cap.set(cv2.CAP_PROP_POS_FRAMES, int(win["start"] * fps))
        count = int((win["end"] - win["start"]) * fps)
        label = f"{n}/{len(windows)}  " + " + ".join(win["labels"])
        for i in range(count):
            ok, frame = cap.read()
            if not ok:
                break
            frame = caption(frame, label)
            if i < fade_frames:                                  # fade in from black
                frame = (frame * ((i + 1) / fade_frames)).astype(np.uint8)
            elif i >= count - fade_frames:                       # fade out
                frame = (frame * ((count - i) / fade_frames)).astype(np.uint8)
            out.write(frame); written += 1
    for f in title_card(w, h, "The end", int(fps)):
        out.write(f); written += 1
    cap.release(); out.release()
    return written / fps                                         # length of the reel in seconds

Three ideas in plan_windows: each event gets a few seconds before (the build-up) and fewer after; windows that overlap or nearly touch are merged into one clip so you never see the same second twice; and clips are clamped to the video so you never ask for time that does not exist.

Cell 2: from video to reel

"""Full chain: video -> detector -> goal events -> highlight reel."""
import cv2
 
from detector import find_ball
from events import EventBus
from goal_detector import GoalDetector
from highlights import plan_windows, render_reel
from scene import GOAL
 
 
def find_goals(video_path):
    cap = cv2.VideoCapture(video_path)
    fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
    goals = []
    bus = EventBus()
    bus.on("goal", lambda zone, t: goals.append((t, "GOAL")))
    det = GoalDetector(GOAL, bus, confirm_frames=3, cooldown=3.0)
    i = 0
    while True:
        ok, frame = cap.read()
        if not ok:
            break
        det.update(find_ball(frame), i / fps)
        i += 1
    cap.release()
    return goals, i / fps
 
 
if __name__ == "__main__":
    import sys
    video = sys.argv[1] if len(sys.argv) > 1 else "match.mp4"
    goals, length = find_goals(video)
    print("goals found:", [(round(t, 1), l) for t, l in goals])
    windows = plan_windows(goals, length)
    print("windows:", [(round(w["start"], 1), round(w["end"], 1), w["labels"]) for w in windows])
    seconds = render_reel(video, windows, "highlights.mp4")
    print(f"wrote highlights.mp4, {seconds:.1f} s long")

Run python make_reel.py match.mp4. It printed this for us:

goals found: [(4.1, 'GOAL'), (8.7, 'GOAL')]
windows: [(0.1, 10.0, ['GOAL', 'GOAL'])]
wrote highlights.mp4, 12.4 s long

The two goals were 4.6 seconds apart, so their windows overlapped and became one clip with both labels. With real goals minutes apart you would get separate clips.

Cell 3: keep the sound (ffmpeg)

OpenCV cannot write sound, and its mp4v files do not play in every browser. For a reel people will actually watch, use ffmpeg. It is a separate program, installed once (from ffmpeg.org, or winget install ffmpeg, brew install ffmpeg, sudo apt install ffmpeg).

"""Same reel, but with ffmpeg: keeps the sound and makes a file every browser and phone can play (H.264 + AAC).
Install ffmpeg first (ffmpeg.org, or: winget install ffmpeg / brew install ffmpeg / apt install ffmpeg)."""
import os
import shutil
import subprocess
import tempfile
 
 
def ffmpeg_available():
    return shutil.which("ffmpeg") is not None
 
 
def cut_with_ffmpeg(video_path, windows, out_path, fade=0.2):
    if not ffmpeg_available():
        raise RuntimeError("ffmpeg not found. Install it, or use render_reel() from highlights.py (no sound).")
    tmp = tempfile.mkdtemp()
    parts = []
    for n, w in enumerate(windows):
        part = os.path.join(tmp, f"part{n:03d}.mp4")
        dur = w["end"] - w["start"]
        cmd = ["ffmpeg", "-y", "-loglevel", "error",
               "-ss", f"{w['start']:.3f}", "-t", f"{dur:.3f}", "-i", video_path,
               "-vf", f"fade=t=in:st=0:d={fade},fade=t=out:st={max(dur - fade, 0):.3f}:d={fade}",
               "-af", f"afade=t=in:st=0:d={fade},afade=t=out:st={max(dur - fade, 0):.3f}:d={fade}",
               "-c:v", "libx264", "-pix_fmt", "yuv420p", "-c:a", "aac", part]
        subprocess.run(cmd, check=True)                           # a list, not a string: no shell, no quoting bugs
        parts.append(part)
    listfile = os.path.join(tmp, "list.txt")
    with open(listfile, "w") as f:
        for p in parts:
            f.write(f"file '{p}'\n")
    subprocess.run(["ffmpeg", "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", listfile,
                    "-c", "copy", out_path], check=True)
    shutil.rmtree(tmp, ignore_errors=True)
    return out_path
 
 
def duration(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path],
                         capture_output=True, text=True, check=True)
    return float(out.stdout.strip())

Use it like this:

from highlights import plan_windows
from reel_ffmpeg import cut_with_ffmpeg
 
windows = plan_windows([(4.1, "GOAL")], video_seconds=10.0, before=2, after=2)
cut_with_ffmpeg("match.mp4", windows, "reel.mp4")

We tested it on a video with a 440 Hz tone: the 4-second reel came out 4.02 s long with an H.264 video stream and an AAC audio stream. (The captions and title cards come from the OpenCV version; ffmpeg adds fades. Combining both is an extension.)

Cell 4: tests

import cv2
import numpy as np
from highlights import plan_windows, caption, title_card, render_reel
 
bad = 0
def check(name, ok, extra=""):
    global bad; bad += not ok; print("PASS" if ok else "FAIL", name, extra)
 
w = plan_windows([(20, "GOAL")], 100, before=4, after=3)
check("one event, one clip of 7 s", len(w) == 1 and w[0]["start"] == 16 and w[0]["end"] == 23, w)
w = plan_windows([(2, "GOAL")], 100, before=4, after=3)
check("clip clamps to the start of the video", w[0]["start"] == 0 and w[0]["end"] == 5, w)
w = plan_windows([(98, "GOAL")], 100, before=4, after=3)
check("clip clamps to the end of the video", w[0]["end"] == 100, w)
w = plan_windows([(20, "GOAL"), (25, "GOAL")], 100, before=4, after=3)
check("overlapping windows are merged", len(w) == 1 and w[0]["start"] == 16 and w[0]["end"] == 28 and len(w[0]["labels"]) == 2, w)
w = plan_windows([(20, "GOAL"), (40, "GOAL")], 100)
check("far apart events stay separate", len(w) == 2)
w = plan_windows([(40, "B"), (20, "A")], 100)
check("events are sorted first", [x["labels"] for x in w] == [["A"], ["B"]])
w = plan_windows([(23, "A"), (31, "B")], 100, before=4, after=3, merge_gap=1.0)
check("a gap of exactly the merge gap merges (26 vs 27)", len(w) == 1, w)
w = plan_windows([(10, "A"), (30, "B"), (50, "C")], 100, max_total=15)
check("max_total drops later clips to fit", len(w) == 2 and sum(x["end"] - x["start"] for x in w) <= 15, w)
check("no events, no windows", plan_windows([], 100) == [])
check("event past the end of the video is ignored", plan_windows([(150, "X")], 100, before=4, after=3) == [])
 
frame = np.full((360, 640, 3), 100, np.uint8)
c = caption(frame, "GOAL 4.1 s")
check("caption changes the picture but not the original", (c != frame).any() and (frame == 100).all())
t = title_card(640, 360, "Match highlights", 10)
check("title card has the right number of frames", len(t) == 10 and t[0].shape == (360, 640, 3))
 
secs = render_reel("match.mp4", plan_windows([(4.1, "GOAL"), (8.7, "GOAL")], 10.0, before=1.5, after=1.5), "t.mp4")
cap = cv2.VideoCapture("t.mp4"); n = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)); cap.release()
check("reel length = 1.5 s title + clips + 1 s end", abs(secs - (1.5 + 3.0 + 3.0 + 1.0)) < 0.5 and n > 200, (round(secs, 2), n))
print("failures:", bad)

All thirteen should print PASS. Add a test of your own: what should happen when two events are exactly merge_gap apart?

Questions: who decides what is a "highlight": the code or you? What other events could make a highlight (a save, a tackle, a fast sprint from lesson 45)? How long should a reel be before people stop watching? Could the reel ever be unfair to a player?

If it goes wrong

Error	Cause	Fix
Reel has no sound	OpenCV writes silent video	Use the ffmpeg version
Reel won't play in the browser	`mp4v` codec	Re-encode with ffmpeg (H.264 and AAC)
`FileNotFoundError: ffmpeg`	Not installed or not on PATH	Install it and restart the terminal
Clip starts too late	Event time is late (lesson 40 offset) or before is too small	Use the offset; raise `before`
Same moment shown twice	Windows not merged	Check `merge_gap` and the sort order
Zero clips	No goals found, or all outside the video	Print the events and the video length
Reel is too long	Many clips	Use `max_total` or a ranking rule
Jumpy audio	Cuts without fades	Raise `fade`

Try it

Add a second event type (a fast sprint, lesson 45) with its own caption
Rank clips by importance and keep the best within the budget
Combine the OpenCV captions with the ffmpeg audio

Check what you learnt

Why start the clip a few seconds before the goal?
Why merge windows that overlap?
Why use ffmpeg instead of only OpenCV?
What must you have before sharing the reel online?
Who decides what counts as a highlight?

Conclusion

A highlight reel is automation that makes something. Each event becomes a cut point, a window adds build-up before and celebration after, overlapping windows merge, captions and fades smooth the joins, and ffmpeg exports the result with sound. The rules are yours, and so is the responsibility to get consent before sharing.

What next?

Next: Bias, fairness audit & model card. Unit 10 starts: bias, fairness and writing a model card.